From 19f086773e9640ad1fe930526b665977597ceb5c Mon Sep 17 00:00:00 2001 From: MODELBEAST Date: Sun, 12 Jul 2026 22:38:52 +1000 Subject: [PATCH] Headless control: mb CLI, AGENTS.md handover, serve scripts; verified-current image models MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - mb: zero-dependency python CLI for the full API (ops/upload/run/wait/follow/ download incl. folder assets, retry/cancel, settings). Schema-aware -p k=v coercion. Works from any tailnet machine via MB_HOST. Tested end-to-end (upload -> blender_convert -> download). - AGENTS.md: complete handover brief for other agents using this box as an asset factory — endpoints, CLI reference, catalog, recipes, lanes/etiquette. - scripts/serve.sh (headless start/restart) + scripts/install_launchagent.sh (optional boot persistence, owner-run) - flux_local upgraded to mflux 0.18 reality: flux2-klein-4b default (Apache, UNGATED — runs with zero keys), klein-9b, schnell/schnell-4bit community quant, dev/krea-dev. Research verdict: FLUX.1-dev no longer competitive (Elo ~1027) vs klein ~1083-1119 vs nano-banana ~1154. - openrouter_image rewritten to the dedicated Image API (POST /api/v1/images): b64_json parsing, seed, resolution/aspect, exact usage.cost logging; model enum: gemini-2.5-flash-image / 3.1-flash-image (NB2) / 3-pro-image Co-Authored-By: Claude Fable 5 --- AGENTS.md | 117 ++++++++ README.md | 8 + mb | 256 ++++++++++++++++++ scripts/install_launchagent.sh | 31 +++ scripts/serve.sh | 12 + server/operators/flux_local/manifest.json | 12 +- server/operators/flux_local/run.py | 60 ++-- .../operators/openrouter_image/manifest.json | 8 +- server/operators/openrouter_image/run.py | 67 ++--- 9 files changed, 499 insertions(+), 72 deletions(-) create mode 100644 AGENTS.md create mode 100755 mb create mode 100755 scripts/install_launchagent.sh create mode 100755 scripts/serve.sh diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..9e98721 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,117 @@ +# AGENTS.md — Using the M3 Ultra asset factory (MODELBEAST) + +**Audience: any agent (or human) that needs 3D models, gaussian splats, camera-pose +datasets, generated/cleaned images, or format conversions.** This machine — an +M3 Ultra Mac Studio, 256GB unified memory — runs MODELBEAST, a job server that +wraps local Apple-Silicon ML tools and cloud APIs behind one HTTP API. You send +files and job requests; it returns finished assets. + +## 1. Reaching it + +- **Base URL (tailnet):** `http://100.89.131.57:8777` — from the machine itself, `http://localhost:8777`. +- **No auth.** The tailnet is the security boundary. Do not expose this port publicly. +- Web UI at the same URL (humans). Agents use the REST API or the `mb` CLI. +- If the server is down (connection refused): on the M3 run `~/Documents/MODELBEAST/scripts/serve.sh` (or ask the owner to run `scripts/install_launchagent.sh` once for boot persistence). + +## 2. The `mb` CLI (recommended) + +One stdlib-only python3 file at repo root — copy it anywhere (`scp m3ultra:~/Documents/MODELBEAST/mb .`) and point it at the box: + +```bash +export MB_HOST=http://100.89.131.57:8777 + +./mb ops -v # operator catalog + every parameter (LIVE truth — trust this over any doc) +./mb upload photo.jpg # -> prints asset id +./mb run fal_trellis2 --asset -p resolution=1024 --wait --download out/ +./mb run flux_local -p prompt="a brass astrolabe" -p seed=7 --follow +./mb jobs # recent jobs ./mb log -f # stream a log +./mb assets --kind model # browse assets ./mb get -o out/ +./mb retry / ./mb cancel / ./mb rm-job +``` + +- `mb run --file local.png ...` uploads and runs in one step. +- `--wait` blocks and prints output asset ids; `--download DIR` also fetches them; `--follow` streams the live log. Exit codes: 0 done, 2 error, 3 cancelled. +- `-p k=v` is type-coerced against the operator's schema; `--params-json '{...}'` for anything complex. + +### Raw REST (if you can't run python) +``` +GET /api/operators # catalog incl. params_schema (JSON Schema) +POST /api/assets # multipart file upload -> asset +GET /api/assets # list (outputs have parent_job=) +GET /api/assets/{id}/file[?member=f] # download (folder assets: JSON listing, then ?member=) +POST /api/jobs # {"operator": id, "asset_id": id|null, "params": {...}} +GET /api/jobs/{id} # status: queued|running|done|error|cancelled + log +POST /api/jobs/{id}/cancel | /retry # control +``` +Poll `GET /api/jobs/{id}` every ~2s. When `done`, your outputs are the assets whose `parent_job` equals the job id. + +## 3. What it can make (operator catalog — run `mb ops -v` for live params) + +| Goal | Operator(s) | Notes | +|---|---|---| +| **image → 3D mesh (GLB)** | `fal_trellis2` (SOTA open, PBR, $0.25–0.35) · `fal_hunyuan3d_v21` (~90s, PBR) · `fal_rodin` (hero-grade, $0.40+) · `fal_trellis` ($0.02 draft) · local: `trellis_mac`, `sf3d` | **Best practice: run `fal_bg_remove` first** and feed the cutout — biggest quality lever. Local ops need the owner's HF token; fal ops need FAL_KEY (check `mb ops` for `[needs ...]` flags) | +| **video → gaussian splat** | `ffmpeg_frames` → `colmap_poses` → `brush_train` | 100% local/free. Orbit video, 2–3 fps sampling, 150–300 frames. 30k steps = quality, 1500 = fast preview | +| **video → camera poses** | `ffmpeg_frames` → `colmap_poses` | Outputs a COLMAP dataset folder (cameras/images/points3D) | +| **prompt → image** | `flux_local` (on-device, free, seed-reproducible; `flux2-klein-4b` works with zero keys) · `openrouter_image` (nano-banana family, ~$0.04–0.24/img) · `fal_text_image` (Ideogram, readable text) | A/B: same prompt through several, then judge | +| **image cleanup** | `fal_bg_remove` (cutout) · `fal_upscale` (faithful) · `fal_image_edit` (prompt edits: "remove the sticker") | All sub-cent to low-cent | +| **3D format conversion** | `blender_convert` | GLB/GLTF/OBJ/FBX/STL/PLY/USD/BLEND in → glb/fbx/obj/usd/usdz/stl/ply/blend out | +| **media inspection** | `ffprobe` | codec/resolution/fps/duration JSON | + +## 4. Recipes + +**Photo → clean game-ready mesh (the standard chain):** +```bash +CUT=$(./mb run fal_bg_remove --file product.jpg --wait | awk '/image/{print $1}' | tail -1) +./mb run fal_trellis2 --asset $CUT -p resolution=1024 --wait --download out/ +``` + +**Orbit video → splat:** +```bash +VID=$(./mb upload orbit.mp4 | awk '{print $1}') +FRAMES=$(./mb run ffmpeg_frames --asset $VID -p fps=2.5 -p max_frames=250 --wait | awk '/frames/{print $1}' | tail -1) +DS=$(./mb run colmap_poses --asset $FRAMES --wait | awk '/colmap_dataset/{print $1}' | tail -1) +./mb run brush_train --asset $DS -p total_steps=30000 --wait --download out/ +``` + +**Local vs cloud image A/B (fixed prompt):** +```bash +P="isometric low-poly tavern, warm lantern light" +./mb run flux_local -p prompt="$P" -p model=flux2-klein-4b -p seed=7 --wait +./mb run openrouter_image -p prompt="$P" -p model=google/gemini-2.5-flash-image --wait +# then compare in the web UI (⊞ Compare) or download both +``` + +**Drop-folder ingest (no API needed):** copy files into `~/Documents/MODELBEAST/data/inbox/` on the M3 — they auto-register as assets within ~6s (size-stable check). Useful for rsync/UE/Blender exports. + +## 5. Job semantics you must respect + +- **Concurrency lanes:** `gpu` jobs run ONE at a time (Metal contention — trellis_mac, sf3d, brush_train, flux_local); `cpu` 3 at once; `net` (cloud APIs) 6 at once. Queued jobs wait their turn — don't spam retries because something is `queued`. +- **First-run weight downloads** on local ML operators can take 10–30+ min (klein ~15GB, schnell ~24GB, TRELLIS ~15GB). The job log shows download progress; be patient, don't cancel. +- Logs stream into the job record (`mb log ID -f`). Secrets are redacted server-side. +- A job that ends `error` has the reason in `error` + last log lines. `mb retry ID` re-runs it with identical inputs. +- Server restarts re-queue anything that was running. Jobs are cheap to re-run — assets are the durable artifacts. + +## 6. Keys, gating, and cost + +- Gated operators are visible but refuse to run with a clear error until the owner adds the key in ⚙ Settings (`fal_key`, `openrouter_key`, `hf_token`). Check gating live via `mb ops` (`[needs ...]`). +- Rough costs when unlocked: fal_trellis $0.02 · hunyuan v21 ~$0.05–0.10 · trellis2 $0.25–0.35 · rodin $0.40+ · bg_remove/upscale sub-cent · nano-banana ~$0.039 · nano-banana-pro ~$0.134 · everything local = $0. +- **Never ask for or handle the raw keys.** They live in the server's settings vault. + +## 7. Etiquette & limits (you are a guest on someone's workstation) + +- Don't delete assets/jobs you didn't create. +- Big batches: stagger `gpu` work; the owner may be using the machine. +- Disk: weights + assets are large; if you generate hundreds of assets, download what you need and `mb rm-asset` your intermediates. +- The box also hosts Ollama (`http://100.89.131.57:11434`, qwen3:235b/32b/8b) if you need local LLM inference — separate service, same etiquette. +- Benchmarks so far: `BENCHMARKS.md`. Architecture and roadmap: `README.md`, `HANDOFF.md`, `PLAN.md`. + +## 8. Ops runbook (on the M3) + +```bash +cd ~/Documents/MODELBEAST +./scripts/serve.sh # start/restart headless (log: data/server.log) +./tests/smoke.sh # 12-check regression suite (~30s, isolated data dir) +./mb jobs # what's running +git pull && cd web && npm run build && cd .. && ./scripts/serve.sh # deploy an update +``` +Repo: `ssh://git@100.71.119.27:222/monster/modelbeast.git`. Heavy tools reinstall via `scripts/install_*.sh` (vendor/, venvs/, bin/ are not in git). diff --git a/README.md b/README.md index 8eb18e8..e67993f 100644 --- a/README.md +++ b/README.md @@ -12,6 +12,14 @@ Open http://localhost:8777 (or `http://:8777` from any device on t Drag/drop/paste any video, image, or 3D file; or drop files into `data/inbox/` for auto-ingest. Pick an operator, tune its parameters, run. Add API keys / HuggingFace token under ⚙ Settings. Use ⊞ Compare to view several outputs side by side. +## Headless / agents + +Everything is driveable without the UI via the zero-dependency [`mb`](mb) CLI (works from any tailnet machine with `MB_HOST=http://100.89.131.57:8777`) or raw REST. **[AGENTS.md](AGENTS.md) is the complete handover brief** — endpoints, `mb` reference, operator catalog, recipes, job semantics, etiquette. Server ops: `scripts/serve.sh` (start/restart headless), `scripts/install_launchagent.sh` (optional boot persistence, owner-run). + +```bash +./mb run fal_trellis2 --file photo.jpg -p resolution=1024 --wait --download out/ +``` + ## Develop - Backend: `server/` — FastAPI + SQLite (`data/modelbeast.db`), job runner runs operators as subprocesses across concurrency lanes (gpu=1, cpu=3, net=6). Settings/secrets in `server/settings.py` (env-injected, log-redacted). diff --git a/mb b/mb new file mode 100755 index 0000000..9c2f66b --- /dev/null +++ b/mb @@ -0,0 +1,256 @@ +#!/usr/bin/env python3 +"""mb — headless CLI for MODELBEAST. Zero dependencies (python3 stdlib only), +works from any machine that can reach the server (tailnet included). + + export MB_HOST=http://100.89.131.57:8777 # default: http://localhost:8777 + + mb ops [-v] list operators (+params with -v) + mb assets [--kind image] list assets + mb upload FILE [FILE...] upload files, print asset ids + mb run OP [--asset ID | --file F] [-p k=v ...] [--params-json J] + [--wait] [--follow] [--download DIR] + mb job ID job status json + mb jobs [-n 20] recent jobs + mb log ID [-f] print job log (-f = follow to completion) + mb wait ID [--download DIR] block until job finishes (exit 0/2/3) + mb get ASSET_ID [-o PATH] download an asset (folders: all members) + mb rm-asset ID / mb rm-job ID delete + mb retry ID / mb cancel ID job control + mb settings show settings (secrets masked) + mb set KEY VALUE set a setting (e.g. keys — value visible in + shell history; prefer the web UI for secrets) + +Typical: mb run flux_local -p prompt="a brass astrolabe" -p seed=7 --wait --download out/ +Exit codes: 0 ok/done, 1 usage/HTTP error, 2 job error, 3 job cancelled. +""" +import argparse +import json +import mimetypes +import os +import sys +import time +import urllib.error +import urllib.request +import uuid +from pathlib import Path + +HOST = os.environ.get("MB_HOST", "http://localhost:8777").rstrip("/") + + +def http(method, path, body=None): + req = urllib.request.Request(HOST + path, method=method) + data = None + if body is not None: + data = json.dumps(body).encode() + req.add_header("Content-Type", "application/json") + try: + with urllib.request.urlopen(req, data=data, timeout=120) as r: + ct = r.headers.get("content-type", "") + raw = r.read() + return json.loads(raw) if "json" in ct else raw + except urllib.error.HTTPError as e: + sys.exit(f"HTTP {e.code} {method} {path}: {e.read().decode()[:500]}") + except urllib.error.URLError as e: + sys.exit(f"cannot reach {HOST} ({e.reason}) — is the server up? set MB_HOST?") + + +def upload_file(path): + p = Path(path) + if not p.is_file(): + sys.exit(f"no such file: {p}") + boundary = uuid.uuid4().hex + ctype = mimetypes.guess_type(p.name)[0] or "application/octet-stream" + body = (f"--{boundary}\r\nContent-Disposition: form-data; name=\"file\"; " + f"filename=\"{p.name}\"\r\nContent-Type: {ctype}\r\n\r\n").encode() + body += p.read_bytes() + f"\r\n--{boundary}--\r\n".encode() + req = urllib.request.Request(HOST + "/api/assets", data=body, method="POST") + req.add_header("Content-Type", f"multipart/form-data; boundary={boundary}") + try: + with urllib.request.urlopen(req, timeout=600) as r: + return json.loads(r.read()) + except urllib.error.HTTPError as e: + sys.exit(f"upload failed: HTTP {e.code}: {e.read().decode()[:300]}") + + +def coerce(op, key, value): + """Coerce k=v strings using the operator's JSON schema, else best-effort JSON.""" + prop = (op.get("params_schema", {}).get("properties", {}) or {}).get(key, {}) + t = prop.get("type") + try: + if t == "integer": + return int(value) + if t == "number": + return float(value) + if t == "boolean": + return value.lower() in ("1", "true", "yes", "on") + if t == "string": + return value + return json.loads(value) + except (ValueError, json.JSONDecodeError): + return value + + +def outputs_of(job_id): + return [a for a in http("GET", "/api/assets") if a.get("parent_job") == job_id] + + +def download_asset(asset, dest_dir): + dest_dir = Path(dest_dir) + dest_dir.mkdir(parents=True, exist_ok=True) + url = f"{HOST}/api/assets/{asset['id']}/file" + with urllib.request.urlopen(url, timeout=600) as r: + ct = r.headers.get("content-type", "") + data = r.read() + if "json" in ct: + listing = json.loads(data) + if listing.get("folder"): + sub = dest_dir / asset["name"] + sub.mkdir(exist_ok=True) + for member in listing["files"]: + murl = url + "?member=" + urllib.request.quote(member) + with urllib.request.urlopen(murl, timeout=600) as r: + (sub / member).write_bytes(r.read()) + print(f" ↓ {sub} ({len(listing['files'])} files)") + return + out = dest_dir / asset["name"] + out.write_bytes(data) + print(f" ↓ {out} ({len(data)} bytes)") + + +def wait_for(job_id, follow=False, download=None): + printed = 0 + last_status = None + while True: + j = http("GET", f"/api/jobs/{job_id}") + if follow: + log = j.get("log") or "" + if len(log) > printed: + sys.stdout.write(log[printed:]) + sys.stdout.flush() + printed = len(log) + elif j["status"] != last_status: + print(f"[{job_id}] {j['status']}") + last_status = j["status"] + if j["status"] in ("done", "error", "cancelled"): + if j["status"] == "error": + print(f"ERROR: {j.get('error')}\n--- last log ---\n{(j.get('log') or '')[-1500:]}", + file=sys.stderr) + sys.exit(2) + if j["status"] == "cancelled": + sys.exit(3) + outs = outputs_of(job_id) + print(f"done — {len(outs)} output(s):") + for a in outs: + print(f" {a['id']} {a['kind']:<14} {a['name']}") + if download: + for a in outs: + download_asset(a, download) + return + time.sleep(2) + + +def main(): + ap = argparse.ArgumentParser(prog="mb", add_help=True) + sub = ap.add_subparsers(dest="cmd", required=True) + + sub.add_parser("ops").add_argument("-v", action="store_true") + p = sub.add_parser("assets"); p.add_argument("--kind") + p = sub.add_parser("upload"); p.add_argument("files", nargs="+") + p = sub.add_parser("run") + p.add_argument("operator") + p.add_argument("--asset"); p.add_argument("--file") + p.add_argument("-p", "--param", action="append", default=[]) + p.add_argument("--params-json") + p.add_argument("--wait", action="store_true") + p.add_argument("--follow", action="store_true") + p.add_argument("--download") + p = sub.add_parser("job"); p.add_argument("id") + p = sub.add_parser("jobs"); p.add_argument("-n", type=int, default=20) + p = sub.add_parser("log"); p.add_argument("id"); p.add_argument("-f", action="store_true") + p = sub.add_parser("wait"); p.add_argument("id"); p.add_argument("--download") + p = sub.add_parser("get"); p.add_argument("id"); p.add_argument("-o", default=".") + p = sub.add_parser("rm-asset"); p.add_argument("id") + p = sub.add_parser("rm-job"); p.add_argument("id") + p = sub.add_parser("retry"); p.add_argument("id") + p = sub.add_parser("cancel"); p.add_argument("id") + sub.add_parser("settings") + p = sub.add_parser("set"); p.add_argument("key"); p.add_argument("value") + a = ap.parse_args() + + if a.cmd == "ops": + for op in http("GET", "/api/operators"): + gate = f" [needs {','.join(op['requires_env'])}]" if op.get("requires_env") else "" + inputs = ",".join(op["accepts"]) or "none (prompt-driven)" + print(f"{op['id']:<20} {op.get('category','?'):<11} in:{inputs:<18} " + f"lane:{op.get('resources','cpu')}{gate}") + if a.v: + for k, d in (op.get("params_schema", {}).get("properties", {}) or {}).items(): + dv = f" (default {d['default']})" if "default" in d else "" + en = f" one of {d['enum']}" if "enum" in d else "" + print(f" -p {k}=<{d.get('type','any')}>{en}{dv} {d.get('description','')}") + elif a.cmd == "assets": + for x in http("GET", "/api/assets"): + if a.kind and x["kind"] != a.kind: + continue + print(f"{x['id']} {x['kind']:<14} {x['size']:>10} {x['name']}") + elif a.cmd == "upload": + for f in a.files: + asset = upload_file(f) + print(f"{asset['id']} {asset['kind']} {asset['name']}") + elif a.cmd == "run": + ops = {o["id"]: o for o in http("GET", "/api/operators")} + if a.operator not in ops: + sys.exit(f"unknown operator '{a.operator}'. Run: mb ops") + op = ops[a.operator] + asset_id = a.asset + if a.file: + asset_id = upload_file(a.file)["id"] + print(f"uploaded → {asset_id}") + if op["accepts"] and not asset_id: + sys.exit(f"{a.operator} needs an input ({','.join(op['accepts'])}): --asset ID or --file F") + params = {} + if a.params_json: + params.update(json.loads(a.params_json)) + for kv in a.param: + if "=" not in kv: + sys.exit(f"bad -p '{kv}' (want k=v)") + k, v = kv.split("=", 1) + params[k] = coerce(op, k, v) + job = http("POST", "/api/jobs", + {"operator": a.operator, "asset_id": asset_id, "params": params}) + print(f"job {job['id']} queued") + if a.wait or a.follow or a.download: + wait_for(job["id"], follow=a.follow, download=a.download) + elif a.cmd == "job": + print(json.dumps(http("GET", f"/api/jobs/{a.id}"), indent=2)) + elif a.cmd == "jobs": + for j in http("GET", "/api/jobs")[: a.n]: + print(f"{j['id']} {j['status']:<9} {j['operator']}") + elif a.cmd == "log": + if a.f: + wait_for(a.id, follow=True) + else: + print(http("GET", f"/api/jobs/{a.id}").get("log") or "(empty)") + elif a.cmd == "wait": + wait_for(a.id, download=a.download) + elif a.cmd == "get": + assets = {x["id"]: x for x in http("GET", "/api/assets")} + if a.id not in assets: + sys.exit("no such asset") + download_asset(assets[a.id], a.o) + elif a.cmd == "rm-asset": + http("DELETE", f"/api/assets/{a.id}"); print("deleted") + elif a.cmd == "rm-job": + http("DELETE", f"/api/jobs/{a.id}"); print("deleted") + elif a.cmd == "retry": + j = http("POST", f"/api/jobs/{a.id}/retry"); print(f"new job {j['id']}") + elif a.cmd == "cancel": + http("POST", f"/api/jobs/{a.id}/cancel"); print("cancel requested") + elif a.cmd == "settings": + print(json.dumps(http("GET", "/api/settings"), indent=2)) + elif a.cmd == "set": + http("PUT", "/api/settings", {a.key: a.value}); print("saved") + + +if __name__ == "__main__": + main() diff --git a/scripts/install_launchagent.sh b/scripts/install_launchagent.sh new file mode 100755 index 0000000..773561b --- /dev/null +++ b/scripts/install_launchagent.sh @@ -0,0 +1,31 @@ +#!/usr/bin/env bash +# OPTIONAL (owner runs this by hand): keep MODELBEAST running across reboots +# via a per-user LaunchAgent. Remove with: +# launchctl bootout gui/$(id -u)/com.modelbeast.server +# rm ~/Library/LaunchAgents/com.modelbeast.server.plist +set -euo pipefail +cd "$(dirname "$0")/.." +ROOT=$(pwd) +PLIST="$HOME/Library/LaunchAgents/com.modelbeast.server.plist" +mkdir -p "$HOME/Library/LaunchAgents" "$ROOT/data" +cat > "$PLIST" < + + + Labelcom.modelbeast.server + ProgramArguments + /opt/homebrew/bin/uv + runuvicornserver.main:app + --host0.0.0.0 + --port8777 + + WorkingDirectory$ROOT + KeepAlive + RunAtLoad + StandardOutPath$ROOT/data/server.log + StandardErrorPath$ROOT/data/server.log + +EOF +launchctl bootout "gui/$(id -u)/com.modelbeast.server" 2>/dev/null || true +launchctl bootstrap "gui/$(id -u)" "$PLIST" +echo "LaunchAgent installed + started. Server will survive reboots." diff --git a/scripts/serve.sh b/scripts/serve.sh new file mode 100755 index 0000000..2205274 --- /dev/null +++ b/scripts/serve.sh @@ -0,0 +1,12 @@ +#!/usr/bin/env bash +# Start (or restart) the MODELBEAST server headless on port 8777. +set -euo pipefail +cd "$(dirname "$0")/.." +lsof -ti:8777 | xargs kill 2>/dev/null || true +sleep 1 +nohup /opt/homebrew/bin/uv run uvicorn server.main:app --host 0.0.0.0 --port 8777 \ + > data/server.log 2>&1 & +sleep 2 +curl -sf -o /dev/null http://localhost:8777/api/operators \ + && echo "MODELBEAST up: http://$( /Applications/Tailscale.app/Contents/MacOS/Tailscale ip -4 2>/dev/null || echo localhost):8777 (log: data/server.log)" \ + || { echo "FAILED to start — tail data/server.log:"; tail -20 data/server.log; exit 1; } diff --git a/server/operators/flux_local/manifest.json b/server/operators/flux_local/manifest.json index 5d592db..071babd 100644 --- a/server/operators/flux_local/manifest.json +++ b/server/operators/flux_local/manifest.json @@ -2,7 +2,7 @@ "id": "flux_local", "name": "FLUX (local, MLX)", "category": "generate", - "description": "Prompt → image entirely on this Mac via mflux (MLX-native FLUX). schnell = 2-4 steps fast draft; dev = higher quality, 20-28 steps. BOTH are HF-gated now (verified 2026-07): accept the license on huggingface.co/black-forest-labs, then set the HuggingFace token in Settings. First run downloads weights (~24GB). No API cost, no cloud.", + "description": "Prompt → image entirely on this Mac via mflux 0.18 (MLX). flux2-klein-4b = Apache/UNGATED, works out of the box, better than FLUX.1-schnell (~15GB first download). schnell-4bit = ungated community quant. schnell/dev/krea-dev/klein-9b are HF-gated (license accept + HF token in Settings). No API cost, no cloud. Elo context: klein ~1083-1119, FLUX.1-dev ~1027, nano-banana ~1154.", "accepts": [], "produces": ["image"], "resources": "gpu", @@ -12,13 +12,13 @@ "type": "object", "properties": { "prompt": {"type": "string", "default": "", "description": "What to generate"}, - "model": {"type": "string", "enum": ["schnell", "dev"], "default": "schnell", "description": "schnell = fast/ungated; dev = best quality, HF-gated"}, - "steps": {"type": "integer", "default": 4, "minimum": 1, "maximum": 50, "description": "Inference steps (schnell: 2-4, dev: 20-28)"}, + "model": {"type": "string", "enum": ["flux2-klein-4b", "flux2-klein-9b", "schnell", "schnell-4bit", "dev", "krea-dev"], "default": "flux2-klein-4b", "description": "klein-4b + schnell-4bit are ungated; the rest need the HF token"}, + "steps": {"type": "integer", "default": 4, "minimum": 1, "maximum": 50, "description": "klein/schnell: 2-4; dev/krea-dev: 20-28"}, "width": {"type": "integer", "default": 1024, "minimum": 256, "maximum": 2048}, "height": {"type": "integer", "default": 1024, "minimum": 256, "maximum": 2048}, - "seed": {"type": "integer", "default": 42, "description": "Random seed (fixed seed = reproducible A/B tests)"}, - "quantize": {"type": "string", "enum": ["none", "8", "4"], "default": "8", "description": "Weight quantization: 8-bit ≈ full quality, half the memory; none = bf16"}, - "guidance": {"type": "number", "default": 3.5, "minimum": 0, "maximum": 10, "description": "Guidance scale (dev only; schnell ignores it)"} + "seed": {"type": "integer", "default": 42, "description": "Fixed seed = reproducible A/B tests"}, + "quantize": {"type": "string", "enum": ["none", "8", "4"], "default": "8", "description": "FLUX.1 models only: 8-bit ≈ full quality at half memory"}, + "guidance": {"type": "number", "default": 3.5, "minimum": 0, "maximum": 10, "description": "dev/krea-dev only"} } } } diff --git a/server/operators/flux_local/run.py b/server/operators/flux_local/run.py index 8498f42..ced9ebd 100644 --- a/server/operators/flux_local/run.py +++ b/server/operators/flux_local/run.py @@ -16,45 +16,57 @@ if not p.get("prompt"): print("ERROR: prompt is required") sys.exit(1) -model = p.get("model", "schnell") +model = p.get("model", "flux2-klein-4b") steps = int(p.get("steps", 4)) outdir = Path(a.outdir) -out_png = outdir / f"flux_{model}_s{p.get('seed', 42)}.png" +out_png = outdir / f"flux_{model.replace('/', '_')}_s{p.get('seed', 42)}.png" +bindir = Path(sys.executable).parent + +if model.startswith("flux2-"): + cli = bindir / "mflux-generate-flux2" + model_args = ["--model", model] + quant_args = [] # klein is small; skip quantization flags + guidance_args = [] # distilled klein: guidance fixed at 1.0 +elif model == "schnell-4bit": + cli = bindir / "mflux-generate" + # ungated community pre-quantized weights (~10GB) — no HF license wall + model_args = ["--model", "dhairyashil/FLUX.1-schnell-mflux-4bit", + "--base-model", "schnell"] + quant_args = [] + guidance_args = [] +else: + cli = bindir / "mflux-generate" + model_args = ["--model", model] + quant = str(p.get("quantize", "8")) + quant_args = ["--quantize", quant] if quant in ("4", "8") else [] + guidance_args = (["--guidance", str(p.get("guidance", 3.5))] + if model in ("dev", "krea-dev") else []) -# mflux-generate lives next to this venv's python -cli = Path(sys.executable).parent / "mflux-generate" if not cli.exists(): print(f"ERROR: mflux not installed ({cli} missing). Run scripts/install_mflux.sh") sys.exit(1) -cmd = [ - str(cli), - "--model", model, - "--prompt", p["prompt"], - "--steps", str(steps), - "--width", str(p.get("width", 1024)), - "--height", str(p.get("height", 1024)), - "--seed", str(p.get("seed", 42)), - "--output", str(out_png), -] -quant = str(p.get("quantize", "8")) -if quant in ("4", "8"): - cmd += ["--quantize", quant] -if model == "dev": - cmd += ["--guidance", str(p.get("guidance", 3.5))] +cmd = [str(cli), *model_args, + "--prompt", p["prompt"], + "--steps", str(steps), + "--width", str(p.get("width", 1024)), + "--height", str(p.get("height", 1024)), + "--seed", str(p.get("seed", 42)), + "--output", str(out_png), + *quant_args, *guidance_args] print("+", " ".join(cmd), flush=True) -print("(first run downloads weights from HuggingFace — can take a while)", flush=True) +print("(first run per model downloads weights from HuggingFace)", flush=True) t0 = time.time() res = subprocess.run(cmd, stdout=sys.stdout, stderr=subprocess.STDOUT) if res.returncode != 0: - print("HINT: if this failed with a gated-repo/auth error, the model needs an " - "HF license accept + token (Settings → HuggingFace token). schnell is ungated.") + print("HINT: gated-repo/auth errors mean this model needs an HF license accept " + "+ token (Settings → HuggingFace token). flux2-klein-4b and schnell-4bit " + "are ungated and need nothing.") sys.exit(res.returncode) elapsed = round(time.time() - t0, 1) if not out_png.exists(): - # mflux may append suffixes; grab any png it wrote pngs = sorted(outdir.glob("*.png")) if not pngs: print("ERROR: no image produced") @@ -64,5 +76,5 @@ if not out_png.exists(): (outdir / "result.json").write_text(json.dumps({"outputs": [ {"path": out_png.name, "meta": {"tool": "mflux", "model": model, "steps": steps, - "seed": p.get("seed", 42), "quantize": quant, "seconds": elapsed}}]})) + "seed": p.get("seed", 42), "seconds": elapsed}}]})) print(f"done in {elapsed}s: {out_png.name}", flush=True) diff --git a/server/operators/openrouter_image/manifest.json b/server/operators/openrouter_image/manifest.json index 4748467..07933c5 100644 --- a/server/operators/openrouter_image/manifest.json +++ b/server/operators/openrouter_image/manifest.json @@ -2,7 +2,7 @@ "id": "openrouter_image", "name": "OpenRouter · nano-banana", "category": "generate", - "description": "Prompt → image via OpenRouter (Google nano-banana / Gemini image models). Use with the same prompt+seedless run as FLUX (local) and Compare mode for A/B tests. Needs OPENROUTER_API_KEY.", + "description": "Prompt → image via OpenRouter's Image API. nano-banana ≈$0.039/img, nano-banana 2 ≈$0.05-0.08, nano-banana-pro ≈$0.134 (1-2K). Exact cost is reported in the job log. Pair with FLUX (local) on the same prompt + Compare mode for A/B tests. Needs OPENROUTER_API_KEY.", "accepts": [], "produces": ["image"], "resources": "net", @@ -12,7 +12,11 @@ "type": "object", "properties": { "prompt": {"type": "string", "default": "", "description": "What to generate"}, - "model": {"type": "string", "enum": ["google/gemini-2.5-flash-image", "google/gemini-3-pro-image-preview"], "default": "google/gemini-2.5-flash-image", "description": "nano-banana (flash) or nano-banana-pro (gemini 3)"} + "model": {"type": "string", "enum": ["google/gemini-2.5-flash-image", "google/gemini-3.1-flash-image", "google/gemini-3-pro-image"], "default": "google/gemini-2.5-flash-image", "description": "nano-banana / nano-banana-2 / nano-banana-pro"}, + "resolution": {"type": "string", "enum": ["1K", "2K", "4K"], "default": "1K", "description": "Output resolution tier (4K: pro/NB2 only, costs more)"}, + "aspect_ratio": {"type": "string", "enum": ["1:1", "16:9", "9:16", "4:3", "3:4", "3:2", "2:3"], "default": "1:1"}, + "seed": {"type": "integer", "description": "Seed (best-effort reproducibility)"}, + "n": {"type": "integer", "default": 1, "minimum": 1, "maximum": 4, "description": "Number of images"} } } } diff --git a/server/operators/openrouter_image/run.py b/server/operators/openrouter_image/run.py index b9dbaff..c4fef65 100644 --- a/server/operators/openrouter_image/run.py +++ b/server/operators/openrouter_image/run.py @@ -4,6 +4,7 @@ import json import os import sys import time +import urllib.error import urllib.request from pathlib import Path @@ -25,17 +26,22 @@ if not key: model = p.get("model", "google/gemini-2.5-flash-image") body = { "model": model, - "messages": [{"role": "user", "content": p["prompt"]}], - "modalities": ["image", "text"], + "prompt": p["prompt"], + "resolution": p.get("resolution", "1K"), + "aspect_ratio": p.get("aspect_ratio", "1:1"), + "n": int(p.get("n", 1)), } +if p.get("seed") not in (None, ""): + body["seed"] = int(p["seed"]) + req = urllib.request.Request( - "https://openrouter.ai/api/v1/chat/completions", + "https://openrouter.ai/api/v1/images", data=json.dumps(body).encode(), headers={"Authorization": f"Bearer {key}", "Content-Type": "application/json", - "X-Title": "MODELBEAST"}, + "X-OpenRouter-Title": "MODELBEAST"}, method="POST", ) -print(f"calling {model} via OpenRouter ...", flush=True) +print(f"calling {model} via OpenRouter Image API ...", flush=True) t0 = time.time() try: with urllib.request.urlopen(req, timeout=300) as resp: @@ -46,44 +52,25 @@ except urllib.error.HTTPError as e: elapsed = round(time.time() - t0, 1) outdir = Path(a.outdir) -(outdir / "openrouter_result.json").write_text(json.dumps( - {k: v for k, v in result.items() if k != "choices"} | - {"note": "choices omitted here; images extracted below"}, indent=2)) +outputs = [] +for i, item in enumerate(result.get("data", [])): + b64 = item.get("b64_json") + if not b64: + continue + media = item.get("media_type", "image/png") + ext = ".png" if "png" in media else ".webp" if "webp" in media else ".jpg" + short = model.split("/")[-1] + name = f"{short}_{i}{ext}" if len(result.get("data", [])) > 1 else f"{short}{ext}" + (outdir / name).write_bytes(base64.b64decode(b64)) + outputs.append({"path": name, "meta": {"tool": "openrouter", "model": model, + "seconds": elapsed, + "cost_usd": result.get("usage", {}).get("cost")}}) - -def data_urls(obj, acc): - """Collect data:image/... URLs from any nesting (message.images, content parts).""" - if isinstance(obj, dict): - for v in obj.values(): - data_urls(v, acc) - elif isinstance(obj, list): - for v in obj: - data_urls(v, acc) - elif isinstance(obj, str) and obj.startswith("data:image/"): - acc.append(obj) - - -urls: list[str] = [] -data_urls(result.get("choices", []), urls) -if not urls: - text = "" - try: - text = result["choices"][0]["message"].get("content") or "" - except (KeyError, IndexError): - pass - print("ERROR: no image in response.", ("Model said: " + str(text)[:800]) if text else "") +if not outputs: + print("ERROR: no image in response:") print(json.dumps(result, indent=2)[:1500]) sys.exit(1) -outputs = [] -for i, u in enumerate(urls): - header, b64 = u.split(",", 1) - ext = ".png" if "png" in header else ".webp" if "webp" in header else ".jpg" - name = f"nano_banana_{i}{ext}" if len(urls) > 1 else f"nano_banana{ext}" - (outdir / name).write_bytes(base64.b64decode(b64)) - outputs.append({"path": name, "meta": {"tool": "openrouter", "model": model, - "seconds": elapsed}}) - usage = result.get("usage", {}) -print(f"done in {elapsed}s — {len(outputs)} image(s); usage: {json.dumps(usage)}", flush=True) +print(f"done in {elapsed}s — {len(outputs)} image(s), cost ${usage.get('cost', '?')}", flush=True) (outdir / "result.json").write_text(json.dumps({"outputs": outputs}))