MODELBEAST 19f086773e Headless control: mb CLI, AGENTS.md handover, serve scripts; verified-current image models

- mb: zero-dependency python CLI for the full API (ops/upload/run/wait/follow/
  download incl. folder assets, retry/cancel, settings). Schema-aware -p k=v
  coercion. Works from any tailnet machine via MB_HOST. Tested end-to-end
  (upload -> blender_convert -> download).
- AGENTS.md: complete handover brief for other agents using this box as an
  asset factory — endpoints, CLI reference, catalog, recipes, lanes/etiquette.
- scripts/serve.sh (headless start/restart) + scripts/install_launchagent.sh
  (optional boot persistence, owner-run)
- flux_local upgraded to mflux 0.18 reality: flux2-klein-4b default (Apache,
  UNGATED — runs with zero keys), klein-9b, schnell/schnell-4bit community
  quant, dev/krea-dev. Research verdict: FLUX.1-dev no longer competitive
  (Elo ~1027) vs klein ~1083-1119 vs nano-banana ~1154.
- openrouter_image rewritten to the dedicated Image API (POST /api/v1/images):
  b64_json parsing, seed, resolution/aspect, exact usage.cost logging; model
  enum: gemini-2.5-flash-image / 3.1-flash-image (NB2) / 3-pro-image

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

2026-07-12 22:38:52 +10:00

4.8 KiB

Raw Blame History

MODELBEAST

Local-first web app that turns videos, images, and 3D files into meshes, splats, mocap, and rigged characters on the M3 Ultra. See PLAN.md for the full verified tool matrix and roadmap, and HANDOFF.md for the agent build brief (phases 1–4 instructions).

Run

/opt/homebrew/bin/uv run uvicorn server.main:app --host 0.0.0.0 --port 8777

Open http://localhost:8777 (or http://<tailscale-ip>:8777 from any device on the tailnet).

Drag/drop/paste any video, image, or 3D file; or drop files into data/inbox/ for auto-ingest. Pick an operator, tune its parameters, run. Add API keys / HuggingFace token under ⚙ Settings. Use ⊞ Compare to view several outputs side by side.

Headless / agents

Everything is driveable without the UI via the zero-dependency mb CLI (works from any tailnet machine with MB_HOST=http://100.89.131.57:8777) or raw REST. AGENTS.md is the complete handover brief — endpoints, mb reference, operator catalog, recipes, job semantics, etiquette. Server ops: scripts/serve.sh (start/restart headless), scripts/install_launchagent.sh (optional boot persistence, owner-run).

./mb run fal_trellis2 --file photo.jpg -p resolution=1024 --wait --download out/

Develop

Backend: server/ — FastAPI + SQLite (data/modelbeast.db), job runner runs operators as subprocesses across concurrency lanes (gpu=1, cpu=3, net=6). Settings/secrets in server/settings.py (env-injected, log-redacted).
Frontend: web/ — React + Vite + three.js + @mkkellogg/gaussian-splats-3d. After editing: cd web && npm run build (the server serves web/dist).
Data: data/assets/ (store), data/jobs/ (job workdirs), data/inbox/ (watch folder). Delete data/ to reset. Tests use MODELBEAST_DATA=<tmp>.
Tests: ./tests/smoke.sh (12 framework checks). Benchmarks in BENCHMARKS.md.
Heavy tools live in vendor/ (repos) + venvs/ (per-tool envs), both gitignored. Reinstall with scripts/install_*.sh.

Setup for the local mesh generators (one-time)

sf3d and trellis_mac are installed but their weights are HuggingFace-gated. Accept the licenses (stabilityai/stable-fast-3d, facebook/dinov3-vitl16-pretrain-lvd1689m, briaai/RMBG-2.0), then huggingface-cli login or paste an HF token in Settings. Cloud fal_* operators need FAL_KEY in Settings.

Operators

Each subfolder of server/operators/ with a manifest.json is an operator. The UI auto-renders its parameter form from params_schema (JSON Schema) and filters by the selected asset's kind (accepts).

Contract: the runner invokes <python> run.py --input <asset> --outdir <jobdir> --params '<json>'. Write outputs into the outdir; optionally write result.json ({"outputs": [{"path": ..., "name": ..., "meta": ...}]}) to control what gets registered as assets. stdout/stderr become the job log.

Manifest fields: id, name, category, description, accepts, produces, resources (gpu/cpu/net lane), requires_env (gates the operator until the key is set), python (absolute venv path for heavy tools), params_schema. Operators can tag output asset kind via result.json output meta.kind (e.g. splat, colmap_dataset).

Current operators:

id	lane	what
`ffprobe`	cpu	media inspection
`ffmpeg_frames`	cpu	video → frames (fps, mpdecimate dedupe, blur cull, max cap)
`blender_convert`	cpu	universal 3D format conversion via headless Blender
`colmap_poses`	cpu	frames → camera poses + sparse cloud (COLMAP 4.x + GLOMAP)
`brush_train`	gpu	colmap dataset → 3D gaussian splat (Brush, native Metal)
`sf3d`	gpu	image → GLB locally (Stable-Fast-3D, MPS) [HF-gated]
`trellis_mac`	gpu	image → GLB+PBR locally (TRELLIS.2 MPS port) [HF-gated]
`fal_trellis` / `fal_trellis2` / `fal_hunyuan3d` / `fal_hunyuan3d_v21` / `fal_rodin`	net	image → mesh via fal.ai API [needs FAL_KEY]
`fal_bg_remove`	net	image → subject cutout (BiRefNet v2) — run before any image→3D for a big quality jump
`fal_upscale`	net	image → faithful upscale (SeedVR)
`fal_image_edit`	net	image + instruction → edited image (nano-banana)
`fal_text_image`	net	prompt → image (Ideogram v3, readable text) — no input asset needed

Recommended fal chain for best image→3D: fal_bg_remove → (fal_upscale if thin) → fal_trellis2 or fal_hunyuan3d_v21. Note: Hunyuan v21 multi-view is broken on fal (verified 2026-07) — v21 is single-image only; use v2 for multi-view.

Heavy operators get their own uv venv ("python": "<abs venv path>" in the manifest). Remaining roadmap (object_capture, freemocap, retargeting, tripo/meshy character APIs, workflow presets, LLM copilot) is in HANDOFF.md §5–8.

4.8 KiB Raw Blame History Unescape Escape