27564e80eb
BENCHMARKS.md says what a model cost the day it was measured; nothing noticed if a macOS/torch update or a thermal fault halved a node. perfcheck runs the whole fleet in ~35s off godcheck's 03:30 cron on the m4mini and reports drift into GODCHECK_LATEST.md. Probes matmul (fp16/fp32), memory bandwidth, and SDPA at head_dim 64 vs 56 — the latter turning CorridorKey's fast-path cliff into a permanent canary: it confirms the padding win fleet-wide (1.96x-5.26x) and tells us if a future torch closes it. Runs on venvs/rmbg/bin/python, already identical fleet-wide, so nothing new is installed (nothing lands on the disk-tight m1max). First cross-machine capability table for all 6 nodes. The M3 Ultra is ~2.2x the M1 Ultra on fp16 matmul, but they share ~625 GB/s — so bandwidth-bound stages run alike while compute-bound ones scale. Both Ultras reach only ~78% of spec bandwidth on a single kernel; the smaller Macs hit ~88%. Measuring a fleet that is doing real work is the whole problem, and naive benchmarking here is off by 11x: - min, not median: a concurrent trellis_mac job dragged a median-of-5 matmul from ~24500 to ~2150 GFLOP/s, which reads exactly like a catastrophic regression. - n=4096 not 2048: 2048 is dispatch-bound and swung 48% run-to-run; 4096 reproduces to 0.1% even while contended. - sdpa 16x2048 not 8x1024: sub-ms probes are dispatch noise — 8x1024 gave ratios of 0.79/3.95/2.35 on three runs of one machine, the first "proving" 56 is faster. - busy nodes are excluded, not blamed: GPU contention is invisible to load average (M3 Ultra read load 2.45 with its GPU pinned), so bench.py samples ioreg GPU% before it touches the GPU — our own matmul pins the device, so ordering is the trick. - baselines are the median of recent history, not a saved best: the M4 Pro also serves Ollama and is bimodal (~3200 vs ~5500 fp32), so a best-observed baseline pins to a lucky outlier and alerts forever. - a regression must repeat before it is believed ([~] watching -> [!] CONFIRMED). Validated by re-running the fleet against its own baselines: zero false alarms, including a sweep where the M3 Ultra read 43 GB/s under load and was correctly marked BUSY rather than reported as a 93% regression. Also found: the m4mini is the only node with Tailscale SSH (RunSSH: true) and it does NOT propagate remote exit codes — `ssh m4mini "exit 7"` returns 0, so any `if ssh m4mini ...` test silently always passes. Test on output instead, which is what godcheck already does (and why it is unaffected). It also cannot ssh to itself, so run_fleet detects its own tailnet IP and benches the local node via the shell. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|---|---|---|
| .claude | ||
| docs | ||
| scripts | ||
| server | ||
| tests | ||
| web | ||
| .gitignore | ||
| AGENTS.md | ||
| BENCHMARKS.md | ||
| CLUSTER.md | ||
| CORRIDORKEY.md | ||
| HANDOFF2.md | ||
| HANDOFF_HY3D_MLX.md | ||
| HANDOFF.md | ||
| HARDWARE.md | ||
| mb | ||
| PERFCHECK.md | ||
| PLAN.md | ||
| pyproject.toml | ||
| README.md | ||
| uv.lock | ||
MODELBEAST
Local-first web app that turns videos, images, and 3D files into meshes, splats, mocap, and rigged characters on the M3 Ultra. See PLAN.md for the full verified tool matrix and roadmap, and HANDOFF.md for the agent build brief (phases 1–4 instructions).
🖥️ Multi-Mac fleet? Read CLUSTER.md first — the local-first policy, the M3/M1/M4 node roles, and how the central queue routes jobs across machines. 💾 What runs on which Mac? HARDWARE.md — min/recommended RAM per operator across every Apple Silicon tier (8GB → 256GB). 🎬 Green-screen keying → 3D: CORRIDORKEY.md — CorridorKey status, our Ultra tuning, and the video→cutout→mesh/splat use-case map.
Run
/opt/homebrew/bin/uv run uvicorn server.main:app --host 0.0.0.0 --port 8777
Open http://localhost:8777 (or http://<tailscale-ip>:8777 from any device on the tailnet). Prefer scripts/serve.sh — it runs headless and prints the one-time owner password on first start.
Drag/drop/paste any video, image, or 3D file; or drop files into data/inbox/ for auto-ingest. Pick an operator, tune its parameters, run. Add API keys / HuggingFace token under ⚙ Settings. Use ⊞ Compare to view several outputs side by side, and 📊 Dashboard for a live CPU/RAM/GPU/queue snapshot.
Accounts & sharing
Login is required. The first scripts/serve.sh creates an owner (monster) and prints a one-time password to data/server.log — change it in ⚙ Settings → Users (or uv run python scripts/users.py passwd monster). The owner can add guest accounts for friends: guests are local-only — they can run only on-device operators (never the owner's paid fal/OpenRouter keys, enforced server-side), see only their own assets/jobs, and have a per-user concurrent-job cap. So mates cost you electricity, nothing else. Public hosting via a VPS + Caddy is in docs/VPS.md; the CUDA-worker offload plan is in docs/CUDA-WORKER.md.
Headless / agents
Everything is driveable without the UI via the zero-dependency mb CLI (works from any tailnet machine with MB_HOST=http://100.89.131.57:8777) or raw REST. AGENTS.md is the complete handover brief — endpoints, mb reference, operator catalog, recipes, job semantics, etiquette. Server ops: scripts/serve.sh (start/restart headless), scripts/install_launchagent.sh (optional boot persistence, owner-run).
export MB_HOST=http://100.89.131.57:8777 MB_TOKEN=mbt_... # token from ⚙ Settings → Users
./mb run fal_trellis2 --file photo.jpg -p resolution=1024 --wait --download out/
Develop
- Backend:
server/— FastAPI + SQLite (data/modelbeast.db), job runner runs operators as subprocesses across concurrency lanes (gpu=1, cpu=3, net=6). Settings/secrets inserver/settings.py(env-injected, log-redacted). - Frontend:
web/— React + Vite + three.js +@mkkellogg/gaussian-splats-3d. After editing:cd web && npm run build(the server servesweb/dist). - Data:
data/assets/(store),data/jobs/(job workdirs),data/inbox/(watch folder). Deletedata/to reset. Tests useMODELBEAST_DATA=<tmp>. - Tests:
./tests/smoke.sh(12 framework checks). Benchmarks in BENCHMARKS.md. - Heavy tools live in
vendor/(repos) +venvs/(per-tool envs), both gitignored. Reinstall withscripts/install_*.sh.
Setup for the local mesh generators (one-time)
sf3d and trellis_mac are installed but their weights are HuggingFace-gated. Accept the licenses (stabilityai/stable-fast-3d, facebook/dinov3-vitl16-pretrain-lvd1689m, briaai/RMBG-2.0), then huggingface-cli login or paste an HF token in Settings. Cloud fal_* operators need FAL_KEY in Settings.
Operators
Each subfolder of server/operators/ with a manifest.json is an operator. The UI auto-renders its parameter form from params_schema (JSON Schema) and filters by the selected asset's kind (accepts).
Contract: the runner invokes <python> run.py --input <asset> --outdir <jobdir> --params '<json>'. Write outputs into the outdir; optionally write result.json ({"outputs": [{"path": ..., "name": ..., "meta": ...}]}) to control what gets registered as assets. stdout/stderr become the job log.
Manifest fields: id, name, category, description, accepts, produces, resources (gpu/cpu/net lane), requires_env (gates the operator until the key is set), python (absolute venv path for heavy tools), params_schema. Operators can tag output asset kind via result.json output meta.kind (e.g. splat, colmap_dataset).
Current operators:
| id | lane | what |
|---|---|---|
ffprobe |
cpu | media inspection |
ffmpeg_frames |
cpu | video → frames (fps, mpdecimate dedupe, blur cull, max cap) |
blender_convert |
cpu | universal 3D format conversion via headless Blender |
colmap_poses |
cpu | frames → camera poses + sparse cloud (COLMAP 4.x + GLOMAP) |
brush_train |
gpu | colmap dataset → 3D gaussian splat (Brush, native Metal) |
sf3d |
gpu | image → GLB locally (Stable-Fast-3D, MPS) [HF-gated] |
trellis_mac |
gpu | image → GLB+PBR locally (TRELLIS.2 MPS port) [HF-gated] |
hunyuan3d_mlx |
gpu | image → GLB+PBR locally (Hunyuan3D 2.1, native MLX — runs on M1 too; weights public, no HF login) |
fal_trellis / fal_trellis2 / fal_hunyuan3d / fal_hunyuan3d_v21 / fal_rodin |
net | image → mesh via fal.ai API [needs FAL_KEY] |
fal_bg_remove |
net | image → subject cutout (BiRefNet v2) — run before any image→3D for a big quality jump |
fal_upscale |
net | image → faithful upscale (SeedVR) |
fal_image_edit |
net | image + instruction → edited image (nano-banana) |
fal_text_image |
net | prompt → image (Ideogram v3, readable text) — no input asset needed |
Recommended fal chain for best image→3D: fal_bg_remove → (fal_upscale if thin) → fal_trellis2 or fal_hunyuan3d_v21. Note: Hunyuan v21 multi-view is broken on fal (verified 2026-07) — v21 is single-image only; use v2 for multi-view.
Heavy operators get their own uv venv ("python": "<abs venv path>" in the manifest). Remaining roadmap (object_capture, freemocap, retargeting, tripo/meshy character APIs, workflow presets, LLM copilot) is in HANDOFF.md §5–8.