monster/modelbeast
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
605b1ae347
modelbeast/README.md
MODELBEAST 605b1ae347 Phase 1 + framework: settings/secrets, queue lanes, job mgmt, inbox, 8 new operators 
Framework:
- server/settings.py: key/value settings + secrets, env-injected into operator
  subprocesses, secret values masked in API and redacted from job logs
- runner: gpu/cpu/net concurrency lanes, job cancel/retry/delete, multi-input,
  graceful 'not installed' error when a tool venv is missing
- db: settings table, asset_ids column (migrated), MODELBEAST_DATA test override
- main: settings + job-action endpoints, inbox watch folder auto-ingest
- store: operators can tag output asset kind (splat, colmap_dataset)

Operators (11 total):
- fal_trellis/trellis2/hunyuan3d/rodin via shared _lib/fal_common.py (verified
  params + endpoint ids; recursive result-URL extractor handles per-endpoint keys)
- sf3d, trellis_mac: local MPS image-to-3D, installed with Metal kernels built,
  gated on owner HuggingFace auth
- colmap_poses (COLMAP 4.x + GLOMAP global mapper), brush_train (native Metal 3DGS)
- Scan pipeline validated end-to-end through the UI: frames -> colmap (48/48
  registered, 0.6px) -> brush -> splat.ply -> in-app SplatViewer

Frontend:
- Settings modal, operator gating (lock + disabled run when requires_env unmet),
  job cancel/retry/delete, Compare grid (multi-select side-by-side viewers),
  SplatViewer (gaussian-splats-3d, Ply format forced for extensionless URLs)

Tooling: scripts/install_{colmap,brush,sf3d,trellis_mac}.sh; vendor/ + venvs/
gitignored; tests/smoke.sh (12 checks passing); BENCHMARKS.md

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 21:42:27 +10:00

3.6 KiB
Raw Blame History

MODELBEAST

Local-first web app that turns videos, images, and 3D files into meshes, splats, mocap, and rigged characters on the M3 Ultra. See PLAN.md for the full verified tool matrix and roadmap, and HANDOFF.md for the agent build brief (phases 1–4 instructions).

Run

/opt/homebrew/bin/uv run uvicorn server.main:app --host 0.0.0.0 --port 8777

Open http://localhost:8777 (or http://<tailscale-ip>:8777 from any device on the tailnet).

Drag/drop/paste any video, image, or 3D file; or drop files into data/inbox/ for auto-ingest. Pick an operator, tune its parameters, run. Add API keys / HuggingFace token under ⚙ Settings. Use ⊞ Compare to view several outputs side by side.

Develop

  • Backend: server/ — FastAPI + SQLite (data/modelbeast.db), job runner runs operators as subprocesses across concurrency lanes (gpu=1, cpu=3, net=6). Settings/secrets in server/settings.py (env-injected, log-redacted).
  • Frontend: web/ — React + Vite + three.js + @mkkellogg/gaussian-splats-3d. After editing: cd web && npm run build (the server serves web/dist).
  • Data: data/assets/ (store), data/jobs/ (job workdirs), data/inbox/ (watch folder). Delete data/ to reset. Tests use MODELBEAST_DATA=<tmp>.
  • Tests: ./tests/smoke.sh (12 framework checks). Benchmarks in BENCHMARKS.md.
  • Heavy tools live in vendor/ (repos) + venvs/ (per-tool envs), both gitignored. Reinstall with scripts/install_*.sh.

Setup for the local mesh generators (one-time)

sf3d and trellis_mac are installed but their weights are HuggingFace-gated. Accept the licenses (stabilityai/stable-fast-3d, facebook/dinov3-vitl16-pretrain-lvd1689m, briaai/RMBG-2.0), then huggingface-cli login or paste an HF token in Settings. Cloud fal_* operators need FAL_KEY in Settings.

Operators

Each subfolder of server/operators/ with a manifest.json is an operator. The UI auto-renders its parameter form from params_schema (JSON Schema) and filters by the selected asset's kind (accepts).

Contract: the runner invokes <python> run.py --input <asset> --outdir <jobdir> --params '<json>'. Write outputs into the outdir; optionally write result.json ({"outputs": [{"path": ..., "name": ..., "meta": ...}]}) to control what gets registered as assets. stdout/stderr become the job log.

Manifest fields: id, name, category, description, accepts, produces, resources (gpu/cpu/net lane), requires_env (gates the operator until the key is set), python (absolute venv path for heavy tools), params_schema. Operators can tag output asset kind via result.json output meta.kind (e.g. splat, colmap_dataset).

Current operators:

id lane what
ffprobe cpu media inspection
ffmpeg_frames cpu video → frames (fps, mpdecimate dedupe, blur cull, max cap)
blender_convert cpu universal 3D format conversion via headless Blender
colmap_poses cpu frames → camera poses + sparse cloud (COLMAP 4.x + GLOMAP)
brush_train gpu colmap dataset → 3D gaussian splat (Brush, native Metal)
sf3d gpu image → GLB locally (Stable-Fast-3D, MPS) [HF-gated]
trellis_mac gpu image → GLB+PBR locally (TRELLIS.2 MPS port) [HF-gated]
fal_trellis / fal_trellis2 / fal_hunyuan3d / fal_rodin net image → mesh via fal.ai API [needs FAL_KEY]

Heavy operators get their own uv venv ("python": "<abs venv path>" in the manifest). Remaining roadmap (object_capture, freemocap, retargeting, tripo/meshy character APIs, workflow presets, LLM copilot) is in HANDOFF.md §5–8.