MODELBEAST 605b1ae347 Phase 1 + framework: settings/secrets, queue lanes, job mgmt, inbox, 8 new operators

Framework:
- server/settings.py: key/value settings + secrets, env-injected into operator
  subprocesses, secret values masked in API and redacted from job logs
- runner: gpu/cpu/net concurrency lanes, job cancel/retry/delete, multi-input,
  graceful 'not installed' error when a tool venv is missing
- db: settings table, asset_ids column (migrated), MODELBEAST_DATA test override
- main: settings + job-action endpoints, inbox watch folder auto-ingest
- store: operators can tag output asset kind (splat, colmap_dataset)

Operators (11 total):
- fal_trellis/trellis2/hunyuan3d/rodin via shared _lib/fal_common.py (verified
  params + endpoint ids; recursive result-URL extractor handles per-endpoint keys)
- sf3d, trellis_mac: local MPS image-to-3D, installed with Metal kernels built,
  gated on owner HuggingFace auth
- colmap_poses (COLMAP 4.x + GLOMAP global mapper), brush_train (native Metal 3DGS)
- Scan pipeline validated end-to-end through the UI: frames -> colmap (48/48
  registered, 0.6px) -> brush -> splat.ply -> in-app SplatViewer

Frontend:
- Settings modal, operator gating (lock + disabled run when requires_env unmet),
  job cancel/retry/delete, Compare grid (multi-select side-by-side viewers),
  SplatViewer (gaussian-splats-3d, Ply format forced for extensionless URLs)

Tooling: scripts/install_{colmap,brush,sf3d,trellis_mac}.sh; vendor/ + venvs/
gitignored; tests/smoke.sh (12 checks passing); BENCHMARKS.md

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-07-12 21:42:27 +10:00

2.3 KiB

Raw Blame History

MODELBEAST Benchmarks (M3 Ultra, 256GB)

First measurements on this machine, recorded 2026-07-12. Fixtures are synthetic (Blender-rendered Suzanne), so quality numbers are not representative of real photography — these validate that the pipeline runs and how fast.

Scan track (fully validated, no gated weights)

Stage	Input	Settings	Result
`colmap_poses`	48 frames @ 800×600	sequential matcher, global (GLOMAP) mapper, OPENCV, CPU SIFT	48/48 images registered, 1479 points, 0.60px mean reprojection error; global mapper step ~1.0s; full job a few seconds
`brush_train`	above colmap_dataset	1500 steps, max_res 800, sh 2	~30–60s wall, 465KB splat.ply, renders in the in-app SplatViewer

A full-quality brush_train run is 30000 steps (the default) — expect minutes, and a much crisper splat than the 1500-step preview above.

Mesh-gen (installed; first real run blocked on owner HuggingFace auth)

Operator	Install	Runtime status
`sf3d`	venv + Metal texture_baker/uv_unwrapper kernels compiled OK; torch 2.13 MPS available	Runs end-to-end; weights gated — `stabilityai/stable-fast-3d` returns `GatedRepoError` until the owner accepts the license + sets an HF token. Then expect seconds-to-a-minute on MPS.
`trellis_mac`	setup.sh built .venv (py3.11) + mtl* Metal kernels; torch 2.13 MPS available	Runs end-to-end; weights gated — needs HF access to `facebook/dinov3-vitl16-pretrain-lvd1689m` + `briaai/RMBG-2.0`. Expect ~3–5 min/gen once authed (M4 Pro reference; M3 Ultra should match or beat).
`fal_*` (trellis / trellis2 / hunyuan3d / rodin)	none (API)	Gated on `FAL_KEY`. Verified param surfaces; ~1s–1min server-side per fal docs.

To unblock the gated local operators

Accept the model licenses on HuggingFace (one-time, usually instant):
Either huggingface-cli login on the machine, or paste an HF token into Settings → "HuggingFace token" (injected as HF_TOKEN for the operators).

Method

Timings are wall-clock from the job runner (started_at→finished_at), single job at a time (gpu lane = 1). Re-run tests/smoke.sh for the framework regression suite (12 checks, ~30s).