Commit Graph

17 Commits

Author SHA1 Message Date
m3ultra
06f33be865 pool cpu lane across nodes; hunyuan Studio-quality defaults
runner: gpu AND cpu are now a node pool. Per-node cpu_slots (primary 3,
helpers 2, nodes.json-overridable); net stays primary-only. Python-less ops
(ffmpeg/ffprobe) now run remotely on the node's system python3. Verified: 9
concurrent ffmpeg_frames distributed 4 local / 2 m1 / 2 m4.

hunyuan3d_mlx: default to Studio-quality (octree 384, texture 4096, remesh
120k) — 4096 bake verified watchdog-free on M3 Ultra; big quality gain
(defined face, 120k faces). remesh_faces now a param. All param-overridable.
2026-07-16 10:59:44 +10:00
m3ultra
573afa7b7a hunyuan3d_mlx operator (local MLX image→3D) + cluster docs
- server/operators/hunyuan3d_mlx: native-MLX Hunyuan3D 2.1, both stages, PBR.
  Verified end-to-end on M3 Ultra: 260s total, 40k-face GLB w/ 2048² PBR.
  Weights public (no HF login). Runs on M1 (MLX-native).
- scripts/install_hunyuan3d_mlx.sh: uv py3.11 venv + MLX-path deps.
- CLUSTER.md: local-first policy + M3/M1/M4 fleet roles + central-queue howto.
- BENCHMARKS/README/AGENTS: hunyuan3d_mlx rows + trellis head-to-head.
2026-07-16 10:10:43 +10:00
m3ultra
19cf9ed21e 🔑 agents get keys: shared agent token minting + guest-visible job outputs
scripts/agent_token.py mints the 'agents' guest user + token non-interactively,
writes data/agent.env (0600, gitignored, never printed). Fixes _register_outputs
dropping user_id — guest job outputs were registered ownerless, so guests could
never see or download their own results.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 23:30:04 +10:00
MODELBEAST
5f46939321 runner: gpu lane becomes a node pool (local + remote workers)
The gpu lane is now a pool of nodes -- this Mac plus any remote workers in
nodes.json (gitignored, primary-only). A gpu job acquires the first free node
that supports its operator; local runs as before, a remote node has its inputs
rsynced out, runs the operator's run.py over ssh (keys sourced from the node's
.env.remote, kept off the process table), then its outputs are rsynced back and
registered locally exactly as for a local job. One job per node (one Metal
device each).

- server/remote.py (new): node loading, cached health checks, prepare/cmd/collect
- runner: _acquire_gpu_node + a remote branch in _run_job. lane_of now treats
  'gpu' as a real lane -- it was dropped from LANE_LIMITS (which holds only the
  semaphore lanes cpu/net), so gpu jobs were silently falling back to the cpu
  lane and the pool was never reached. A node is reserved before the health
  await so two concurrent jobs can't grab the same one.
- sysinfo: gpu lane capacity = pool size, plus per-node status for the dashboard

Verified: all 7 gpu operators route to the pool; remote_cmd/node_supports and
nodes.json parsing correct; acquisition is race-free; a scratch-DB server boots
clean and logs the pool.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 13:38:34 +10:00
MODELBEAST
210d80484c runner: resolve relative operator python paths against repo root (multi-node portability)
Manifest "python" can now be a repo-relative path (venvs/mflux/bin/python) that
resolves against db.ROOT, so the same operator code runs on any machine/checkout
(M3 Ultra, M1 Ultra worker). All 6 venv operators switched to relative paths.
Prep for the M1 Ultra second node.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 11:47:39 +10:00
MODELBEAST
514ec6cdcc Security hardening from adversarial review (5-agent workflow)
Fixed confirmed findings before public exposure:
HIGH:
- upload filename path traversal → store.safe_name() strips to basename
- login rate-limit XFF bypass → key on request.client.host + per-username bucket;
  auth.check_login() burns bcrypt time on unknown users (no enumeration)
- cross-user read access → per-user isolation: guests see/use/download/delete only
  their own assets & jobs (owner sees all); WS job events scoped per-user
MEDIUM:
- unbounded upload read → bounded chunked streaming to the 1GB cap
- asset member path check → Path.is_relative_to boundary + ownership gate
- WS token-in-query-string leak → session-cookie-only WS auth
LOW:
- retry_job bypassed the per-user job cap → cap now checked on retry
- wholesale API-key injection → env_for_operator injects a paid key only to
  operators that declare it (guest local jobs never receive fal/OpenRouter keys)
- session revocation → users.session_epoch, bumped on password change
- int() 500s → 400; net-lane defense-in-depth (guests blocked by requires_env AND
  resources==net, so a mis-tagged paid op is still blocked)
+ public /api/health for serve.sh & proxy; docs/VPS.md; mb MB_TOKEN bearer auth

tests/smoke.sh: 34 checks passing incl. all new hardening.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 12:25:13 +10:00
MODELBEAST
810400fb60 Auth + dashboard (HANDOFF2 phases 1-2): guests local-only, system snapshot
Phase 1 — multi-user auth:
- server/auth.py: bcrypt passwords, itsdangerous signed-cookie sessions, sha256
  bearer tokens, FastAPI current_user/require_owner deps, login rate limit
- users + api_tokens tables + jobs/assets.user_id (additive migrations)
- HARD RULE enforced server-side: guests are local-only — /api/operators filters
  out requires_env operators, POST /api/jobs 403s cloud ops for non-owners (proven
  via direct POST in smoke.sh, not just UI). Settings owner-only. auth_secret
  hidden from the settings API. Per-user active-job cap (owner exempt). Own-asset/
  own-job checks. WS auth via cookie or ?token=. Owner bootstrap prints pw once.
- mb-ready: bearer MB_TOKEN; scripts/users.py for out-of-band management
- Frontend: Login gate, header user chip + logout, guest note, username on jobs,
  Users panel in Settings (owner)

Phase 2 — dashboard:
- server/sysinfo.py: psutil CPU/RAM/disk + macmon Apple GPU (util/power/temp, no
  sudo), computed lane occupancy, 24h job summary, recent jobs w/ output thumbs;
  all cached (5s stats, 5min du). /api/system + /api/jobs/recent.
- Dashboard.jsx: snapshot-on-refresh (no polling) — stat cards, per-core strip,
  lane strip, running/queued, recent grid.

tests/smoke.sh rewritten for auth: 28 checks passing incl. all guest-security
rules. Browser-verified owner + guest + dashboard.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 12:06:52 +10:00
MODELBEAST
764ae3faf5 trellis_mac: unblock — dinov3 lvd1689m granted; add HF_HUB_DISABLE_XET + OMP guards
dinov3-vitl16-pretrain-lvd1689m access granted (the natural-image variant TRELLIS.2
needs — note the approval email named the sat493m satellite variant, but lvd1689m
came through too). Added HF_HUB_DISABLE_XET=1 (same xet download bug as flux) +
KMP_DUPLICATE_LIB_OK/OMP_NUM_THREADS=1 guards (torch + Metal-kernel omp). First
local TRELLIS.2 generation now running.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 09:44:13 +10:00
MODELBEAST
f0bac67edb Local mflux operators: seedvr2_upscale + mflux_image_edit (free, ungated)
Two new local image-prep operators through the already-installed mflux 0.18 (no
new heavy install, ungated weights):
- seedvr2_upscale: mflux-upscale-seedvr2 (ByteDance SeedVR2, MIT) — faithful
  local upscale, free alternative to cloud fal_upscale
- mflux_image_edit: mflux-generate-qwen-edit (Qwen-Image-Edit, Apache-2.0) —
  local prompt-based image editing, free alternative to fal_image_edit

Research verdict on dinov3 (recorded): waiting for Meta approval is the only
legitimate path — TRELLIS.2's DiT is trained on DINOv3 ViT-L/16 features so the
encoder can't be swapped, and the only ungated mirrors are unauthorized
re-uploads. trellis_mac stays blocked on the pending gate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 01:37:56 +10:00
MODELBEAST
1cc0b80d89 bg_remove_local: local RMBG-2.0 background removal on MPS
New operator: briaai/RMBG-2.0 image->transparent cutout, fully local on Apple
Silicon (weights unlocked by owner HF license). Free local alternative to the
cloud fal_bg_remove; the recommended pre-pass before SF3D / image->3D. Verified:
clean 1024px cutout of the FLUX astrolabe (alpha 0-255, 86.6% removed, thin rings
preserved). Same OpenMP guardrails as sf3d. scripts/install_rmbg.sh; venvs/rmbg.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 01:32:38 +10:00
MODELBEAST
21938a4e17 sf3d: fix macOS OpenMP segfault — single-threaded OMP + KMP_DUPLICATE_LIB_OK
torch and the compiled texture_baker/uv_unwrapper each link a libomp. Allowing
the duplicate runtime clears 'OMP: Error #15' but the duplicate then corrupts the
multithreaded CPU LAPACK path (torch.linalg.svd MPS fallback) -> segfault with no
Python traceback. Forcing OMP_NUM_THREADS=1 + MKL_NUM_THREADS=1 removes the
threading collision. Verified: clean 1.3MB GLB (13298 verts) in ~5s on M3 Ultra,
9GB peak. SF3D local image->3D now works (weights unlocked by owner HF license).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 01:16:40 +10:00
MODELBEAST
c42f066723 flux_local: HF_HUB_DISABLE_XET fix + full local FLUX lineup benchmarks
- run.py: force HF_HUB_DISABLE_XET=1 (xet chunked downloader fails on BFL repos
  with 'Unable to parse string as hex hash value'; HTTP path is reliable)
- BENCHMARKS.md: warm generation times for all 5 local FLUX models on M3 Ultra
  (Klein 4B 9.1s, schnell-4bit 18.5s, Klein 9B 18.7s, schnell 20.4s, dev 108s).
  Verdict: Klein 9B = best hero-asset quality (beats schnell at same speed),
  Klein 4B = volume workhorse.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 00:37:24 +10:00
MODELBEAST
19f086773e Headless control: mb CLI, AGENTS.md handover, serve scripts; verified-current image models
- mb: zero-dependency python CLI for the full API (ops/upload/run/wait/follow/
  download incl. folder assets, retry/cancel, settings). Schema-aware -p k=v
  coercion. Works from any tailnet machine via MB_HOST. Tested end-to-end
  (upload -> blender_convert -> download).
- AGENTS.md: complete handover brief for other agents using this box as an
  asset factory — endpoints, CLI reference, catalog, recipes, lanes/etiquette.
- scripts/serve.sh (headless start/restart) + scripts/install_launchagent.sh
  (optional boot persistence, owner-run)
- flux_local upgraded to mflux 0.18 reality: flux2-klein-4b default (Apache,
  UNGATED — runs with zero keys), klein-9b, schnell/schnell-4bit community
  quant, dev/krea-dev. Research verdict: FLUX.1-dev no longer competitive
  (Elo ~1027) vs klein ~1083-1119 vs nano-banana ~1154.
- openrouter_image rewritten to the dedicated Image API (POST /api/v1/images):
  b64_json parsing, seed, resolution/aspect, exact usage.cost logging; model
  enum: gemini-2.5-flash-image / 3.1-flash-image (NB2) / 3-pro-image

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 22:38:52 +10:00
MODELBEAST
76f065d3a6 A/B image-gen: local FLUX (mflux/MLX) + OpenRouter nano-banana operators
- flux_local: prompt->image fully on-device via mflux (schnell/dev, steps,
  quantize, seed, size). Both FLUX repos are HF-gated as of 2026-07 (schnell
  included) — clean GatedRepoError surfaces with a hint; needs owner HF token.
- openrouter_image: prompt->image via OpenRouter chat/completions with
  modalities [image,text]; parses data-URL images from the response; model
  picker for nano-banana / nano-banana-pro. Gated on OPENROUTER_API_KEY.
- settings: openrouter_key added to vault (masked/redacted/env-injected)
- scripts/install_mflux.sh; venv installed
- A/B flow: run both with the same prompt, judge in Compare mode

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 22:31:19 +10:00
MODELBEAST
7ea3c8b935 fal panel: 5 new operators, image outputs, grouped UI, no-input ops + review fixes
Operators (16 total):
- fal_hunyuan3d_v21: Hunyuan3D 2.1 single-image (live-verified dash variant,
  ~90s; v21 multi-view is broken on fal — v2 stays the multi-view path)
- fal_bg_remove (BiRefNet v2): subject cutout pre-pass — the biggest quality
  lever before image->3D
- fal_upscale (SeedVR faithful upscale), fal_image_edit (nano-banana prompt
  edits), fal_text_image (Ideogram v3 — first no-input operator)
- fal_common: collect='images' mode; recursive URL extractor now takes the
  wanted extension set

UI: operator dropdown grouped by category (optgroup); operators with
accepts: [] run without a selected asset ('No input asset needed' note);
run-button/launch logic updated accordingly.

Review fixes (Opus Phase 1 commit):
- runner._run_lane crashed (TypeError) when a queued job was deleted before
  the worker picked it up
- PUT /api/settings treated empty string as a masked placeholder, making
  secrets impossible to clear from the UI
- store.register_file mutated the caller's meta dict via pop

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 21:51:16 +10:00
MODELBEAST
605b1ae347 Phase 1 + framework: settings/secrets, queue lanes, job mgmt, inbox, 8 new operators
Framework:
- server/settings.py: key/value settings + secrets, env-injected into operator
  subprocesses, secret values masked in API and redacted from job logs
- runner: gpu/cpu/net concurrency lanes, job cancel/retry/delete, multi-input,
  graceful 'not installed' error when a tool venv is missing
- db: settings table, asset_ids column (migrated), MODELBEAST_DATA test override
- main: settings + job-action endpoints, inbox watch folder auto-ingest
- store: operators can tag output asset kind (splat, colmap_dataset)

Operators (11 total):
- fal_trellis/trellis2/hunyuan3d/rodin via shared _lib/fal_common.py (verified
  params + endpoint ids; recursive result-URL extractor handles per-endpoint keys)
- sf3d, trellis_mac: local MPS image-to-3D, installed with Metal kernels built,
  gated on owner HuggingFace auth
- colmap_poses (COLMAP 4.x + GLOMAP global mapper), brush_train (native Metal 3DGS)
- Scan pipeline validated end-to-end through the UI: frames -> colmap (48/48
  registered, 0.6px) -> brush -> splat.ply -> in-app SplatViewer

Frontend:
- Settings modal, operator gating (lock + disabled run when requires_env unmet),
  job cancel/retry/delete, Compare grid (multi-select side-by-side viewers),
  SplatViewer (gaussian-splats-3d, Ply format forced for extensionless URLs)

Tooling: scripts/install_{colmap,brush,sf3d,trellis_mac}.sh; vendor/ + venvs/
gitignored; tests/smoke.sh (12 checks passing); BENCHMARKS.md

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 21:42:27 +10:00
MODELBEAST
d0569dc746 Phase 0 baseline: FastAPI+SQLite operator pipeline, React/three.js UI, ffprobe/ffmpeg_frames/blender_convert operators
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 21:05:22 +10:00