monster/modelbeast
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

benchmarks: CorridorKey MLX Ultra tuning — tiled512+compile patch wins (1.47x M3, best quality, 2.3GB)

Browse Source
This commit is contained in:
m3ultra 2026-07-16 21:38:08 +10:00
parent 876a666e9a
commit 175e1eeac4
1 changed files with 15 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
BENCHMARKS.md
View File

@ -47,6 +47,21 @@ First-run downloads (one-time): schnell/dev ~31GB & ~16–19 min each, Klein 9B
**Verdict:** at defaults `trellis_mac` wins on quality (crisper face + geometry, richer color); `hunyuan3d_mlx` is faster, ~3× lighter, and the only local 3D op that runs on M1. `sf3d` stays the ~5s draft tier. All free/offline; fal cloud for on-demand SOTA without the local wait.
## CorridorKey (neural green-screen keyer, MLX) — Ultra tuning (2026-07-16)
Corridor Digital's keyer (`vendor/corridorkey`, 14.4k★) with the native MLX backend (`corridorkey-mlx`, resolved from git — not on PyPI). Benchmarked on a synthetic 12-frame 2048² green-screen set with **exact ground-truth alpha** (RMBG mermaid cutout + soft-alpha stripes/disk over chroma green); hint = 8× downscaled truth. Scores = alpha MAE / soft-IoU vs truth, steady-state after 2-frame warmup.
| config | M3 s/f | M1 s/f | MAE ↓ | IoU ↑ | peak GB |
|---|---|---|---|---|---|
| full 2048 (stock default) | 3.81 | 4.97 | 0.0091 | 0.925 | **28.2** |
| full 1024 | 0.61 | 0.75 | 0.0111 | 0.908 | 3.7 |
| full 512 | 0.25 | 0.31 | 0.0139 | 0.885 | 2.6 |
| tiled 512 (stock: compile forced off) | 3.64 | 4.65 | **0.0082** | **0.932** | 2.5 |
| **tiled 512 + our compile patch** | **2.48** | 4.19 | **0.0082** | **0.932** | **2.3** |
**Findings:** (1) `compile` gains nothing full-frame at 2048 on Ultras (within noise) — the documented "1.5–2×" is a small-res/laptop figure. (2) **Tiled-512 is the QUALITY winner**, not just the memory fallback — best alpha accuracy, model at native tile scale over full-res input. (3) Upstream hard-codes `compile=False` in tiled mode; tiles are fixed-shape so compilation applies — our 1-line patch (`vendor/corridorkey-mlx`, branch `modelbeast`, editable-installed into the app venv on M3+M1) makes tiled **1.47× faster on M3** (3.64→2.48 s/f), 1.11× on M1, output bit-identical. (4) tile 1024 tested worse (quality + speed) — 512 is the sweet spot. (5) full-frame 2048 peaks **28 GB** → 32GB Macs (M1 Max) should run **tiled** (2.3 GB) — which is also the best-quality config anyway.
**Fleet verdict:** best-quality config = `tile_size=512, overlap=64` + our patch: M3 ~0.40 fps, M1 ~0.24 fps at 2048², IoU 0.932, 2.3 GB — runs on every node including the M4 24GB. Throughput mode: full-1024 (1.65/1.34 fps, IoU 0.908). MLX gaps: blue-screen checkpoint + despill/despeckle not yet on MLX (torch backend covers those). License: CC BY-NC-SA (non-commercial).
### Phase D — hunyuan Studio tuning (2026-07-16, M3 Ultra) → new operator defaults
Raised the config from the laptop-tuned defaults to `octree_resolution 384` + `remesh_faces 120000` + `texture_size 4096`. **4096² bake works on the Studio GPU — no Metal command-buffer watchdog** (the existing `extract_textiles` tiling handles it; `uv_feature_map` never needed patching). Result: **380s** (shape 160 + tex 221), peak 20.2GB, 21.5MB GLB, 78k verts / **120k faces**, 4096² baseColor+MR. Quality jump is real — the melted face gains defined eyes + structure, tail geometry sharpens, textures crisper; closes most of the gap to trellis (trellis still edges the face). Cost: ~46% slower + ~3× file size vs the 40k/2048 default. **These are now the `hunyuan3d_mlx` operator defaults** (all still param-overridable; drop to `remesh_faces 40000`/`texture_size 2048` for fast drafts). **4096 confirmed watchdog-free on the M1 Ultra too** (751s total — M1 runs it at ~2× M3 time, so speed-critical jobs prefer M3, which is first in the pool). Defaults are fleet-safe on both Ultras.