CORRIDORKEY.md: status, Ultra tuning findings, video->3D use-case map

CORRIDORKEY.md

@@ -0,0 +1,50 @@
+# CorridorKey in MODELBEAST — status, tuning findings, and use-case map
+
+*Working notes, 2026-07-16. We're just getting started — expect this doc to grow as we try things.*
+
+## What it is
+[CorridorKey](https://github.com/nikopueringer/CorridorKey) (Corridor Digital, 14.4k★): a neural green/blue-screen keyer that recovers **true color in semi-transparent pixels** (motion blur, hair, defocus) instead of a binary mask. Native MLX on Apple Silicon. Outputs pro compositing passes: linear EXR alpha (`/Matte`), straight FG color (`/FG`), premultiplied RGBA (`/Processed`).
+
+## Current status in our fleet
+- Installed + benchmarked on **M3 Ultra and M1 Ultra** (`vendor/corridorkey`, venv via `uv sync --extra mlx`; 399MB checkpoint in `CorridorKeyModule/checkpoints/`).
+- **Our engine patch** (honor `compile` in tiled mode): `vendor/corridorkey-mlx` branch `modelbeast`, editable-installed into both nodes' venvs. **1.47× faster on M3, bit-identical output.**
+- **Self-owned fork:** `monster/corridorkey-mrp-mlx` on the Gitea (full upstream history + patch).
+- **Best config (both Ultras + any ≥8GB node):** `tile_size=512, overlap=64` → top quality (IoU 0.932 vs ground truth) at 2.3GB peak. Throughput mode: full-frame 1024. Full numbers: [BENCHMARKS.md](BENCHMARKS.md).
+- **Deferred (deliberately, while we learn more):** upstream PR of the patch (check overlap with their open PRs #8/#11 first — maintainer is active and receptive); `corridorkey` queue operator; GVM trial.
+- **MLX gaps today:** blue-screen checkpoint, despill/despeckle → torch/MPS backend covers those if needed. License: **CC BY-NC-SA (non-commercial)**.
+
+## The core question: keyed video → 3D models?
+
+**Yes — this works today with existing operators.** The chain:
+
+```
+green-screen video ──ffmpeg_frames──▶ frames ──CorridorKey──▶ per-frame RGBA cutouts
+       (true edge color, motion-blur-aware)          │
+                                                     ├─▶ best frame ──hunyuan3d_mlx / trellis_mac──▶ textured GLB
+                                                     ├─▶ all frames ──colmap_poses──▶ brush_train──▶ clean object splat
+                                                     └─▶ frame set ──▶ sprite sheets / datasets / comps
+```
+
+Our #1 quality rule has always been "bg-remove before 3D-gen." CorridorKey is that rule **upgraded for video**: RMBG guesses at edges of a single image; CorridorKey *recovers* them from chroma, per frame, with motion blur handled.
+
+**The honest caveat:** it needs an actual green/blue screen behind the subject. RMBG keys anything; CorridorKey keys *filmed-for-keying* footage — but does it at a fidelity RMBG can't touch. (The no-green-screen version of this is GVM — see use case 7.)
+
+## Use-case map (ranked by fit to our pipeline)
+
+1. **Turntable capture rig → hero 3D** ⭐ *the killer combo.* Object on a turntable in front of a green screen → every rotation frame becomes a perfect cutout → best/sharpest frame (ffmpeg blur-cull) → `hunyuan3d_mlx`/`trellis_mac`. Beats single-photo capture: you pick the ideal angle *after* shooting, from hundreds of candidates.
+2. **Clean gaussian splats / photogrammetry.** Background features poison SfM and splat training. Pre-keyed frames → `colmap_poses` → `brush_train` = object-only splats with no background floaters. (Apple Object Capture also accepts masks.)
+3. **Multi-view 3D.** A rotating subject gives calibrated-ish multi-view cutouts → fal Hunyuan multi-view today; local multi-view later.
+4. **Person/character reference capture.** Film someone on green → pristine per-frame mattes → reference sets for character modeling, texture projection, or the video→rigged-character track in [PLAN.md](PLAN.md).
+5. **Sprite/billboard factories.** Keyed animation frames → RGBA sprite sheets for games — pairs with the FLUX "volume/sprites" lane.
+6. **The reverse pipeline: composite INTO our 3D.** CorridorKey's native purpose — key real footage, then place the person *inside* renders of MODELBEAST-generated scenes (Blender comps). Assets out, humans in.
+7. **GVM: keying WITHOUT a green screen** 🏔️ *the 256GB flex.* The optional ~80GB Generative Video Matting module mattes arbitrary footage automatically — no chroma needed. Almost nobody can run it (Niko built it on a 96GB RTX Pro 6000); the M3 Ultra can. If it works on MPS, "any video → clean subject → 3D" with no capture rig. Untested; genuine expedition.
+8. **Dataset/training-data factory.** Volume RGBA cutouts from video → LoRA/fine-tune datasets, asset libraries — one turntable session = hundreds of clean training images.
+9. **Key-then-upscale chain.** Key first, `seedvr2_upscale` the FG only (no background pollution in the upscale), then 3D-gen — the existing "upscale if thin" practice, sharpened.
+
+## Next experiments (in rough order)
+1. Shoot/simulate a **real turntable clip** → run chain end-to-end → compare resulting mesh vs single-photo baseline. (Can simulate with a Blender-rendered turntable over green if no footage handy.)
+2. `corridorkey` **operator** (video/frames in → RGBA sequence out, tiled-512 default) so the queue can fan keying across all three nodes.
+3. **GVM on the M3** — install the ~80GB module, see if MPS survives it.
+4. **Upstream PR** once we've lived with the patch a bit.
+
+*Fleet routing: [CLUSTER.md](CLUSTER.md) · RAM guide: [HARDWARE.md](HARDWARE.md) · numbers: [BENCHMARKS.md](BENCHMARKS.md)*

README.md

@@ -4,6 +4,7 @@ Local-first web app that turns videos, images, and 3D files into meshes, splats,
 
 > 🖥️ **Multi-Mac fleet? Read [CLUSTER.md](CLUSTER.md) first** — the local-first policy, the M3/M1/M4 node roles, and how the central queue routes jobs across machines.
 > 💾 **What runs on which Mac?** [HARDWARE.md](HARDWARE.md) — min/recommended RAM per operator across every Apple Silicon tier (8GB → 256GB).
+> 🎬 **Green-screen keying → 3D:** [CORRIDORKEY.md](CORRIDORKEY.md) — CorridorKey status, our Ultra tuning, and the video→cutout→mesh/splat use-case map.
 
 ## Run