From 7819e12a5a513a675c056e4db0d94a7d7a431295 Mon Sep 17 00:00:00 2001
From: type-two <jingkohn@gmail.com>
Date: Fri, 17 Jul 2026 01:10:26 +1000
Subject: [PATCH] Lane B / B1: pose-model bake-off + SMPL decision gate
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Fresh SOTA survey (per PLAN — distrust the snapshot). Findings:
- MoCapAnything V2 (MIT code+weights, no SMPL, BVH-direct onto arbitrary
  rig) is the pick — sidesteps the SMPL commercial-license hard-stop and
  collapses the SMPL->mixamorig mapper + most of Lane C retarget.
- GVHMR: best accuracy but non-commercial code license -> reference only.
- GEM-X: commercial-clean + perfect output but CUDA-only -> off-fleet.

Stopped at the John-decision gate (docs/DECISIONS.md): confirm engine,
clear m3ultra disk (97% full), supply test footage. Inference eyeball,
B2 service-ize, B3 jitter remain.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---
 PROGRESS.md        |  12 +++++-
 docs/B1_BAKEOFF.md | 103 +++++++++++++++++++++++++++++++++++++++++++++
 docs/DECISIONS.md  |  47 +++++++++++++++++++++
 docs/LICENSES.md   |   6 ++-
 4 files changed, 165 insertions(+), 3 deletions(-)
 create mode 100644 docs/B1_BAKEOFF.md
 create mode 100644 docs/DECISIONS.md

diff --git a/PROGRESS.md b/PROGRESS.md
index ad2219f..b144725 100644
--- a/PROGRESS.md
+++ b/PROGRESS.md
@@ -6,13 +6,23 @@ One lane per session. Tick on completion. See PLAN.md for acceptance checks.
       spec/mixamorig_bones.json (65 bones, `mixamorig:` colon-prefixed). — 2026-07-17
 - [x] **Lane A** — capture + ingest. docs/CAPTURE.md protocol, ingest.py
       (probe → 30fps normalize → rename → rsync to m3ultra queue → LEDGER.tsv). — 2026-07-17
-- [ ] **Lane B** — pose engine (m3ultra). Model bake-off, service-ize, jitter pass.
+- [~] **Lane B** — pose engine (m3ultra). B1 research + selection DONE (2026-07-17):
+      picked **MoCapAnything V2** (MIT/MIT, no SMPL, BVH-direct onto mixamorig) — see
+      `docs/B1_BAKEOFF.md`. **Stopped at the John-decision gate** (`docs/DECISIONS.md`):
+      confirm engine + clear m3ultra disk (97% full) + supply test footage. Inference
+      eyeball, B2 (service-ize), B3 (jitter) remain.
 - [ ] **Lane C** — retarget + cleanup (Blender). bvh → mixamorig FBX + QC turntable.
 - [ ] **Lane D** — clip factory. `mocap` CLI end-to-end; games + Unreal integration.
 - [ ] Lane E — ML upgrades (post-v1, gated).
 - [ ] Lane F — UX (optional).
 
 ## Notes for the next lane
+- **MoCapAnything is BVH-direct onto an arbitrary rig.** If it hits mixamorig directly, do
+  NOT build `spec/smpl_to_mixamorig.json` (B2 task 2) or Lane C's constraint-retarget rig —
+  both are obsoleted. Verify the direct-to-mixamorig path first; only build a mapper if it
+  can't. (`# ponytail: don't write the retarget machinery a BVH-direct model deletes.`)
+- **Check MPS + global-trajectory FIRST next session** (the two unproven risks in B1_BAKEOFF).
+  MoCapAnything ships a ~160MB sample + HF demo — smoke-test without waiting on John's footage.
 - Ground-truth skeleton naming is **`mixamorig:`** (colon), armature object named `Armature`.
   Lane C's name-map must handle colon vs underscore variants (PLAN warned about this).
 - Blender headless entrypoint: `/Applications/Blender.app/Contents/MacOS/Blender --background --python <script>`.
diff --git a/docs/B1_BAKEOFF.md b/docs/B1_BAKEOFF.md
new file mode 100644
index 0000000..02f38ba
--- /dev/null
+++ b/docs/B1_BAKEOFF.md
@@ -0,0 +1,103 @@
+# B1 — pose-model bake-off
+
+**Status:** research + selection done (2026-07-17). Inference eyeball NOT yet run —
+blocked on 3 preconditions (see bottom + `DECISIONS.md`). This lane does **not** tick until
+the chosen model produces a right-way-up BVH on a real clip.
+
+The PLAN told this lane to distrust the July-2024 snapshot and search fresh. It paid off:
+two models that post-date the snapshot change the whole calculus — one of them **removes
+the SMPL licensing hard-stop the plan was built around.**
+
+## Selection criteria (PLAN order)
+1. permissive code license (must ship in commercial games)
+2. world-grounded global trajectory (not pose-in-place)
+3. runs on Apple Silicon (m3ultra — MPS or acceptable CPU; no NVIDIA on the fleet)
+4. SMPL-dependency status (the legal rail)
+
+## Candidate matrix
+
+| Model | World-grounded | Code lic. | Weights lic. | SMPL? | Apple Silicon | Output | Commercial-ship |
+|---|---|---|---|---|---|---|---|
+| **MoCapAnything V2** ⭐ | trajectories (global vs camera **unverified**) | MIT | **MIT** | **No** | standard PyTorch, no custom CUDA ops → MPS-portable *with effort* | **BVH rotations onto an arbitrary target rig** (feed it mixamorig) + hands | **YES (MIT/MIT)** |
+| GVHMR | Yes — SOTA, beats WHAM 18–28% WA-MPJPE | **research-only** | needs SMPL | Yes | plain PyTorch, MPS-portable w/ effort | SMPL params (rot) + global traj (.pt) | **NO** — code is non-commercial *and* SMPL-gated |
+| GEM-X (NVIDIA) | Yes | Apache-2.0 | NVIDIA Open Model Lic. (NVIDIA states commercial-ready) | **No** (SOMA body model) | **CUDA 12.1+ only, NVIDIA arch** — will not run on the fleet | 77-joint axis-angle + world root (m), incl. hands | Yes, but **off-fleet** |
+| WHAM | Yes (autoregressive; GVHMR's predecessor) | research/NC (not re-verified) | needs SMPL | Yes | PyTorch, MPS-portable | SMPL + traj | NO (gate) — dominated by GVHMR |
+| TRAM | Yes (SLAM-based camera+human) | mixed; +DROID-SLAM | needs SMPL | Yes | heavier (SLAM stage) | SMPL + traj | NO (gate) — heaviest, not pursued |
+
+Sources verified this session: GVHMR repo LICENSE (custom research-only, "prohibited for
+commercial use", contact xwzhou@zju.edu.cn); GEM-X model card (Apache code / NVIDIA Open
+Model License weights / SOMA / CUDA-only); MoCapAnything V2 weights card
+(`kehong/MoCapAnythingV2-weights`, **License: mit**, author Kehong Gong = paper author) and
+code repo `github.com/phongdaot/MocapAnything` (MIT). WHAM/TRAM licenses not re-verified —
+both are SMPL-gated and dominated by GVHMR on the world-grounded metric, so not installed.
+
+## The two findings that matter
+
+**1. GVHMR (the accuracy leader) is double-blocked for us.** Its own code license is
+non-commercial — that's a wall *independent of* SMPL, and the joints-only Path-B from the
+PLAN can't get around a non-commercial *code* license. John's clips ship in commercial
+games. GVHMR is therefore a research/accuracy *reference* only, unless John negotiates a
+paid ZJU commercial license. It is **not** a v1 shipping engine.
+
+**2. MoCapAnything V2 dissolves the SMPL gate.** MIT code + MIT weights, commercial use
+permitted, **no SMPL files anywhere** in the pipeline or dependency list (torch/opencv/
+trimesh only). It is category-agnostic: you give it a monocular video **plus the target
+rig as a prompt**, and it emits a rotation-based BVH that drives *that rig* directly, via
+constraint-aware IK. Cross-species too (it retargets across heterogeneous rigs — a free
+preview of the Lane E4 centaur work).
+
+### Why #2 is bigger than "gate cleared"
+If we prompt MoCapAnything with the **mixamorig** skeleton (`spec/mixamorig_bones.json` /
+`female_game.glb`), it emits **mixamorig BVH directly**. That means:
+- `spec/smpl_to_mixamorig.json` (B2 task 2) is **unnecessary** — there is no SMPL
+  intermediate to map.
+- Lane C's core constraint-retarget (bvh→mixamorig) **largely collapses** — the BVH is
+  already on-skeleton. Lane C shrinks to: import → foot-contact fix → jitter → trim →
+  export FBX.
+
+The fresh search didn't just clear the legal hold-up; it deleted a spec file and most of a
+lane. `# ponytail win: don't build the SMPL→mixamorig mapper or the Lane C retarget rig
+until we've confirmed MoCapAnything can't hit mixamorig directly.`
+
+**Caveat (honest):** getting it to emit *exactly* the treaty's mixamorig (joint order,
+Y-up, faces −Z, bone-roll) will still need a rig-prompt config and a convention
+reconciliation pass. "Collapses" = "greatly simplifies," not "free."
+
+## Recommendation
+
+**Primary: MoCapAnything V2.** Only candidate that is commercial-clean AND on-fleet AND
+BVH-direct. Accept two risks, both verifiable cheaply at inference (below).
+**Reference (optional): GVHMR** under its research license, purely to benchmark how much
+accuracy we're leaving on the table — informs whether a paid ZJU license is ever worth it.
+**Shelved: GEM-X** — revisit only if the fleet ever gains an NVIDIA GPU (it's the cleanest
+option the day CUDA is available; zero-cloud rules out renting it).
+
+## Must-verify at the inference eyeball (the other half of B1)
+These are why the lane isn't ticked. Cheap to settle — MoCapAnything ships a ~160 MB sample
+dataset and an HF demo Space, so we don't even need John's footage to answer them:
+1. **Global trajectory?** Treaty requires world-space root motion. MoCapAnything says
+   "temporally coherent trajectories" — confirm the character *travels through the world*,
+   not runs-in-place. If camera-space only, we need a grounding pass (or fall back to GVHMR
+   for locomotion clips). **This is the #1 risk.**
+2. **MPS reality.** Standard PyTorch predicts yes, but the PLAN's rule is "check MPS early —
+   it decides everything." Expect some ops to need CPU fallback; time it (<2× realtime is
+   the bar).
+3. **Quality.** The linked code is a re-implementation ("use as reference, not
+   reproduction"). Eyeball walk / dance / pick-up-object in Blender before trusting it.
+
+## Preconditions before the install/inference session
+- **m3ultra disk: 29 GB free of 926 GB (97% full).** Marginal for one torch env + weights;
+  too tight for two. Clear space first (see `DECISIONS.md`).
+- **Test footage:** inbox is empty. Sample data covers the smoke test; John's walk / dance /
+  pick-up-object clips are needed for the real quality call and B2/B3/acceptance.
+- **Engine decision:** confirm MoCapAnything-primary (vs. paying for GVHMR-grade). See
+  `DECISIONS.md`.
+
+## Sources
+- [GVHMR](https://github.com/zju3dv/GVHMR) · [GVHMR paper](https://arxiv.org/html/2409.06662v1)
+- [NVIDIA GEM-X](https://huggingface.co/nvidia/GEM-X)
+- [MoCapAnything project](https://animotionlab.github.io/MoCapAnything/) ·
+  [V2 weights (MIT)](https://huggingface.co/kehong/MoCapAnythingV2-weights) ·
+  [code](https://github.com/phongdaot/MocapAnything) ·
+  [paper](https://arxiv.org/abs/2512.10881)
+- [TRAM](https://github.com/yufu-wang/tram)
diff --git a/docs/DECISIONS.md b/docs/DECISIONS.md
new file mode 100644
index 0000000..595fdc8
--- /dev/null
+++ b/docs/DECISIONS.md
@@ -0,0 +1,47 @@
+# Decisions needing John
+
+Lane B stops here per the PLAN's John-decision protocol. Full analysis in `B1_BAKEOFF.md`.
+
+## Headline: the SMPL hard-stop is (probably) avoided
+
+The gate the plan feared — "SOTA model needs SMPL, which is research-licensed, so a
+commercial license decision blocks everything" — **does not fire for the recommended
+engine.** The fresh search turned up **MoCapAnything V2**: MIT code + MIT weights,
+commercial use permitted, **no SMPL anywhere**, and it emits BVH straight onto our
+mixamorig rig. So the remaining decision is lighter and strategic, not a legal wall.
+
+## Decision 1 — pose engine (the real choice)
+
+| Option | Pros | Cons |
+|---|---|---|
+| **A. MoCapAnything V2** (recommended) | commercial-clean (MIT/MIT), no SMPL, BVH-direct onto mixamorig (kills a spec file + most of Lane C), runs on the fleet, cross-species bonus for centaurs | newer/less battle-tested; linked code is a re-implementation; global-trajectory + MPS not yet proven (cheap to verify) |
+| B. Pay for GVHMR-grade | best measured accuracy (beats WHAM 18–28%) | GVHMR code is **non-commercial** → must negotiate a paid ZJU license *and* separately clear SMPL. Two negotiations for one engine. |
+| C. GEM-X | cleanest license + best output format | **CUDA-only** — needs an NVIDIA box the fleet doesn't have; renting cloud CUDA breaks the zero-cloud rule |
+
+**Recommendation: A.** Go MoCapAnything V2. It's the only option that is commercial-clean,
+on-fleet, and BVH-direct at once. Verify its two unknowns (global trajectory, MPS) on the
+sample data next session before committing B2 code to it. Keep GVHMR as a research-only
+accuracy yardstick if you ever want to know what B would buy.
+
+→ **John: confirm "go MoCapAnything," or say if you want the paid-GVHMR route explored.**
+
+## Decision 2 — clear m3ultra disk (blocker)
+
+`~` on m3ultra is **97% full — 29 GB free of 926 GB.** Enough for one torch env + weights,
+too tight to be comfortable, and MODELBEAST shares the box. Please free some space (or point
+me at what's safe to prune) before the install session.
+
+→ **John: free space on m3ultra, or greenlight me to find prunable caches.**
+
+## Decision 3 — test footage (blocker for the real quality call)
+
+`capture/inbox/` is empty. I can smoke-test MoCapAnything on its bundled sample, but the
+honest walk/dance/pick-up-object quality verdict — and all of B2/B3 + the acceptance check —
+needs your clips. Film the PLAN's three per `docs/CAPTURE.md` (T-pose bookends, feet in
+frame) and drop them in the inbox, then `python3 ingest.py`.
+
+→ **John: film + ingest walk / dance / pick-up-object when convenient.**
+
+---
+Once Decision 1 is confirmed and 2–3 are unblocked, the next session installs MoCapAnything
+on m3ultra, proves MPS + global trajectory, and proceeds to B2 (service-ize) / B3 (jitter).
diff --git a/docs/LICENSES.md b/docs/LICENSES.md
index 460932c..4aa2076 100644
--- a/docs/LICENSES.md
+++ b/docs/LICENSES.md
@@ -8,8 +8,10 @@ the pose-model rows, Lane E fills any training-dataset rows. See PLAN.md §Groun
 
 | Component | Version | License | Commercial OK? | Notes | Checked |
 |---|---|---|---|---|---|
-| _pose model (Lane B1 winner)_ | — | — | — | fill after bake-off | — |
-| SMPL / SMPL-X body model | — | research (Meshcapade) | **NO** by default | commercial needs Meshcapade license; or use joints-only path (PLAN §5) | — |
+| **MoCapAnything** (B1 pick) | V2 | code MIT + weights MIT | **YES** | `kehong/MoCapAnythingV2-weights` states `License: mit`; code `github.com/phongdaot/MocapAnything` MIT. No SMPL. BVH-direct onto arbitrary rig. Re-verify no SMPL/AMASS-derived weights taint at install. | 2026-07-17 |
+| GVHMR (rejected — NC) | siga24 | custom research-only | **NO** | "prohibited for commercial use"; commercial via xwzhou@zju.edu.cn. Also SMPL-gated. Reference/accuracy yardstick only. | 2026-07-17 |
+| GEM-X (shelved — CUDA) | X | code Apache-2.0 + weights NVIDIA Open Model Lic. | yes, but off-fleet | Uses NVIDIA **SOMA** body model, not SMPL; NVIDIA states commercial-ready. CUDA-only → won't run on Apple Silicon. | 2026-07-17 |
+| SMPL / SMPL-X body model | — | research (Meshcapade) | **NO** by default | **Avoided** by the MoCapAnything path — no SMPL files touched. Gate only returns if we ever switch to a SMPL-based engine (GVHMR/WHAM/TRAM). | 2026-07-17 |
 | AMASS dataset | — | academic | **verify** before any shipped training | Lane E only | — |
 | Mixamo clips | — | Adobe ToS | ship-in-game OK; **not** ML training data | — | — |