mocapgod/docs/B1_BAKEOFF.md

B1 — pose-model bake-off

Status: research + selection done (2026-07-17). Inference eyeball NOT yet run — blocked on 3 preconditions (see bottom + DECISIONS.md). This lane does not tick until the chosen model produces a right-way-up BVH on a real clip.

The PLAN told this lane to distrust the July-2024 snapshot and search fresh. It paid off: two models that post-date the snapshot change the whole calculus — one of them removes the SMPL licensing hard-stop the plan was built around.

Selection criteria (PLAN order)

1. permissive code license (must ship in commercial games)
2. world-grounded global trajectory (not pose-in-place)
3. runs on Apple Silicon (m3ultra — MPS or acceptable CPU; no NVIDIA on the fleet)
4. SMPL-dependency status (the legal rail)

Candidate matrix

Model	World-grounded	Code lic.	Weights lic.	SMPL?	Apple Silicon	Output	Commercial-ship
MoCapAnything V2 ⭐	trajectories (global vs camera unverified)	MIT	MIT	No	standard PyTorch, no custom CUDA ops → MPS-portable with effort	BVH rotations onto an arbitrary target rig (feed it mixamorig) + hands	YES (MIT/MIT)
GVHMR	Yes — SOTA, beats WHAM 18–28% WA-MPJPE	research-only	needs SMPL	Yes	plain PyTorch, MPS-portable w/ effort	SMPL params (rot) + global traj (.pt)	NO — code is non-commercial and SMPL-gated
GEM-X (NVIDIA)	Yes	Apache-2.0	NVIDIA Open Model Lic. (NVIDIA states commercial-ready)	No (SOMA body model)	CUDA 12.1+ only, NVIDIA arch — will not run on the fleet	77-joint axis-angle + world root (m), incl. hands	Yes, but off-fleet
WHAM	Yes (autoregressive; GVHMR's predecessor)	research/NC (not re-verified)	needs SMPL	Yes	PyTorch, MPS-portable	SMPL + traj	NO (gate) — dominated by GVHMR
TRAM	Yes (SLAM-based camera+human)	mixed; +DROID-SLAM	needs SMPL	Yes	heavier (SLAM stage)	SMPL + traj	NO (gate) — heaviest, not pursued

Sources verified this session: GVHMR repo LICENSE (custom research-only, "prohibited for commercial use", contact xwzhou@zju.edu.cn); GEM-X model card (Apache code / NVIDIA Open Model License weights / SOMA / CUDA-only); MoCapAnything V2 weights card (kehong/MoCapAnythingV2-weights, License: mit, author Kehong Gong = paper author) and code repo github.com/phongdaot/MocapAnything (MIT). WHAM/TRAM licenses not re-verified — both are SMPL-gated and dominated by GVHMR on the world-grounded metric, so not installed.

The two findings that matter

1. GVHMR (the accuracy leader) is double-blocked for us. Its own code license is non-commercial — that's a wall independent of SMPL, and the joints-only Path-B from the PLAN can't get around a non-commercial code license. John's clips ship in commercial games. GVHMR is therefore a research/accuracy reference only, unless John negotiates a paid ZJU commercial license. It is not a v1 shipping engine.

2. MoCapAnything V2 dissolves the SMPL gate. MIT code + MIT weights, commercial use permitted, no SMPL files anywhere in the pipeline or dependency list (torch/opencv/ trimesh only). It is category-agnostic: you give it a monocular video plus the target rig as a prompt, and it emits a rotation-based BVH that drives that rig directly, via constraint-aware IK. Cross-species too (it retargets across heterogeneous rigs — a free preview of the Lane E4 centaur work).

Why #2 is bigger than "gate cleared"

If we prompt MoCapAnything with the mixamorig skeleton (spec/mixamorig_bones.json / female_game.glb), it emits mixamorig BVH directly. That means:

• spec/smpl_to_mixamorig.json (B2 task 2) is unnecessary — there is no SMPL intermediate to map.
• Lane C's core constraint-retarget (bvh→mixamorig) largely collapses — the BVH is already on-skeleton. Lane C shrinks to: import → foot-contact fix → jitter → trim → export FBX.

The fresh search didn't just clear the legal hold-up; it deleted a spec file and most of a lane. # ponytail win: don't build the SMPL→mixamorig mapper or the Lane C retarget rig until we've confirmed MoCapAnything can't hit mixamorig directly.

Caveat (honest): getting it to emit exactly the treaty's mixamorig (joint order, Y-up, faces −Z, bone-roll) will still need a rig-prompt config and a convention reconciliation pass. "Collapses" = "greatly simplifies," not "free."

Recommendation

Primary: MoCapAnything V2. Only candidate that is commercial-clean AND on-fleet AND BVH-direct. Accept two risks, both verifiable cheaply at inference (below). Reference (optional): GVHMR under its research license, purely to benchmark how much accuracy we're leaving on the table — informs whether a paid ZJU license is ever worth it. Shelved: GEM-X — revisit only if the fleet ever gains an NVIDIA GPU (it's the cleanest option the day CUDA is available; zero-cloud rules out renting it).

Must-verify at the inference eyeball (the other half of B1)

These are why the lane isn't ticked. Cheap to settle — MoCapAnything ships a ~160 MB sample dataset and an HF demo Space, so we don't even need John's footage to answer them:

1. Global trajectory? Treaty requires world-space root motion. MoCapAnything says "temporally coherent trajectories" — confirm the character travels through the world, not runs-in-place. If camera-space only, we need a grounding pass (or fall back to GVHMR for locomotion clips). This is the #1 risk.
2. MPS reality. Standard PyTorch predicts yes, but the PLAN's rule is "check MPS early — it decides everything." Expect some ops to need CPU fallback; time it (<2× realtime is the bar).
3. Quality. The linked code is a re-implementation ("use as reference, not reproduction"). Eyeball walk / dance / pick-up-object in Blender before trusting it.

Preconditions before the install/inference session

• m3ultra disk: 29 GB free of 926 GB (97% full). Marginal for one torch env + weights; too tight for two. Clear space first (see DECISIONS.md).
• Test footage: inbox is empty. Sample data covers the smoke test; John's walk / dance / pick-up-object clips are needed for the real quality call and B2/B3/acceptance.
• Engine decision: confirm MoCapAnything-primary (vs. paying for GVHMR-grade). See DECISIONS.md.

Sources

• GVHMR · GVHMR paper
• NVIDIA GEM-X
• MoCapAnything project · V2 weights (MIT) · code · paper
• TRAM