4cc67b90fa
B1's MoCapAnything pick reversed (animal model, wrong for human capture); license analysis recorded in LICENSES.md. John chose Path B: run HSMR->SKEL locally on ultra (MPS), ship rig-motion only, accept the non-commercial licence risk as interim until a commercial SKEL licence request goes out (Path A, the clean upgrade). Engine installed under .engine/ (gitignored): uv venv py3.10, torch 2.13 MPS-green, ungated weights, smoke_test.py with Apple-Silicon bypasses (detectron2 stubbed with full-frame bbox, headless pyrender stubbed -> QC in Blender) -- see docs/B_ENGINE_SETUP.md. Remaining before B ships: John's gated SKEL v1.1.1 download, then the forward-pass smoke test (MPS end-to-end, timing, pose sanity). Known caveat: HSMR is per-frame so root is camera-space cam_t, NOT world-grounded -> Lane C derives root from foot-contact. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
50 lines
3.8 KiB
Markdown
50 lines
3.8 KiB
Markdown
# MOCAPGOD progress ledger
|
|
|
|
One lane per session. Tick on completion. See PLAN.md for acceptance checks.
|
|
|
|
- [x] **Lane 0** — bootstrap. git + .gitignore, docs/LICENSES.md skeleton,
|
|
spec/mixamorig_bones.json (65 bones, `mixamorig:` colon-prefixed). — 2026-07-17
|
|
- [x] **Lane A** — capture + ingest. docs/CAPTURE.md protocol, ingest.py
|
|
(probe → 30fps normalize → rename → rsync to m3ultra queue → LEDGER.tsv). — 2026-07-17
|
|
- [ ] **Lane B** — pose engine. **Decision made 2026-07-17 (session 3): Path B** — engine =
|
|
**HSMR → SKEL**, run local, ship rig-motion only, John accepts NC-licence risk
|
|
(`docs/DECISIONS.md` Decision 1 RESOLVED). Env is 90% up in `.engine/HSMR` (uv venv,
|
|
torch 2.13 **MPS verified green**, ungated weights pulled). **Still open before B ships:**
|
|
(1) John downloads gated **SKEL v1.1.1** → `.engine/HSMR/data_inputs/body_models/skel/`;
|
|
(2) fix the person-detector (detectron2 + chumpy failed to build — see gotchas);
|
|
(3) real forward-pass smoke test on an HSMR demo image; then B2 (service-ize → BVH) + B3
|
|
(jitter). Lanes C and D remain blocked on B.
|
|
- [ ] **Lane C** — retarget + cleanup (Blender). bvh → mixamorig FBX + QC turntable.
|
|
- [ ] **Lane D** — clip factory. `mocap` CLI end-to-end; games + Unreal integration.
|
|
- [ ] Lane E — ML upgrades (post-v1, gated).
|
|
- [ ] Lane F — UX (optional).
|
|
|
|
## Notes for the next lane
|
|
- **Lane D can't be started until B + C ship + test footage exists** — it's the CLI that wraps
|
|
ingest → pose_engine.py → retarget.py, and the latter two don't exist yet. Session 3 (the
|
|
"Lane D" session) correctly built nothing; it locked Decision 1 and de-risked MPS instead.
|
|
- **SKEL version trap:** HSMR needs **v1.1.1** exactly (`changelog_v1.1.1.txt`); older = silent
|
|
incompat. Source: `.engine/HSMR/docs/SETUP.md` (register skel.is.tue.mpg.de → download.php).
|
|
- **Engine wiring + ARM bypasses live in `docs/B_ENGINE_SETUP.md`** — `smoke_test.py` already
|
|
stubs the un-buildable detectron2 (full-frame bbox) and headless pyrender (QC → Blender), so
|
|
the detector swap + real QC are B2 polish, **not** blockers for a centered demo clip.
|
|
- **World-grounded gap (important):** HSMR is per-frame → root is camera-space `cam_t`, NOT the
|
|
treaty's world-grounded trajectory. Fine for in-place DJ moves; walking/locomotion needs Lane C
|
|
to derive root from foot-contact (or a grounding pass). Carry into Lane C design.
|
|
- **HSMR is installed on ultra, not m3ultra** (PLAN Ground-Rule 1 puts the pose engine on
|
|
m3ultra). Likely because m3ultra's disk reclaim waits on the reboot (Decision 2). Next B
|
|
session decides: move it to m3ultra, or consciously host inference on ultra.
|
|
- **VOID** (was: "MoCapAnything is BVH-direct, skip the retarget rig"). Reversed — the model
|
|
is animal-only and not our engine. Whatever human model we pick emits **SMPL or SKEL, not
|
|
mixamorig**, so **Lane C's constraint-retarget IS required** and
|
|
`spec/{smpl,skel}_to_mixamorig.json` is back on the menu. HSMR's SKEL output (a biomechanical
|
|
skeleton) is the likely source format if John goes SKEL.
|
|
- **Check MPS + global-trajectory FIRST** for whichever human model John's decision unlocks
|
|
(HSMR has an HF Space + demo to smoke-test before John's footage). These two risks are
|
|
unchanged by the engine swap.
|
|
- Ground-truth skeleton naming is **`mixamorig:`** (colon), armature object named `Armature`.
|
|
Lane C's name-map must handle colon vs underscore variants (PLAN warned about this).
|
|
- Blender headless entrypoint: `/Applications/Blender.app/Contents/MacOS/Blender --background --python <script>`.
|
|
- House style for Blender scripts: `~/Documents/character_kit/scripts/merge_anims.py`
|
|
(note its units warning: mixamorig head sits at ~0.0505 native units — don't rescale via parent).
|