monster/mocapgod
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
4cc67b90fa
mocapgod/PROGRESS.md
type-two 4cc67b90fa Lane B: resolve decision gate -> Path B (local HSMR/SKEL, accept NC risk) 
B1's MoCapAnything pick reversed (animal model, wrong for human capture);
license analysis recorded in LICENSES.md. John chose Path B: run HSMR->SKEL
locally on ultra (MPS), ship rig-motion only, accept the non-commercial
licence risk as interim until a commercial SKEL licence request goes out
(Path A, the clean upgrade).

Engine installed under .engine/ (gitignored): uv venv py3.10, torch 2.13
MPS-green, ungated weights, smoke_test.py with Apple-Silicon bypasses
(detectron2 stubbed with full-frame bbox, headless pyrender stubbed -> QC in
Blender) -- see docs/B_ENGINE_SETUP.md.

Remaining before B ships: John's gated SKEL v1.1.1 download, then the
forward-pass smoke test (MPS end-to-end, timing, pose sanity). Known caveat:
HSMR is per-frame so root is camera-space cam_t, NOT world-grounded -> Lane C
derives root from foot-contact.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 02:25:55 +10:00

3.8 KiB
Raw Blame History

MOCAPGOD progress ledger

One lane per session. Tick on completion. See PLAN.md for acceptance checks.

  • Lane 0 — bootstrap. git + .gitignore, docs/LICENSES.md skeleton, spec/mixamorig_bones.json (65 bones, mixamorig: colon-prefixed). — 2026-07-17
  • Lane A — capture + ingest. docs/CAPTURE.md protocol, ingest.py (probe → 30fps normalize → rename → rsync to m3ultra queue → LEDGER.tsv). — 2026-07-17
  • Lane B — pose engine. Decision made 2026-07-17 (session 3): Path B — engine = HSMR → SKEL, run local, ship rig-motion only, John accepts NC-licence risk (docs/DECISIONS.md Decision 1 RESOLVED). Env is 90% up in .engine/HSMR (uv venv, torch 2.13 MPS verified green, ungated weights pulled). Still open before B ships: (1) John downloads gated SKEL v1.1.1 → .engine/HSMR/data_inputs/body_models/skel/; (2) fix the person-detector (detectron2 + chumpy failed to build — see gotchas); (3) real forward-pass smoke test on an HSMR demo image; then B2 (service-ize → BVH) + B3 (jitter). Lanes C and D remain blocked on B.
  • Lane C — retarget + cleanup (Blender). bvh → mixamorig FBX + QC turntable.
  • Lane D — clip factory. mocap CLI end-to-end; games + Unreal integration.
  • Lane E — ML upgrades (post-v1, gated).
  • Lane F — UX (optional).

Notes for the next lane

  • Lane D can't be started until B + C ship + test footage exists — it's the CLI that wraps ingest → pose_engine.py → retarget.py, and the latter two don't exist yet. Session 3 (the "Lane D" session) correctly built nothing; it locked Decision 1 and de-risked MPS instead.
  • SKEL version trap: HSMR needs v1.1.1 exactly (changelog_v1.1.1.txt); older = silent incompat. Source: .engine/HSMR/docs/SETUP.md (register skel.is.tue.mpg.de → download.php).
  • Engine wiring + ARM bypasses live in docs/B_ENGINE_SETUP.md — smoke_test.py already stubs the un-buildable detectron2 (full-frame bbox) and headless pyrender (QC → Blender), so the detector swap + real QC are B2 polish, not blockers for a centered demo clip.
  • World-grounded gap (important): HSMR is per-frame → root is camera-space cam_t, NOT the treaty's world-grounded trajectory. Fine for in-place DJ moves; walking/locomotion needs Lane C to derive root from foot-contact (or a grounding pass). Carry into Lane C design.
  • HSMR is installed on ultra, not m3ultra (PLAN Ground-Rule 1 puts the pose engine on m3ultra). Likely because m3ultra's disk reclaim waits on the reboot (Decision 2). Next B session decides: move it to m3ultra, or consciously host inference on ultra.
  • VOID (was: "MoCapAnything is BVH-direct, skip the retarget rig"). Reversed — the model is animal-only and not our engine. Whatever human model we pick emits SMPL or SKEL, not mixamorig, so Lane C's constraint-retarget IS required and spec/{smpl,skel}_to_mixamorig.json is back on the menu. HSMR's SKEL output (a biomechanical skeleton) is the likely source format if John goes SKEL.
  • Check MPS + global-trajectory FIRST for whichever human model John's decision unlocks (HSMR has an HF Space + demo to smoke-test before John's footage). These two risks are unchanged by the engine swap.
  • Ground-truth skeleton naming is mixamorig: (colon), armature object named Armature. Lane C's name-map must handle colon vs underscore variants (PLAN warned about this).
  • Blender headless entrypoint: /Applications/Blender.app/Contents/MacOS/Blender --background --python <script>.
  • House style for Blender scripts: ~/Documents/character_kit/scripts/merge_anims.py (note its units warning: mixamorig head sits at ~0.0505 native units — don't rescale via parent).