mocapgod/docs/RUNBOOK.md
type-two c918ac3dcd lane C+D: retarget + mocap CLI + docs; fix: FBX take carries the clip name (was Armature|Scene for every clip — bank collisions in take-addressed consumers)
First generated-footage clip proven through the full chain: Wan2.1 T2V (m3ultra, prompt-engineered to CAPTURE.md rules) -> ./mocap -> wan_walk_t1.fbx -> merge_anims bank (49 tracks). PROGRESS.md bug closed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 14:06:38 +10:00

3.6 KiB
Raw Permalink Blame History

MOCAPGOD runbook — film → clip, one command

The whole v1 pipeline on one page. Everything runs on ultra.

The one command

cd ~/Documents/MOCAPGOD
./mocap capture/inbox/<vid>.mp4 --name dj_scratch_01            # female rig, in-place
./mocap <vid>.mov --name spin --character male --travel         # male rig, keep world travel

Options: --character female|male (default female), --travel (keep root translation; default in-place), --no-qc, --gender (override pose gender), --selfcheck.

Output: clips_out/<name>.fbx (anim-only, Mixamo "Without Skin") + clips_out/<name>_qc.mp4. House rule: watch the QC mp4 before the clip enters any bank. Not soup = good.

What it chains (all proven; mocap is just the glue)

Lane Script Does Emits Runs in
A ingest.py probe → 30fps h264 → rename → queue. Moves the source to capture/done/. queue/<date>_<name>.mp4 system python3
B pose_engine.py HSMR/SKEL monocular recovery + SKEL→BVH + jitter out/<stem>/motion.bvh + motion.json + params.npz .engine/HSMR/.venv (MPS)
C retarget.py BVH → mixamorig (constraint + bake) → FBX + turntable QC clips_out/<name>.fbx + _qc.mp4 headless Blender 5

Timing: pose engine dominates (~1.6× realtime on ultra/MPS); a 10s clip ≈ a couple of minutes.

Prerequisites (one-time)

  • SKEL v1.1.1 at .engine/HSMR/data_inputs/body_models/skel/skel_male.pkl (+ female). Gated download — see docs/B_ENGINE_SETUP.md. This is the only hard blocker if the engine is fresh.
  • HSMR env built: .engine/setup_hsmr.sh --patch (idempotent). Blender at /Applications/Blender.app.
  • Filming protocol: docs/CAPTURE.md (landscape, feet in frame, T-pose bookends, one move/take).

Known-good vs. rough edges (honest)

  • In-place clips are first-class. HSMR is per-frame → root is camera-space, not world-grounded (motion.json: root_space:"camera"). DJ moves / dancing-on-the-spot come out clean.
  • Travel (--travel) is best-effort until a foot-contact grounding pass exists — walking clips may drift/slide. Foot-fix is a logged stub today (retarget.py foot_fix); tune it against a real walk clip (Lane C deferred).
  • Fingers are Lane E — the body clip has static hands. Fine for v1 NPC/DJ body moves.
  • Male rig (--character malehum_character.glb) is unverified until a male clip is run; female (female_game.glb) is the proven path.

Deferred hooks — build these when real footage lands (not faked in the CLI)

Each is off by default and needs a real clip to verify, so none ship as dead flags today.

  1. Clip-bank GLB merge (--merge, → 90sDJsim / three.js games). Games load GLB, so this is the "lands in-game" step. Needs a small merge script first: character_kit/scripts/merge_anims.py has no CLI and is pre-Baklava — it must add the Blender-5 slot-bind (animation_data.action_slot = act.slots[0], verified in Lane C) or the merged GLB won't animate. Then merge clips_out/<name>.fbx onto the character's bank GLB and export.
  2. --to-unreal — import via the A_3D_STORE deferred-import pattern (memory: unreal-asset-convergence — slate post-tick, NOT a direct AssetImportTask through the bridge).
  3. --publish-3god — depot pattern from procity publish.py.

John's homework (blocks the real quality verdict)

Film walk / dance / pick-up-object per docs/CAPTURE.md, drop in capture/inbox/, run mocap. Until then the pipeline is proven on the HSMR gymnasts demo only — machinery works; the honest "is it good enough for my DJ NPCs" call needs footage of you.