4af9033b68
--merge folds clips_out/*.fbx onto the character's bank GLB via character_kit's merge_anims.py (already CLI'd + slot-bound; fps pin added there this session) -> clips_out/<character>_bank.glb. Rebuilt from scratch each run, so it's idempotent. This is the 'lands in 90sDJsim' step: games load GLB. Verified from the exported glTF JSON rather than Blender (which resamples on read): 27 animations (24 base + 3 ours), no duplicate names, all 30.0 keys/sec, the 317-frame clip keeps 317 keys over 10.567s, our clips drive real bone motion (delta 48-184u). Also: mocap now reports a DANGLING venv symlink honestly (uv pruned its managed CPython 3.10, which makes Path.exists() read as 'missing'), selfcheck covers the bank path, RUNBOOK documents --merge + the env gotcha, PROGRESS records all three bugs and the lesson (check artifacts, not green log lines). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
71 lines
4.3 KiB
Markdown
71 lines
4.3 KiB
Markdown
# MOCAPGOD runbook — film → clip, one command
|
||
|
||
The whole v1 pipeline on one page. Everything runs on **ultra**.
|
||
|
||
## The one command
|
||
```
|
||
cd ~/Documents/MOCAPGOD
|
||
./mocap capture/inbox/<vid>.mp4 --name dj_scratch_01 # female rig, in-place
|
||
./mocap <vid>.mov --name spin --character male --travel # male rig, keep world travel
|
||
```
|
||
Options: `--character female|male` (default female), `--travel` (keep root translation; default
|
||
in-place), `--no-qc`, `--gender` (override pose gender), `--selfcheck`.
|
||
|
||
**Output:** `clips_out/<name>.fbx` (anim-only, Mixamo "Without Skin") + `clips_out/<name>_qc.mp4`.
|
||
**House rule:** watch the QC mp4 before the clip enters any bank. Not soup = good.
|
||
|
||
## What it chains (all proven; `mocap` is just the glue)
|
||
| Lane | Script | Does | Emits | Runs in |
|
||
|---|---|---|---|---|
|
||
| A | `ingest.py` | probe → 30fps h264 → rename → queue. **Moves** the source to `capture/done/`. | `queue/<date>_<name>.mp4` | system python3 |
|
||
| B | `pose_engine.py` | HSMR/SKEL monocular recovery + SKEL→BVH + jitter | `out/<stem>/motion.bvh` + `motion.json` + `params.npz` | `.engine/HSMR/.venv` (MPS) |
|
||
| C | `retarget.py` | BVH → mixamorig (constraint + bake) → FBX + turntable QC | `clips_out/<name>.fbx` + `_qc.mp4` | headless Blender 5 |
|
||
|
||
Timing: pose engine dominates (~1.6× realtime on ultra/MPS); a 10s clip ≈ a couple of minutes.
|
||
|
||
## Prerequisites (one-time)
|
||
- **SKEL v1.1.1** at `.engine/HSMR/data_inputs/body_models/skel/skel_male.pkl` (+ female). Gated
|
||
download — see `docs/B_ENGINE_SETUP.md`. This is the only hard blocker if the engine is fresh.
|
||
- HSMR env built: `.engine/setup_hsmr.sh --patch` (idempotent). Blender at `/Applications/Blender.app`.
|
||
- Filming protocol: `docs/CAPTURE.md` (landscape, **feet in frame**, T-pose bookends, one move/take).
|
||
|
||
## Known-good vs. rough edges (honest)
|
||
- **In-place clips are first-class.** HSMR is per-frame → root is camera-space, not world-grounded
|
||
(`motion.json: root_space:"camera"`). DJ moves / dancing-on-the-spot come out clean.
|
||
- **Travel (`--travel`) is best-effort** until a foot-contact grounding pass exists — walking clips
|
||
may drift/slide. Foot-fix is a logged stub today (`retarget.py foot_fix`); tune it against a real
|
||
walk clip (Lane C deferred).
|
||
- **Fingers are Lane E** — the body clip has static hands. Fine for v1 NPC/DJ body moves.
|
||
- Male rig (`--character male` → `hum_character.glb`) is **unverified** until a male clip is run;
|
||
female (`female_game.glb`) is the proven path.
|
||
|
||
## `--merge` — the clip-bank GLB (BUILT + verified 2026-07-25)
|
||
```
|
||
./mocap capture/inbox/move.mp4 --name dj_scratch_01 --merge
|
||
```
|
||
Folds **every** `clips_out/*.fbx` onto the character's bank GLB via
|
||
`character_kit/scripts/merge_anims.py` → `clips_out/<character>_bank.glb`. Games load GLB, so this
|
||
is the "lands in 90sDJsim" step: point the game at that file.
|
||
|
||
- Rebuilt from scratch each run (base rig + current `clips_out/` contents) — **idempotent, no
|
||
compounding**. Prune `clips_out/` to control what ships.
|
||
- Verified from the exported glTF JSON: female bank = **27 animations** (24 base + 3 ours), no
|
||
duplicate names, all **30.0 keys/sec**, our clips drive real bone motion (Δ 48–184u).
|
||
- Needs the Blender-5 slot-bind and the 30fps pin — both now **in** `merge_anims.py`. Without the
|
||
slot-bind the GLB animates nothing; without the fps pin every rebuild resampled 30→24fps.
|
||
|
||
## Deferred hooks — build when real footage lands (not faked in the CLI)
|
||
1. **`--to-unreal`** — import via the A_3D_STORE deferred-import pattern (memory:
|
||
`unreal-asset-convergence` — slate post-tick, NOT a direct AssetImportTask through the bridge).
|
||
3. **`--publish-3god`** — depot pattern from procity `publish.py`.
|
||
|
||
## Environment gotcha (bit us 2026-07-25)
|
||
`uv` pruned its managed **CPython 3.10**, leaving `.engine/HSMR/.venv/bin/python` a *dangling
|
||
symlink* — so Lane B dies and `Path.exists()` reports the venv "missing". `mocap` now says so
|
||
explicitly. Rebuild with **`./setup_hsmr.sh`** (re-runnable; SKEL files + weights survive).
|
||
|
||
## John's homework (blocks the real quality verdict)
|
||
Film **walk / dance / pick-up-object** per `docs/CAPTURE.md`, drop in `capture/inbox/`, run `mocap`.
|
||
Until then the pipeline is proven on the HSMR `gymnasts` demo only — machinery works; the honest
|
||
"is it good enough for my DJ NPCs" call needs footage of you.
|