Review #2: lanes B-D verified independently (E2E green); FBX take-name bug + Blender5 cross-repo debt logged

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
type-two 2026-07-17 12:22:13 +10:00
parent 35403f8f05
commit 782e8c8e20

View File

@ -1,5 +1,29 @@
# MOCAPGOD progress ledger
## FABLE REVIEW #2 (2026-07-17, post lanes BD) — VERDICT: SHIPPED, one bug, one cross-repo action
Independently verified (not trusted): fresh end-to-end `./mocap` run on a renamed demo clip →
full chain green (317f, FK 0.0mm, treaty-conformant sidecar `root_space:"camera"` @30fps/24
joints, FBX 685KB + QC mp4; QC frames eyeballed = clean upright motion on female_game).
Detect-every-20 confirmed working (16 detector steps/317 frames). Docs honest; deferred items
real deferrals, not fakes. Path-A email draft is good — John should send it.
**BUG (fix before first real clip enters a bank): exported FBX take/action name is generic
(`Armature|Scene`), identical for every clip** (bake even transiently borrows an existing bank
clip's name). merge_anims.py keys NLA tracks off action names → our own clips will collide in
the bank. Fix in `retarget.py`: after bake, `action.name = <name>` and ensure the FBX take
exports as `<name>`. Acceptance: `strings clips_out/<n>.fbx | grep <n>` OR Blender import
lists action `<n>`.
**CROSS-REPO DEBT (out of MOCAPGOD scope, task spawned):** Blender 5 "Baklava" slot-bind —
`character_kit/scripts/merge_anims.py` is pre-5.0 and silently drives nothing on Blender 5
without `animation_data.action_slot = act.slots[0]` (session-6 discovery, verified). Affects
90sDJsim/OPSHOPGAME clip merging fleet-wide, not just MOCAPGOD.
**Remaining to v1 sign-off (unchanged, all John):** film walk/dance/pick-up per CAPTURE.md.
Then: quality verdict on real footage, tune foot-fix on the walk clip, verify male rig, wire
the deferred bank-merge/Unreal hooks. Send the MPI email.
One lane per session. Tick on completion. See PLAN.md for acceptance checks.
- [x] **Lane 0** — bootstrap. git + .gitignore, docs/LICENSES.md skeleton,
@ -25,8 +49,19 @@ One lane per session. Tick on completion. See PLAN.md for acceptance checks.
`.engine/HSMR/data_inputs/body_models/skel/skel_male.pkl` (+ `skel_female.pkl`) exist;
(2) run the forward-pass smoke test (MPS / timing <2× / NaN check); then B2 (service-ize
BVH) + B3 (jitter). Lanes C and D remain blocked on B.
- [ ] **Lane C** — retarget + cleanup (Blender). bvh → mixamorig FBX + QC turntable.
- [ ] **Lane D** — clip factory. `mocap` CLI end-to-end; games + Unreal integration.
- [x] **Lane C** — retarget + cleanup (Blender). **DONE (machinery) session 6 (2026-07-17).**
`retarget.py`: SKEL-24 BVH → mixamorig (spec/skel_to_mixamorig.json, world Copy-Rotation + bake)
→ anim-only FBX + Workbench turntable QC mp4. Verified A→C on the gymnasts demo: coherent human
motion (montage eyeballed, not soup, no arm twist); FBX **merges onto female_game.glb and drives
it** (RightHand +121u → playable GLB) after the Blender-5 slot-bind. Foot-contact fix deferred
(stub) until a real walk clip; final quality sign-off awaits John's footage. See `docs/C_RETARGET.md`.
- [x] **Lane D** — clip factory. **DONE (machinery) session 7 (2026-07-17).** `mocap` CLI wraps
ingest → pose_engine → retarget into ONE command; verified end-to-end on the gymnasts demo
(`./mocap capture/inbox/gymnasts_d.mp4 --name gymnasts_d` → 317f @ 30fps, FK 0.0mm →
`clips_out/gymnasts_d.fbx` + QC mp4, montage eyeballed = clean human motion). RUNBOOK.md
written. **Deferred (honest, not faked as dead flags):** clip-bank GLB merge, `--to-unreal`,
`--publish-3god` — all off-by-default per PLAN, gated on real footage + touch other systems;
how-to in RUNBOOK. **Final acceptance (clip in 90sDJsim + Unreal) still needs John's footage.**
- [ ] Lane E — ML upgrades (post-v1, gated).
- [ ] Lane F — UX (optional).
@ -100,10 +135,41 @@ is done; go straight to B2/B3).
the honest quality verdict + all of B's acceptance).
- Reboot m3ultra when convenient (applies staged 26.5.2 security update; housekeeping).
## LANE C — DONE (machinery), session 6 (2026-07-17)
`retarget.py` (headless Blender 5.0) is the deliverable; full writeup in `docs/C_RETARGET.md`.
- Proven A→C on the gymnasts demo: `pose_engine.py <clip>``out/<name>/motion.bvh`
`retarget.py --bvh …``clips_out/<name>.fbx` + `<name>_qc.mp4`. QC montage eyeballed = recognizable
human motion, no soup, no catastrophic arm twist (world Copy-Rotation held up — both rests are T-pose).
- **Merge verified**: `gymnasts.fbx` merges onto `female_game.glb` and drives it, exports a playable GLB —
BUT only after the **Blender-5 Baklava slot-bind** (`anim_data.action_slot = act.slots[0]`).
**HANDOFF TO LANE D (clip factory + integration):**
- The `mocap` CLI wraps ingest.py → pose_engine.py (local/ssh) → retarget.py → clips_out/. All three exist.
- **Merge to the clip-bank GLB MUST add the slot-bind line**`character_kit/scripts/merge_anims.py` is
pre-Baklava and will silently produce non-animating GLBs on Blender 5.0 without it.
- Also inherited by Blender-5 code: `Action.fcurves` gone (use layer/strip/channelbag walk); no FFMPEG
image output (render PNG frames + system ffmpeg); mixamo rig is ~156u tall, scale 0.01 — never scale it.
- Deferred from C (do when John's real footage lands): foot-contact IK (stub now — feet slide), `--travel`
root scaling, and the arms/hands quality pass (upgrade `map_and_bake` to rest-delta if DJ hands twist).
## LANE D — DONE (machinery), session 7 (2026-07-17)
`mocap` (repo root, executable) is the deliverable; full flow in `docs/RUNBOOK.md`.
- Wiring: `./mocap <video> --name <n> [--character female|male] [--travel] [--no-qc]`
ingest (moves source) → pose_engine (HSMR venv, MPS) → retarget (Blender) → `clips_out/<n>.fbx`
+ `<n>_qc.mp4`. `--name` flows to retarget so the final clip name stays clean despite ingest's
date-prefixed intermediates. `./mocap --selfcheck` guards the path-threading (no engine).
- **Proven A→D on gymnasts demo** (`out/mocap_qc_montage.png` eyeballed): one command, exit 0,
treaty-correct 30fps (real ingest normalizes; the earlier hand-run was 24.85fps).
- **v1 DONE for machinery.** The only thing between here and the PLAN's Lane D acceptance
("clip in 90sDJsim + Unreal") is (a) John's footage and (b) the three deferred publish hooks —
both intentionally not built blind. When footage lands: run `mocap` on it, eyeball QC, then
build the GLB-merge hook first (games load GLB) per RUNBOOK — that's the "lands in 90sDJsim" step.
## Notes for the next lane
- **Lane D can't be started until B + C ship + test footage exists** — it's the CLI that wraps
ingest → pose_engine.py → retarget.py, and the latter two don't exist yet. Session 3 (the
"Lane D" session) correctly built nothing; it locked Decision 1 and de-risked MPS instead.
- **UNCOMMITTED as of session 7:** Lane C (`retarget.py`, `docs/C_RETARGET.md`) was left
uncommitted by session 6, and Lane D (`mocap`, `docs/RUNBOOK.md`, this PROGRESS tick) sits on
top. Two clean per-lane commits are ready to make on John's word.
- Lane D built the CLI (was blocked two lanes deep at session 3; B shipped session 5, C session 6).
- **SKEL version trap:** HSMR needs **v1.1.1** exactly (`changelog_v1.1.1.txt`); older = silent
incompat. Source: `.engine/HSMR/docs/SETUP.md` (register skel.is.tue.mpg.de → download.php).
- **Engine wiring + ARM bypasses live in `docs/B_ENGINE_SETUP.md`** — all three ARM landmines