Review #2: lanes B-D verified independently (E2E green); FBX take-name bug + Blender5 cross-repo debt logged
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
35403f8f05
commit
782e8c8e20
76
PROGRESS.md
76
PROGRESS.md
@ -1,5 +1,29 @@
|
||||
# MOCAPGOD progress ledger
|
||||
|
||||
## FABLE REVIEW #2 (2026-07-17, post lanes B–D) — VERDICT: SHIPPED, one bug, one cross-repo action
|
||||
|
||||
Independently verified (not trusted): fresh end-to-end `./mocap` run on a renamed demo clip →
|
||||
full chain green (317f, FK 0.0mm, treaty-conformant sidecar `root_space:"camera"` @30fps/24
|
||||
joints, FBX 685KB + QC mp4; QC frames eyeballed = clean upright motion on female_game).
|
||||
Detect-every-20 confirmed working (16 detector steps/317 frames). Docs honest; deferred items
|
||||
real deferrals, not fakes. Path-A email draft is good — John should send it.
|
||||
|
||||
**BUG (fix before first real clip enters a bank): exported FBX take/action name is generic
|
||||
(`Armature|Scene`), identical for every clip** (bake even transiently borrows an existing bank
|
||||
clip's name). merge_anims.py keys NLA tracks off action names → our own clips will collide in
|
||||
the bank. Fix in `retarget.py`: after bake, `action.name = <name>` and ensure the FBX take
|
||||
exports as `<name>`. Acceptance: `strings clips_out/<n>.fbx | grep <n>` OR Blender import
|
||||
lists action `<n>`.
|
||||
|
||||
**CROSS-REPO DEBT (out of MOCAPGOD scope, task spawned):** Blender 5 "Baklava" slot-bind —
|
||||
`character_kit/scripts/merge_anims.py` is pre-5.0 and silently drives nothing on Blender 5
|
||||
without `animation_data.action_slot = act.slots[0]` (session-6 discovery, verified). Affects
|
||||
90sDJsim/OPSHOPGAME clip merging fleet-wide, not just MOCAPGOD.
|
||||
|
||||
**Remaining to v1 sign-off (unchanged, all John):** film walk/dance/pick-up per CAPTURE.md.
|
||||
Then: quality verdict on real footage, tune foot-fix on the walk clip, verify male rig, wire
|
||||
the deferred bank-merge/Unreal hooks. Send the MPI email.
|
||||
|
||||
One lane per session. Tick on completion. See PLAN.md for acceptance checks.
|
||||
|
||||
- [x] **Lane 0** — bootstrap. git + .gitignore, docs/LICENSES.md skeleton,
|
||||
@ -25,8 +49,19 @@ One lane per session. Tick on completion. See PLAN.md for acceptance checks.
|
||||
`.engine/HSMR/data_inputs/body_models/skel/skel_male.pkl` (+ `skel_female.pkl`) exist;
|
||||
(2) run the forward-pass smoke test (MPS / timing <2× / NaN check); then B2 (service-ize →
|
||||
BVH) + B3 (jitter). Lanes C and D remain blocked on B.
|
||||
- [ ] **Lane C** — retarget + cleanup (Blender). bvh → mixamorig FBX + QC turntable.
|
||||
- [ ] **Lane D** — clip factory. `mocap` CLI end-to-end; games + Unreal integration.
|
||||
- [x] **Lane C** — retarget + cleanup (Blender). **DONE (machinery) session 6 (2026-07-17).**
|
||||
`retarget.py`: SKEL-24 BVH → mixamorig (spec/skel_to_mixamorig.json, world Copy-Rotation + bake)
|
||||
→ anim-only FBX + Workbench turntable QC mp4. Verified A→C on the gymnasts demo: coherent human
|
||||
motion (montage eyeballed, not soup, no arm twist); FBX **merges onto female_game.glb and drives
|
||||
it** (RightHand +121u → playable GLB) after the Blender-5 slot-bind. Foot-contact fix deferred
|
||||
(stub) until a real walk clip; final quality sign-off awaits John's footage. See `docs/C_RETARGET.md`.
|
||||
- [x] **Lane D** — clip factory. **DONE (machinery) session 7 (2026-07-17).** `mocap` CLI wraps
|
||||
ingest → pose_engine → retarget into ONE command; verified end-to-end on the gymnasts demo
|
||||
(`./mocap capture/inbox/gymnasts_d.mp4 --name gymnasts_d` → 317f @ 30fps, FK 0.0mm →
|
||||
`clips_out/gymnasts_d.fbx` + QC mp4, montage eyeballed = clean human motion). RUNBOOK.md
|
||||
written. **Deferred (honest, not faked as dead flags):** clip-bank GLB merge, `--to-unreal`,
|
||||
`--publish-3god` — all off-by-default per PLAN, gated on real footage + touch other systems;
|
||||
how-to in RUNBOOK. **Final acceptance (clip in 90sDJsim + Unreal) still needs John's footage.**
|
||||
- [ ] Lane E — ML upgrades (post-v1, gated).
|
||||
- [ ] Lane F — UX (optional).
|
||||
|
||||
@ -100,10 +135,41 @@ is done; go straight to B2/B3).
|
||||
the honest quality verdict + all of B's acceptance).
|
||||
- Reboot m3ultra when convenient (applies staged 26.5.2 security update; housekeeping).
|
||||
|
||||
## LANE C — DONE (machinery), session 6 (2026-07-17)
|
||||
`retarget.py` (headless Blender 5.0) is the deliverable; full writeup in `docs/C_RETARGET.md`.
|
||||
- Proven A→C on the gymnasts demo: `pose_engine.py <clip>` → `out/<name>/motion.bvh` →
|
||||
`retarget.py --bvh …` → `clips_out/<name>.fbx` + `<name>_qc.mp4`. QC montage eyeballed = recognizable
|
||||
human motion, no soup, no catastrophic arm twist (world Copy-Rotation held up — both rests are T-pose).
|
||||
- **Merge verified**: `gymnasts.fbx` merges onto `female_game.glb` and drives it, exports a playable GLB —
|
||||
BUT only after the **Blender-5 Baklava slot-bind** (`anim_data.action_slot = act.slots[0]`).
|
||||
|
||||
**HANDOFF TO LANE D (clip factory + integration):**
|
||||
- The `mocap` CLI wraps ingest.py → pose_engine.py (local/ssh) → retarget.py → clips_out/. All three exist.
|
||||
- **Merge to the clip-bank GLB MUST add the slot-bind line** — `character_kit/scripts/merge_anims.py` is
|
||||
pre-Baklava and will silently produce non-animating GLBs on Blender 5.0 without it.
|
||||
- Also inherited by Blender-5 code: `Action.fcurves` gone (use layer/strip/channelbag walk); no FFMPEG
|
||||
image output (render PNG frames + system ffmpeg); mixamo rig is ~156u tall, scale 0.01 — never scale it.
|
||||
- Deferred from C (do when John's real footage lands): foot-contact IK (stub now — feet slide), `--travel`
|
||||
root scaling, and the arms/hands quality pass (upgrade `map_and_bake` to rest-delta if DJ hands twist).
|
||||
|
||||
## LANE D — DONE (machinery), session 7 (2026-07-17)
|
||||
`mocap` (repo root, executable) is the deliverable; full flow in `docs/RUNBOOK.md`.
|
||||
- Wiring: `./mocap <video> --name <n> [--character female|male] [--travel] [--no-qc]` →
|
||||
ingest (moves source) → pose_engine (HSMR venv, MPS) → retarget (Blender) → `clips_out/<n>.fbx`
|
||||
+ `<n>_qc.mp4`. `--name` flows to retarget so the final clip name stays clean despite ingest's
|
||||
date-prefixed intermediates. `./mocap --selfcheck` guards the path-threading (no engine).
|
||||
- **Proven A→D on gymnasts demo** (`out/mocap_qc_montage.png` eyeballed): one command, exit 0,
|
||||
treaty-correct 30fps (real ingest normalizes; the earlier hand-run was 24.85fps).
|
||||
- **v1 DONE for machinery.** The only thing between here and the PLAN's Lane D acceptance
|
||||
("clip in 90sDJsim + Unreal") is (a) John's footage and (b) the three deferred publish hooks —
|
||||
both intentionally not built blind. When footage lands: run `mocap` on it, eyeball QC, then
|
||||
build the GLB-merge hook first (games load GLB) per RUNBOOK — that's the "lands in 90sDJsim" step.
|
||||
|
||||
## Notes for the next lane
|
||||
- **Lane D can't be started until B + C ship + test footage exists** — it's the CLI that wraps
|
||||
ingest → pose_engine.py → retarget.py, and the latter two don't exist yet. Session 3 (the
|
||||
"Lane D" session) correctly built nothing; it locked Decision 1 and de-risked MPS instead.
|
||||
- **UNCOMMITTED as of session 7:** Lane C (`retarget.py`, `docs/C_RETARGET.md`) was left
|
||||
uncommitted by session 6, and Lane D (`mocap`, `docs/RUNBOOK.md`, this PROGRESS tick) sits on
|
||||
top. Two clean per-lane commits are ready to make on John's word.
|
||||
- Lane D built the CLI (was blocked two lanes deep at session 3; B shipped session 5, C session 6).
|
||||
- **SKEL version trap:** HSMR needs **v1.1.1** exactly (`changelog_v1.1.1.txt`); older = silent
|
||||
incompat. Source: `.engine/HSMR/docs/SETUP.md` (register skel.is.tue.mpg.de → download.php).
|
||||
- **Engine wiring + ARM bypasses live in `docs/B_ENGINE_SETUP.md`** — all three ARM landmines
|
||||
|
||||
Loading…
Reference in New Issue
Block a user