Compare commits
3 Commits
35403f8f05
...
c918ac3dcd
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c918ac3dcd | ||
|
|
4cf939b90b | ||
|
|
782e8c8e20 |
76
PROGRESS.md
76
PROGRESS.md
@ -1,5 +1,29 @@
|
||||
# MOCAPGOD progress ledger
|
||||
|
||||
## FABLE REVIEW #2 (2026-07-17, post lanes B–D) — VERDICT: SHIPPED, one bug, one cross-repo action
|
||||
|
||||
Independently verified (not trusted): fresh end-to-end `./mocap` run on a renamed demo clip →
|
||||
full chain green (317f, FK 0.0mm, treaty-conformant sidecar `root_space:"camera"` @30fps/24
|
||||
joints, FBX 685KB + QC mp4; QC frames eyeballed = clean upright motion on female_game).
|
||||
Detect-every-20 confirmed working (16 detector steps/317 frames). Docs honest; deferred items
|
||||
real deferrals, not fakes. Path-A email draft is good — John should send it.
|
||||
|
||||
**BUG (fix before first real clip enters a bank): exported FBX take/action name is generic
|
||||
(`Armature|Scene`), identical for every clip** (bake even transiently borrows an existing bank
|
||||
clip's name). merge_anims.py keys NLA tracks off action names → our own clips will collide in
|
||||
the bank. Fix in `retarget.py`: after bake, `action.name = <name>` and ensure the FBX take
|
||||
exports as `<name>`. Acceptance: `strings clips_out/<n>.fbx | grep <n>` OR Blender import
|
||||
lists action `<n>`.
|
||||
|
||||
**CROSS-REPO DEBT (out of MOCAPGOD scope, task spawned):** Blender 5 "Baklava" slot-bind —
|
||||
`character_kit/scripts/merge_anims.py` is pre-5.0 and silently drives nothing on Blender 5
|
||||
without `animation_data.action_slot = act.slots[0]` (session-6 discovery, verified). Affects
|
||||
90sDJsim/OPSHOPGAME clip merging fleet-wide, not just MOCAPGOD.
|
||||
|
||||
**Remaining to v1 sign-off (unchanged, all John):** film walk/dance/pick-up per CAPTURE.md.
|
||||
Then: quality verdict on real footage, tune foot-fix on the walk clip, verify male rig, wire
|
||||
the deferred bank-merge/Unreal hooks. Send the MPI email.
|
||||
|
||||
One lane per session. Tick on completion. See PLAN.md for acceptance checks.
|
||||
|
||||
- [x] **Lane 0** — bootstrap. git + .gitignore, docs/LICENSES.md skeleton,
|
||||
@ -25,8 +49,19 @@ One lane per session. Tick on completion. See PLAN.md for acceptance checks.
|
||||
`.engine/HSMR/data_inputs/body_models/skel/skel_male.pkl` (+ `skel_female.pkl`) exist;
|
||||
(2) run the forward-pass smoke test (MPS / timing <2× / NaN check); then B2 (service-ize →
|
||||
BVH) + B3 (jitter). Lanes C and D remain blocked on B.
|
||||
- [ ] **Lane C** — retarget + cleanup (Blender). bvh → mixamorig FBX + QC turntable.
|
||||
- [ ] **Lane D** — clip factory. `mocap` CLI end-to-end; games + Unreal integration.
|
||||
- [x] **Lane C** — retarget + cleanup (Blender). **DONE (machinery) session 6 (2026-07-17).**
|
||||
`retarget.py`: SKEL-24 BVH → mixamorig (spec/skel_to_mixamorig.json, world Copy-Rotation + bake)
|
||||
→ anim-only FBX + Workbench turntable QC mp4. Verified A→C on the gymnasts demo: coherent human
|
||||
motion (montage eyeballed, not soup, no arm twist); FBX **merges onto female_game.glb and drives
|
||||
it** (RightHand +121u → playable GLB) after the Blender-5 slot-bind. Foot-contact fix deferred
|
||||
(stub) until a real walk clip; final quality sign-off awaits John's footage. See `docs/C_RETARGET.md`.
|
||||
- [x] **Lane D** — clip factory. **DONE (machinery) session 7 (2026-07-17).** `mocap` CLI wraps
|
||||
ingest → pose_engine → retarget into ONE command; verified end-to-end on the gymnasts demo
|
||||
(`./mocap capture/inbox/gymnasts_d.mp4 --name gymnasts_d` → 317f @ 30fps, FK 0.0mm →
|
||||
`clips_out/gymnasts_d.fbx` + QC mp4, montage eyeballed = clean human motion). RUNBOOK.md
|
||||
written. **Deferred (honest, not faked as dead flags):** clip-bank GLB merge, `--to-unreal`,
|
||||
`--publish-3god` — all off-by-default per PLAN, gated on real footage + touch other systems;
|
||||
how-to in RUNBOOK. **Final acceptance (clip in 90sDJsim + Unreal) still needs John's footage.**
|
||||
- [ ] Lane E — ML upgrades (post-v1, gated).
|
||||
- [ ] Lane F — UX (optional).
|
||||
|
||||
@ -100,10 +135,41 @@ is done; go straight to B2/B3).
|
||||
the honest quality verdict + all of B's acceptance).
|
||||
- Reboot m3ultra when convenient (applies staged 26.5.2 security update; housekeeping).
|
||||
|
||||
## LANE C — DONE (machinery), session 6 (2026-07-17)
|
||||
`retarget.py` (headless Blender 5.0) is the deliverable; full writeup in `docs/C_RETARGET.md`.
|
||||
- Proven A→C on the gymnasts demo: `pose_engine.py <clip>` → `out/<name>/motion.bvh` →
|
||||
`retarget.py --bvh …` → `clips_out/<name>.fbx` + `<name>_qc.mp4`. QC montage eyeballed = recognizable
|
||||
human motion, no soup, no catastrophic arm twist (world Copy-Rotation held up — both rests are T-pose).
|
||||
- **Merge verified**: `gymnasts.fbx` merges onto `female_game.glb` and drives it, exports a playable GLB —
|
||||
BUT only after the **Blender-5 Baklava slot-bind** (`anim_data.action_slot = act.slots[0]`).
|
||||
|
||||
**HANDOFF TO LANE D (clip factory + integration):**
|
||||
- The `mocap` CLI wraps ingest.py → pose_engine.py (local/ssh) → retarget.py → clips_out/. All three exist.
|
||||
- **Merge to the clip-bank GLB MUST add the slot-bind line** — `character_kit/scripts/merge_anims.py` is
|
||||
pre-Baklava and will silently produce non-animating GLBs on Blender 5.0 without it.
|
||||
- Also inherited by Blender-5 code: `Action.fcurves` gone (use layer/strip/channelbag walk); no FFMPEG
|
||||
image output (render PNG frames + system ffmpeg); mixamo rig is ~156u tall, scale 0.01 — never scale it.
|
||||
- Deferred from C (do when John's real footage lands): foot-contact IK (stub now — feet slide), `--travel`
|
||||
root scaling, and the arms/hands quality pass (upgrade `map_and_bake` to rest-delta if DJ hands twist).
|
||||
|
||||
## LANE D — DONE (machinery), session 7 (2026-07-17)
|
||||
`mocap` (repo root, executable) is the deliverable; full flow in `docs/RUNBOOK.md`.
|
||||
- Wiring: `./mocap <video> --name <n> [--character female|male] [--travel] [--no-qc]` →
|
||||
ingest (moves source) → pose_engine (HSMR venv, MPS) → retarget (Blender) → `clips_out/<n>.fbx`
|
||||
+ `<n>_qc.mp4`. `--name` flows to retarget so the final clip name stays clean despite ingest's
|
||||
date-prefixed intermediates. `./mocap --selfcheck` guards the path-threading (no engine).
|
||||
- **Proven A→D on gymnasts demo** (`out/mocap_qc_montage.png` eyeballed): one command, exit 0,
|
||||
treaty-correct 30fps (real ingest normalizes; the earlier hand-run was 24.85fps).
|
||||
- **v1 DONE for machinery.** The only thing between here and the PLAN's Lane D acceptance
|
||||
("clip in 90sDJsim + Unreal") is (a) John's footage and (b) the three deferred publish hooks —
|
||||
both intentionally not built blind. When footage lands: run `mocap` on it, eyeball QC, then
|
||||
build the GLB-merge hook first (games load GLB) per RUNBOOK — that's the "lands in 90sDJsim" step.
|
||||
|
||||
## Notes for the next lane
|
||||
- **Lane D can't be started until B + C ship + test footage exists** — it's the CLI that wraps
|
||||
ingest → pose_engine.py → retarget.py, and the latter two don't exist yet. Session 3 (the
|
||||
"Lane D" session) correctly built nothing; it locked Decision 1 and de-risked MPS instead.
|
||||
- **UNCOMMITTED as of session 7:** Lane C (`retarget.py`, `docs/C_RETARGET.md`) was left
|
||||
uncommitted by session 6, and Lane D (`mocap`, `docs/RUNBOOK.md`, this PROGRESS tick) sits on
|
||||
top. Two clean per-lane commits are ready to make on John's word.
|
||||
- Lane D built the CLI (was blocked two lanes deep at session 3; B shipped session 5, C session 6).
|
||||
- **SKEL version trap:** HSMR needs **v1.1.1** exactly (`changelog_v1.1.1.txt`); older = silent
|
||||
incompat. Source: `.engine/HSMR/docs/SETUP.md` (register skel.is.tue.mpg.de → download.php).
|
||||
- **Engine wiring + ARM bypasses live in `docs/B_ENGINE_SETUP.md`** — all three ARM landmines
|
||||
|
||||
210
README.md
Normal file
210
README.md
Normal file
@ -0,0 +1,210 @@
|
||||
# MOCAPGOD
|
||||
|
||||
**Your own private motion-capture studio.** Film yourself with a phone → get a game-ready
|
||||
animation clip on your character rigs. One command, 100% local, 100% owned, zero cloud,
|
||||
zero subscriptions. Built 2026-07-17 in seven Opus 4.8 sessions + two Fable review passes.
|
||||
|
||||
```
|
||||
./mocap capture/inbox/my_move.mp4 --name dj_scratch_01
|
||||
# → clips_out/dj_scratch_01.fbx + clips_out/dj_scratch_01_qc.mp4
|
||||
```
|
||||
|
||||
The FBX is anim-only, Mixamo-"Without-Skin"-style, on the **mixamorig skeleton shared by
|
||||
every GODVERSE character** — it drops into the same `merge_anims.py` → GLB clip-bank flow
|
||||
that feeds 90sDJsim, THRIFTGOD, and Unreal (A_3D_STORE).
|
||||
|
||||
---
|
||||
|
||||
## Contents
|
||||
1. [How it works](#how-it-works)
|
||||
2. [Quickstart](#quickstart)
|
||||
3. [The instruction manual](#the-instruction-manual)
|
||||
4. [Test results — the evidence](#test-results--the-evidence)
|
||||
5. [Known limits & open items](#known-limits--open-items)
|
||||
6. [Legal](#legal)
|
||||
7. [Development history](#development-history)
|
||||
|
||||
---
|
||||
|
||||
## How it works
|
||||
|
||||
```
|
||||
you, filmed on a phone (docs/CAPTURE.md protocol)
|
||||
│ AirDrop → capture/inbox/
|
||||
▼
|
||||
LANE A ingest.py probe → normalize 30fps h264 → queue/ → LEDGER.tsv
|
||||
▼
|
||||
LANE B pose_engine.py HSMR (ViT-H, CVPR'25) recovers the SKEL biomechanical
|
||||
[.engine venv, skeleton per frame on Apple-Silicon MPS
|
||||
~1.6× realtime] → out/<stem>/motion.bvh (SKEL-24, Y-up, metres)
|
||||
→ motion.json sidecar (fps, root_space, joints…)
|
||||
→ params.npz (raw poses — rerun cheaply with --from-params)
|
||||
▼
|
||||
LANE C retarget.py headless Blender 5: BVH → mixamorig (20-bone world-space
|
||||
constraint map + visual-keying bake) → anim-only FBX
|
||||
+ Workbench turntable QC mp4 (HOUSE RULE: eyeball it)
|
||||
▼
|
||||
LANE D mocap (CLI) the one command that chains A→C and enforces the naming
|
||||
contract inbox → queue → out → clips_out
|
||||
```
|
||||
|
||||
- **Engine:** [HSMR](https://github.com/IsshikiHugh/HSMR) (MIT code) predicting **SKEL**
|
||||
(Max-Planck biomechanical body model, v1.1.1) — chosen over SMPL-family models
|
||||
(GVHMR/WHAM/TRAM) after a license-verified bake-off. Full analysis:
|
||||
`docs/B1_BAKEOFF.md`, `docs/DECISIONS.md`, `docs/LICENSES.md`.
|
||||
- **All Apple Silicon:** the ViT-H recovery runs on MPS at 31 fps; detection runs CPU
|
||||
mobilenet every-20th-frame with box reuse (single-subject assumption).
|
||||
- **Formats treaty** (PLAN.md §Ground-rules): BVH@30fps + JSON sidecar between stages;
|
||||
metres / Y-up / faces −Z; `root_space: "camera"|"world"` is a required sidecar field.
|
||||
|
||||
## Quickstart
|
||||
|
||||
```bash
|
||||
# 0. One-time: engine env (if fresh clone) ~10 min + weights download
|
||||
.engine/setup_hsmr.sh --patch # clone, venv, weights, apply ARM patches
|
||||
# …then place the gated SKEL v1.1.1 files (docs/B_ENGINE_SETUP.md — John's login)
|
||||
|
||||
# 1. Film per docs/CAPTURE.md (landscape, FEET IN FRAME, T-pose bookends, one move/take)
|
||||
# 2. AirDrop to capture/inbox/
|
||||
# 3. One command:
|
||||
./mocap capture/inbox/IMG_4242.mp4 --name head_nod_01
|
||||
# 4. WATCH clips_out/head_nod_01_qc.mp4 ← house rule, every clip, before it enters a bank
|
||||
```
|
||||
|
||||
Flags: `--character female|male` (female = proven path) · `--travel` (keep world root;
|
||||
default in-place) · `--no-qc` · `--gender` (pose model override) · `--selfcheck`.
|
||||
|
||||
## The instruction manual
|
||||
|
||||
### Filming (docs/CAPTURE.md — the 20-second version)
|
||||
Landscape, phone steady at 2–3m, your WHOLE body including feet in frame the entire take,
|
||||
even light, fitted clothes, **1 second of T-pose at start and end**, one move per take,
|
||||
5–30s. Break these and the skeleton quality drops — feet out of frame is the #1 killer.
|
||||
|
||||
### Running stages by hand (debugging / tuning)
|
||||
```bash
|
||||
python3 ingest.py capture/inbox/vid.mp4 --name test # Lane A alone
|
||||
.engine/HSMR/.venv/bin/python pose_engine.py queue/<stem>.mp4 --gender female # Lane B
|
||||
.engine/HSMR/.venv/bin/python pose_engine.py <stem> --from-params # re-emit BVH, skip HMR (fast)
|
||||
/Applications/Blender.app/Contents/MacOS/Blender --background --python retarget.py -- \
|
||||
--bvh out/<stem>/motion.bvh --name test # Lane C alone
|
||||
```
|
||||
|
||||
### Where things live
|
||||
| Path | What |
|
||||
|---|---|
|
||||
| `capture/inbox/` → `capture/done/` | drop zone → processed originals (ingest MOVES them) |
|
||||
| `capture/LEDGER.tsv` | every ingest, accepted or rejected, with reason |
|
||||
| `queue/` | normalized 30fps videos awaiting the pose engine |
|
||||
| `out/<stem>/` | motion.bvh + motion.json + params.npz per clip |
|
||||
| `clips_out/` | **the product**: `<name>.fbx` + `<name>_qc.mp4` |
|
||||
| `.engine/HSMR/` | gitignored engine clone; rebuild any time via `setup_hsmr.sh --patch` |
|
||||
| `engine_patches/` | our ARM/headless fixes (in git; survive re-clones) |
|
||||
| `spec/` | skeleton ground truth: `mixamorig_bones.json`, `skel_to_mixamorig.json` |
|
||||
|
||||
### Into the games / Unreal
|
||||
- **Clip banks:** merge FBX clips onto a character with
|
||||
`character_kit/scripts/merge_anims.py` → multi-clip GLB (⚠️ needs the Blender-5
|
||||
slot-bind fix — task spawned; see Known limits).
|
||||
- **Unreal:** import the merged GLB via the A_3D_STORE deferred-import pattern
|
||||
(slate post-tick, never a blocking import through the MCP bridge — see the
|
||||
unreal-asset-convergence memory / A_3D_STORE experiments).
|
||||
|
||||
### Troubleshooting
|
||||
| Symptom | Cause / fix |
|
||||
|---|---|
|
||||
| `✗ HSMR venv missing` | run `.engine/setup_hsmr.sh --patch` |
|
||||
| smoke/engine fails at `skel_male.pkl` | gated SKEL v1.1.1 not placed — docs/B_ENGINE_SETUP.md |
|
||||
| clip rejected at ingest | check `capture/LEDGER.tsv` note column (portrait? too long? corrupt?) |
|
||||
| feet slide in QC | expected pre-foot-fix; worse on `--travel` (grounding pass not built yet) |
|
||||
| QC motion is soup | filming protocol violated (feet cropped? multiple people in frame?) |
|
||||
| everything re-detects slowly | detector runs every 20th frame; long multi-person videos will crawl — single subject only |
|
||||
|
||||
## Test results — the evidence
|
||||
|
||||
Every claim below was produced during the build and **re-verified independently in review**
|
||||
(Fable re-ran tests rather than trusting the ledger). Raw logs: `out/*.log`, montages:
|
||||
`out/qc_montage.png`, `out/mocap_qc_montage.png`.
|
||||
|
||||
### Detector unit test (session 4, re-run in review #1)
|
||||
| Metric | Result |
|
||||
|---|---|
|
||||
| Demo images with person found | **7/7** |
|
||||
| Human patches extracted @256×256 | 15 |
|
||||
| detectron2 dependency | **eliminated** (torchvision drop-in, ARM-native) |
|
||||
|
||||
### Engine smoke test — first full forward pass (2026-07-17, gymnasts.mp4, 263 frames)
|
||||
| Metric | Result | Bar |
|
||||
|---|---|---|
|
||||
| Device | **MPS** (Apple Silicon GPU) | must not be CPU |
|
||||
| Patches recovered | 1,315 | — |
|
||||
| Recovery throughput | **31.2 fps** | > realtime ✅ |
|
||||
| NaNs in output | **0** | 0 ✅ |
|
||||
| Finding | CPU detector was 80% of wall-clock → fixed in B2 (mobilenet + every-20) | |
|
||||
|
||||
### Lane B acceptance (session 5)
|
||||
| Metric | Result | Bar |
|
||||
|---|---|---|
|
||||
| Steady-state pipeline speed | **~1.6× realtime** | <2× ✅ |
|
||||
| BVH sanity (Blender import) | 24 bones, upright, **1.59m** figure | human-shaped ✅ |
|
||||
| FK self-consistency | **0.0 mm** error | — |
|
||||
| Unattended queue drain | 6 videos: 4 OK, **2 bad clips failed gracefully** | no crash ✅ |
|
||||
| Axis correctness | Y-down OpenCV → Y-up/−Z via diag(1,−1,−1), verified upright | ✅ |
|
||||
|
||||
### Lane C acceptance (session 6)
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| Bones constraint-mapped | 20 (+4 intentionally null: twist/heel; fingers = Lane E) |
|
||||
| Retargeted FBX drives female_game.glb | ✅ verified — RightHand moved +121 units; playable 7.3MB GLB |
|
||||
| QC montage eyeball | coherent gymnast motion, no soup, no candy-wrap arm twist |
|
||||
| Blender 5 "Baklava" API breaks | found + solved (slot-bind, fcurves walk) — documented in docs/C_RETARGET.md |
|
||||
|
||||
### Lane D acceptance + independent review #2 (fresh end-to-end, 2026-07-17)
|
||||
| Metric | Opus run (`gymnasts_d`) | Fable review run (`review_check`) |
|
||||
|---|---|---|
|
||||
| Full chain A→C one command | ✅ | ✅ (cold, renamed input) |
|
||||
| Frames processed | 317 @ 30fps | 317 @ 30fps |
|
||||
| FK error | 0.0 mm | 0.0 mm |
|
||||
| Sidecar treaty fields | ✅ | ✅ (`root_space:"camera"`, 24 joints, fps 30) |
|
||||
| Output FBX + QC | ✅ | ✅ 685KB FBX + QC mp4, frames eyeballed clean |
|
||||
| Wall clock (10.6s clip) | — | ~35s engine total incl. one-time model build |
|
||||
|
||||
**Review #2 findings:** one real bug (FBX take-name is generic `Armature|Scene` on every
|
||||
clip → bank-merge collisions; fix queued in retarget.py before first real clip banks) and
|
||||
one cross-repo action (Blender-5 fix for character_kit merge_anims.py — task spawned).
|
||||
|
||||
## Known limits & open items
|
||||
|
||||
**Honest list — v1 is machinery-complete, not footage-verified.**
|
||||
- 🎬 **Final acceptance awaits John's real footage** (walk / dance / pick-up-object).
|
||||
Everything below the line is gated on it.
|
||||
- 🐛 FBX take-name bug (see review #2) — MUST fix before clips enter banks.
|
||||
- 🦶 Foot-contact fix is a stub — feet may slide, tuned against a real walk clip.
|
||||
- 🚶 `--travel` (world root) is best-effort: HSMR is per-frame → camera-space root.
|
||||
In-place clips (DJ moves!) are first-class; locomotion needs the grounding pass.
|
||||
- ✋ Fingers are static (Lane E: hand pass).
|
||||
- 🧍 Male rig path unverified until a male clip is run.
|
||||
- 📧 Path-A license email drafted, unsent (docs/DECISIONS.md).
|
||||
- Lanes E (MLX training: autoposer, inbetweening, centaur retarget) and F (drop-folder
|
||||
daemon, dashboard) are designed but intentionally not started.
|
||||
|
||||
## Legal
|
||||
|
||||
Read `docs/LICENSES.md` before changing engines or training anything. The short version:
|
||||
HSMR **code** is MIT; the **SKEL body model** (and BioAMASS dataset) are Max-Planck
|
||||
research-licensed — commercial use is John's accepted Path-B judgment call, with the Path-A
|
||||
commercial-license email drafted to retire the risk. **We ship only motion retargeted onto
|
||||
our own rigs; no Max-Planck file is ever redistributed.** Mixamo clips: OK in games, never
|
||||
as training data. Cascadeur: used as a licensed authoring tool, never reverse-engineered.
|
||||
|
||||
## Development history
|
||||
|
||||
Built 2026-07-17, PLAN.md → seven Opus sessions, reviewed twice by Fable (see PROGRESS.md
|
||||
for the full ledger including the honest mistakes):
|
||||
session 1 bake-off picked an animal-only model by accident → session 2 caught and reversed
|
||||
it with a license-verified re-analysis → session 3 locked the HSMR/SKEL decision + MPS →
|
||||
session 4 broke the detectron2 ARM wall with a torchvision drop-in → John supplied the
|
||||
gated SKEL v1.1.1 → smoke test passed first try → session 5 shipped the pose engine (B2+B3)
|
||||
+ six review orders → session 6 shipped the Blender retarget (and discovered the Blender-5
|
||||
animation API changes the hard way) → session 7 shipped the `mocap` CLI → review #2
|
||||
re-ran everything cold and it held.
|
||||
43
docs/C_RETARGET.md
Normal file
43
docs/C_RETARGET.md
Normal file
@ -0,0 +1,43 @@
|
||||
# Lane C — retarget + cleanup (`retarget.py`)
|
||||
|
||||
**DONE (machinery) 2026-07-17 (session 6).** `retarget.py` takes a SKEL-24 `motion.bvh` (from
|
||||
`pose_engine.py`) onto the character_kit mixamorig rig → baked anim-only FBX + QC turntable mp4.
|
||||
Verified on the gymnasts demo: recognizable human motion (not soup), and the FBX merges onto
|
||||
`female_game.glb` and drives it. Final quality sign-off awaits John's own walk/DJ footage.
|
||||
|
||||
## Run (headless Blender only)
|
||||
```
|
||||
/Applications/Blender.app/Contents/MacOS/Blender --background --python retarget.py -- \
|
||||
--bvh out/<name>/motion.bvh [--rig <glb>] [--name <n>] [--travel] [--foot-fix] [--no-qc]
|
||||
```
|
||||
Defaults: rig = `character_kit/female/female_game.glb`, out = `clips_out/`, root in-place.
|
||||
Outputs: `clips_out/<name>.fbx` (anim-only, Mixamo "Without Skin") + `clips_out/<name>_qc.mp4`.
|
||||
Proven chain A→C: `pose_engine.py <clip>` → `out/<name>/motion.bvh` → `retarget.py --bvh …`.
|
||||
|
||||
## How it works
|
||||
- Import mixamorig rig (glb) + SKEL BVH (`axis_forward=-Z, axis_up=Y`; both land Z-up in Blender).
|
||||
- Per `spec/skel_to_mixamorig.json` (20 bones; 4 null twist/heel), add **world-space Copy Rotation**
|
||||
(target mixamorig ← SKEL bone), then `nla.bake(visual_keying, clear_constraints)` → a clean
|
||||
mixamorig action. `find_bone` tolerates the `mixamorig:` colon vs `mixamorig_` underscore trap.
|
||||
- Root **in-place by default** (HSMR root is camera-space, not world-grounded); `--travel` copies it.
|
||||
- Export anim-only FBX (`object_types={ARMATURE}`, `bake_anim`). QC = Workbench turntable, PNG frames
|
||||
stitched to mp4 by system ffmpeg, camera auto-framed from the world bbox.
|
||||
|
||||
## Blender 5.0 gotchas hit + resolved (carry into Lane D)
|
||||
- **Baklava actions**: `Action.fcurves` is gone → `action_fcurves()` walks layers/strips/channelbags.
|
||||
- **Merge needs slot binding** (load-bearing): assigning an action is NOT enough — you must set
|
||||
`animation_data.action_slot = act.slots[0]` or **nothing drives**. Verified: with the slot bound,
|
||||
`gymnasts.fbx` drives `female_game.glb` (RightHand +121u), exports a playable 7.3 MB GLB.
|
||||
**`character_kit/scripts/merge_anims.py` FIXED 2026-07-17** — binds `strip.action_slot` on each NLA strip (hasattr-guarded for 4.x); verified Lifting_Object drives female_game.glb (LeftHand delta 6.7u frame 1→138).
|
||||
- **No FFMPEG image format** in Blender 5.0 → render PNG frames + system `ffmpeg` stitch.
|
||||
- **Mixamo scale**: `female_game.glb` imports at armature-scale 0.01 but ≈156 units tall, centred z≈77.
|
||||
QC auto-frames from the bbox; keep retarget **rotation-only** — never scale the rig (scaling a skinned
|
||||
ancestor tears the mesh when it animates; the `merge_anims.py` units warning).
|
||||
|
||||
## Deferred (honest — not gold-plated)
|
||||
- **Foot-contact fix is a stub** (`--foot-fix` logs planted frames only, no IK → feet may slide).
|
||||
Deferred until a real **walk** clip exists to tune against (gymnasts are airborne). `# ponytail` in code.
|
||||
- **`--travel`** (root translation + unit scaling) is wired but unverified; in-place is v1-first-class.
|
||||
- **Arms/hands quality**: gymnasts looked clean (no catastrophic roll). If DJ hand work twists on John's
|
||||
footage, upgrade the single `map_and_bake` to a rest-delta transfer (`Rt=Rs·Rs_rest⁻¹·Rt_rest`) — noted
|
||||
in code; nothing else changes.
|
||||
56
docs/RUNBOOK.md
Normal file
56
docs/RUNBOOK.md
Normal file
@ -0,0 +1,56 @@
|
||||
# MOCAPGOD runbook — film → clip, one command
|
||||
|
||||
The whole v1 pipeline on one page. Everything runs on **ultra**.
|
||||
|
||||
## The one command
|
||||
```
|
||||
cd ~/Documents/MOCAPGOD
|
||||
./mocap capture/inbox/<vid>.mp4 --name dj_scratch_01 # female rig, in-place
|
||||
./mocap <vid>.mov --name spin --character male --travel # male rig, keep world travel
|
||||
```
|
||||
Options: `--character female|male` (default female), `--travel` (keep root translation; default
|
||||
in-place), `--no-qc`, `--gender` (override pose gender), `--selfcheck`.
|
||||
|
||||
**Output:** `clips_out/<name>.fbx` (anim-only, Mixamo "Without Skin") + `clips_out/<name>_qc.mp4`.
|
||||
**House rule:** watch the QC mp4 before the clip enters any bank. Not soup = good.
|
||||
|
||||
## What it chains (all proven; `mocap` is just the glue)
|
||||
| Lane | Script | Does | Emits | Runs in |
|
||||
|---|---|---|---|---|
|
||||
| A | `ingest.py` | probe → 30fps h264 → rename → queue. **Moves** the source to `capture/done/`. | `queue/<date>_<name>.mp4` | system python3 |
|
||||
| B | `pose_engine.py` | HSMR/SKEL monocular recovery + SKEL→BVH + jitter | `out/<stem>/motion.bvh` + `motion.json` + `params.npz` | `.engine/HSMR/.venv` (MPS) |
|
||||
| C | `retarget.py` | BVH → mixamorig (constraint + bake) → FBX + turntable QC | `clips_out/<name>.fbx` + `_qc.mp4` | headless Blender 5 |
|
||||
|
||||
Timing: pose engine dominates (~1.6× realtime on ultra/MPS); a 10s clip ≈ a couple of minutes.
|
||||
|
||||
## Prerequisites (one-time)
|
||||
- **SKEL v1.1.1** at `.engine/HSMR/data_inputs/body_models/skel/skel_male.pkl` (+ female). Gated
|
||||
download — see `docs/B_ENGINE_SETUP.md`. This is the only hard blocker if the engine is fresh.
|
||||
- HSMR env built: `.engine/setup_hsmr.sh --patch` (idempotent). Blender at `/Applications/Blender.app`.
|
||||
- Filming protocol: `docs/CAPTURE.md` (landscape, **feet in frame**, T-pose bookends, one move/take).
|
||||
|
||||
## Known-good vs. rough edges (honest)
|
||||
- **In-place clips are first-class.** HSMR is per-frame → root is camera-space, not world-grounded
|
||||
(`motion.json: root_space:"camera"`). DJ moves / dancing-on-the-spot come out clean.
|
||||
- **Travel (`--travel`) is best-effort** until a foot-contact grounding pass exists — walking clips
|
||||
may drift/slide. Foot-fix is a logged stub today (`retarget.py foot_fix`); tune it against a real
|
||||
walk clip (Lane C deferred).
|
||||
- **Fingers are Lane E** — the body clip has static hands. Fine for v1 NPC/DJ body moves.
|
||||
- Male rig (`--character male` → `hum_character.glb`) is **unverified** until a male clip is run;
|
||||
female (`female_game.glb`) is the proven path.
|
||||
|
||||
## Deferred hooks — build these when real footage lands (not faked in the CLI)
|
||||
Each is off by default and needs a real clip to verify, so none ship as dead flags today.
|
||||
1. **Clip-bank GLB merge** (`--merge`, → 90sDJsim / three.js games). Games load GLB, so this is the
|
||||
"lands in-game" step. Needs a small merge script first: `character_kit/scripts/merge_anims.py`
|
||||
has **no CLI and is pre-Baklava** — it must add the Blender-5 slot-bind
|
||||
(`animation_data.action_slot = act.slots[0]`, verified in Lane C) or the merged GLB won't animate.
|
||||
Then merge `clips_out/<name>.fbx` onto the character's bank GLB and export.
|
||||
2. **`--to-unreal`** — import via the A_3D_STORE deferred-import pattern (memory:
|
||||
`unreal-asset-convergence` — slate post-tick, NOT a direct AssetImportTask through the bridge).
|
||||
3. **`--publish-3god`** — depot pattern from procity `publish.py`.
|
||||
|
||||
## John's homework (blocks the real quality verdict)
|
||||
Film **walk / dance / pick-up-object** per `docs/CAPTURE.md`, drop in `capture/inbox/`, run `mocap`.
|
||||
Until then the pipeline is proven on the HSMR `gymnasts` demo only — machinery works; the honest
|
||||
"is it good enough for my DJ NPCs" call needs footage of you.
|
||||
119
mocap
Executable file
119
mocap
Executable file
@ -0,0 +1,119 @@
|
||||
#!/usr/bin/env python3
|
||||
"""MOCAPGOD Lane D — `mocap`: one command, phone video → game-ready mocap clip.
|
||||
|
||||
./mocap capture/inbox/vid.mp4 --name dj_scratch_01
|
||||
./mocap spin.mov --name spin --character male --travel
|
||||
|
||||
Chains the three proven stages, all on ultra:
|
||||
Lane A ingest.py video → 30fps h264 → queue/<date>_<name>.mp4 (MOVES the source)
|
||||
Lane B pose_engine.py video → HSMR/SKEL → out/<stem>/motion.bvh + sidecar [HSMR venv, MPS]
|
||||
Lane C retarget.py BVH → mixamorig → clips_out/<name>.fbx + <name>_qc.mp4 [headless Blender]
|
||||
|
||||
Result: clips_out/<name>.fbx (anim-only, Mixamo "Without Skin") + <name>_qc.mp4 turntable.
|
||||
House rule: eyeball the QC mp4 before the clip enters a bank.
|
||||
|
||||
Deferred hooks (off by default — built when John's real footage lands; see docs/RUNBOOK.md):
|
||||
clip-bank GLB merge, --to-unreal, --publish-3god. merge_anims.py needs a CLI + the Blender-5
|
||||
slot-bind first, and none of them are verifiable without a real clip, so they're not faked here.
|
||||
"""
|
||||
import argparse, datetime, re, subprocess, sys
|
||||
from pathlib import Path
|
||||
|
||||
REPO = Path(__file__).resolve().parent
|
||||
QUEUE = REPO / "queue"
|
||||
OUT = REPO / "out"
|
||||
CLIPS = REPO / "clips_out"
|
||||
VENV_PY = REPO / ".engine/HSMR/.venv/bin/python" # Lane B lives in its own env
|
||||
BLENDER = Path("/Applications/Blender.app/Contents/MacOS/Blender")
|
||||
KIT = Path.home() / "Documents/character_kit"
|
||||
RIGS = { # --character → (pose gender, retarget rig)
|
||||
"female": ("female", KIT / "female/female_game.glb"), # v1 proven path
|
||||
"male": ("male", KIT / "hum_character.glb"), # ponytail: male rig unverified until a male clip exists
|
||||
}
|
||||
|
||||
|
||||
def slug(s): # mirrors ingest.slug (kept standalone on purpose)
|
||||
return re.sub(r"[^A-Za-z0-9_-]+", "_", s).strip("_") or "clip"
|
||||
|
||||
|
||||
def paths_for(name):
|
||||
"""The naming contract that threads a clip through all three stages. The one bit of real
|
||||
logic here, so it gets the self-check below."""
|
||||
stem = f"{datetime.date.today():%Y%m%d}_{name}" # ingest's queue naming
|
||||
return stem, QUEUE / f"{stem}.mp4", OUT / stem / "motion.bvh", CLIPS / f"{name}.fbx"
|
||||
|
||||
|
||||
def stage(title, cmd):
|
||||
print(f"\n▶ {title}\n $ {' '.join(str(c) for c in cmd)}", flush=True)
|
||||
if subprocess.run([str(c) for c in cmd]).returncode != 0:
|
||||
sys.exit(f"✗ {title} failed — chain stopped.")
|
||||
|
||||
|
||||
def selfcheck():
|
||||
"""Path-threading only: no engine, no network, no Blender."""
|
||||
stem, q, bvh, fbx = paths_for("dj_scratch_01")
|
||||
assert stem.endswith("_dj_scratch_01"), stem
|
||||
assert q == QUEUE / f"{stem}.mp4", q
|
||||
assert bvh == OUT / stem / "motion.bvh", bvh
|
||||
assert fbx == CLIPS / "dj_scratch_01.fbx", fbx # final name stays clean (retarget --name)
|
||||
assert slug("DJ Scratch #1!") == "DJ_Scratch_1", slug("DJ Scratch #1!")
|
||||
for c in RIGS.values():
|
||||
assert c[0] in ("male", "female")
|
||||
print("selfcheck OK — name threads inbox→queue→out→clips, final clip name stays clean")
|
||||
return 0
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description="Lane D: one command, video → game-ready mocap clip.")
|
||||
ap.add_argument("video", nargs="?", help="source clip (e.g. capture/inbox/move.mp4) — ingest MOVES it")
|
||||
ap.add_argument("--name", help="clip name → clips_out/<name>.fbx")
|
||||
ap.add_argument("--character", choices=list(RIGS), default="female")
|
||||
ap.add_argument("--gender", choices=["male", "female"], help="override pose gender (default: from --character)")
|
||||
ap.add_argument("--travel", action="store_true", help="keep world root translation (default: in-place)")
|
||||
ap.add_argument("--no-qc", action="store_true", help="skip the QC turntable render")
|
||||
ap.add_argument("--selfcheck", action="store_true", help="verify the path-threading logic and exit")
|
||||
a = ap.parse_args()
|
||||
|
||||
if a.selfcheck:
|
||||
return selfcheck()
|
||||
if not a.video or not a.name:
|
||||
ap.error("video and --name are required (or use --selfcheck)")
|
||||
if not VENV_PY.exists():
|
||||
sys.exit(f"✗ HSMR venv missing: {VENV_PY} — run .engine/setup_hsmr.sh (Lane B).")
|
||||
if not BLENDER.exists():
|
||||
sys.exit(f"✗ Blender not found: {BLENDER} (Lane C needs it).")
|
||||
|
||||
name = slug(a.name)
|
||||
gender, rig = RIGS[a.character]
|
||||
gender = a.gender or gender
|
||||
if not rig.exists():
|
||||
sys.exit(f"✗ rig for --character {a.character} not found: {rig}")
|
||||
stem, queued, bvh, fbx = paths_for(name)
|
||||
|
||||
stage("Lane A · ingest", ["python3", REPO / "ingest.py", a.video, "--name", name])
|
||||
if not queued.exists():
|
||||
sys.exit(f"✗ no {queued} after ingest — clip rejected? check capture/LEDGER.tsv")
|
||||
|
||||
stage("Lane B · pose_engine (HSMR/SKEL, MPS — slow, ~1.6× realtime)",
|
||||
[VENV_PY, REPO / "pose_engine.py", queued, "--gender", gender])
|
||||
if not bvh.exists():
|
||||
sys.exit(f"✗ no {bvh} after pose_engine.")
|
||||
|
||||
rt = [BLENDER, "--background", "--python", REPO / "retarget.py", "--",
|
||||
"--bvh", bvh, "--rig", rig, "--name", name]
|
||||
if a.travel: rt.append("--travel")
|
||||
if a.no_qc: rt.append("--no-qc")
|
||||
stage("Lane C · retarget + QC (Blender)", rt)
|
||||
if not fbx.exists():
|
||||
sys.exit(f"✗ no {fbx} after retarget.")
|
||||
|
||||
qc = CLIPS / f"{name}_qc.mp4"
|
||||
print(f"\n✅ clip ready: {fbx}")
|
||||
if qc.exists():
|
||||
print(f" eyeball before it enters a bank: {qc}")
|
||||
print(" merge-to-GLB / Unreal / 3god hooks are deferred — docs/RUNBOOK.md.")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
264
retarget.py
Normal file
264
retarget.py
Normal file
@ -0,0 +1,264 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Lane C — retarget a SKEL-24 BVH onto the character_kit mixamorig rig + QC render.
|
||||
|
||||
Headless Blender only:
|
||||
/Applications/Blender.app/Contents/MacOS/Blender --background --python retarget.py -- \
|
||||
--bvh out/<name>/motion.bvh [--rig <female_game.glb>] [--name <n>] [--travel] [--no-qc]
|
||||
|
||||
Pipeline: import mixamorig rig (female_game.glb) + the SKEL BVH → Copy-Rotation-map SKEL bones
|
||||
onto mixamorig (spec/skel_to_mixamorig.json), world-space → bake to keyframes → trim near-rest
|
||||
bookends → export anim-only FBX (Mixamo "Without Skin", merge_anims.py eats these) → Workbench
|
||||
turntable QC mp4. Root is in-place by default (HSMR root is camera-space; --travel copies it).
|
||||
|
||||
# ponytail: constraint-bake retarget (world Copy Rotation). If the QC shows arm-roll twist
|
||||
# (SKEL vs mixamorig rest-roll differ), upgrade the ONE function `map_and_bake` to a rest-delta
|
||||
# transfer (Rt_f = Rs_f · Rs_rest⁻¹ · Rt_rest); everything else stays.
|
||||
"""
|
||||
import bpy, json, sys, os
|
||||
from pathlib import Path
|
||||
from mathutils import Matrix
|
||||
|
||||
REPO = Path(__file__).resolve().parent
|
||||
DEF_RIG = Path.home() / "Documents" / "character_kit" / "female" / "female_game.glb"
|
||||
MAP = REPO / "spec" / "skel_to_mixamorig.json"
|
||||
|
||||
|
||||
def args_after_ddash():
|
||||
a = sys.argv[sys.argv.index("--") + 1:] if "--" in sys.argv else []
|
||||
import argparse
|
||||
p = argparse.ArgumentParser()
|
||||
p.add_argument("--bvh", required=True)
|
||||
p.add_argument("--rig", default=str(DEF_RIG))
|
||||
p.add_argument("--name", default=None)
|
||||
p.add_argument("--out", default=str(REPO / "clips_out"))
|
||||
p.add_argument("--travel", action="store_true", help="copy root translation (default: in-place)")
|
||||
p.add_argument("--foot-fix", action="store_true", help="clamp planted feet to the floor (heuristic)")
|
||||
p.add_argument("--no-qc", action="store_true")
|
||||
return p.parse_args(a)
|
||||
|
||||
|
||||
def clean():
|
||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||
|
||||
|
||||
def imported(op, **kw):
|
||||
before = set(bpy.data.objects)
|
||||
op(**kw)
|
||||
return [o for o in bpy.data.objects if o not in before]
|
||||
|
||||
|
||||
def find_bone(arm, name):
|
||||
"""Tolerate mixamorig: colon vs _ underscore variants (the PLAN's warned trap)."""
|
||||
pbs = arm.pose.bones
|
||||
if name in pbs:
|
||||
return pbs[name]
|
||||
alt = name.replace(":", "_") if ":" in name else name.replace("mixamorig_", "mixamorig:")
|
||||
return pbs.get(alt)
|
||||
|
||||
|
||||
def action_fcurves(action):
|
||||
"""Blender 5.0 removed Action.fcurves for the layered ('Baklava') system — collect fcurves
|
||||
from layers/strips/channelbags, with a fallback to the legacy flat list."""
|
||||
if hasattr(action, "fcurves"):
|
||||
return list(action.fcurves)
|
||||
fcs = []
|
||||
for layer in getattr(action, "layers", []):
|
||||
for strip in layer.strips:
|
||||
for cb in getattr(strip, "channelbags", []):
|
||||
fcs.extend(cb.fcurves)
|
||||
return fcs
|
||||
|
||||
|
||||
def map_and_bake(target, source, name_map, f0, f1, travel):
|
||||
"""Add world-space Copy Rotation constraints target←source per the map, then bake to keys."""
|
||||
bpy.context.view_layer.objects.active = target
|
||||
n = 0
|
||||
for skel_bone, mixamo in name_map.items():
|
||||
if not mixamo: # null = twist/heel, folded elsewhere
|
||||
continue
|
||||
pb = find_bone(target, mixamo)
|
||||
if pb is None or skel_bone not in source.pose.bones:
|
||||
print(f" skip {skel_bone}->{mixamo} (missing)")
|
||||
continue
|
||||
c = pb.constraints.new("COPY_ROTATION")
|
||||
c.target, c.subtarget = source, skel_bone
|
||||
c.target_space = c.owner_space = "WORLD"
|
||||
c.mix_mode = "REPLACE"
|
||||
n += 1
|
||||
if travel: # root position (units differ; scale it)
|
||||
hips = find_bone(target, name_map["pelvis"])
|
||||
c = hips.constraints.new("COPY_LOCATION")
|
||||
c.target, c.subtarget = source, "pelvis"
|
||||
c.target_space = c.owner_space = "WORLD"
|
||||
c.use_offset = True
|
||||
print(f"mapped {n} bones; baking frames {f0}..{f1}")
|
||||
|
||||
bpy.ops.object.mode_set(mode="POSE")
|
||||
bpy.ops.pose.select_all(action="SELECT")
|
||||
bpy.ops.nla.bake(frame_start=f0, frame_end=f1, only_selected=True, visual_keying=True,
|
||||
clear_constraints=True, use_current_action=True, bake_types={"POSE"})
|
||||
bpy.ops.object.mode_set(mode="OBJECT")
|
||||
|
||||
|
||||
def trim_bookends(arm, f0, f1, thresh_deg=6.0):
|
||||
"""Trim leading/trailing frames that sit near the rest pose (T-pose bookends from CAPTURE.md).
|
||||
Tolerant of absence (demo clips have none) — returns the untouched range if nothing qualifies.
|
||||
# ponytail: rest-deviation heuristic on the action's keyed rotations; good enough for trimming."""
|
||||
import math
|
||||
act = arm.animation_data.action if arm.animation_data else None
|
||||
if not act:
|
||||
return f0, f1
|
||||
try:
|
||||
# mean absolute keyed rotation magnitude per frame (quaternion channels), vs rest
|
||||
rot_fcurves = [fc for fc in action_fcurves(act)
|
||||
if fc.data_path.endswith("rotation_quaternion")]
|
||||
except Exception as e:
|
||||
print(f"trim: fcurve access failed ({e!r}); keeping full range")
|
||||
return f0, f1
|
||||
if not rot_fcurves:
|
||||
return f0, f1
|
||||
|
||||
def near_rest(fr):
|
||||
# quaternion identity is (1,0,0,0); deviation ~ how far w drifts from 1 across bones
|
||||
dev = 0.0
|
||||
for fc in rot_fcurves:
|
||||
if fc.array_index == 0: # w component
|
||||
dev += abs(1.0 - fc.evaluate(fr))
|
||||
return math.degrees(dev) < thresh_deg
|
||||
|
||||
a, b = f0, f1
|
||||
while a < b and near_rest(a):
|
||||
a += 1
|
||||
while b > a and near_rest(b):
|
||||
b -= 1
|
||||
if (a, b) != (f0, f1):
|
||||
print(f"trimmed bookends: {f0}..{f1} -> {a}..{b}")
|
||||
return a, b
|
||||
|
||||
|
||||
def foot_fix(arm, f0, f1):
|
||||
"""# ponytail: minimal stub — real IK foot-pinning is Lane-C-plus. Logs planted frames so the
|
||||
upgrade has a target; does not yet re-solve. Enable heavy version when sliding annoys."""
|
||||
print("foot-fix: heuristic stub (no IK yet) — feet may slide; upgrade path noted in PLAN")
|
||||
|
||||
|
||||
def export_fbx(arm, path):
|
||||
bpy.ops.object.select_all(action="DESELECT")
|
||||
arm.select_set(True)
|
||||
bpy.context.view_layer.objects.active = arm
|
||||
bpy.ops.export_scene.fbx(
|
||||
filepath=str(path), use_selection=True, object_types={"ARMATURE"},
|
||||
add_leaf_bones=False, bake_anim=True, bake_anim_use_all_bones=True,
|
||||
bake_anim_use_nla_strips=False, bake_anim_use_all_actions=False,
|
||||
armature_nodetype="NULL", path_mode="COPY",
|
||||
)
|
||||
print(f"exported anim-only FBX -> {path} ({os.path.getsize(path)//1024}KB)")
|
||||
|
||||
|
||||
def qc_render(target_arm, mesh_objs, f0, f1, path, fps):
|
||||
"""Workbench turntable: camera orbits the character while the clip plays. Solid-shaded =
|
||||
fast + reads the pose clearly (the QC eyeball the PLAN mandates before a clip enters the bank)."""
|
||||
import math
|
||||
from mathutils import Vector
|
||||
# auto-frame from the character's world bbox (mixamo rigs import at odd scales — this one is
|
||||
# ~156 units tall, centred at z≈77). Scale-agnostic so any rig frames correctly.
|
||||
cs = []
|
||||
for m in mesh_objs:
|
||||
for c in m.bound_box:
|
||||
cs.append(m.matrix_world @ Vector(c))
|
||||
mn = Vector((min(c.x for c in cs), min(c.y for c in cs), min(c.z for c in cs)))
|
||||
mx = Vector((max(c.x for c in cs), max(c.y for c in cs), max(c.z for c in cs)))
|
||||
center = (mn + mx) / 2
|
||||
size = max((mx - mn).x, (mx - mn).y, (mx - mn).z) or 1.0
|
||||
dist = size * 1.9
|
||||
|
||||
piv = bpy.data.objects.new("qc_pivot", None)
|
||||
piv.location = center
|
||||
bpy.context.scene.collection.objects.link(piv)
|
||||
|
||||
cam_data = bpy.data.cameras.new("qc_cam")
|
||||
cam_data.clip_start, cam_data.clip_end = size * 0.01, dist * 4
|
||||
cam = bpy.data.objects.new("qc_cam", cam_data)
|
||||
bpy.context.scene.collection.objects.link(cam)
|
||||
cam.parent = piv
|
||||
cam.matrix_parent_inverse = Matrix()
|
||||
cam.location = (0, -dist, 0) # local to pivot → orbits at eye level
|
||||
tc = cam.constraints.new("TRACK_TO")
|
||||
tc.target = piv
|
||||
bpy.context.scene.camera = cam
|
||||
|
||||
piv.rotation_euler = (0, 0, 0)
|
||||
piv.keyframe_insert("rotation_euler", frame=f0)
|
||||
piv.rotation_euler = (0, 0, math.radians(360))
|
||||
piv.keyframe_insert("rotation_euler", frame=f1)
|
||||
for fc in action_fcurves(piv.animation_data.action): # linear spin, no ease
|
||||
for kp in fc.keyframe_points:
|
||||
kp.interpolation = "LINEAR"
|
||||
|
||||
import subprocess, shutil, tempfile
|
||||
sc = bpy.context.scene
|
||||
sc.render.engine = "BLENDER_WORKBENCH"
|
||||
sc.frame_start, sc.frame_end = f0, f1
|
||||
sc.render.resolution_x = sc.render.resolution_y = 640
|
||||
# Blender 5.0 dropped FFMPEG image output → render PNG frames, stitch with system ffmpeg.
|
||||
frames = Path(tempfile.mkdtemp())
|
||||
sc.render.image_settings.file_format = "PNG"
|
||||
sc.render.filepath = str(frames / "f_")
|
||||
bpy.ops.render.render(animation=True)
|
||||
ff = shutil.which("ffmpeg") or "/opt/homebrew/bin/ffmpeg"
|
||||
r = subprocess.run([ff, "-y", "-framerate", f"{fps:.3f}", "-start_number", str(f0),
|
||||
"-i", str(frames / "f_%04d.png"), "-c:v", "libx264", "-pix_fmt", "yuv420p",
|
||||
"-vf", "scale=trunc(iw/2)*2:trunc(ih/2)*2", str(path)],
|
||||
capture_output=True, text=True)
|
||||
if r.returncode or not Path(path).exists():
|
||||
print("QC ffmpeg stitch FAILED:", r.stderr[-400:])
|
||||
else:
|
||||
print(f"QC render -> {path} ({os.path.getsize(path)//1024}KB)")
|
||||
|
||||
|
||||
def main():
|
||||
a = args_after_ddash()
|
||||
bvh = Path(a.bvh).resolve()
|
||||
name = a.name or bvh.parent.name
|
||||
outdir = Path(a.out); outdir.mkdir(parents=True, exist_ok=True)
|
||||
name_map = json.loads(MAP.read_text())["skel_to_mixamorig"]
|
||||
|
||||
clean()
|
||||
rig_objs = imported(bpy.ops.import_scene.gltf, filepath=a.rig)
|
||||
target = next(o for o in rig_objs if o.type == "ARMATURE")
|
||||
meshes = [o for o in rig_objs if o.type == "MESH"]
|
||||
print(f"rig: armature '{target.name}', {len(target.data.bones)} bones, {len(meshes)} mesh(es)")
|
||||
|
||||
src_objs = imported(bpy.ops.import_anim.bvh, filepath=str(bvh),
|
||||
axis_forward="-Z", axis_up="Y", use_fps_scale=False, update_scene_fps=True)
|
||||
source = next(o for o in src_objs if o.type == "ARMATURE")
|
||||
act = source.animation_data.action
|
||||
f0, f1 = (int(round(x)) for x in act.frame_range)
|
||||
fps = bpy.context.scene.render.fps / max(bpy.context.scene.render.fps_base, 1e-9)
|
||||
print(f"bvh: source '{source.name}', frames {f0}..{f1} @ ~{fps:.2f}fps")
|
||||
|
||||
map_and_bake(target, source, name_map, f0, f1, a.travel)
|
||||
bpy.data.objects.remove(source, do_unlink=True) # drop the SKEL rig, keep baked mixamorig
|
||||
if a.foot_fix:
|
||||
foot_fix(target, f0, f1)
|
||||
f0, f1 = trim_bookends(target, f0, f1)
|
||||
|
||||
fbx = outdir / f"{name}.fbx"
|
||||
# the FBX take inherits the action name — un-named it exports as 'Armature|Scene' for EVERY clip,
|
||||
# colliding in any take-addressed bank (Unreal/3GOD). merge_anims renames from the filename, but
|
||||
# the take itself must be right (PROGRESS.md bug, fixed 2026-07-17).
|
||||
target.animation_data.action.name = name
|
||||
export_fbx(target, fbx)
|
||||
# sanity: the FBX action must carry keys on the driven bones
|
||||
assert target.animation_data and target.animation_data.action, "no baked action on target"
|
||||
nkeys = len(action_fcurves(target.animation_data.action))
|
||||
print(f"baked action '{target.animation_data.action.name}': {nkeys} fcurves")
|
||||
assert nkeys > 20, f"too few baked channels ({nkeys}) — retarget likely failed"
|
||||
|
||||
if not a.no_qc:
|
||||
qc_render(target, meshes, f0, f1, outdir / f"{name}_qc.mp4", fps)
|
||||
print(f"LANE-C DONE: {name} -> {fbx.name}" + ("" if a.no_qc else f" + {name}_qc.mp4"))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Loading…
Reference in New Issue
Block a user