mocapgod/docs/C_RETARGET.md
type-two c918ac3dcd lane C+D: retarget + mocap CLI + docs; fix: FBX take carries the clip name (was Armature|Scene for every clip — bank collisions in take-addressed consumers)
First generated-footage clip proven through the full chain: Wan2.1 T2V (m3ultra, prompt-engineered to CAPTURE.md rules) -> ./mocap -> wan_walk_t1.fbx -> merge_anims bank (49 tracks). PROGRESS.md bug closed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 14:06:38 +10:00

3.2 KiB

Lane C — retarget + cleanup (retarget.py)

DONE (machinery) 2026-07-17 (session 6). retarget.py takes a SKEL-24 motion.bvh (from pose_engine.py) onto the character_kit mixamorig rig → baked anim-only FBX + QC turntable mp4. Verified on the gymnasts demo: recognizable human motion (not soup), and the FBX merges onto female_game.glb and drives it. Final quality sign-off awaits John's own walk/DJ footage.

Run (headless Blender only)

/Applications/Blender.app/Contents/MacOS/Blender --background --python retarget.py -- \
    --bvh out/<name>/motion.bvh [--rig <glb>] [--name <n>] [--travel] [--foot-fix] [--no-qc]

Defaults: rig = character_kit/female/female_game.glb, out = clips_out/, root in-place. Outputs: clips_out/<name>.fbx (anim-only, Mixamo "Without Skin") + clips_out/<name>_qc.mp4. Proven chain A→C: pose_engine.py <clip>out/<name>/motion.bvhretarget.py --bvh ….

How it works

  • Import mixamorig rig (glb) + SKEL BVH (axis_forward=-Z, axis_up=Y; both land Z-up in Blender).
  • Per spec/skel_to_mixamorig.json (20 bones; 4 null twist/heel), add world-space Copy Rotation (target mixamorig ← SKEL bone), then nla.bake(visual_keying, clear_constraints) → a clean mixamorig action. find_bone tolerates the mixamorig: colon vs mixamorig_ underscore trap.
  • Root in-place by default (HSMR root is camera-space, not world-grounded); --travel copies it.
  • Export anim-only FBX (object_types={ARMATURE}, bake_anim). QC = Workbench turntable, PNG frames stitched to mp4 by system ffmpeg, camera auto-framed from the world bbox.

Blender 5.0 gotchas hit + resolved (carry into Lane D)

  • Baklava actions: Action.fcurves is gone → action_fcurves() walks layers/strips/channelbags.
  • Merge needs slot binding (load-bearing): assigning an action is NOT enough — you must set animation_data.action_slot = act.slots[0] or nothing drives. Verified: with the slot bound, gymnasts.fbx drives female_game.glb (RightHand +121u), exports a playable 7.3 MB GLB. character_kit/scripts/merge_anims.py FIXED 2026-07-17 — binds strip.action_slot on each NLA strip (hasattr-guarded for 4.x); verified Lifting_Object drives female_game.glb (LeftHand delta 6.7u frame 1→138).
  • No FFMPEG image format in Blender 5.0 → render PNG frames + system ffmpeg stitch.
  • Mixamo scale: female_game.glb imports at armature-scale 0.01 but ≈156 units tall, centred z≈77. QC auto-frames from the bbox; keep retarget rotation-only — never scale the rig (scaling a skinned ancestor tears the mesh when it animates; the merge_anims.py units warning).

Deferred (honest — not gold-plated)

  • Foot-contact fix is a stub (--foot-fix logs planted frames only, no IK → feet may slide). Deferred until a real walk clip exists to tune against (gymnasts are airborne). # ponytail in code.
  • --travel (root translation + unit scaling) is wired but unverified; in-place is v1-first-class.
  • Arms/hands quality: gymnasts looked clean (no catastrophic roll). If DJ hand work twists on John's footage, upgrade the single map_and_bake to a rest-delta transfer (Rt=Rs·Rs_rest⁻¹·Rt_rest) — noted in code; nothing else changes.