7d0a3b7f7c
pose_engine.py: queue video -> HSMR/SKEL -> BVH@fps + sidecar JSON per the treaty. - detection: torchvision mobilenet (engine_patches), every-Nth frame + single best-box reuse (single-subject) -> ~1 pose/frame aligned timeline. mobilenet swap + detect-every address the CPU-detector wall-clock finding. - recovery: HSMR ViT-H on MPS -> SKEL params (poses q46, betas, cam_t). - SKEL forward -> per-joint global orientations; rigid-bone BVH via rest-aware FK (offset[j]=rest_J[j]-rest_J[parent]; W[j]=Rg[j]@rest_R[j]^T; local Q=W[parent]^T@W[j]). Verified: rest-aware FK reconstructs SKEL joints to ~4cm (residual = SKEL biomechanical coupled translations, which rigid BVH can't carry), in-BVH FK self-consistency 0.0mm. - camera->world axis fix: HSMR emits Y-down (OpenCV, verified head.y<pelvis.y); diag(1,-1,-1) -> Y-up, faces -Z (treaty). Blender import: 24 bones, upright, 1.59m, motion present. - B3 jitter: savgol on hemisphere-aligned quaternions (rotations) + root xyz separately. - sidecar: root_space=camera (per-frame HSMR), joint_order, root_trajectory, model, fps. - build models once + per-video try/except: queue of 6 drained unattended, 2 bad clips (no-person, garbage) failed gracefully, 4 produced valid BVH; heartbeat per clip. - runtime steady-state ~1.6x realtime on ultra (detect+recover), under the <2x bar. - --selfcheck (engine-free): BVH FK exact + Euler round-trip exact. spec/skel_to_mixamorig.json: SKEL-24 -> mixamorig name map for Lane C's retarget (20 mapped, 4 null twist/heel; validated against both skeletons). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
39 lines
2.1 KiB
JSON
39 lines
2.1 KiB
JSON
{
|
|
"comment": "SKEL 24-joint -> mixamorig name map for Lane C's constraint retarget (pose_engine.py emits a SKEL-native BVH; Lane C maps it onto the character_kit mixamorig rig). null = a SKEL joint with no direct mixamorig target (twist/heel DoF folded elsewhere). mixamorig bones absent here (Spine1, Head, fingers, *_End) are not driven by SKEL — see notes.",
|
|
"source_skeleton": "SKEL-24 (skel/kin_skel.py order)",
|
|
"target_skeleton": "mixamorig (spec/mixamorig_bones.json, colon-prefixed, 65 joints)",
|
|
"skel_to_mixamorig": {
|
|
"pelvis": "mixamorig:Hips",
|
|
"lumbar_body": "mixamorig:Spine",
|
|
"thorax": "mixamorig:Spine2",
|
|
"head": "mixamorig:Neck",
|
|
"scapula_r": "mixamorig:RightShoulder",
|
|
"humerus_r": "mixamorig:RightArm",
|
|
"ulna_r": "mixamorig:RightForeArm",
|
|
"radius_r": null,
|
|
"hand_r": "mixamorig:RightHand",
|
|
"scapula_l": "mixamorig:LeftShoulder",
|
|
"humerus_l": "mixamorig:LeftArm",
|
|
"ulna_l": "mixamorig:LeftForeArm",
|
|
"radius_l": null,
|
|
"hand_l": "mixamorig:LeftHand",
|
|
"femur_r": "mixamorig:RightUpLeg",
|
|
"tibia_r": "mixamorig:RightLeg",
|
|
"talus_r": "mixamorig:RightFoot",
|
|
"calcn_r": null,
|
|
"toes_r": "mixamorig:RightToeBase",
|
|
"femur_l": "mixamorig:LeftUpLeg",
|
|
"tibia_l": "mixamorig:LeftLeg",
|
|
"talus_l": "mixamorig:LeftFoot",
|
|
"calcn_l": null,
|
|
"toes_l": "mixamorig:LeftToeBase"
|
|
},
|
|
"notes": {
|
|
"spine": "SKEL has 2 trunk joints (lumbar_body, thorax); mixamorig has 3 (Spine, Spine1, Spine2). lumbar_body->Spine, thorax->Spine2; Lane C interpolates Spine1 (or leaves identity).",
|
|
"neck_head": "SKEL 'head' is a single thorax-relative joint -> mapped to mixamorig:Neck. mixamorig:Head is left to Lane C (identity, or a small follow of Neck).",
|
|
"radius_calcn": "radius_* (forearm pronation/supination twist) and calcn_* (heel) have no direct mixamorig bone; their DoF fold into ForeArm/Foot in mixamorig. null = skip in the constraint map.",
|
|
"fingers": "SKEL has no fingers -> mixamorig hand/finger bones stay undriven; fingers arrive in Lane E1.",
|
|
"toe_end_headtop": "mixamorig *_End tips (HeadTop_End, LeftToe_End, RightToe_End) are leaf markers, not driven."
|
|
}
|
|
}
|