mocapgod

monster/mocapgod

Fork 0

Commit Graph

Author	SHA1	Message	Date
type-two	7d0a3b7f7c	Lane B / session-5 item 5: pose_engine.py — SKEL BVH + sidecar (B2) + jitter (B3) pose_engine.py: queue video -> HSMR/SKEL -> BVH@fps + sidecar JSON per the treaty. - detection: torchvision mobilenet (engine_patches), every-Nth frame + single best-box reuse (single-subject) -> ~1 pose/frame aligned timeline. mobilenet swap + detect-every address the CPU-detector wall-clock finding. - recovery: HSMR ViT-H on MPS -> SKEL params (poses q46, betas, cam_t). - SKEL forward -> per-joint global orientations; rigid-bone BVH via rest-aware FK (offset[j]=rest_J[j]-rest_J[parent]; W[j]=Rg[j]@rest_R[j]^T; local Q=W[parent]^T@W[j]). Verified: rest-aware FK reconstructs SKEL joints to ~4cm (residual = SKEL biomechanical coupled translations, which rigid BVH can't carry), in-BVH FK self-consistency 0.0mm. - camera->world axis fix: HSMR emits Y-down (OpenCV, verified head.y<pelvis.y); diag(1,-1,-1) -> Y-up, faces -Z (treaty). Blender import: 24 bones, upright, 1.59m, motion present. - B3 jitter: savgol on hemisphere-aligned quaternions (rotations) + root xyz separately. - sidecar: root_space=camera (per-frame HSMR), joint_order, root_trajectory, model, fps. - build models once + per-video try/except: queue of 6 drained unattended, 2 bad clips (no-person, garbage) failed gracefully, 4 produced valid BVH; heartbeat per clip. - runtime steady-state ~1.6x realtime on ultra (detect+recover), under the <2x bar. - --selfcheck (engine-free): BVH FK exact + Euler round-trip exact. spec/skel_to_mixamorig.json: SKEL-24 -> mixamorig name map for Lane C's retarget (20 mapped, 4 null twist/heel; validated against both skeletons). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-17 10:41:08 +10:00
type-two	f8037f7b85	Lane B / session-5 item 1: durable-ize the ARM engine fixes into git The torchvision detector drop-in, headless pyrender/OpenGL shims, and the detector+smoke tests existed only on ultra's disk inside gitignored .engine/. Captured byte-identical copies into engine_patches/ (mirrors the HSMR subtree) and moved setup_hsmr.sh to the repo root (tracked, path-relative) with a --patch step that overlays them onto the clone. Acceptance PASSED: fresh shallow clone of HSMR in a temp dir + './setup_hsmr.sh --patch' + validated venv -> test_detector.py green (7/7 demo imgs, 15 patches), no hand edits. A re-clone can no longer silently drop the ARM fixes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-17 09:34:39 +10:00

Author

SHA1

Message

Date

type-two

7d0a3b7f7c

Lane B / session-5 item 5: pose_engine.py — SKEL BVH + sidecar (B2) + jitter (B3)

pose_engine.py: queue video -> HSMR/SKEL -> BVH@fps + sidecar JSON per the treaty.
- detection: torchvision mobilenet (engine_patches), every-Nth frame + single best-box
  reuse (single-subject) -> ~1 pose/frame aligned timeline. mobilenet swap + detect-every
  address the CPU-detector wall-clock finding.
- recovery: HSMR ViT-H on MPS -> SKEL params (poses q46, betas, cam_t).
- SKEL forward -> per-joint global orientations; rigid-bone BVH via rest-aware FK
  (offset[j]=rest_J[j]-rest_J[parent]; W[j]=Rg[j]@rest_R[j]^T; local Q=W[parent]^T@W[j]).
  Verified: rest-aware FK reconstructs SKEL joints to ~4cm (residual = SKEL biomechanical
  coupled translations, which rigid BVH can't carry), in-BVH FK self-consistency 0.0mm.
- camera->world axis fix: HSMR emits Y-down (OpenCV, verified head.y<pelvis.y); diag(1,-1,-1)
  -> Y-up, faces -Z (treaty). Blender import: 24 bones, upright, 1.59m, motion present.
- B3 jitter: savgol on hemisphere-aligned quaternions (rotations) + root xyz separately.
- sidecar: root_space=camera (per-frame HSMR), joint_order, root_trajectory, model, fps.
- build models once + per-video try/except: queue of 6 drained unattended, 2 bad clips
  (no-person, garbage) failed gracefully, 4 produced valid BVH; heartbeat per clip.
- runtime steady-state ~1.6x realtime on ultra (detect+recover), under the <2x bar.
- --selfcheck (engine-free): BVH FK exact + Euler round-trip exact.

spec/skel_to_mixamorig.json: SKEL-24 -> mixamorig name map for Lane C's retarget
(20 mapped, 4 null twist/heel; validated against both skeletons).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-07-17 10:41:08 +10:00

type-two

f8037f7b85

Lane B / session-5 item 1: durable-ize the ARM engine fixes into git

The torchvision detector drop-in, headless pyrender/OpenGL shims, and the
detector+smoke tests existed only on ultra's disk inside gitignored .engine/.
Captured byte-identical copies into engine_patches/ (mirrors the HSMR subtree)
and moved setup_hsmr.sh to the repo root (tracked, path-relative) with a
--patch step that overlays them onto the clone.

Acceptance PASSED: fresh shallow clone of HSMR in a temp dir + './setup_hsmr.sh
--patch' + validated venv -> test_detector.py green (7/7 demo imgs, 15 patches),
no hand edits. A re-clone can no longer silently drop the ARM fixes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-07-17 09:34:39 +10:00

2 Commits