From 2dbd99579baf0b93b0c9d3ad8a4b4f2dd284d9f6 Mon Sep 17 00:00:00 2001 From: type-two Date: Fri, 17 Jul 2026 08:48:34 +1000 Subject: [PATCH] Review: session-5 orders from Fable review of lanes 0-B Co-Authored-By: Claude Fable 5 --- PROGRESS.md | 59 +++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 50 insertions(+), 9 deletions(-) diff --git a/PROGRESS.md b/PROGRESS.md index 9cf4cb5..e1d1450 100644 --- a/PROGRESS.md +++ b/PROGRESS.md @@ -8,26 +8,67 @@ One lane per session. Tick on completion. See PLAN.md for acceptance checks. (probe → 30fps normalize → rename → rsync to m3ultra queue → LEDGER.tsv). — 2026-07-17 - [ ] **Lane B** — pose engine. **Decision made 2026-07-17 (session 3): Path B** — engine = **HSMR → SKEL**, run local, ship rig-motion only, John accepts NC-licence risk - (`docs/DECISIONS.md` Decision 1 RESOLVED). Env is 90% up in `.engine/HSMR` (uv venv, - torch 2.13 **MPS verified green**, ungated weights pulled). **Still open before B ships:** - (1) John downloads gated **SKEL v1.1.1** → `.engine/HSMR/data_inputs/body_models/skel/`; - (2) fix the person-detector (detectron2 + chumpy failed to build — see gotchas); - (3) real forward-pass smoke test on an HSMR demo image; then B2 (service-ize → BVH) + B3 - (jitter). Lanes C and D remain blocked on B. + (`docs/DECISIONS.md` Decision 1 RESOLVED). Env up in `.engine/HSMR` (uv venv, torch 2.13 + **MPS green**, ungated weights pulled). **Detector RESOLVED (session 4):** torchvision + Faster R-CNN drop-in replaces the unbuildable detectron2; `test_detector.py` is green today + (7/7 demo imgs → 15 patches, **no SKEL needed**). `smoke_test.py` now runs real detection + + patch-crop and reaches the **exact SKEL gate**; chumpy cleared as a non-blocker. **Only open + before B ships:** (1) John downloads gated **SKEL v1.1.1** so + `.engine/HSMR/data_inputs/body_models/skel/skel_male.pkl` (+ `skel_female.pkl`) exist; + (2) run the forward-pass smoke test (MPS / timing <2× / NaN check); then B2 (service-ize → + BVH) + B3 (jitter). Lanes C and D remain blocked on B. - [ ] **Lane C** — retarget + cleanup (Blender). bvh → mixamorig FBX + QC turntable. - [ ] **Lane D** — clip factory. `mocap` CLI end-to-end; games + Unreal integration. - [ ] Lane E — ML upgrades (post-v1, gated). - [ ] Lane F — UX (optional). +## SESSION 5 ORDERS — from Fable's review (2026-07-17, supersede older notes where they conflict) + +Review verdict: Lanes 0/A solid; session 2's reversal of the B1 animal-model blunder was the +right call, well executed; ARM engineering verified independently (test_detector.py re-run +green 7/7 by reviewer). Proceed as follows, in order: + +1. **Durable-ize the gitignored engine fixes (do first, ~30 min).** The torchvision detector, + `_headless.py`, `test_detector.py`, `smoke_test.py` exist ONLY on ultra's disk inside the + gitignored `.engine/`. Move canonical copies into git at `engine_patches/` and make + `setup_hsmr.sh` apply them after clone. Acceptance: fresh clone in a temp dir + script → + `test_detector.py` green with no hand edits. +2. **Draft the Path-A email (parallel risk-retirement, ~15 min).** Write the SKEL indie + commercial-license request to ps-license@tue.mpg.de into `docs/DECISIONS.md` (project + description, solo Australian dev, indie revenue, SKEL-in-pipeline-not-redistributed usage). + John sends it himself. Path B continues meanwhile; a granted license converts B→A with zero + rework. Do not wait on the reply. +3. **Reconcile the split-brain queue (~30 min).** Lane A's ingest.py rsyncs to m3ultra's queue, + but the engine (and MPS verification) is on ultra — nothing consumes that queue. For v1: + default ingest to LOCAL `queue/` on ultra (`--remote-queue` flag preserves the m3ultra path), + note the deviation from PLAN Ground-rule 1 in PLAN.md with a `# ponytail: v1 inference on + ultra; migrate to m3ultra when a real throughput need appears` marker. Update CAPTURE.md. +4. **Treaty amendment (small, load-bearing).** `motion.json` sidecar gains REQUIRED field + `root_space: "camera" | "world"`. HSMR emits `"camera"`. Lane C must read it and, for + locomotion clips, derive world root from foot contacts (already in notes). v1 quality bar: + in-place clips are first-class; traveling clips best-effort until a grounding pass exists. +5. **If (and only if) SKEL v1.1.1 files are in place** (`data_inputs/body_models/skel/ + skel_male.pkl`): run smoke_test.py — MPS forward pass, runtime <2× realtime on demo video, + NaN check — then proceed straight into B2 (params.npz → `spec/skel_to_mixamorig.json` map → + BVH@30fps + sidecar per treaty) and B3 (One-Euro/Sav-Gol jitter pass, root smoothed + separately). If SKEL absent: finish items 1–4, verify Decision-3 footage status, stop clean. + +**John's homework (all three block different things):** +- SKEL v1.1.1 download (the ONLY hard blocker; instructions in `docs/B_ENGINE_SETUP.md`). +- Film walk / dance / pick-up-object per `docs/CAPTURE.md`, drop in `capture/inbox/` (blocks + the honest quality verdict + all of B's acceptance). +- Reboot m3ultra when convenient (applies staged 26.5.2 security update; housekeeping). + ## Notes for the next lane - **Lane D can't be started until B + C ship + test footage exists** — it's the CLI that wraps ingest → pose_engine.py → retarget.py, and the latter two don't exist yet. Session 3 (the "Lane D" session) correctly built nothing; it locked Decision 1 and de-risked MPS instead. - **SKEL version trap:** HSMR needs **v1.1.1** exactly (`changelog_v1.1.1.txt`); older = silent incompat. Source: `.engine/HSMR/docs/SETUP.md` (register skel.is.tue.mpg.de → download.php). -- **Engine wiring + ARM bypasses live in `docs/B_ENGINE_SETUP.md`** — `smoke_test.py` already - stubs the un-buildable detectron2 (full-frame bbox) and headless pyrender (QC → Blender), so - the detector swap + real QC are B2 polish, **not** blockers for a centered demo clip. +- **Engine wiring + ARM bypasses live in `docs/B_ENGINE_SETUP.md`** — all three ARM landmines + are now cleared (session 4): detectron2 → **torchvision drop-in** (real detection, not a + full-frame stub), the pyrender/EGL fake is centralized in `.engine/HSMR/_headless.py`, and + chumpy is confirmed a non-blocker. `test_detector.py` proves the detector path green, SKEL-free. - **World-grounded gap (important):** HSMR is per-frame → root is camera-space `cam_t`, NOT the treaty's world-grounded trajectory. Fine for in-place DJ moves; walking/locomotion needs Lane C to derive root from foot-contact (or a grounding pass). Carry into Lane C design.