Commit Graph

12 Commits

Author SHA1 Message Date
m3ultra
2cad2f7b02 Merge branch 'lane/b-recon' 2026-07-16 09:01:59 +10:00
m3ultra
e28dc78e06 Merge branch 'lane/a-media' 2026-07-16 09:01:59 +10:00
m3ultra
bb542960c2 Lane B (M2 + M8 geometry): COLMAP reconstruction, geometry math, graceful degradation
Implements spec M2 (frame sampling, COLMAP orchestration, TXT-model parsing, scene
normalization, pose interpolation, PLY + DB export) and the M8 geometry functions.

geometry.py (M8): slerp_pose (shortest-arc, double-cover), ray_from_pixel (COLMAP
+y-down back-projection), triangulate_rays (closest-point + parallel guard),
nearest_point_on_ray (in-front radius cylinder). Frozen colmap_to_threejs untouched.

frames.py (M2): variance-of-Laplacian sharpness + windowed sharpest-frame sampling.

sfm.py (M2): hand-written COLMAP images/cameras/points3D parsers; normalize_scene
(centroid->0, camera sphere r->10, up->+Y) as one similarity transform over points+poses;
interpolate_poses (slerp+lerp, no extrapolation); build-aware COLMAP CLI orchestration
(3.x/4.x option detection, CPU SIFT, single-camera-per-folder, undistort->TXT, largest
component by images.bin header count); run_reconstruct with full graceful degradation
(COLMAP absent / <60% / <2 videos / no frames / missing files -> poses left untouched).

Tests: 61 pass (24 foundation + 37 lane-B) with independent oracles (scipy Slerp,
projection-inverse, closed-form geometry) and every degradation/DB-safety branch covered.
Validated end-to-end against real COLMAP 4.1.0 (6 components, largest selected, weak ->
graceful degradation, exit 0).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 08:40:46 +10:00
m3ultra
6ec55c70e8 lane D: status -> merged (M7 done, 5c2d7c6; review findings fixed)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 08:38:47 +10:00
m3ultra
5c2d7c6ea3 lane D (M7): audio moment detection + pluggable AI classification
Fills events_ai.py against the frozen M7 contract:
- detect_candidates: RMS x spectral-flux, local-max over a 10s neighborhood,
  >=P90, with an AC-RMS (std) silence/DC floor so flat audio yields no events.
  Matches the fixture ground-truth pulses [3,10,17] within ~0.02s (<0.5s).
- GeminiClassifier (native video), ClaudeClassifier (claude-opus-4-8,
  messages.parse, adaptive thinking, no temperature), LocalClassifier
  (OpenAI-compatible, 6 frames, 1 retry). SDKs imported lazily.
- get_classifier: FESTIVAL4D_CLASSIFIER selection; unconfigured -> None
  (candidates-only, logged).
- prepare_inputs: ffmpeg 3s clip (<=720p, audio kept) + 6 JPEGs (<=768px).
- run_events: reference-audio detect -> classify (insert-once in final state,
  since db.py has no update-event helper); per-candidate exception isolation;
  whole-track replaces machine events (keeps user edits), windowed is additive.

test_events_ai.py: candidate-vs-ground-truth (no ffmpeg/API), stub-classifier
orchestration, exception isolation, degradation, provider selection, silence.
Also updates the now-obsolete api stub test (test_detect_events_endpoint) to the
real M3 {result, events} shape; frozen api.py unchanged (see CR-1). pytest: 90 passed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 08:34:56 +10:00
m3ultra
88caecb660 lane A status: M1 complete, ready_to_merge (isolated worktree)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 01:15:23 +10:00
m3ultra
157c783e25 Lane A (M1): ingest + GCC-PHAT audio sync
Fills the foundation stubs in ingest.py and audio_sync.py.

- ingest: ffprobe probe + mono 16 kHz WAV extraction per video (ffmpeg);
  idempotent run_ingest reuses existing rows so synthetic -> ingest -> sync
  works without unique-constraint collisions.
- sync: PHAT-whitened GCC-PHAT pairwise offsets with sub-sample parabolic
  peak refinement, scored by peak-to-second-peak ratio; confidence-weighted
  global least-squares solve with cycle-consistency rejection (>50 ms);
  windowed drift estimation (ppm) with a 5 ppm deadband; disconnected
  sync-graph components -> offset_ms=None (spec pitfall #5).
- Persists offset_ms/drift_ppm/sync_confidence via db.update_video_sync and
  exports data/work/sync.json.

Verified on the synthetic fixture: synthetic -> ingest -> sync recovers the
ground-truth offsets (0 / +1370 / -842 ms) to <0.001 ms and drift 0 ppm,
well inside the spec M1 tolerances (+/-10 ms, +/-3 ppm). pytest: 43 passed
(24 foundation + 19 new in test_ingest.py and test_audio_sync.py).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 01:13:56 +10:00
m3ultra
4399bc90e2 lane C: session start — verified frozen contracts against live synthetic API
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 00:55:42 +10:00
m3ultra
5fa7301148 status: foundation merged to main (M0 + M3 + frozen contracts)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 00:46:41 +10:00
m3ultra
cf6a6fb2c8 Foundation (M0 + M3 + frozen contracts): scaffold, synthetic fixtures, full API
What now works:
- M0 scaffold: pyproject (all spec deps), uv/py3.12 env, `python -m festival4d`
  CLI registering synthetic|ingest|sync|reconstruct|events|serve. Vite hello page.
- Synthetic fixture (synthetic.py): 3 shifted-audio videos (offsets 0/+1370/-842 ms),
  camera-arc poses, stage point cloud -> points.ply, seeded events + anchors,
  ground_truth.json. `python -m festival4d synthetic` populates data/ + DB.
- DB schema exactly per spec §2 (db.py) + CRUD helpers all lanes use.
- M3 API (api.py) full against synthetic data: manifest/poses/pointcloud/anchors/
  events/detect/annotations; Range-capable video serving (206 verified); CORS for any
  localhost origin.
- Frozen geometry contract: geometry.colmap_to_threejs (M5 math) + unit test (3 known
  vectors, random round-trip, scipy oracle); mirrored frontend/src/lib/pose.js with
  identical POSE_TEST_VECTORS. Lane-B stubs: slerp_pose, ray_from_pixel, triangulate_rays,
  nearest_point_on_ray.
- Classifier contract (events_ai.py): MomentClassification model + MomentClassifier
  protocol + Gemini/Claude/Local provider stubs.
- Lane-owned modules stubbed with final signatures (ingest, audio_sync, frames, sfm,
  events_ai); cli/api catch NotImplementedError and degrade gracefully.
- plan/CHANGE_REQUESTS.md created; plan/status/foundation.md updated.

Acceptance: pytest 24 passed; serve endpoints verified via curl + browser (video seek,
manifest fetch cross-origin, pose.js self-test, 0 console errors).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 00:45:55 +10:00
m3ultra
47bec93bc5 Add coordination loop: run order (DIRECTIVES.md), per-agent status protocol
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 00:08:54 +10:00
m3ultra
e31e8ac559 Festival 4D build spec + parallel lane plan for Opus agents
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 00:06:54 +10:00