monster/festifun
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5c2d7c6ea3
festifun/plan/status/lane-D.md
m3ultra 5c2d7c6ea3 lane D (M7): audio moment detection + pluggable AI classification 
Fills events_ai.py against the frozen M7 contract:
- detect_candidates: RMS x spectral-flux, local-max over a 10s neighborhood,
  >=P90, with an AC-RMS (std) silence/DC floor so flat audio yields no events.
  Matches the fixture ground-truth pulses [3,10,17] within ~0.02s (<0.5s).
- GeminiClassifier (native video), ClaudeClassifier (claude-opus-4-8,
  messages.parse, adaptive thinking, no temperature), LocalClassifier
  (OpenAI-compatible, 6 frames, 1 retry). SDKs imported lazily.
- get_classifier: FESTIVAL4D_CLASSIFIER selection; unconfigured -> None
  (candidates-only, logged).
- prepare_inputs: ffmpeg 3s clip (<=720p, audio kept) + 6 JPEGs (<=768px).
- run_events: reference-audio detect -> classify (insert-once in final state,
  since db.py has no update-event helper); per-candidate exception isolation;
  whole-track replaces machine events (keeps user edits), windowed is additive.

test_events_ai.py: candidate-vs-ground-truth (no ffmpeg/API), stub-classifier
orchestration, exception isolation, degradation, provider selection, silence.
Also updates the now-obsolete api stub test (test_detect_events_endpoint) to the
real M3 {result, events} shape; frozen api.py unchanged (see CR-1). pytest: 90 passed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 08:34:56 +10:00

4.2 KiB
Raw Blame History

Status — lane-D

Round 1 — 2026-07-16 — STATUS: ready_to_merge

Adversarial review (self-run, 5 lenses → verify): 3 findings, all confirmed & fixed — (1) detect_candidates fabricated ~1 candidate/s on silent/DC/noise audio → added an AC-RMS (std) floor MIN_SIGNAL_STD; silence/DC/noise now return [], fixture unchanged [3,10,17]; (2) degradation test's raised tripwire was swallowed by except Exception → switched to a call-recorder (assert prepared == []); (3) windowed test ran on a fresh DB → now seeds an out-of-window ai event and asserts it survives. pytest 90 passed after fixes.

Directives acknowledged: round 0 of plan/DIRECTIVES.md (foundation merged; lanes may start; lane D is small + fully independent; merge when acceptance passes — no waiting on siblings).

Acceptance checklist (plan/lane-D-events.md + spec M7) — implementation complete, verified:

  • detect_candidates finds candidates within ±0.5 s of fixture ground-truth pulses [3.0, 10.0, 17.0]. Evidence: test_events_ai.py::test_detect_candidates_matches_ground_truth (recall+precision+count). RMS×spectral-flux, centered-10 s-window local-max, ≥ P90.
  • MomentClassification + MomentClassifier frozen contract preserved verbatim; import of events_ai pulls in ZERO heavy libs/SDKs (all lazy). Evidence: import-cost probe → [].
  • Three providers implemented against the installed SDKs (verified surfaces): Gemini (google-genai 2.11.0, Part.from_bytes(data,mime_type) + generate_content structured output), Claude (anthropic 0.116.0, messages.parse(output_format=…, thinking=adaptive, max_tokens) → parsed_output, NO temperature, model claude-opus-4-8), Local (openai 2.45.0, chat.completions.create json_object, 6 image_url parts, 1 retry).
  • Provider selection FESTIVAL4D_CLASSIFIER=gemini|claude|local (default gemini); unconfigured → candidates-only + log line. Evidence: test_get_classifier_* (5 tests).
  • Per-candidate exception isolation. Evidence: test_run_events_isolates_per_candidate_failure (2 ai + 1 fallback candidate; batch of 3 completes despite one raising).
  • run_events fills the frozen entrypoint (cli.py + api.py dispatch); writes via db helpers only (insert-once in final state — db.py has no update-event helper; end state matches spec).
  • pytest green with NO network: 89 passed across 5 consecutive full-suite runs.
  • Real-path end-to-end (ffmpeg-decoded AAC audio, not in-memory): isolated 20 s fixture → run_events() degraded → candidates at 3.008/10.016/17.013 s (≤0.016 s error), exactly 3, no spurious. Evidence: scratchpad/e2e_verify.py → RESULT PASS.

Done this round: implemented backend/festival4d/events_ai.py (candidate detection + 3 providers + get_classifier + prepare_inputs + run_events orchestration) and backend/tests/test_events_ai.py (15 tests). Detection algorithm locked via a scratch experiment before coding. Confirmed Claude call shape via claude-api skill and all three SDK surfaces via introspection. Currently running a 5-lens adversarial review workflow (spec-compliance / provider-SDK / detection-orchestration / contract-ownership / test-quality) before merge.

Coordinator note (out-of-strict-ownership edit): updated backend/tests/test_api.py test_detect_degrades_gracefully → test_detect_events_endpoint. That foundation test asserted the STUB behavior ("note" in data) of POST /api/events/detect; my landing replaces the stub with real detection returning {result, events}. The frozen api.py app is UNCHANGED — only the now-obsolete stub assertion was updated. Flagged here for transparency; see plan/CHANGE_REQUESTS.md CR-1.

Note on shared tree: this working directory holds all four lanes' uncommitted work concurrently (A/B/C/D). I will stage and commit ONLY my owned files. One transient test_audio_sync.py (lane A) failure was observed once mid-run (concurrent write) but the suite is deterministically green (89/89 ×5); not caused by lane D.

Blockers / questions for coordinator: none.

Next: address any confirmed review findings, then merge lane/d-events → main.