festifun/plan/DIRECTIVES.md

Coordinator Directives

Written only by the coordinator (the human's planning session). Agents: read the latest round at session start and after every milestone via git fetch origin && git show origin/main:plan/DIRECTIVES.md. The newest round addressed to your lane overrides older rounds and, where in conflict, your lane brief. Never edit this file.

---

Round 4 — 2026-07-17 — Fresh-eyes upgrades landed; PHASE 5 begins (M10–M17)

Coordinator work landed directly on main (self-reviewed + live-verified):

• 95b7178 — sync solver caches one rFFT per signal across the pairwise GCC-PHAT stage (1.8× on 6 cams × 10 min; equivalence locked by a new unequal-lengths test).
• c24af80 — audio is now the master clock (GET /api/audio/{id} AAC + WebAudio; t_global from AudioContext.currentTime; all <video> muted picture-only; graceful fallback) + spatial audio (🎧 per-camera HRTF panners at live poses, listener = viewer camera, gains 1/√N). Verified live: clock drives playback, mid-play source switching is continuous, panners sit at the correct camera positions, toggle on/off mid-play is clean.
• db42c42 — chore: python-multipart locked (declared-but-unlocked drift), launch configs.
• Suite is 121 passed; frontend build clean. The test floor is now 121.

Contract amendments (coordinator-ratified, additive only): new route GET /api/audio/{id}; transport.js is now the WebAudio clock owner (its header comment is the reference); the window.__f4d debug hook is available in production builds via localStorage f4dDebug=1. test_api.py's manifest key-set lock is unchanged.

Directives — Phase 5 (spec: plan/20-phase5.md; lane briefs: plan/lane-E/F/G-*.md):

1. Now: foundation2 only (one agent, branch foundation2, scope in 20-phase5.md). Same rule as phase 1: contracts correctness beats speed — three lanes will hard-depend on your route shapes, stubs, hooks, and UI wiring points. Maintain plan/status/foundation2.md.
2. After foundation2 merges — lanes in priority order (all three concurrently if capacity allows): E (director — longest, earliest contract validation), G (capsule — the shareable payoff), F (social UX — smallest, pure frontend).
3. Worktree rule is MANDATORY this time: every parallel agent creates its own git worktree before its first edit (Round 1 standing lesson; the phase-2 shared-tree collision cost us a stash cleanup). Single-agent phases (5a, 5c) may use the main working directory.
4. Merge policy unchanged: merge to main per-milestone where the lane brief allows it; never wait on another lane. Then integration2 (branch integration2), which owns the real-footage field test and the v0.2.0 tag proposal — failures go to plan/ISSUES.md, not into hasty fixes.
5. Standing protocol (status files, evidence discipline, blocker rule, DIRECTIVES re-read after every milestone) is unchanged from Round 0.

For the human: (a) the Round 3 focused-browser sync check is still open — now also put on headphones, enable 🎧 3D audio, and fly around; (b) real overlapping clips for the field test remain the gating item for integration2; (c) optional: a trained splat for a real project makes photo mode (M14) and the capsule (M17) dramatically more impressive — scripts/splat_via_modelbeast.sh.

Round 3 — 2026-07-16 — Integration review: v0.1.0 ACCEPTED; polish + field test next

Coordinator review of the integration work (independently verified): main == origin/main (d313c8d), tag v0.1.0 on e5d6b2c. Re-ran the suite: 101 passed; frontend build clean. The M8 acceptance test is faithful (forward-projects ground truth, no self-confirmation); resolve.py is clean orchestration over lane B's frozen primitives; CR-1/CR-2 correctly adjudicated and strictly additive; camPath's Catmull-Rom parameterization maps exactly to three.js segment lookup (checked). The live-browser M8/M9 verification methodology was sound given the hidden-tab rAF throttle. v0.1.0 accepted.

Ratification: Round 2 was written by the integration agent; DIRECTIVES.md is coordinator-only. Content verified accurate — ratified as written. Rule stands for future rounds.

Review findings (all polish; none block the tag):

1. Anchor stacking — the one real wart. Each annotation creates a NEW anchor, so the documented two-view "refine" flow leaves BOTH the view-1 fallback anchor and the view-2 triangulated anchor, identically labeled (reproduced: 2 anchors for 1 object). On real footage the fallback can be far off and there is no way to remove it — no DELETE /api/anchors/{id}, no UI delete.
2. docs/ideas.md (spec §5 future-extensions note) was never created.
3. Human focused-browser M4 sync eyeball still open (carried from Round 2).

Directives — Phase 4 (one agent, branch phase4/polish-field-test):

1. Polish pass (small, do first):
   • When an annotation resolves for an event that already has a resolved anchor from a prior annotation of the same event, update that anchor in place (supersede the fallback with the triangulation) instead of creating a sibling. Keep per-annotation resolved_anchor_id history consistent.
   • Add DELETE /api/anchors/{id} + a small delete affordance in the UI (e.g. in the annotation panel or anchor list). Additive API only; file a CR entry noting it.
   • Create docs/ideas.md (gaussian splatting, MP4 fly-through export, realtime ingest, multi-modal audio analysis — from the spec's future-extensions list).
   • Suite must stay ≥ 101; add tests for supersede + delete.
2. Field test protocol (the real milestone): when the user drops 2–4 real overlapping clips into data/raw/, run ingest → sync → reconstruct → events → serve; record in plan/status/phase4.md: per-pair sync confidence, offsets, COLMAP registration rate, which milestones survived contact with real footage. File failures as entries in a new plan/ISSUES.md rather than rushing fixes (per plan/10-integration.md item 4).
3. Maintain plan/status/phase4.md; same evidence discipline.

For the human (unchanged + one new): (a) open the app in a normal focused browser window once and confirm the sync panel stays green during continuous playback; (b) shoot or collect 2–4 overlapping clips of the same performance for the field test; (c) optionally set GEMINI_API_KEY done — see addendum.

Round 3 addendum (coordinator, same day) — real API keys wired + first field finding:

• A local .env now exists at the repo root (gitignored, chmod 600, never commit) with GEMINI_API_KEY (default provider) and OpenRouter creds mapped to the local provider (FESTIVAL4D_OPENAI_*, model google/gemini-2.5-flash). Load with set -a; . ./.env; set +a before python -m festival4d events. Phase-4 agent: the file is machine-local — if it's missing in your worktree, ask the coordinator, don't recreate keys.
• Field finding #1 (fixed by coordinator hotfix): Google retired gemini-2.5-flash for new API keys (404 "no longer available to new users"). The Gemini model is now env-overridable (FESTIVAL4D_GEMINI_MODEL) with default gemini-3.1-flash-lite — verified live: inline video + JSON-schema structured output OK; 3/3 synthetic candidates classified correctly (light_show/bass_drop, coherent descriptions). Suite still 101 passed. The degradation path also behaved exactly as designed during the failure — good lane D work.
• Phase-4 agent: consider a plan/ISSUES.md entry template seeded from this finding's shape (symptom → root cause → fix → evidence).

Round 2 — 2026-07-16 — Integration complete; v0.1.0 shipped

Integration merged to main (e5d6b2c) and tagged v0.1.0. M8 (annotation→3D correction + triangulation) and M9 (keyframed camera paths) are done; README rewritten for v0.1.0. Both change requests adjudicated and applied (CR-1, CR-2 — additive only; timebase/schema-columns/ pose/API shapes otherwise unchanged).

Verification (coordinator-run on merged tree): 101 backend tests pass (up from the 96 floor); two-view stage-corner recovery err 0.00000 (< 0.2); frontend build clean; live in-browser M8 triangulation (gap 0.21) and M9 path (camera lands exactly on keyframe at t); M4 sync ~16 ms via deterministic pump. Worktrees removed, debris stash dropped, frontend/dist gitignored.

One open item for the human (not code-blocking): the automated browser pane runs document.hidden=true, which throttles requestAnimationFrame, so continuous multi-cam playback sync could only be verified deterministically, not by eye. Open the app in a normal focused browser window once (python -m festival4d serve + cd frontend && npm run dev) and confirm the top-right sync panel stays green (<50 ms) during continuous playback.

Next: the pipeline is ready for real footage — drop 2–4 overlapping concert clips into data/raw/ and run ingest → sync → reconstruct → events → serve (see README).

Round 1 — 2026-07-16 — All lanes merged & verified; integration begins

Coordinator review result: foundation + all four lanes are merged to main (merge commits e28dc78 A, 2cad2f7 B, 9d17b47 C; D fast-forwarded earlier at 5c2d7c6). Independently verified on the merged tree: 96 backend tests pass, synthetic ingest → sync recovers ground-truth offsets exactly (+0 / +1370 / −842 ms, drift 0), events finds 3 candidates and degrades gracefully unconfigured, frontend production build clean. Excellent evidence discipline across all lanes — keep it.

Directives:

1. Integration phase starts now — one agent, branch integration from main, per plan/10-integration.md. Maintain plan/status/integration.md.
2. CR-1 (lane D, test_api.py stub assertion): APPROVED by coordinator. The edit was a direct consequence of M7 landing and touched no frozen contract file. Integration agent: mark the Decision line in plan/CHANGE_REQUESTS.md and confirm the new assertion matches the spec M3 POST /api/events/detect response shape ({result, events}).
3. Integration additions to the plan/10-integration.md scope (small, discovered in review):
   • The shared-worktree collision (see lane A/C status round 0) left a coordinator-stashed debris stash in the main working dir (git stash list — "stale lane A/B debris"). After confirming nothing of value remains, drop it, and remove the now-merged worktrees ../festifun-laneA and ../festifun-laneB (git worktree remove).
   • Lane C's env note: continuous-playback sync was verified deterministically because the automated browser tab was backgrounded. During your full-system pass, verify the M4 sync overlay (<50 ms) once in a real focused browser window.
   • Test count sanity: suite is 96 passed on merge day; it must never drop below that.
4. Worktree protocol lesson (standing): integration runs single-agent, so use the main working directory directly on branch integration. If parallel agents are ever launched again, each MUST create its own git worktree before its first edit — never share a working directory.

Round 0 — 2026-07-16 — Run order & protocol

Execution order:

1. Now: foundation only. Nothing else starts until foundation merges to main. Foundation agent: contracts correctness beats speed — lanes will hard-depend on your API shapes, schema, pose math, and synthetic fixtures.
2. After foundation merges — start lanes in this priority order (all four concurrently if capacity allows; if running fewer sessions, start in this order):
   1. Lane C (viewer) — longest lane and the earliest validation of the API contract; starts first.
   2. Lane A (media) — the core real-footage capability; small surface, should finish early.
   3. Lane D (events) — small and fully independent.
   4. Lane B (recon) — heaviest external dependency (COLMAP) and the most graceful-degradation slack; safe to start last.
3. Merge policy: merge to main as soon as your lane's acceptance passes — do not wait for other lanes. Expected finish order: A, D, then C, then B.
4. After all four merge: integration.

Standing protocol (all agents):

• Maintain plan/status/<your-lane>.md per plan/status/TEMPLATE.md — session start, every milestone, every blocker, before merge. Push status commits to your branch immediately; the coordinator reviews between rounds by reading these files.
• If blocked > 30 minutes on an ambiguity, write the blocker to your status file, push, and move to another task in your lane rather than guessing on a frozen contract.
• Evidence discipline: a checked acceptance box must name the test or the exact command + output that proves it.

(Future rounds will be appended above this line as the coordinator reviews status files.)