5c2d7c6ea3
Fills events_ai.py against the frozen M7 contract:
- detect_candidates: RMS x spectral-flux, local-max over a 10s neighborhood,
>=P90, with an AC-RMS (std) silence/DC floor so flat audio yields no events.
Matches the fixture ground-truth pulses [3,10,17] within ~0.02s (<0.5s).
- GeminiClassifier (native video), ClaudeClassifier (claude-opus-4-8,
messages.parse, adaptive thinking, no temperature), LocalClassifier
(OpenAI-compatible, 6 frames, 1 retry). SDKs imported lazily.
- get_classifier: FESTIVAL4D_CLASSIFIER selection; unconfigured -> None
(candidates-only, logged).
- prepare_inputs: ffmpeg 3s clip (<=720p, audio kept) + 6 JPEGs (<=768px).
- run_events: reference-audio detect -> classify (insert-once in final state,
since db.py has no update-event helper); per-candidate exception isolation;
whole-track replaces machine events (keeps user edits), windowed is additive.
test_events_ai.py: candidate-vs-ground-truth (no ffmpeg/API), stub-classifier
orchestration, exception isolation, degradation, provider selection, silence.
Also updates the now-obsolete api stub test (test_detect_events_endpoint) to the
real M3 {result, events} shape; frozen api.py unchanged (see CR-1). pytest: 90 passed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|---|---|---|
| backend | ||
| frontend | ||
| plan | ||
| .gitignore | ||
| OPUS_BUILD_INSTRUCTIONS.md | ||
| pyproject.toml | ||
| README.md | ||
| uv.lock | ||
Festival 4D
Turn multiple fan-shot smartphone videos of the same concert into a synchronized, explorable 4D experience: time-aligned multi-video playback, 3D scene reconstruction with camera poses, a free-roam "god's eye" viewer, AR-style overlays projected onto each video, and AI-tagged moments on a shared timeline.
Status: foundation phase. The scaffold, database, frozen contracts (API, pose math, DB schema), and a synthetic fixture generator are in place. Feature lanes (media sync, reconstruction, viewer, AI events) build on top. The full README with the real-footage workflow is written in the integration phase (M9). See
OPUS_BUILD_INSTRUCTIONS.mdfor the canonical spec andplan/for the execution plan.
Prerequisites
- Python 3.11+ and
uv(or venv + pip) - ffmpeg / ffprobe on your
PATH(required) - COLMAP (optional — reconstruction degrades gracefully without it)
- A classifier API key (optional —
GEMINI_API_KEYby default; seeevents_ai.py) - Node 18+ for the frontend
Quickstart (synthetic demo — no footage needed)
# 1. backend env
uv venv --python 3.12
uv pip install -e ".[dev]"
# 2. generate the synthetic fixture project (fake videos + poses + point cloud + events)
uv run python -m festival4d synthetic
# 3. serve the API (http://127.0.0.1:8000)
uv run python -m festival4d serve
# 4. in another terminal, the frontend
cd frontend
npm install
npm run dev # http://localhost:5173
Backend CLI
python -m festival4d synthetic # generate the synthetic fixture (M0)
python -m festival4d ingest # probe videos + extract audio (lane A / M1)
python -m festival4d sync # GCC-PHAT audio alignment (lane A / M1)
python -m festival4d reconstruct # COLMAP SfM + pose export (lane B / M2)
python -m festival4d events # audio candidates + AI classify (lane D / M7)
python -m festival4d serve # FastAPI app (M3)
Tests
uv run pytest