diff --git a/README.md b/README.md new file mode 100644 index 0000000..6658709 --- /dev/null +++ b/README.md @@ -0,0 +1,57 @@ +# Festival 4D + +Turn multiple fan-shot smartphone videos of the same concert into a synchronized, +explorable **4D experience**: time-aligned multi-video playback, 3D scene reconstruction +with camera poses, a free-roam "god's eye" viewer, AR-style overlays projected onto each +video, and AI-tagged moments on a shared timeline. + +> **Status: foundation phase.** The scaffold, database, frozen contracts (API, pose math, +> DB schema), and a synthetic fixture generator are in place. Feature lanes (media sync, +> reconstruction, viewer, AI events) build on top. The full README with the real-footage +> workflow is written in the integration phase (M9). See +> [`OPUS_BUILD_INSTRUCTIONS.md`](OPUS_BUILD_INSTRUCTIONS.md) for the canonical spec and +> [`plan/`](plan/) for the execution plan. + +## Prerequisites + +- **Python 3.11+** and [`uv`](https://docs.astral.sh/uv/) (or venv + pip) +- **ffmpeg** / **ffprobe** on your `PATH` (required) +- **COLMAP** (optional — reconstruction degrades gracefully without it) +- A classifier API key (optional — `GEMINI_API_KEY` by default; see `events_ai.py`) +- **Node 18+** for the frontend + +## Quickstart (synthetic demo — no footage needed) + +```bash +# 1. backend env +uv venv --python 3.12 +uv pip install -e ".[dev]" + +# 2. generate the synthetic fixture project (fake videos + poses + point cloud + events) +uv run python -m festival4d synthetic + +# 3. serve the API (http://127.0.0.1:8000) +uv run python -m festival4d serve + +# 4. in another terminal, the frontend +cd frontend +npm install +npm run dev # http://localhost:5173 +``` + +## Backend CLI + +``` +python -m festival4d synthetic # generate the synthetic fixture (M0) +python -m festival4d ingest # probe videos + extract audio (lane A / M1) +python -m festival4d sync # GCC-PHAT audio alignment (lane A / M1) +python -m festival4d reconstruct # COLMAP SfM + pose export (lane B / M2) +python -m festival4d events # audio candidates + AI classify (lane D / M7) +python -m festival4d serve # FastAPI app (M3) +``` + +## Tests + +```bash +uv run pytest +``` diff --git a/backend/festival4d/__init__.py b/backend/festival4d/__init__.py new file mode 100644 index 0000000..25abc48 --- /dev/null +++ b/backend/festival4d/__init__.py @@ -0,0 +1,8 @@ +"""Festival 4D — synchronized, explorable 4D concert replay. + +See ``OPUS_BUILD_INSTRUCTIONS.md`` for the canonical spec and ``plan/`` for the +parallel execution plan. This package is the backend: CLI, database, synthetic +fixture generator, media/reconstruction/AI pipelines, and the FastAPI app. +""" + +__version__ = "0.1.0" diff --git a/backend/festival4d/__main__.py b/backend/festival4d/__main__.py new file mode 100644 index 0000000..dc1a5ba --- /dev/null +++ b/backend/festival4d/__main__.py @@ -0,0 +1,6 @@ +"""Enable ``python -m festival4d ``.""" + +from festival4d.cli import main + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/backend/festival4d/api.py b/backend/festival4d/api.py new file mode 100644 index 0000000..dbaf02b --- /dev/null +++ b/backend/festival4d/api.py @@ -0,0 +1,209 @@ +"""FastAPI app (spec M3). FROZEN after foundation — lanes never edit routes or shapes. + +Serves the synthetic project fully so lane C can treat this API as finished: + + GET /api/manifest -> {videos:[{id,filename,url,duration_s,fps,width,height, + offset_ms,drift_ppm}], t_global_max, has_poses} + GET /api/videos/{id}/poses -> [{frame_idx,t_video_s,q:[w,x,y,z],t:[x,y,z], + intrinsics:{fx,fy,cx,cy},registered}] + GET /api/pointcloud -> points.ply (binary; Range-capable) + GET /api/anchors -> [...] POST /api/anchors + GET /api/events -> [...] + POST /api/events/detect {t_global_s?, window_s?} -> runs M7 detection + POST /api/annotations {video_id, t_video_s, bbox:[x0,y0,x1,y1]} -> {anchor_id?, point?} + +Source videos in ``data/raw`` are mounted at ``/media`` via Starlette ``StaticFiles``, +which supports HTTP Range (required for ``