cf6a6fb2c8
What now works: - M0 scaffold: pyproject (all spec deps), uv/py3.12 env, `python -m festival4d` CLI registering synthetic|ingest|sync|reconstruct|events|serve. Vite hello page. - Synthetic fixture (synthetic.py): 3 shifted-audio videos (offsets 0/+1370/-842 ms), camera-arc poses, stage point cloud -> points.ply, seeded events + anchors, ground_truth.json. `python -m festival4d synthetic` populates data/ + DB. - DB schema exactly per spec §2 (db.py) + CRUD helpers all lanes use. - M3 API (api.py) full against synthetic data: manifest/poses/pointcloud/anchors/ events/detect/annotations; Range-capable video serving (206 verified); CORS for any localhost origin. - Frozen geometry contract: geometry.colmap_to_threejs (M5 math) + unit test (3 known vectors, random round-trip, scipy oracle); mirrored frontend/src/lib/pose.js with identical POSE_TEST_VECTORS. Lane-B stubs: slerp_pose, ray_from_pixel, triangulate_rays, nearest_point_on_ray. - Classifier contract (events_ai.py): MomentClassification model + MomentClassifier protocol + Gemini/Claude/Local provider stubs. - Lane-owned modules stubbed with final signatures (ingest, audio_sync, frames, sfm, events_ai); cli/api catch NotImplementedError and degrade gracefully. - plan/CHANGE_REQUESTS.md created; plan/status/foundation.md updated. Acceptance: pytest 24 passed; serve endpoints verified via curl + browser (video seek, manifest fetch cross-origin, pose.js self-test, 0 console errors). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
58 lines
2.1 KiB
Markdown
58 lines
2.1 KiB
Markdown
# Festival 4D
|
|
|
|
Turn multiple fan-shot smartphone videos of the same concert into a synchronized,
|
|
explorable **4D experience**: time-aligned multi-video playback, 3D scene reconstruction
|
|
with camera poses, a free-roam "god's eye" viewer, AR-style overlays projected onto each
|
|
video, and AI-tagged moments on a shared timeline.
|
|
|
|
> **Status: foundation phase.** The scaffold, database, frozen contracts (API, pose math,
|
|
> DB schema), and a synthetic fixture generator are in place. Feature lanes (media sync,
|
|
> reconstruction, viewer, AI events) build on top. The full README with the real-footage
|
|
> workflow is written in the integration phase (M9). See
|
|
> [`OPUS_BUILD_INSTRUCTIONS.md`](OPUS_BUILD_INSTRUCTIONS.md) for the canonical spec and
|
|
> [`plan/`](plan/) for the execution plan.
|
|
|
|
## Prerequisites
|
|
|
|
- **Python 3.11+** and [`uv`](https://docs.astral.sh/uv/) (or venv + pip)
|
|
- **ffmpeg** / **ffprobe** on your `PATH` (required)
|
|
- **COLMAP** (optional — reconstruction degrades gracefully without it)
|
|
- A classifier API key (optional — `GEMINI_API_KEY` by default; see `events_ai.py`)
|
|
- **Node 18+** for the frontend
|
|
|
|
## Quickstart (synthetic demo — no footage needed)
|
|
|
|
```bash
|
|
# 1. backend env
|
|
uv venv --python 3.12
|
|
uv pip install -e ".[dev]"
|
|
|
|
# 2. generate the synthetic fixture project (fake videos + poses + point cloud + events)
|
|
uv run python -m festival4d synthetic
|
|
|
|
# 3. serve the API (http://127.0.0.1:8000)
|
|
uv run python -m festival4d serve
|
|
|
|
# 4. in another terminal, the frontend
|
|
cd frontend
|
|
npm install
|
|
npm run dev # http://localhost:5173
|
|
```
|
|
|
|
## Backend CLI
|
|
|
|
```
|
|
python -m festival4d synthetic # generate the synthetic fixture (M0)
|
|
python -m festival4d ingest # probe videos + extract audio (lane A / M1)
|
|
python -m festival4d sync # GCC-PHAT audio alignment (lane A / M1)
|
|
python -m festival4d reconstruct # COLMAP SfM + pose export (lane B / M2)
|
|
python -m festival4d events # audio candidates + AI classify (lane D / M7)
|
|
python -m festival4d serve # FastAPI app (M3)
|
|
```
|
|
|
|
## Tests
|
|
|
|
```bash
|
|
uv run pytest
|
|
```
|