m3ultra cf6a6fb2c8 Foundation (M0 + M3 + frozen contracts): scaffold, synthetic fixtures, full API

What now works:
- M0 scaffold: pyproject (all spec deps), uv/py3.12 env, `python -m festival4d`
  CLI registering synthetic|ingest|sync|reconstruct|events|serve. Vite hello page.
- Synthetic fixture (synthetic.py): 3 shifted-audio videos (offsets 0/+1370/-842 ms),
  camera-arc poses, stage point cloud -> points.ply, seeded events + anchors,
  ground_truth.json. `python -m festival4d synthetic` populates data/ + DB.
- DB schema exactly per spec §2 (db.py) + CRUD helpers all lanes use.
- M3 API (api.py) full against synthetic data: manifest/poses/pointcloud/anchors/
  events/detect/annotations; Range-capable video serving (206 verified); CORS for any
  localhost origin.
- Frozen geometry contract: geometry.colmap_to_threejs (M5 math) + unit test (3 known
  vectors, random round-trip, scipy oracle); mirrored frontend/src/lib/pose.js with
  identical POSE_TEST_VECTORS. Lane-B stubs: slerp_pose, ray_from_pixel, triangulate_rays,
  nearest_point_on_ray.
- Classifier contract (events_ai.py): MomentClassification model + MomentClassifier
  protocol + Gemini/Claude/Local provider stubs.
- Lane-owned modules stubbed with final signatures (ingest, audio_sync, frames, sfm,
  events_ai); cli/api catch NotImplementedError and degrade gracefully.
- plan/CHANGE_REQUESTS.md created; plan/status/foundation.md updated.

Acceptance: pytest 24 passed; serve endpoints verified via curl + browser (video seek,
manifest fetch cross-origin, pose.js self-test, 0 console errors).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-07-16 00:45:55 +10:00

2.1 KiB

Raw Blame History

Festival 4D

Turn multiple fan-shot smartphone videos of the same concert into a synchronized, explorable 4D experience: time-aligned multi-video playback, 3D scene reconstruction with camera poses, a free-roam "god's eye" viewer, AR-style overlays projected onto each video, and AI-tagged moments on a shared timeline.

Status: foundation phase. The scaffold, database, frozen contracts (API, pose math, DB schema), and a synthetic fixture generator are in place. Feature lanes (media sync, reconstruction, viewer, AI events) build on top. The full README with the real-footage workflow is written in the integration phase (M9). See OPUS_BUILD_INSTRUCTIONS.md for the canonical spec and plan/ for the execution plan.

Prerequisites

Python 3.11+ and uv (or venv + pip)
ffmpeg / ffprobe on your PATH (required)
COLMAP (optional — reconstruction degrades gracefully without it)
A classifier API key (optional — GEMINI_API_KEY by default; see events_ai.py)
Node 18+ for the frontend

Quickstart (synthetic demo — no footage needed)

# 1. backend env
uv venv --python 3.12
uv pip install -e ".[dev]"

# 2. generate the synthetic fixture project (fake videos + poses + point cloud + events)
uv run python -m festival4d synthetic

# 3. serve the API (http://127.0.0.1:8000)
uv run python -m festival4d serve

# 4. in another terminal, the frontend
cd frontend
npm install
npm run dev            # http://localhost:5173

Backend CLI

python -m festival4d synthetic     # generate the synthetic fixture (M0)
python -m festival4d ingest        # probe videos + extract audio      (lane A / M1)
python -m festival4d sync          # GCC-PHAT audio alignment           (lane A / M1)
python -m festival4d reconstruct   # COLMAP SfM + pose export           (lane B / M2)
python -m festival4d events        # audio candidates + AI classify     (lane D / M7)
python -m festival4d serve         # FastAPI app                        (M3)

Tests

uv run pytest