bb542960c2
Implements spec M2 (frame sampling, COLMAP orchestration, TXT-model parsing, scene normalization, pose interpolation, PLY + DB export) and the M8 geometry functions. geometry.py (M8): slerp_pose (shortest-arc, double-cover), ray_from_pixel (COLMAP +y-down back-projection), triangulate_rays (closest-point + parallel guard), nearest_point_on_ray (in-front radius cylinder). Frozen colmap_to_threejs untouched. frames.py (M2): variance-of-Laplacian sharpness + windowed sharpest-frame sampling. sfm.py (M2): hand-written COLMAP images/cameras/points3D parsers; normalize_scene (centroid->0, camera sphere r->10, up->+Y) as one similarity transform over points+poses; interpolate_poses (slerp+lerp, no extrapolation); build-aware COLMAP CLI orchestration (3.x/4.x option detection, CPU SIFT, single-camera-per-folder, undistort->TXT, largest component by images.bin header count); run_reconstruct with full graceful degradation (COLMAP absent / <60% / <2 videos / no frames / missing files -> poses left untouched). Tests: 61 pass (24 foundation + 37 lane-B) with independent oracles (scipy Slerp, projection-inverse, closed-form geometry) and every degradation/DB-safety branch covered. Validated end-to-end against real COLMAP 4.1.0 (6 components, largest selected, weak -> graceful degradation, exit 0). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|---|---|---|
| backend | ||
| frontend | ||
| plan | ||
| .gitignore | ||
| OPUS_BUILD_INSTRUCTIONS.md | ||
| pyproject.toml | ||
| README.md | ||
| uv.lock | ||
Festival 4D
Turn multiple fan-shot smartphone videos of the same concert into a synchronized, explorable 4D experience: time-aligned multi-video playback, 3D scene reconstruction with camera poses, a free-roam "god's eye" viewer, AR-style overlays projected onto each video, and AI-tagged moments on a shared timeline.
Status: foundation phase. The scaffold, database, frozen contracts (API, pose math, DB schema), and a synthetic fixture generator are in place. Feature lanes (media sync, reconstruction, viewer, AI events) build on top. The full README with the real-footage workflow is written in the integration phase (M9). See
OPUS_BUILD_INSTRUCTIONS.mdfor the canonical spec andplan/for the execution plan.
Prerequisites
- Python 3.11+ and
uv(or venv + pip) - ffmpeg / ffprobe on your
PATH(required) - COLMAP (optional — reconstruction degrades gracefully without it)
- A classifier API key (optional —
GEMINI_API_KEYby default; seeevents_ai.py) - Node 18+ for the frontend
Quickstart (synthetic demo — no footage needed)
# 1. backend env
uv venv --python 3.12
uv pip install -e ".[dev]"
# 2. generate the synthetic fixture project (fake videos + poses + point cloud + events)
uv run python -m festival4d synthetic
# 3. serve the API (http://127.0.0.1:8000)
uv run python -m festival4d serve
# 4. in another terminal, the frontend
cd frontend
npm install
npm run dev # http://localhost:5173
Backend CLI
python -m festival4d synthetic # generate the synthetic fixture (M0)
python -m festival4d ingest # probe videos + extract audio (lane A / M1)
python -m festival4d sync # GCC-PHAT audio alignment (lane A / M1)
python -m festival4d reconstruct # COLMAP SfM + pose export (lane B / M2)
python -m festival4d events # audio candidates + AI classify (lane D / M7)
python -m festival4d serve # FastAPI app (M3)
Tests
uv run pytest