cf6a6fb2c8
What now works: - M0 scaffold: pyproject (all spec deps), uv/py3.12 env, `python -m festival4d` CLI registering synthetic|ingest|sync|reconstruct|events|serve. Vite hello page. - Synthetic fixture (synthetic.py): 3 shifted-audio videos (offsets 0/+1370/-842 ms), camera-arc poses, stage point cloud -> points.ply, seeded events + anchors, ground_truth.json. `python -m festival4d synthetic` populates data/ + DB. - DB schema exactly per spec §2 (db.py) + CRUD helpers all lanes use. - M3 API (api.py) full against synthetic data: manifest/poses/pointcloud/anchors/ events/detect/annotations; Range-capable video serving (206 verified); CORS for any localhost origin. - Frozen geometry contract: geometry.colmap_to_threejs (M5 math) + unit test (3 known vectors, random round-trip, scipy oracle); mirrored frontend/src/lib/pose.js with identical POSE_TEST_VECTORS. Lane-B stubs: slerp_pose, ray_from_pixel, triangulate_rays, nearest_point_on_ray. - Classifier contract (events_ai.py): MomentClassification model + MomentClassifier protocol + Gemini/Claude/Local provider stubs. - Lane-owned modules stubbed with final signatures (ingest, audio_sync, frames, sfm, events_ai); cli/api catch NotImplementedError and degrade gracefully. - plan/CHANGE_REQUESTS.md created; plan/status/foundation.md updated. Acceptance: pytest 24 passed; serve endpoints verified via curl + browser (video seek, manifest fetch cross-origin, pose.js self-test, 0 console errors). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
72 lines
3.1 KiB
Python
72 lines
3.1 KiB
Python
"""COLMAP orchestration, model parsing, scene normalization, pose export (spec M2, lane B).
|
|
|
|
STUB — lane B fills the bodies; the public signatures below are frozen (``cli.py`` calls
|
|
``run_reconstruct``).
|
|
|
|
Pipeline (spec M2, for lane B's reference):
|
|
- Sample frames (``frames.sample_frames``), then drive the COLMAP CLI via subprocess:
|
|
``feature_extractor`` (OPENCV model, single camera per folder) -> ``sequential_matcher``
|
|
within each video + ``exhaustive_matcher`` across -> ``mapper`` -> ``model_converter`` to
|
|
a TXT model. Parse ``images.txt`` / ``cameras.txt`` / ``points3D.txt`` by hand (no
|
|
pycolmap).
|
|
- **Normalize** the scene: centroid -> origin, camera bounding sphere radius -> 10, world-up
|
|
-> average camera up. Apply the same similarity transform to poses and points.
|
|
- **Interpolate** poses for unregistered frames (slerp rotation, lerp translation, mark
|
|
``registered=False``); never extrapolate past the first/last registered frame.
|
|
- Export ``config.POINTS_PLY`` (binary little-endian, matching ``synthetic.write_ply``'s
|
|
format) and fill ``camera_poses`` via ``db.set_poses``.
|
|
- **Failure handling** (spec M2): COLMAP optional at runtime (``shutil.which("colmap")``);
|
|
if it registers < 60% of frames or < 2 videos into one model, print a clear diagnostic and
|
|
leave existing poses untouched — never corrupt the DB. The app degrades to "synced videos,
|
|
no 3D".
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import logging
|
|
from pathlib import Path
|
|
|
|
log = logging.getLogger("festival4d.sfm")
|
|
|
|
|
|
def colmap_available() -> bool:
|
|
"""Whether the COLMAP binary is on PATH (spec M2 optional-at-runtime rule)."""
|
|
import shutil
|
|
|
|
return shutil.which("colmap") is not None
|
|
|
|
|
|
def parse_images_txt(path: Path) -> list[dict]:
|
|
"""Parse a COLMAP ``images.txt`` into per-image pose records (spec M2, lane B)."""
|
|
raise NotImplementedError("lane B (M2): implement parse_images_txt")
|
|
|
|
|
|
def parse_cameras_txt(path: Path) -> dict[int, dict]:
|
|
"""Parse a COLMAP ``cameras.txt`` into ``camera_id -> intrinsics`` (spec M2, lane B)."""
|
|
raise NotImplementedError("lane B (M2): implement parse_cameras_txt")
|
|
|
|
|
|
def parse_points3d_txt(path: Path):
|
|
"""Parse a COLMAP ``points3D.txt`` into ``(points Nx3, colors Nx3)`` (spec M2, lane B)."""
|
|
raise NotImplementedError("lane B (M2): implement parse_points3d_txt")
|
|
|
|
|
|
def normalize_scene(points, poses):
|
|
"""Similarity transform: centroid->origin, camera sphere radius->10, up->+Y (spec M2)."""
|
|
raise NotImplementedError("lane B (M2): implement normalize_scene")
|
|
|
|
|
|
def run_colmap(frames_dir: Path, workspace: Path) -> Path:
|
|
"""Drive the COLMAP CLI and return the exported TXT-model directory (spec M2, lane B)."""
|
|
raise NotImplementedError("lane B (M2): implement run_colmap")
|
|
|
|
|
|
def run_reconstruct() -> dict:
|
|
"""Full reconstruction: sample frames, run COLMAP, normalize, interpolate, export.
|
|
|
|
Entrypoint for ``python -m festival4d reconstruct``. Degrades gracefully when COLMAP is
|
|
absent or the reconstruction is too weak (spec M2 failure handling): existing poses are
|
|
left untouched. Returns a summary dict.
|
|
"""
|
|
raise NotImplementedError("lane B (M2): implement run_reconstruct")
|