monster/festifun
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge foundation3 (phase 6a / M18): Friend-Tracks fixture ground truth, contracts, stubs (CR-6 ratified)

Browse Source 
Coordinator-verified: 209 passed independently; build clean; live boot on the
marker fixture (btn-friends hidden, solve degrades, has_tracks false); blink
protocol constants match contract #4 bit-for-bit; CR-6 lock edits ratified.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
type-two 2026-07-17 21:38:06 +10:00
commit 9bc1dae41d
17 changed files with 1276 additions and 20 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

85
backend/festival4d/api.py
View File

@ -23,6 +23,17 @@ Phase 5 (foundation2; shapes frozen in plan/20-phase5.md contracts):
DELETE /api/paths/{id}
PATCH /api/anchors/{id} {label?, color?} -> updated anchor dict
Phase 6 (foundation3 / M18; shapes frozen in plan/30-phase6.md contract #2):
GET /api/tracks -> [{id, marker_key, label, color, points:[{t_global_s,x,y,z,
quality,views}]}] (points ordered by t_global_s)
PATCH /api/tracks/{id} {label?, color?} -> updated track dict (404 on missing)
DELETE /api/tracks/{id} -> {deleted: id} (404 on missing)
POST /api/tracks/solve -> {result, tracks[, note]} — runs lane H's detect->solve
(tracker_solve.run_solve); while lane H is stubbed it
degrades to a VALID empty result + "note" (house pattern).
manifest gains "has_tracks": bool.
Source videos in ``data/raw`` are mounted at ``/media`` via Starlette ``StaticFiles``,
which supports HTTP Range (required for ``<video>`` seeking; verify ``curl -H "Range:
bytes=0-100"`` -> 206). Stubbed lane entrypoints (events detection, M8 annotation
@ -118,6 +129,11 @@ class PathIn(BaseModel):
path_json: str = Field(alias="json")
class TrackPatch(BaseModel):
label: str | None = None
color: str | None = None
# ---------------------------------------------------------------------------
# Serialization helpers
# ---------------------------------------------------------------------------
@ -162,6 +178,24 @@ def _event_dict(e) -> dict:
}
def _track_dict(t) -> dict:
"""A track with its points inlined (phase 6 contract #2), ordered by t_global_s."""
return {
"id": t.id,
"marker_key": t.marker_key,
"label": t.label,
"color": t.color,
"points": [
{
"t_global_s": p.t_global_s,
"x": p.x, "y": p.y, "z": p.z,
"quality": p.quality, "views": p.views,
}
for p in db.get_track_points(t.id)
],
}
# ---------------------------------------------------------------------------
# Routes
# ---------------------------------------------------------------------------
@ -183,6 +217,8 @@ def manifest() -> dict:
"has_capture": capture.capture_enabled(),
"has_poses": db.has_poses(),
"has_splat": config.SPLAT_PLY.exists(),
"has_tracks": db.has_tracks(), # phase 6 (M18): any solved friend track ⇒ frontend loads them
# Always False from the live server; the capsule (M17) bakes a manifest with true,
# which makes the frontend hide all write UI (phase-5 contract #5).
"capsule": False,
@ -465,3 +501,52 @@ def delete_anchor(anchor_id: int) -> dict:
if not db.delete_anchor(anchor_id):
raise HTTPException(status_code=404, detail=f"no anchor with id={anchor_id}")
return {"deleted": anchor_id}
# ---------------------------------------------------------------------------
# Phase 6 (foundation3 / M18). Friend tracks. Shapes frozen in plan/30-phase6.md contract #2.
# ---------------------------------------------------------------------------
@app.get("/api/tracks")
def get_tracks() -> list[dict]:
"""All friend tracks with their points inlined (contract #2)."""
return [_track_dict(t) for t in db.get_tracks()]
@app.patch("/api/tracks/{track_id}")
def patch_track(track_id: int, payload: TrackPatch) -> dict:
"""Rename / recolor a track in place (M21 friends panel)."""
try:
track = db.patch_track(track_id, label=payload.label, color=payload.color)
except KeyError:
raise HTTPException(status_code=404, detail=f"no track with id={track_id}")
return _track_dict(track)
@app.delete("/api/tracks/{track_id}")
def delete_track(track_id: int) -> dict:
"""Delete a track and its points (M21)."""
if not db.delete_track(track_id):
raise HTTPException(status_code=404, detail=f"no track with id={track_id}")
return {"deleted": track_id}
@app.post("/api/tracks/solve")
def solve_tracks() -> dict:
"""Run marker detection + track solving (M19+M20) and return the resulting tracks.
Dispatches into lane H's ``tracker_solve.run_solve`` (which itself consumes
``tracker_detect``'s detections). While lane H is stubbed both raise
``NotImplementedError``; we catch it and degrade to a VALID empty result + a note, so the
frozen route never 500s — the house pattern (mirrors ``POST /api/events/detect``)."""
from festival4d import tracker_solve
try:
result = tracker_solve.run_solve()
except NotImplementedError as exc:
log.info("track solving not implemented yet: %s", exc)
return {
"result": None,
"note": "friend-track solving not implemented yet (lane H / M19-M20)",
"tracks": [_track_dict(t) for t in db.get_tracks()],
}
return {"result": result, "tracks": [_track_dict(t) for t in db.get_tracks()]}

4
backend/festival4d/capsule.py
View File

@ -6,7 +6,7 @@ Contract: :func:`build_capsule` writes a self-contained bundle to ``out_dir`` (d
- copy ``frontend/dist`` (error clearly if missing — tell the user to ``npm run build``);
- copy media from ``config.RAW_DIR``;
- bake every read-only API response to files at the *same relative paths*
(``api/manifest``, ``api/events``, ``api/anchors``, ``api/beats``,
(``api/manifest``, ``api/events``, ``api/anchors``, ``api/tracks``, ``api/beats``,
``api/videos/{id}/poses``, ``api/pointcloud``, ``api/splat``, ``api/audio/{id}`` —
extracting audio via ``ingest.extract_audio_hq`` if not yet cached);
- inject ``<script>window.__F4D_API_BASE__=""</script>`` into the copied ``index.html``;
@ -265,6 +265,8 @@ def build_capsule(out_dir: str | Path | None = None,
_bake_json(api_dir / "manifest", manifest)
_bake_json(api_dir / "events", api.get_events())
_bake_json(api_dir / "anchors", api.get_anchors())
# Friend tracks (M18/M20) — read-only in a capsule so ribbons + follow still work zero-backend.
_bake_json(api_dir / "tracks", api.get_tracks())
for v in videos:
_bake_json(api_dir / "videos" / str(v.id) / "poses", api.video_poses(v.id))

27
backend/festival4d/cli.py
View File

@ -5,7 +5,7 @@ to (in ``ingest.py``, ``audio_sync.py``, ``sfm.py``, ``events_ai.py``); they nev
edit this file or ``api.py``.
Subcommands: ``synthetic | ingest | sync | reconstruct | events | features | direct |
capsule | serve``.
track | capsule | serve``.
Every subcommand dispatches to a single lane entrypoint. While a lane is still a stub,
its entrypoint raises :class:`NotImplementedError`; we catch that here and print a clear
@ -76,6 +76,23 @@ def _cmd_direct(args: argparse.Namespace) -> int:
return 0
def _cmd_track(args: argparse.Namespace) -> int:
"""Detect wearable markers and solve them into 3D friend tracks (lane H / M19-M20).
Runs detect -> solve by default; ``--detect-only`` / ``--solve-only`` run one stage.
Both stages dispatch into lane-H stubs, so while unimplemented this prints the
"not implemented yet" message and exits 2 (same as every other stubbed subcommand)."""
from festival4d import tracker_detect, tracker_solve
if not args.solve_only:
result = tracker_detect.run_detect()
log.info("track detect: %s", result)
if not args.detect_only:
result = tracker_solve.run_solve()
log.info("track solve: %s", result)
return 0
def _cmd_capsule(args: argparse.Namespace) -> int:
from festival4d import capsule
@ -134,6 +151,14 @@ def build_parser() -> argparse.ArgumentParser:
help="seconds of lead-in before each event (default 2.0)")
p_dir.set_defaults(func=_cmd_direct)
p_trk = sub.add_parser("track", help="detect markers + solve friend tracks (lane H / M19-M20)")
trk_mode = p_trk.add_mutually_exclusive_group()
trk_mode.add_argument("--detect-only", dest="detect_only", action="store_true",
help="run marker detection only (write detections.json)")
trk_mode.add_argument("--solve-only", dest="solve_only", action="store_true",
help="run track solving only (from an existing detections.json)")
p_trk.set_defaults(func=_cmd_track, detect_only=False, solve_only=False)
p_cap = sub.add_parser("capsule", help="bake a zero-backend shareable bundle (lane G / M17)")
p_cap.add_argument("--out", default=None, help="output dir (default data/capsule/)")
p_cap.set_defaults(func=_cmd_capsule)

2
backend/festival4d/config.py
View File

@ -46,6 +46,8 @@ POINTS_PLY = WORK_DIR / "points.ply" # reconstructed / synthetic point cl
SPLAT_PLY = WORK_DIR / "splat.ply" # optional 3DGS splat (trained externally, e.g. via MODELBEAST — see docs/modelbeast-crossover.md); viewer prefers it over the point cloud
SYNC_JSON = WORK_DIR / "sync.json" # exported sync solution (lane A)
GROUND_TRUTH_JSON = WORK_DIR / "ground_truth.json" # synthetic fixture ground truth
DETECTIONS_JSON = WORK_DIR / "detections.json" # marker detections (phase 6 / M19, contract #3)
TRACK_TRUTH_JSON = WORK_DIR / "track_truth.json" # synthetic marker ground truth (phase 6 / M18)
# ---------------------------------------------------------------------------
# Constants

140
backend/festival4d/db.py
View File

@ -13,6 +13,8 @@ Schema (spec §2)::
events(id, t_global_s, duration_s, event_type, confidence, description, source)
annotations(id, video_id FK, t_video_s, x0,y0,x1,y1, resolved_anchor_id FK NULL)
paths(id, name, created_at, path_json) # phase 5 (M16): saved camPath JSON, as text
tracks(id, marker_key, label, color, created_at) # phase 6 (M18/M20): friend tracks
track_points(id, track_id FK, t_global_s, x,y,z, quality, views) # ordered by t_global_s
Engine management: :func:`init_engine` (re)binds the module to a SQLite file. It defaults
to ``config.DB_PATH`` but tests point it at a temp file. Helpers open and commit their own
@ -143,6 +145,43 @@ class SavedPath(Base):
path_json: Mapped[str] = mapped_column(String, nullable=False)
class Track(Base):
"""A friend track (phase 6 contract #2): a wearable marker's identity + metadata.
``marker_key`` is the detector-emitted identity — ``"hue:<opencv_hue>"`` for a color
marker or ``"code:<id>"`` for a blink badge (see ``tracker_detect``). The per-timestep
3D path lives in :class:`TrackPoint`. ``label`` / ``color`` are user-editable.
"""
__tablename__ = "tracks"
id: Mapped[int] = mapped_column(Integer, primary_key=True)
marker_key: Mapped[str] = mapped_column(String, nullable=False)
label: Mapped[str | None] = mapped_column(String, nullable=True)
color: Mapped[str | None] = mapped_column(String, nullable=True)
created_at: Mapped[str] = mapped_column(String, nullable=False)
class TrackPoint(Base):
"""One solved position of a track at a master-timeline instant (phase 6 contract #2).
Coordinates are **Three.js scene space**. ``views`` = number of cameras used to solve the
point (1 ⇒ single-view ground-plane fallback, contract #5); ``quality`` is 0..1.
Points are always read ordered by ``t_global_s``.
"""
__tablename__ = "track_points"
id: Mapped[int] = mapped_column(Integer, primary_key=True)
track_id: Mapped[int] = mapped_column(ForeignKey("tracks.id"), nullable=False, index=True)
t_global_s: Mapped[float] = mapped_column(Float, nullable=False)
x: Mapped[float] = mapped_column(Float, nullable=False)
y: Mapped[float] = mapped_column(Float, nullable=False)
z: Mapped[float] = mapped_column(Float, nullable=False)
quality: Mapped[float | None] = mapped_column(Float, nullable=True)
views: Mapped[int | None] = mapped_column(Integer, nullable=True)
# ---------------------------------------------------------------------------
# Engine / session management
# ---------------------------------------------------------------------------
@ -508,6 +547,107 @@ def delete_path(path_id: int) -> bool:
return True
# ---------------------------------------------------------------------------
# Friend tracks (phase 6 / M18 contract #2). Written by the solver (lane H / M20),
# read by the API + frontend. TrackPoints are always returned ordered by t_global_s.
# ---------------------------------------------------------------------------
def add_track(marker_key: str, label: str | None = None, color: str | None = None) -> Track:
"""Insert a track row (no points yet) and return it."""
with session_scope() as s:
track = Track(marker_key=marker_key, label=label, color=color, created_at=_now_iso())
s.add(track)
s.flush()
s.refresh(track)
return track
def get_tracks() -> list[Track]:
"""All tracks, ordered by id."""
with session_scope() as s:
return list(s.scalars(select(Track).order_by(Track.id)))
def get_track(track_id: int) -> Track | None:
with session_scope() as s:
return s.get(Track, track_id)
def get_track_points(track_id: int) -> list[TrackPoint]:
"""A track's points, ordered by ``t_global_s`` (contract #2)."""
with session_scope() as s:
return list(
s.scalars(
select(TrackPoint)
.where(TrackPoint.track_id == track_id)
.order_by(TrackPoint.t_global_s)
)
)
def set_track_points(track_id: int, points: Iterable[dict]) -> int:
"""Replace all points for a track with ``points`` (solver output, M20). Returns the count.
Each dict needs ``t_global_s, x, y, z`` and optionally ``quality`` (default None) and
``views`` (default None). Atomic: the delete + inserts share one transaction, so a caller
that raises mid-way never leaves a half-written track. Raises ``KeyError`` if the track
is unknown.
"""
rows = list(points)
with session_scope() as s:
if s.get(Track, track_id) is None:
raise KeyError(f"no track with id={track_id}")
s.execute(delete(TrackPoint).where(TrackPoint.track_id == track_id))
for p in rows:
s.add(
TrackPoint(
track_id=track_id,
t_global_s=float(p["t_global_s"]),
x=float(p["x"]), y=float(p["y"]), z=float(p["z"]),
quality=None if p.get("quality") is None else float(p["quality"]),
views=None if p.get("views") is None else int(p["views"]),
)
)
return len(rows)
def patch_track(track_id: int, label: str | None = None, color: str | None = None) -> Track:
"""Rename / recolor a track (M21 panel). Only provided fields change.
Raises ``KeyError`` if the id is unknown.
"""
with session_scope() as s:
track = s.get(Track, track_id)
if track is None:
raise KeyError(f"no track with id={track_id}")
if label is not None:
track.label = label
if color is not None:
track.color = color
s.flush()
s.refresh(track)
return track
def delete_track(track_id: int) -> bool:
"""Delete a track and its points. Returns ``True`` if a row was deleted, else ``False``.
SQLite doesn't cascade by default, so the points are removed explicitly.
"""
with session_scope() as s:
track = s.get(Track, track_id)
if track is None:
return False
s.execute(delete(TrackPoint).where(TrackPoint.track_id == track_id))
s.delete(track)
return True
def has_tracks() -> bool:
"""Whether any track exists (drives ``manifest.has_tracks``)."""
with session_scope() as s:
return s.scalar(select(Track.id).limit(1)) is not None
# ---------------------------------------------------------------------------
# Annotations
# ---------------------------------------------------------------------------

295
backend/festival4d/synthetic.py
View File

@ -11,11 +11,17 @@ testable without real footage:
- **Point cloud**: random points on the stage box + ground plane, written to
``points.ply`` in the exact binary-little-endian format the real COLMAP path produces.
- **Seeded events + anchors** so the timeline and overlays have data immediately.
- **Two moving markers** (phase 6 / M18) composited into every camera's video *through the
same camera projection the poses use*, plus ``track_truth.json`` — the 3D ground truth lane
H recovers. Marker A is a solid magenta disc (color mode, ``hue:150``); marker B is a
blinking bright disc speaking the frozen OOK protocol with ID 5 (``code:5``). See
:func:`marker_position`, :func:`render_marker_overlay`, :func:`build_track_truth`.
- **``ground_truth.json``**: offsets, drift, audio pulse (bang) times, events, and stage
corners — part of the frozen contract (lanes A and D assert against it).
FROZEN CONTRACT after foundation. The module is factored so the numeric pieces
(audio shifts, poses, PLY round-trip) are unit-testable without invoking ffmpeg.
(audio shifts, poses, PLY round-trip, marker geometry) are unit-testable without invoking
ffmpeg.
"""
from __future__ import annotations
@ -31,8 +37,8 @@ from pathlib import Path
import numpy as np
from festival4d import config, db
from festival4d.geometry import mat_to_quat
from festival4d import config, db, tracker_detect
from festival4d.geometry import mat_to_quat, quat_to_mat, slerp_pose
log = logging.getLogger("festival4d.synthetic")
@ -363,6 +369,219 @@ def read_ply(path: Path) -> tuple[np.ndarray, np.ndarray]:
return points, colors
# ---------------------------------------------------------------------------
# Moving markers (phase 6 / M18). Two wearable markers travel known 3D paths across the
# stage; each camera's frame shows them at the pixel its OWN pose projects the 3D point to,
# so the rendered pixels and ``track_truth.json`` are consistent BY CONSTRUCTION — the marker
# pixel is derived from the ground-truth 3D point through the same projection lane H inverts.
#
# Marker A — solid magenta (#ff00ff) disc, color mode, marker_key "hue:150". Slow circle
# (radius 2, y≈1.5) centered on the stage.
# Marker B — blinking bright disc, blink mode, ID 5, marker_key "code:5". A different path;
# the LED follows the frozen OOK protocol (tracker_detect) in GLOBAL time, so
# every camera sees the same on/off state at the same t_global.
# Discs are small (radius ≈2.5% of frame height ⇒ ≪2% frame area) and composited AFTER all
# existing content, so the 189-test floor (offsets/events/poses/SfM) is untouched (pitfall #6).
# ---------------------------------------------------------------------------
MARKER_A_KEY = tracker_detect.color_marker_key(tracker_detect.MAGENTA_OPENCV_HUE) # "hue:150"
MARKER_A_HEX = "#ff00ff"
MARKER_A_BGR = (255, 0, 255) # OpenCV BGR for magenta (drawn AFTER the hue filter)
MARKER_B_ID = 5
MARKER_B_KEY = tracker_detect.code_marker_key(MARKER_B_ID) # "code:5"
MARKER_B_ON_BGR = (255, 255, 255) # bright white while the LED is ON
# testsrc2's base pattern contains solid full-saturation magenta AND cyan colour bars — the
# EXACT two marker hues (contract #1) — plus other bright bars. Left as-is they swamp lane H's
# colour detection (the magenta bar is a far bigger "magenta blob" than marker A's disc) and
# starve marker B's white-blink luminance contrast. So the base is MUTED before the markers are
# composited on top: saturation crushed (no high-saturation magenta/cyan survives) and dimmed
# (the bright-white blink disc pops in luminance). The fully-saturated discs are then the only
# high-S magenta/cyan and the only bright small blobs in frame — the fixture lane H is graded on
# is actually solvable by construction. Tests assert geometry/audio/PLY, never base pixels.
BASE_SATURATION = 0.28 # hue-filter saturation multiplier applied to the base
BASE_BRIGHTNESS = -0.28 # eq brightness offset applied to the base (dim it)
MARKER_DISC_HEIGHT_FRAC = 0.025 # disc radius / frame height (≈9 px @ 360 -> 0.11% area)
MARKER_A_PERIOD_S = 20.0 # one slow loop across the fixture span
MARKER_B_PERIOD_S = 16.0 # different path + rate so the two never lock together
MARKER_TRUTH_DT_S = 0.05 # track_truth.json sampling step (t_global seconds)
def marker_position(marker_key: str, t_global: float) -> tuple[float, float, float]:
"""Ground-truth 3D position (Three.js scene space) of a marker at ``t_global`` seconds.
Pure and total over all real ``t_global`` (the markers exist for the whole span). This is
THE ground truth: it is what ``track_truth.json`` records and what a correct solver
recovers. FROZEN — lane H's acceptance is measured against these paths.
"""
if marker_key == MARKER_A_KEY:
theta = 2.0 * np.pi * t_global / MARKER_A_PERIOD_S
return (2.0 * float(np.cos(theta)), 1.5, 2.0 * float(np.sin(theta)))
if marker_key == MARKER_B_KEY:
phi = 2.0 * np.pi * t_global / MARKER_B_PERIOD_S
return (
2.4 * float(np.sin(phi)),
1.0 + 0.4 * float(np.sin(2.0 * phi)),
-0.8 + 1.2 * float(np.cos(phi)),
)
raise KeyError(f"unknown marker_key {marker_key!r}")
MARKER_KEYS = (MARKER_A_KEY, MARKER_B_KEY)
def _interp_pose(poses: list[dict], t_video: float):
"""Interpolate a COLMAP pose ``(q, t, (fx,fy,cx,cy))`` at ``t_video`` from ``camera_track``
output (ascending by ``t_video_s``). Mirrors ``resolve._pose_at`` / the frontend ``poseAt``
exactly — slerp the rotation, lerp the translation via the FROZEN ``geometry.slerp_pose`` —
so the marker pixels a solver back-projects land on the same 3D point. Clamps (no extrap.).
"""
def tup(p):
return ([p["qw"], p["qx"], p["qy"], p["qz"]],
[p["tx"], p["ty"], p["tz"]],
(p["fx"], p["fy"], p["cx"], p["cy"]))
if t_video <= poses[0]["t_video_s"]:
return tup(poses[0])
if t_video >= poses[-1]["t_video_s"]:
return tup(poses[-1])
lo, hi = 0, len(poses) - 1
while hi - lo > 1:
mid = (lo + hi) // 2
if poses[mid]["t_video_s"] <= t_video:
lo = mid
else:
hi = mid
a, b = poses[lo], poses[hi]
span = b["t_video_s"] - a["t_video_s"]
alpha = (t_video - a["t_video_s"]) / span if span > 1e-9 else 0.0
q, t = slerp_pose(
[a["qw"], a["qx"], a["qy"], a["qz"]], [a["tx"], a["ty"], a["tz"]],
[b["qw"], b["qx"], b["qy"], b["qz"]], [b["tx"], b["ty"], b["tz"]],
alpha,
)
return list(q), list(t), (a["fx"], a["fy"], a["cx"], a["cy"])
def project_point(q, t, intr, X) -> tuple[float, float, float] | None:
"""Project world point ``X`` through a COLMAP world->camera pose (the inverse of
``geometry.ray_from_pixel``). Returns ``(px, py, z_cam)`` in pixels, or ``None`` if the
point is behind the camera. COLMAP camera axes: +x right, +y down, +z forward.
"""
R = quat_to_mat(q) # world -> cam
Xc = R @ np.asarray(X, dtype=np.float64).reshape(3) + np.asarray(t, dtype=np.float64).reshape(3)
z = float(Xc[2])
if z <= 1e-6:
return None
fx, fy, cx, cy = intr
return fx * float(Xc[0]) / z + cx, fy * float(Xc[1]) / z + cy, z
def _disc_radius_px(height: int) -> int:
return max(6, int(round(MARKER_DISC_HEIGHT_FRAC * height)))
def render_marker_overlay(out_dir: Path, poses: list[dict], offset_ms: float,
width: int, height: int, fps: float, duration_s: float) -> int:
"""Write a transparent BGRA PNG per frame with the two markers drawn at their projected
pixels (phase 6 / M18). Composited over one camera's clip by :func:`render_video`.
For frame ``k`` at local ``t_video = k/fps``: convert to ``t_global`` with the FROZEN
``config.t_global_from_video``, look up each marker's 3D truth at ``t_global``, and project
it through this camera's pose interpolated at ``t_video``. Marker A is always drawn; marker
B is drawn only when its LED is ON per the OOK schedule (``tracker_detect.led_on``, in
global time). Off-frame / behind-camera projections are simply not drawn. Returns frame count.
"""
import cv2
out_dir = Path(out_dir)
out_dir.mkdir(parents=True, exist_ok=True)
radius = _disc_radius_px(height)
n = int(round(duration_s * fps))
for k in range(n):
t_video = k / fps
t_global = config.t_global_from_video(t_video, offset_ms, 0.0)
q, t, intr = _interp_pose(poses, t_video)
img = np.zeros((height, width, 4), dtype=np.uint8) # transparent BGRA
_draw_marker_disc(img, q, t, intr, marker_position(MARKER_A_KEY, t_global),
MARKER_A_BGR, radius)
if tracker_detect.led_on(t_global, MARKER_B_ID):
_draw_marker_disc(img, q, t, intr, marker_position(MARKER_B_KEY, t_global),
MARKER_B_ON_BGR, radius)
if not cv2.imwrite(str(out_dir / f"{k:05d}.png"), img):
raise RuntimeError(f"failed to write marker overlay frame: {out_dir}/{k:05d}.png")
return n
def _draw_marker_disc(img, q, t, intr, X, bgr, radius) -> None:
import cv2
proj = project_point(q, t, intr, X)
if proj is None:
return
px, py, _ = proj
h, w = img.shape[:2]
if not (0.0 <= px < w and 0.0 <= py < h):
return # off this camera's frame ⇒ not visible here
cv2.circle(img, (int(round(px)), int(round(py))), radius,
(int(bgr[0]), int(bgr[1]), int(bgr[2]), 255), thickness=-1, lineType=cv2.LINE_8)
def build_track_truth(g_lo: float, g_hi: float, dt: float = MARKER_TRUTH_DT_S) -> dict:
"""The ``track_truth.json`` body: each marker's 3D path sampled over ``[g_lo, g_hi]``.
Shape (spec M18): ``{"markers": [{"marker_key", "points": [{"t_global_s","x","y","z"}]}]}``
in Three.js scene space. ``generated_by`` is added for provenance (extra keys are fine).
"""
n = max(1, int(round((g_hi - g_lo) / dt)) + 1)
markers = []
for key in MARKER_KEYS:
points = []
for i in range(n):
tg = g_lo + i * dt
x, y, z = marker_position(key, tg)
points.append({"t_global_s": round(tg, 6), "x": x, "y": y, "z": z})
markers.append({"marker_key": key, "points": points})
return {"markers": markers, "generated_by": "festival4d.synthetic"}
def render_blink_clip(out_path: Path, marker_id: int = MARKER_B_ID, fps: float = 30.0,
duration_s: float = 3.0, width: int = 320, height: int = 240) -> Path:
"""Render a small clip of a single blinking badge at frame center (phase 6 / M24 helper).
A fixed-position bright disc blinking ``marker_id`` in the frozen OOK protocol on a black
background — no camera geometry, so lane K's ``badge_selftest --synthetic`` can round-trip
the ID through ``tracker_detect``'s blink decoder without a full fixture. No audio track.
"""
import cv2
if shutil.which("ffmpeg") is None:
raise RuntimeError("ffmpeg not found on PATH — required to render a blink clip")
out_path = Path(out_path)
out_path.parent.mkdir(parents=True, exist_ok=True)
radius = _disc_radius_px(height)
cx, cy = width // 2, height // 2
n = int(round(duration_s * fps))
with tempfile.TemporaryDirectory() as tmp:
frames_dir = Path(tmp)
for k in range(n):
t = k / fps
img = np.zeros((height, width, 3), dtype=np.uint8) # black background
if tracker_detect.led_on(t, marker_id):
cv2.circle(img, (cx, cy), radius, (255, 255, 255), thickness=-1,
lineType=cv2.LINE_8)
cv2.imwrite(str(frames_dir / f"{k:05d}.png"), img)
cmd = [
"ffmpeg", "-y", "-hide_banner", "-loglevel", "error",
"-framerate", str(fps), "-start_number", "0",
"-i", str(frames_dir / "%05d.png"),
"-c:v", "libx264", "-pix_fmt", "yuv420p", "-preset", "veryfast", "-crf", "20",
"-movflags", "+faststart",
str(out_path),
]
subprocess.run(cmd, check=True)
return out_path
# ---------------------------------------------------------------------------
# Video rendering (ffmpeg)
# ---------------------------------------------------------------------------
@ -396,25 +615,30 @@ def _has_drawtext() -> bool:
def render_video(out_path: Path, wav_path: Path, cam_index: int,
width: int, height: int, fps: float, duration_s: float) -> None:
"""Render one synthetic clip: a distinct ``testsrc2`` visual muxed with ``wav_path``."""
width: int, height: int, fps: float, duration_s: float,
poses: list[dict] | None = None, offset_ms: float = 0.0) -> None:
"""Render one synthetic clip: a distinct ``testsrc2`` visual muxed with ``wav_path``.
When ``poses`` is given (this camera's ``camera_track``), the two moving markers (M18) are
composited on top via a per-frame RGBA overlay produced by :func:`render_marker_overlay`
(drawn AFTER the base visual + label, so existing behavior/tests are untouched). ``offset_ms``
aligns the marker schedule to global time for this camera.
"""
if shutil.which("ffmpeg") is None:
raise RuntimeError("ffmpeg not found on PATH — required for the synthetic fixture")
out_path.parent.mkdir(parents=True, exist_ok=True)
# Per-camera hue shift for distinctness, then mute saturation + brightness so the base
# never collides with the fully-saturated markers composited on top (see BASE_* above).
hue = cam_index * 47
vf = f"hue=h={hue}"
vf = f"hue=h={hue}:s={BASE_SATURATION},eq=brightness={BASE_BRIGHTNESS}"
font = _find_font()
if font and _has_drawtext():
vf += (f",drawtext=fontfile='{font}':text='CAM {cam_index}':"
"x=24:y=24:fontsize=44:fontcolor=white:box=1:boxcolor=black@0.5")
src = f"testsrc2=size={width}x{height}:rate={fps}:duration={duration_s}"
cmd = [
"ffmpeg", "-y", "-hide_banner", "-loglevel", "error",
"-f", "lavfi", "-i", src,
"-i", str(wav_path),
"-vf", vf,
common_out = [
"-c:v", "libx264", "-pix_fmt", "yuv420p", "-preset", "veryfast", "-crf", "28",
"-profile:v", "baseline", "-level", "3.1",
"-movflags", "+faststart",
@ -422,6 +646,32 @@ def render_video(out_path: Path, wav_path: Path, cam_index: int,
"-shortest",
str(out_path),
]
if poses is None:
cmd = [
"ffmpeg", "-y", "-hide_banner", "-loglevel", "error",
"-f", "lavfi", "-i", src,
"-i", str(wav_path),
"-vf", vf,
*common_out,
]
subprocess.run(cmd, check=True)
return
# Markers: pre-render the RGBA overlay frames, then composite them over the base visual.
with tempfile.TemporaryDirectory() as ov:
ov_dir = Path(ov)
render_marker_overlay(ov_dir, poses, offset_ms, width, height, fps, duration_s)
cmd = [
"ffmpeg", "-y", "-hide_banner", "-loglevel", "error",
"-f", "lavfi", "-i", src, # 0: base video
"-i", str(wav_path), # 1: audio
"-framerate", str(fps), "-start_number", "0",
"-i", str(ov_dir / "%05d.png"), # 2: marker overlay
"-filter_complex", f"[0:v]{vf}[bg];[bg][2:v]overlay=0:0:eof_action=pass[vo]",
"-map", "[vo]", "-map", "1:a",
*common_out,
]
subprocess.run(cmd, check=True)
@ -481,11 +731,15 @@ def build(base_dir: Path | str | None = None, duration_s: float | None = None,
filename = f"cam{i}.mp4"
out_path = raw / filename
clip = slice_for_offset(master, sr, offset_ms, duration)
poses = camera_track(i, duration, fps, w, h)
if run_ffmpeg:
wav_path = tmp / f"cam{i}.wav"
write_wav(wav_path, clip, sr)
render_video(out_path, wav_path, i, w, h, fps, duration)
# Pass this camera's poses + offset so the M18 markers render through the same
# projection lane H inverts (the marker pixel derives from its 3D truth).
render_video(out_path, wav_path, i, w, h, fps, duration,
poses=poses, offset_ms=float(offset_ms))
probed = ffprobe_video(out_path)
vw, vh, vfps, vdur = (probed["width"], probed["height"],
probed["fps"], probed["duration_s"])
@ -496,7 +750,7 @@ def build(base_dir: Path | str | None = None, duration_s: float | None = None,
filename=filename, duration_s=vdur, fps=vfps, width=vw, height=vh,
offset_ms=float(offset_ms), drift_ppm=0.0, sync_confidence=1.0,
)
db.set_poses(video.id, camera_track(i, duration, fps, w, h))
db.set_poses(video.id, poses)
videos_gt.append({
"video_id": video.id, "filename": filename,
"offset_ms": float(offset_ms), "drift_ppm": 0.0,
@ -537,17 +791,28 @@ def build(base_dir: Path | str | None = None, duration_s: float | None = None,
gt_path = work / "ground_truth.json"
gt_path.write_text(json.dumps(ground_truth, indent=2))
# Friend-track ground truth (M18): the two markers' 3D paths over the union of the
# cameras' global coverage. Written for both ffmpeg and no-ffmpeg builds — it is pure
# geometry (the same paths the markers are rendered from), so tests can check it cheaply.
offsets_s = [o / 1000.0 for o in config.SYNTH_OFFSETS_MS]
track_truth = build_track_truth(min(offsets_s), max(offsets_s) + duration)
tt_path = work / "track_truth.json"
tt_path.write_text(json.dumps(track_truth, indent=2))
summary = {
"base_dir": str(base),
"videos": [v["filename"] for v in videos_gt],
"points_ply": str(work / "points.ply"),
"point_count": int(len(points)),
"ground_truth": str(gt_path),
"track_truth": str(tt_path),
"markers": [m["marker_key"] for m in track_truth["markers"]],
"events": len(SEED_EVENTS),
"anchors": len(STAGE_CORNERS),
}
log.info("synthetic: done — %d videos, %d points, %d events, %d anchors",
len(videos_gt), len(points), len(SEED_EVENTS), len(STAGE_CORNERS))
log.info("synthetic: done — %d videos, %d points, %d events, %d anchors, %d markers",
len(videos_gt), len(points), len(SEED_EVENTS), len(STAGE_CORNERS),
len(track_truth["markers"]))
return summary

171
backend/festival4d/tracker_detect.py Normal file
View File

@ -0,0 +1,171 @@
"""Marker detection (spec M19, lane H) — STUB with the frozen contracts baked in.
foundation3 ships this file with:
- the **concrete, foundation-owned protocol constants + pure encoders** every side of the
system must agree on (the blink OOK protocol #4, the color-mode marker-key convention #1).
``synthetic.py`` renders the fixture markers with these, ``scripts/badge_selftest.py``
(lane K) imports this decoder path, and ``hardware/badge/protocol.h`` (lane K) duplicates
the constants with a documented copy-check. **One source of truth lives here.**
- a **stub** :func:`run_detect` that raises ``NotImplementedError``. Lane H fills the body
(and adds ``backend/tests/test_tracker_detect.py``); it must not change the constants or
the emitted ``marker_key`` conventions, only implement the detection logic.
Nothing in this module raises at import time — ``synthetic.py`` imports the constants and the
``encode_word`` / ``led_on`` helpers, so the import must stay side-effect-free and light
(no OpenCV at module scope; import ``cv2`` inside ``run_detect``).
================================================================================
FROZEN CONTRACTS (plan/30-phase6.md — copy, do not reinterpret)
================================================================================
Contract #1 — Marker hues (color mode)
--------------------------------------
Magenta ``#ff00ff`` and cyan ``#00ffff`` are the two supported fixture hues; real shoots may
configure others via ``FESTIVAL4D_MARKER_HUES`` (comma-separated hex). Detection converts to
HSV and gates on **saturation AND value before hue** (pitfall #2 — stage lighting lies about
color). A color blob's ``marker_key`` is ``"hue:<H>"`` where ``<H>`` is the **OpenCV hue**
(0..179) of the configured marker color, i.e. ``round(matplotlib_hue_degrees / 2)``:
magenta → ``hue:150``, cyan → ``hue:90``. Use :func:`color_marker_key` — never format the
string by hand.
Contract #2 — Schema (what the solver, not the detector, writes; here for context)
----------------------------------------------------------------------------------
``tracks(id INTEGER PK, marker_key TEXT NOT NULL, label TEXT, color TEXT, created_at TEXT)``
``track_points(id INTEGER PK, track_id FK, t_global_s FLOAT, x FLOAT, y FLOAT, z FLOAT,
quality FLOAT, views INTEGER)`` — points ordered by ``t_global_s``;
``views`` = cameras used (1 ⇒ ground-plane fallback); coordinates in **Three.js scene space**.
Contract #3 — detections.json (this module's OUTPUT — write via `db`? No: a JSON file)
--------------------------------------------------------------------------------------
Write ``config.DETECTIONS_JSON`` (``data/work/detections.json``) with EXACTLY this shape::
{"videos": {"<video_id>": [{"t_video_s": float, "marker_key": str,
"cx": float, "cy": float, "area": float,
"conf": float 0..1}]},
"generated_by": "festival4d.tracker_detect"}
``cx`` / ``cy`` are **normalized 0..1** (the bbox-annotation convention: ``cx = px / width``,
``cy = py / height``). ``area`` is the blob area in pixels. Detections at ``t_video`` (the
frame's local time) — the solver converts to ``t_global`` with the FROZEN
``config.t_global_from_video`` helper (pitfall #1). No markers / dark / markerless video →
a valid-empty ``{"videos": {}, "generated_by": ...}``, never a crash.
Contract #4 — Blink protocol (OOK, camera-decodable)
----------------------------------------------------
Bit period **133 ms** (≈4 frames @ 30 fps). One **word** = preamble ``11100`` + 6-bit ID
(MSB first) + even parity = **12 bits**, repeated continuously (~1.6 s/word). ID **63** is
reserved for the sync beacon. Decoding needs **≥24 fps** video and **≥1 full word** of
visibility. Decode by **integrating brightness per bit window using the video's real fps**
(``meta.fps``) — never by counting frames (pitfall #3). Emit ``marker_key = "code:<id>"``
via :func:`code_marker_key`. The constants below are the byte-identical source for
``hardware/badge/protocol.h`` (lane K).
Contract #5 — Ground plane (solver context)
-------------------------------------------
Synthetic ground is ``y = 0`` (scene space). Real projects estimate it as the
5th-percentile-y horizontal plane of the point cloud. Single-view fallback intersects the
pixel ray with this plane; such points get ``views = 1`` and ``quality ≤ 0.5``.
"""
from __future__ import annotations
import logging
import os
log = logging.getLogger("festival4d.tracker_detect")
# ---------------------------------------------------------------------------
# Contract #4 — blink OOK protocol. FROZEN. Byte-identical to hardware/badge/protocol.h.
# ---------------------------------------------------------------------------
BLINK_BIT_PERIOD_S: float = 0.133 # 133 ms bit period (≈4 frames @ 30 fps)
BLINK_PREAMBLE: tuple[int, ...] = (1, 1, 1, 0, 0) # start-of-word marker
BLINK_ID_BITS: int = 6 # payload width (0..63)
BLINK_WORD_BITS: int = len(BLINK_PREAMBLE) + BLINK_ID_BITS + 1 # +1 parity = 12
BLINK_BEACON_ID: int = 63 # reserved: the sync/calibration beacon
BLINK_MIN_FPS: float = 24.0 # decoder needs at least this frame rate
# ---------------------------------------------------------------------------
# Contract #1 — color mode marker hues. OpenCV hue is 0..179 (degrees / 2).
# ---------------------------------------------------------------------------
MAGENTA_OPENCV_HUE: int = 150 # #ff00ff
CYAN_OPENCV_HUE: int = 90 # #00ffff
DEFAULT_MARKER_HUES_HEX: tuple[str, ...] = ("#ff00ff", "#00ffff")
def encode_word(marker_id: int) -> list[int]:
"""The 12-bit OOK word for ``marker_id`` (contract #4). Pure; no I/O.
``preamble(11100) + 6-bit ID MSB-first + even-parity bit``. The parity bit is chosen so
the **whole word has an even number of 1-bits** (parity over preamble+ID). This is the
exact bit sequence a badge emits, ``synthetic.py`` renders, and lane H's decoder inverts.
>>> encode_word(5)
[1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 1, 1]
"""
mid = int(marker_id)
if not 0 <= mid < (1 << BLINK_ID_BITS):
raise ValueError(f"marker_id {marker_id} out of range 0..{(1 << BLINK_ID_BITS) - 1}")
id_bits = [(mid >> (BLINK_ID_BITS - 1 - i)) & 1 for i in range(BLINK_ID_BITS)]
body = list(BLINK_PREAMBLE) + id_bits
parity = sum(body) % 2 # even parity: word weight (incl. parity) is even
return body + [parity]
def led_on(t_seconds: float, marker_id: int, epoch: float = 0.0) -> bool:
"""Whether the badge LED for ``marker_id`` is lit at wall-clock time ``t_seconds``.
The word repeats continuously from ``epoch``; a single physical badge blinks in real
(global) time, so every camera that captures a frame at the same ``t_global`` sees the
same LED state. Used by the synthetic fixture to render marker B faithfully.
"""
bit_index = int((t_seconds - epoch) // BLINK_BIT_PERIOD_S)
word = encode_word(marker_id)
return bool(word[bit_index % BLINK_WORD_BITS])
def color_marker_key(opencv_hue: int) -> str:
"""The ``marker_key`` for a color blob of the given OpenCV hue (contract #1)."""
return f"hue:{int(opencv_hue)}"
def code_marker_key(marker_id: int) -> str:
"""The ``marker_key`` for a decoded blink ID (contract #4)."""
return f"code:{int(marker_id)}"
def marker_hues_hex() -> list[str]:
"""Configured color-mode hues as hex strings (``FESTIVAL4D_MARKER_HUES`` or the default)."""
env = os.environ.get("FESTIVAL4D_MARKER_HUES")
if env:
hues = [h.strip() for h in env.split(",") if h.strip()]
if hues:
return hues
return list(DEFAULT_MARKER_HUES_HEX)
# ---------------------------------------------------------------------------
# Lane H fills these. Signatures FROZEN.
# ---------------------------------------------------------------------------
def run_detect(sample_fps: float = 8.0) -> dict:
"""Detect markers in every video and write ``detections.json`` (spec M19, contract #3).
Lane H implements:
1. Sample frames per video with the ``frames.py`` machinery at ~``sample_fps`` fps
(document the exact rate; ~8 fps comfortably oversamples the 133 ms bit period).
2. **Color mode:** BGR→HSV; gate on saturation AND value *before* hue (pitfall #2);
connected components per configured hue (:func:`marker_hues_hex`); blob centroid →
normalized ``cx``/``cy``; ``marker_key = color_marker_key(<opencv_hue>)``.
3. **Blink mode:** track bright blobs across the sampled window; integrate brightness
per :data:`BLINK_BIT_PERIOD_S` window using the video's **real fps** (``meta.fps``,
pitfall #3); correlate against :data:`BLINK_PREAMBLE`; check even parity; emit
``marker_key = code_marker_key(<id>)``.
4. Write ``config.DETECTIONS_JSON`` in the contract-#3 shape. No markers / dark /
markerless → valid-empty ``{"videos": {}, "generated_by": "festival4d.tracker_detect"}``.
Returns a summary dict (per-video detection counts, decoded blink IDs). Never crash on a
markerless or unreadable video — log and continue.
"""
raise NotImplementedError(
"marker detection not implemented yet (lane H / M19) — fill tracker_detect.run_detect"
)

72
backend/festival4d/tracker_solve.py Normal file
View File

@ -0,0 +1,72 @@
"""Track solving (spec M20, lane H) — STUB with the frozen contract in the docstring.
foundation3 ships the stub; lane H fills :func:`run_solve` (and adds
``backend/tests/test_tracker_solve.py``). This is pure orchestration over the FROZEN
geometry primitives — it must NEVER inline pose math (read ``resolve.py`` first; it is the
exact per-timestep pattern), and it writes through the ``db`` track helpers only.
================================================================================
FROZEN CONTRACTS (plan/30-phase6.md — copy, do not reinterpret)
================================================================================
Input — ``config.DETECTIONS_JSON`` (contract #3, written by ``tracker_detect.run_detect``)::
{"videos": {"<video_id>": [{"t_video_s", "marker_key", "cx", "cy", "area", "conf"}]},
"generated_by": "festival4d.tracker_detect"}
``cx``/``cy`` are normalized 0..1; multiply by the video's width/height for pixel coords.
Output — the ``tracks`` / ``track_points`` tables (contract #2), via ``db`` helpers::
tracks(id, marker_key, label, color, created_at)
track_points(id, track_id, t_global_s, x, y, z, quality, views) # ordered by t_global_s
# coordinates in Three.js scene space; views = cameras used (1 ⇒ ground-plane fallback)
Algorithm (spec M20 — the parts that bite):
1. Load detections; group by ``marker_key``.
2. **Time alignment is the whole game** (pitfall #1): a detection is found at ``t_video``;
association happens at ``t_global``. Convert with the FROZEN
``config.t_global_from_video(t_video, offset_ms, drift_ppm)`` ONLY — never re-derive it.
3. Per time bucket (**≈0.25 s** of ``t_global``), collect same-``marker_key`` detections
across cameras at matching ``t_global``:
- **2+ views → triangulate** through ``geometry`` primitives exactly as ``resolve.py``
does (``geometry.ray_from_pixel`` per view + ``geometry.triangulate_rays``, or a
multi-ray least squares built on them). Record ``views`` and a ``quality`` derived
from the residual/gap.
- **1 view → ray ∩ ground plane** (contract #5): the synthetic ground is ``y = 0``;
real projects estimate the 5th-percentile-y horizontal plane of the point cloud and
record it in the result dict. Such points get ``views = 1`` and ``quality ≤ 0.5``.
4. Smooth each track (moving average or Catmull-Rom — your call; **document which**).
5. Solid color for hue markers, palette color for code markers.
6. Write via ``db.add_track`` + ``db.set_track_points``. **Re-solve replaces that marker's
track** (idempotent — never stack duplicates on repeated runs).
7. Degradation: no detections → zero tracks + a note; never crash.
``POST /api/tracks/solve`` and ``python -m festival4d track`` both run detect→solve
end-to-end once this and ``tracker_detect`` are filled.
"""
from __future__ import annotations
import logging
log = logging.getLogger("festival4d.tracker_solve")
# Poses are keyed every ~0.5 s; bucket association at 0.25 s (spec M20). Kept here so lane H
# and any test reference one number.
BUCKET_S: float = 0.25
GROUND_PLANE_Y: float = 0.0 # synthetic fixture ground (contract #5, scene space)
SINGLE_VIEW_MAX_QUALITY: float = 0.5 # ground-plane fallback quality cap (contract #5)
def run_solve() -> dict:
"""Solve detections into 3D friend tracks (spec M20, see module docstring).
Lane H implements the full detect-output → ``tracks``/``track_points`` pipeline described
above. Returns a summary dict (tracks written, per-track point counts + coverage, the
ground-plane used, and any degradation note). Idempotent: re-running replaces each
marker's track rather than stacking duplicates.
"""
raise NotImplementedError(
"track solving not implemented yet (lane H / M20) — fill tracker_solve.run_solve"
)

5
backend/tests/test_api.py
View File

@ -34,7 +34,10 @@ def test_manifest_shape(client):
data = client.get("/api/manifest").json()
# Exact-set lock: additive manifest fields are fine, but must be added here consciously
# (this assertion has already caught two silent drifts — has_splat and has_capture).
assert set(data) == {"videos", "t_global_max", "has_poses", "has_splat", "has_capture", "capsule"}
# phase 6 (foundation3 / M18): has_tracks added — see plan/CHANGE_REQUESTS.md CR-6.
assert set(data) == {"videos", "t_global_max", "has_poses", "has_splat", "has_capture",
"capsule", "has_tracks"}
assert data["has_tracks"] is False # synthetic build seeds no solved tracks (solver = lane H)
assert data["has_poses"] is True
assert data["capsule"] is False # live server is never a capsule (M17 bakes true)
assert data["has_capture"] is False # capture routes are opt-in (FESTIVAL4D_CAPTURE)

2
backend/tests/test_capsule.py
View File

@ -137,6 +137,8 @@ def test_bundle_structure(bundle):
out, summary = bundle
for rel in ("index.html", "serve.py", "assets/index.js",
"api/manifest", "api/events", "api/anchors",
# phase 6 (foundation3 / M18): friend tracks baked read-only — CR-6.
"api/tracks",
"api/beats", "api/pointcloud", "api/splat",
# the paths listing lives at paths/index.html so per-id files can
# coexist under the same URL prefix (see capsule.py)

351
backend/tests/test_tracks_foundation.py Normal file
View File

@ -0,0 +1,351 @@
"""foundation3 (phase 6 / M18) contract tests — the ground three lanes stand on.
Covers what foundation3 owns and freezes, NOT the lane implementations (detection = lane H's
test_tracker_detect.py; solving = test_tracker_solve.py; ribbons = lane J's live-browser
evidence):
A. tracks / track_points DB helpers (contract #2) — round-trips, replace + delete semantics.
B. /api/tracks routes (contract #2) + POST /api/tracks/solve degradation + manifest.has_tracks.
C. The frozen OOK blink protocol + color marker_key convention (contracts #1, #4) — pure,
no OpenCV. These constants are byte-identical to hardware/badge/protocol.h (lane K).
D. The synthetic fixture (M18 step 1): track_truth.json shape, marker paths, and — the one
that matters most — a projection ROUND-TRIP proving the rendered marker pixels
back-project to the ground-truth 3D point through the frozen geometry primitives. If this
passes, lane H's M20 recovery (median 3D error < 0.3) is achievable by construction.
Hermetic: no ffmpeg (geometry only), temp data dir from conftest. DB-touching tests clean up
after themselves so later modules see the tables as they found them.
"""
from __future__ import annotations
import math
import shutil
import numpy as np
import pytest
from festival4d import config, db, synthetic, tracker_detect
_HAS_FFMPEG = shutil.which("ffmpeg") is not None and shutil.which("ffprobe") is not None
# ===========================================================================
# A. DB helpers (contract #2)
# ===========================================================================
def _use_temp_db(tmp_path):
db.init_engine(tmp_path / "t.db")
db.reset_db()
def test_add_and_get_tracks_ordered_by_id(tmp_path):
_use_temp_db(tmp_path)
a = db.add_track("hue:150", label="Alice", color="#ff00ff")
b = db.add_track("code:5")
assert db.has_tracks() is True
tracks = db.get_tracks()
assert [t.id for t in tracks] == sorted(t.id for t in tracks) == [a.id, b.id]
assert tracks[0].marker_key == "hue:150" and tracks[0].label == "Alice"
assert tracks[1].label is None and tracks[1].color is None # optional fields default None
assert db.get_track(a.id).marker_key == "hue:150"
assert db.get_track(999999) is None
def test_has_tracks_false_on_empty(tmp_path):
_use_temp_db(tmp_path)
assert db.has_tracks() is False
assert db.get_tracks() == []
def test_set_track_points_replaces_and_orders(tmp_path):
_use_temp_db(tmp_path)
t = db.add_track("code:5")
# Insert out of order; helper must return them sorted by t_global_s (contract #2).
n = db.set_track_points(t.id, [
{"t_global_s": 2.0, "x": 1, "y": 2, "z": 3, "quality": 0.9, "views": 2},
{"t_global_s": 1.0, "x": 0, "y": 0, "z": 0, "quality": 0.4, "views": 1},
])
assert n == 2
pts = db.get_track_points(t.id)
assert [p.t_global_s for p in pts] == [1.0, 2.0]
assert pts[0].views == 1 and pts[0].quality == 0.4
assert pts[1].views == 2
# Re-solve replaces (idempotent — never stacks duplicates, per M20 contract).
db.set_track_points(t.id, [{"t_global_s": 5.0, "x": 9, "y": 9, "z": 9}])
pts = db.get_track_points(t.id)
assert len(pts) == 1 and pts[0].t_global_s == 5.0
assert pts[0].quality is None and pts[0].views is None # omitted optionals -> None
def test_set_track_points_unknown_track_raises(tmp_path):
_use_temp_db(tmp_path)
with pytest.raises(KeyError):
db.set_track_points(424242, [{"t_global_s": 0.0, "x": 0, "y": 0, "z": 0}])
def test_patch_track_partial_and_missing(tmp_path):
_use_temp_db(tmp_path)
t = db.add_track("hue:150", label="Old", color="#111111")
db.patch_track(t.id, label="New") # color untouched
got = db.get_track(t.id)
assert got.label == "New" and got.color == "#111111"
db.patch_track(t.id, color="#00ffff")
assert db.get_track(t.id).color == "#00ffff"
with pytest.raises(KeyError):
db.patch_track(424242, label="x")
def test_delete_track_removes_points(tmp_path):
_use_temp_db(tmp_path)
t = db.add_track("code:5")
db.set_track_points(t.id, [{"t_global_s": 0.0, "x": 0, "y": 0, "z": 0}])
assert db.delete_track(t.id) is True
assert db.get_track(t.id) is None
assert db.get_track_points(t.id) == [] # points cascade (SQLite doesn't by default)
assert db.delete_track(t.id) is False # already gone
# ===========================================================================
# B. /api/tracks routes (contract #2)
# ===========================================================================
@pytest.fixture(scope="module")
def client():
from fastapi.testclient import TestClient
from festival4d import api
with TestClient(api.app) as c:
yield c
@pytest.fixture()
def _api_db(client):
"""Point the API at a clean per-test db and restore the shared engine afterwards."""
import tempfile
from pathlib import Path
d = Path(tempfile.mkdtemp(prefix="tracks-api-"))
db.init_engine(d / "api.db")
db.reset_db()
yield
db.init_engine(config.DB_PATH)
def test_get_tracks_shape(client, _api_db):
t = db.add_track("hue:150", label="Alice", color="#ff00ff")
db.set_track_points(t.id, [
{"t_global_s": 1.0, "x": 0.1, "y": 1.5, "z": 2.0, "quality": 0.9, "views": 2},
{"t_global_s": 1.25, "x": 0.2, "y": 1.5, "z": 1.9, "quality": 0.5, "views": 1},
])
listing = client.get("/api/tracks").json()
assert len(listing) == 1
row = listing[0]
assert set(row) == {"id", "marker_key", "label", "color", "points"}
assert row["marker_key"] == "hue:150" and row["label"] == "Alice"
assert [p["t_global_s"] for p in row["points"]] == [1.0, 1.25] # ordered
assert set(row["points"][0]) == {"t_global_s", "x", "y", "z", "quality", "views"}
assert row["points"][1]["views"] == 1
def test_patch_and_delete_routes(client, _api_db):
t = db.add_track("code:5")
patched = client.patch(f"/api/tracks/{t.id}", json={"label": "Bob", "color": "#00ffff"}).json()
assert patched["label"] == "Bob" and patched["color"] == "#00ffff"
# partial patch leaves the other field
assert client.patch(f"/api/tracks/{t.id}", json={"color": "#fff"}).json()["label"] == "Bob"
assert client.patch("/api/tracks/999999", json={"label": "x"}).status_code == 404
assert client.delete(f"/api/tracks/{t.id}").json() == {"deleted": t.id}
assert client.get("/api/tracks").json() == []
assert client.delete(f"/api/tracks/{t.id}").status_code == 404
def test_solve_degrades_while_stubbed(client, _api_db):
# Lane H is a stub (NotImplementedError). The frozen route must NOT 500 — it returns a
# valid empty result + a note (the house pattern, mirrors POST /api/events/detect).
resp = client.post("/api/tracks/solve")
assert resp.status_code == 200
body = resp.json()
assert body["result"] is None
assert "not implemented" in body["note"].lower()
assert body["tracks"] == [] # no tracks solved yet
def test_manifest_has_tracks_reflects_db(client, _api_db):
assert client.get("/api/manifest").json()["has_tracks"] is False
db.add_track("hue:150")
assert client.get("/api/manifest").json()["has_tracks"] is True
# ===========================================================================
# C. Frozen blink OOK protocol + color marker_key (contracts #1, #4)
# ===========================================================================
def test_protocol_constants_match_spec():
assert tracker_detect.BLINK_BIT_PERIOD_S == 0.133
assert tracker_detect.BLINK_PREAMBLE == (1, 1, 1, 0, 0)
assert tracker_detect.BLINK_ID_BITS == 6
assert tracker_detect.BLINK_WORD_BITS == 12 # 5 preamble + 6 id + 1 parity
assert tracker_detect.BLINK_BEACON_ID == 63
assert tracker_detect.BLINK_MIN_FPS == 24.0
assert tracker_detect.MAGENTA_OPENCV_HUE == 150 # #ff00ff, hue 300°/2
assert tracker_detect.CYAN_OPENCV_HUE == 90 # #00ffff, hue 180°/2
def test_encode_word_id5_exact_and_even_parity():
word = tracker_detect.encode_word(5)
assert word == [1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 1, 1] # preamble + 000101 + parity
assert len(word) == 12
for mid in range(0, 64):
w = tracker_detect.encode_word(mid)
assert tuple(w[:5]) == tracker_detect.BLINK_PREAMBLE
assert sum(w) % 2 == 0 # even parity over the whole word
# 6-bit ID MSB-first recovers the value
recovered = sum(bit << (5 - i) for i, bit in enumerate(w[5:11]))
assert recovered == mid
def test_encode_word_rejects_out_of_range():
for bad in (-1, 64, 100):
with pytest.raises(ValueError):
tracker_detect.encode_word(bad)
def test_led_on_matches_encoded_word_windows():
# The LED state at bit-window k equals bit k of the repeating word (global time).
word = tracker_detect.encode_word(5)
period = tracker_detect.BLINK_BIT_PERIOD_S
for k in range(len(word) * 2): # two full words
t_mid = (k + 0.5) * period # sample mid-window
assert tracker_detect.led_on(t_mid, 5) == bool(word[k % len(word)])
def test_marker_key_helpers_and_hue_env(monkeypatch):
assert tracker_detect.color_marker_key(150) == "hue:150"
assert tracker_detect.code_marker_key(5) == "code:5"
monkeypatch.delenv("FESTIVAL4D_MARKER_HUES", raising=False)
assert tracker_detect.marker_hues_hex() == ["#ff00ff", "#00ffff"]
monkeypatch.setenv("FESTIVAL4D_MARKER_HUES", "#ff0000, #00ff00")
assert tracker_detect.marker_hues_hex() == ["#ff0000", "#00ff00"]
# ===========================================================================
# D. Synthetic fixture: track_truth shape, marker paths, projection round-trip
# ===========================================================================
def test_track_truth_shape_and_both_markers():
tt = synthetic.build_track_truth(0.0, 2.0, dt=0.05)
assert set(tt) >= {"markers"}
keys = {m["marker_key"] for m in tt["markers"]}
assert keys == {"hue:150", "code:5"} # marker A (color) + marker B (blink)
for m in tt["markers"]:
ts = [p["t_global_s"] for p in m["points"]]
assert ts == sorted(ts) # monotonically increasing
assert len(ts) == len(set(ts)) # no dup timesteps
assert set(m["points"][0]) == {"t_global_s", "x", "y", "z"} # Three.js scene space
def test_marker_paths_and_unknown_key():
# Marker A: a circle radius 2 at y=1.5 (contract, spec M18).
for tg in (0.0, 3.7, 11.2):
x, y, z = synthetic.marker_position("hue:150", tg)
assert y == pytest.approx(1.5)
assert math.hypot(x, z) == pytest.approx(2.0, abs=1e-9)
# Marker B follows a different path (never identical to A) so the two never lock.
assert synthetic.marker_position("code:5", 3.7) != synthetic.marker_position("hue:150", 3.7)
with pytest.raises(KeyError):
synthetic.marker_position("code:99", 0.0)
def test_marker_discs_are_small():
# Both discs together must be well under 2% of frame area (spec M18: ≤2%, pitfall #6).
w, h = config.SYNTH_VIDEO_W, config.SYNTH_VIDEO_H
r = synthetic._disc_radius_px(h)
two_discs = 2 * math.pi * r * r
assert two_discs / (w * h) < 0.02
def test_projection_round_trip_recovers_ground_truth():
"""THE foundation contract: rendered marker pixels back-project to the 3D truth.
For each marker at several t_global: project its ground-truth 3D point through every
camera's interpolated pose (exactly as render_marker_overlay draws the pixel), then treat
the in-frame pixels as a perfect detector's output and triangulate them back through the
FROZEN geometry primitives. Recovery must be ~exact on the noise-free fixture — this is
what makes lane H's M20 (median 3D error < 0.3) reachable by construction.
"""
from festival4d import geometry
w, h = config.SYNTH_VIDEO_W, config.SYNTH_VIDEO_H
fps, dur = float(config.SYNTH_FPS), config.SYNTH_DURATION_S
# Poses per camera, generated the same way build() does (no DB, no ffmpeg needed).
cam_poses = [synthetic.camera_track(i, dur, fps, w, h)
for i in range(len(config.SYNTH_OFFSETS_MS))]
worst = 0.0
checked = 0
for key in synthetic.MARKER_KEYS:
for t_global in (3.0, 6.5, 10.0, 14.25, 17.5): # inside every camera's valid span
truth = np.array(synthetic.marker_position(key, t_global))
rays = []
for i, offset_ms in enumerate(config.SYNTH_OFFSETS_MS):
t_video = config.t_video_from_global(t_global, offset_ms, 0.0)
q, t, intr = synthetic._interp_pose(cam_poses[i], t_video)
proj = synthetic.project_point(q, t, intr, truth)
if proj is None:
continue
px, py, _ = proj
if not (0.0 <= px < w and 0.0 <= py < h):
continue # off this camera's frame
fx, fy, cx, cy = intr
rays.append(geometry.ray_from_pixel(q, t, fx, fy, cx, cy, px, py))
assert len(rays) >= 2, f"{key} @ {t_global}s: need 2+ views, got {len(rays)}"
point, gap = geometry.triangulate_rays(rays[0][0], rays[0][1], rays[1][0], rays[1][1])
err = float(np.linalg.norm(point - truth))
worst = max(worst, err)
checked += 1
assert checked >= 8
# Noise-free synthetic geometry: recovery is essentially exact, far under the 0.3 M20 bar.
assert worst < 1e-3, f"worst round-trip error {worst:.2e} (>1e-3)"
@pytest.mark.skipif(not _HAS_FFMPEG, reason="ffmpeg/ffprobe required to render the fixture video")
def test_rendered_marker_a_is_the_dominant_magenta_blob(tmp_path):
"""The base-muting contract (BASE_SATURATION / BASE_BRIGHTNESS) actually renders a solvable
fixture: testsrc2's own solid magenta/cyan colour bars must NOT out-mass marker A's disc, or
lane H's colour detector locks onto a bar and M19/M20 fail. This emulates that detector —
largest saturated-magenta blob in a real rendered frame — and asserts it lands on the disc.
Guards against a future un-muting of the base swamping detection.
"""
import cv2
dur = 2.0
synthetic.build(base_dir=tmp_path, duration_s=dur, run_ffmpeg=True)
try:
w, h = config.SYNTH_VIDEO_W, config.SYNTH_VIDEO_H
poses = synthetic.camera_track(0, dur, float(config.SYNTH_FPS), w, h)
t_video = 1.0
q, t, intr = synthetic._interp_pose(poses, t_video)
t_global = config.t_global_from_video(t_video, config.SYNTH_OFFSETS_MS[0], 0.0)
proj = synthetic.project_point(q, t, intr,
synthetic.marker_position(synthetic.MARKER_A_KEY, t_global))
assert proj and 0 <= proj[0] < w and 0 <= proj[1] < h, "marker A not in cam0 frame at t=1s"
cap = cv2.VideoCapture(str(tmp_path / "raw" / "cam0.mp4"))
cap.set(cv2.CAP_PROP_POS_MSEC, t_video * 1000.0)
ok, img = cap.read()
cap.release()
assert ok, "could not read cam0 frame"
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
hh, ss, vv = hsv[:, :, 0].astype(int), hsv[:, :, 1], hsv[:, :, 2]
# OpenCV-hue magenta ≈150, gated on high saturation AND value (pitfall #2).
mask = (((hh >= 145) & (hh <= 155)) & (ss > 170) & (vv > 170)).astype(np.uint8)
n, _lab, stats, cent = cv2.connectedComponentsWithStats(mask, 8)
assert n > 1, "no saturated-magenta blob at all — marker A not rendered?"
biggest = 1 + int(np.argmax(stats[1:, cv2.CC_STAT_AREA]))
cx, cy = cent[biggest]
dist = math.hypot(cx - proj[0], cy - proj[1])
assert dist < 10.0, (
f"dominant magenta blob at ({cx:.0f},{cy:.0f}) is {dist:.0f}px from marker A's "
f"projected centre ({proj[0]:.0f},{proj[1]:.0f}) — base bars are swamping the disc"
)
finally:
db.init_engine(config.DB_PATH) # build() rebound the engine; restore the shared one

4
frontend/index.html
View File

@ -252,12 +252,16 @@
title="Moment FX: bursts and pulses on event markers (M15)">✨</button>
<button class="btn" id="btn-anchors" style="display:none"
title="Anchors & friend tags (M13)">⚓</button>
<button class="btn" id="btn-friends" style="display:none"
title="Friend tracks: ribbons, moving labels, follow-a-friend (M21/M22)">👣 Friends</button>
<input type="file" id="path-file" accept="application/json,.json" hidden />
</div>
<div id="scene-hint">drag to orbit · 1–9 snap to camera · K add keyframe · Esc detach</div>
<div id="annot-panel" class="write-ui"></div>
<!-- M13 anchor manager mounts here (lane F); hidden until anchorPanel.js is filled -->
<div id="anchor-panel" style="display:none"></div>
<!-- M21 friends panel mounts here (lane J); hidden until friendTracks.js is filled -->
<div id="friends-panel" style="display:none"></div>
</div>
<div id="grid-pane">
<div id="video-grid"></div>

41
frontend/src/friendTracks.js Normal file
View File

@ -0,0 +1,41 @@
// Friend tracks (spec M21 ribbons + moving labels + panel, M22 follow-a-friend cam; lane J).
// STUB shipped by foundation3 — lane J fills the body.
//
// main.js imports this, calls friendTracks.init() in buildUI (alongside the other lane
// modules) and friendTracks.update(state.tGlobal) every animation frame — the fx.js pattern
// (after transport.tick(), next to camPath.update / fx.update, before scene3d.update()).
// While stubbed, both are safe no-ops and the hidden #btn-friends button stays hidden; a
// broken or unfilled module degrades to no UI, exactly like the phase-5 stubs.
//
// Contract lane J must honor (frozen by foundation3):
// - state.tracks is loaded at boot by main.js when manifest.has_tracks, in the
// GET /api/tracks shape (contract #2):
// [{ id, marker_key, label, color,
// points: [{ t_global_s, x, y, z, quality, views }] }] // points sorted by t_global_s
// - Scene objects go through scene3d.addObject / scene3d.removeObject ONLY, and every
// helper is registered with scene3d.registerHelper so photo mode (M14) hides ribbons +
// labels. (Read anchorPanel.js for the panel style/UX; do not edit it.)
// - Rename / recolor / delete round-trip through PATCH / DELETE /api/tracks/{id}; those
// controls carry the `write-ui` class so the capsule (M17) hides them while ribbons +
// follow still work.
// - Follow-a-friend (M22) shares scene3d pathMode with camPath — mirror camPath's "someone
// else took the camera" contention checks exactly (pitfall #4), or two drivers fight.
// - update(tGlobal) MUST NEVER THROW: it runs inside the master loop and a throw kills
// playback. Guard the body and disable the module on error (fx.js does this).
export const friendTracks = {
ready: false,
/** Called once by main.js (buildUI). Lane J (M21): unhide #btn-friends, build the friends
* panel in #friends-panel, wire the toggle that renders ribbons + moving labels. */
init() {
// Stub: #btn-friends stays hidden until lane J fills M21. No-op is the safe degrade.
},
/** Called by main.js every animation frame with the master clock. Lane J (M21/M22):
* interpolate each track's position at tGlobal, move ribbons/labels, drive the chase cam.
* MUST NEVER THROW. */
update(_tGlobal) {
// Stub: no tracks rendered yet (M21/M22).
},
};

16
frontend/src/main.js
View File

@ -23,8 +23,10 @@ import { anchorPanel } from "./anchorPanel.js";
import { photoMode } from "./photoMode.js";
import { fx } from "./fx.js";
import { pathsStore } from "./pathsStore.js";
// Phase-6 lane module (stub until lane J fills it — M21 ribbons/panel, M22 follow-a-friend).
import { friendTracks } from "./friendTracks.js";
const laneModules = { director, exportVideo, anchorPanel, photoMode, fx, pathsStore };
const laneModules = { director, exportVideo, anchorPanel, photoMode, fx, pathsStore, friendTracks };
async function getJSON(path) {
const resp = await fetch(API_BASE + path);
@ -58,6 +60,7 @@ async function boot() {
link.style.display = "";
}
state.hasSplat = !!manifest.has_splat;
state.hasTracks = !!manifest.has_tracks; // phase 6 (M18): any solved friend track
// Capsule (M17): a static baked bundle has no write API — hide everything .write-ui.
state.capsule = !!manifest.capsule;
document.body.classList.toggle("capsule", state.capsule);
@ -71,6 +74,16 @@ async function boot() {
state.anchors = anchors;
state.events = events;
// Friend tracks (M18): only fetched when the backend reports solved tracks, so a project
// without them pays nothing. Failure is non-fatal — tracks are an overlay, not core.
if (state.hasTracks) {
try {
state.tracks = await getJSON("/api/tracks");
} catch (err) {
console.warn("[festival4d] friend tracks failed to load", err);
}
}
if (state.hasPoses) {
const lists = await Promise.all(
state.videos.map((v) => getJSON(`/api/videos/${v.id}/poses`))
@ -267,6 +280,7 @@ function startLoop() {
transport.tick();
for (const cell of cells) drawOverlay(cell);
fx.update(state.tGlobal); // M15: playhead-crossing effects (no-op while stubbed)
friendTracks.update(state.tGlobal); // M21/M22: move ribbons/labels + chase cam (no-op while stubbed)
camPath.update(state.tGlobal); // M9: drive the viewer camera before the scene renders
scene3d.update();
timeline.update(state.tGlobal);

2
frontend/src/state.js
View File

@ -26,6 +26,8 @@ export const state = {
poses: {}, // id -> [{frame_idx, t_video_s, q:[w,x,y,z], t:[x,y,z], intrinsics, registered}]
anchors: [], // [{id, label, x, y, z, color}]
events: [], // [{id, t_global_s, duration_s, event_type, confidence, description, source}]
hasTracks: false, // manifest.has_tracks (M18): any solved friend track => load + render tracks
tracks: [], // [{id, marker_key, label, color, points:[{t_global_s,x,y,z,quality,views}]}] (M18)
// transport / playback
tGlobal: 0,

20
plan/CHANGE_REQUESTS.md
View File

@ -111,3 +111,23 @@ Instead append an entry here and work around it locally; the integration agent a
- **Decision:** **APPLIED — RATIFIED by coordinator at merge** (2026-07-17). Same
supersession pattern as CR-1/CR-4; the shape-locked director test is stronger than the
stub assertion it replaces. Suite 169 → **189** on the merged tree.
### CR-6 — foundation3 — 2026-07-17 — Conscious test-lock updates for the two additive M18 contract fields
- **File:** `backend/tests/test_api.py::test_manifest_shape` (manifest exact-set lock) and
`backend/tests/test_capsule.py::test_bundle_structure` (baked-paths list).
- **Problem:** Not a supersession — these are the two *deliberate* lock edits M18 mandates
(spec `plan/30-phase6.md` steps 3 & 7). foundation3 adds an additive manifest field
`has_tracks` (drives the frontend loading friend tracks) and an additive capsule bake
`api/tracks` (so ribbons + follow work zero-backend). The manifest key-set lock is
intentionally exact ("has already caught two silent drifts"), and the capsule
bundle-structure test enumerates baked paths, so both must be updated **consciously in the
same commit** as the source change — the spec calls for exactly this + a CR note.
- **Proposed change:** (a) add `"has_tracks"` to the `test_manifest_shape` exact-set and assert
it is `False` on the live synthetic build (no solver has run — solving is lane H); (b) add
`"api/tracks"` to the `test_bundle_structure` baked-paths tuple so the test now asserts the
new read-only bake exists. No behavior removed; both are stronger post-edit.
- **Workaround in place:** N/A — these are foundation-sanctioned edits to test files (not to a
frozen *contract* file: `db.py`/`api.py`/`capsule.py` were extended additively per M18, and
`synthetic.py`'s marker code is a coordinator-sanctioned M18 edit). Applied directly in the
foundation3 commit per the house pattern; flagged here for integration3 to ratify.
- **Decision:** (integration3 fills this: applied | rejected — rationale)

57
plan/status/foundation3.md Normal file
View File

@ -0,0 +1,57 @@
# Status — foundation3 (Phase 6a / M18)
## Round 6 — 2026-07-17 — STATUS: ready_to_merge
**Directives acknowledged:** round 6 of plan/DIRECTIVES.md. Directive #1: `foundation3` only,
branch `foundation3`, push — do NOT merge. Contracts correctness beats speed; keep 189 green.
**Acceptance checklist** (M18, plan/30-phase6.md §M18):
- [x] 1. Synthetic fixture: two moving markers (A magenta `hue:150`, B blink `code:5`)
composited through each camera's real pose projection; `data/work/track_truth.json`
built from the SAME 3D paths (`synthetic.build_track_truth`, Three.js scene space).
`synthetic.project_point` is the exact inverse of `geometry.ray_from_pixel`. Base
`testsrc2` is saturation/brightness-muted (`BASE_SATURATION`/`BASE_BRIGHTNESS`) so its
own magenta/cyan colour bars don't swamp lane H's colour detector — the discs are the
only high-sat magenta/cyan + only bright small blobs (verified by a real-render test).
- [x] 2. DB: `Track`/`TrackPoint` tables (contract #2) + `add_track`/`get_tracks`/`get_track`/
`get_track_points`/`set_track_points`/`patch_track`/`delete_track`/`has_tracks`.
- [x] 3. API: GET /api/tracks, PATCH/DELETE /api/tracks/{id}, POST /api/tracks/solve (degrades
to valid empty + note while lane H stubbed), manifest `has_tracks`. Key-set lock edit → CR-6.
- [x] 4. CLI: `python -m festival4d track [--detect-only|--solve-only]` → lane-H stubs.
- [x] 5. Backend stubs: `tracker_detect.py`/`tracker_solve.py` — `NotImplementedError` + full
contracts #1–#5 in docstrings; foundation-owned frozen constants (blink OOK, hue keys)
live in `tracker_detect` as the single source of truth (`synthetic` imports them).
- [x] 6. Frontend: `friendTracks.js` stub (never-throws, fx.js pattern) wired into `main.js`
loop; hidden `#btn-friends` + `#friends-panel`; `state.tracks` loaded when `has_tracks`.
`npm run build` clean (27 modules, 851 ms).
- [x] 7. Capsule: `capsule.py` bakes `api/tracks` (additive). Bundle-structure test edit → CR-6.
- [x] 8. Tests: `test_tracks_foundation.py` (20) — DB helpers, routes, solve degradation,
manifest, frozen protocol/encoders, track_truth shape, the **projection round-trip**, and
a real-ffmpeg render test proving marker A is the dominant magenta blob after base-muting.
**Done this round:** all 8 steps land. Evidence:
- Full suite **209 passed, exit 0** (`uv run pytest -q`) — the 14 pre-existing files sum to
exactly 189 (floor held), + 20 new in `test_tracks_foundation.py` (all passed, none skipped).
- `test_projection_round_trip_recovers_ground_truth`: projects both markers' 3D truth through
every camera pose, back-projects via frozen `ray_from_pixel`, triangulates via
`triangulate_rays` — **worst recovery error < 1e-3** across ≥8 samples in 2+ views. Lane H's
M20 target (median 3D error < 0.3) is reachable by construction (~300× margin).
- `encode_word(5) == [1,1,1,0,0,0,0,0,1,0,1,1]`; all 64 IDs preamble+parity+MSB-recover checked.
- Frontend production build clean; manifest `has_tracks` reflects the DB live.
- CR-6 filed (plan/CHANGE_REQUESTS.md): the two sanctioned M18 test-lock edits (manifest key-set,
capsule bundle paths) — additive, flagged for integration3 ratification.
**Recovery note (2026-07-17):** a prior foundation3 attempt left M18 half-done and **uncommitted**
in a stale locked worktree — steps 2/3/4/5/7 (DB/API/CLI/stubs/capsule) + the `synthetic.py`
marker code present, but step 1's ground-truth never validated, steps 6/8 (frontend/tests) not
started, suite never run. This session salvaged it: committed the WIP as checkpoint `6a703c3`,
then completed steps 1/6/8 and verified every inherited contract choice against the spec. The
one real bug found: the base `testsrc2` visual carried solid magenta+cyan bars (the marker hues)
that would have swamped lane H's colour detector — fixed by muting base saturation/brightness
and locked with a real-render regression test (marker A dominant-blob within 2.5 px, 24/24
frames). No work was redone. Do NOT merge — coordinator reviews + merges (DIRECTIVES R6 #3).
**Blockers / questions for coordinator:** none. Ready for review + merge.
**Next:** coordinator reviews `git diff main..foundation3` + this file, merges to `main`.
After merge, lanes H / J / K launch in parallel worktrees (DIRECTIVES R6 #2).