foundation2 (phase 5a): contracts, stubs, hooks & UI wiring for lanes E/F/G
- DB: paths table (contract #3) + helpers; patch_anchor (label/color) - API: GET /api/beats, POST /api/director (valid-empty degradation), /api/paths CRUD (validated POST -> 422), PATCH /api/anchors/{id}; manifest gains capsule:false (test_api.py key-set lock updated consciously) - CLI: features | direct | capsule subcommands -> lane stubs (exit-2 degradation) - Backend stubs with frozen contracts in docstrings: audio_features, director, capsule - Frontend: six lane stub modules imported from main.js; all phase-5 buttons pre-wired hidden in index.html; write-ui/body.capsule read-only mode; __F4D_API_BASE__ runtime override in state.js - Hooks (contract #6): transport.captureAudioStream/releaseAudioStream, scene3d.focusOn, scene3d.setHelpersVisible (+registerHelper; camPath gizmos) - Suite 121 -> 127 passed; npm build clean; live-browser verified (status file) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
545f7a347d
commit
51650c728e
@ -12,6 +12,17 @@ Serves the synthetic project fully so lane C can treat this API as finished:
|
||||
POST /api/events/detect {t_global_s?, window_s?} -> runs M7 detection
|
||||
POST /api/annotations {video_id, t_video_s, bbox:[x0,y0,x1,y1]} -> {anchor_id?, point?}
|
||||
|
||||
Phase 5 (foundation2; shapes frozen in plan/20-phase5.md contracts):
|
||||
|
||||
GET /api/beats -> beats.json body (contract #1); 404 when not yet analyzed
|
||||
POST /api/director {top_n?, lead_s?} -> camPath JSON (contract #2); while lane E
|
||||
is stubbed, degrades to a VALID empty path + "note"
|
||||
GET /api/paths -> [{id, name, created_at}]
|
||||
GET /api/paths/{id} -> {id, name, created_at, json: <parsed camPath JSON>}
|
||||
POST /api/paths {name, json: <camPath JSON as text>} (validated -> 422)
|
||||
DELETE /api/paths/{id}
|
||||
PATCH /api/anchors/{id} {label?, color?} -> updated anchor dict
|
||||
|
||||
Source videos in ``data/raw`` are mounted at ``/media`` via Starlette ``StaticFiles``,
|
||||
which supports HTTP Range (required for ``<video>`` seeking; verify ``curl -H "Range:
|
||||
bytes=0-100"`` -> 206). Stubbed lane entrypoints (events detection, M8 annotation
|
||||
@ -91,6 +102,22 @@ class EventPatch(BaseModel):
|
||||
confidence: float | None = None
|
||||
|
||||
|
||||
class AnchorPatch(BaseModel):
|
||||
label: str | None = None
|
||||
color: str | None = None
|
||||
|
||||
|
||||
class DirectorIn(BaseModel):
|
||||
top_n: int = 8
|
||||
lead_s: float = 2.0
|
||||
|
||||
|
||||
class PathIn(BaseModel):
|
||||
# Wire key is "json" (contract #3); aliased because `json` shadows a BaseModel attr.
|
||||
name: str
|
||||
path_json: str = Field(alias="json")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Serialization helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
@ -156,6 +183,9 @@ def manifest() -> dict:
|
||||
"has_capture": capture.capture_enabled(),
|
||||
"has_poses": db.has_poses(),
|
||||
"has_splat": config.SPLAT_PLY.exists(),
|
||||
# Always False from the live server; the capsule (M17) bakes a manifest with true,
|
||||
# which makes the frontend hide all write UI (phase-5 contract #5).
|
||||
"capsule": False,
|
||||
}
|
||||
|
||||
|
||||
@ -333,6 +363,102 @@ def _existing_event_anchor_id(event_id: int) -> int | None:
|
||||
return None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 5 (foundation2). Shapes are frozen in plan/20-phase5.md "Contracts".
|
||||
# ---------------------------------------------------------------------------
|
||||
@app.get("/api/beats")
|
||||
def get_beats() -> dict:
|
||||
"""The beats.json analysis (contract #1). 404 until `python -m festival4d features` runs."""
|
||||
import json
|
||||
|
||||
if not config.BEATS_JSON.exists():
|
||||
raise HTTPException(status_code=404, detail="no beat analysis (run `python -m festival4d features`)")
|
||||
return json.loads(config.BEATS_JSON.read_text())
|
||||
|
||||
|
||||
@app.post("/api/director")
|
||||
def run_director(payload: DirectorIn) -> dict:
|
||||
"""Auto-director (M11) -> camPath JSON. Degrades to a valid EMPTY path while stubbed,
|
||||
so the response always round-trips through camPath.fromJSON (contract #2)."""
|
||||
from festival4d import director
|
||||
|
||||
try:
|
||||
return director.generate_path(top_n=payload.top_n, lead_s=payload.lead_s)
|
||||
except NotImplementedError as exc:
|
||||
log.info("auto-director not implemented yet: %s", exc)
|
||||
return {"version": 1, "keyframes": [], "note": "auto-director not implemented yet (lane E / M11)"}
|
||||
|
||||
|
||||
def _validate_campath(text: str) -> None:
|
||||
"""422 unless ``text`` is the frozen camPath JSON (contract #2). Lane G owns hardening."""
|
||||
import json
|
||||
|
||||
try:
|
||||
data = json.loads(text)
|
||||
except ValueError:
|
||||
raise HTTPException(status_code=422, detail="path json is not valid JSON")
|
||||
ok = (
|
||||
isinstance(data, dict)
|
||||
and data.get("version") == 1
|
||||
and isinstance(data.get("keyframes"), list)
|
||||
and all(
|
||||
isinstance(k, dict)
|
||||
and isinstance(k.get("t_global"), (int, float))
|
||||
and isinstance(k.get("pos"), list) and len(k["pos"]) == 3
|
||||
and isinstance(k.get("quat"), list) and len(k["quat"]) == 4
|
||||
and isinstance(k.get("fov"), (int, float))
|
||||
for k in data.get("keyframes", [])
|
||||
)
|
||||
)
|
||||
if not ok:
|
||||
raise HTTPException(status_code=422, detail="not a camPath JSON (see plan/20-phase5.md contract #2)")
|
||||
|
||||
|
||||
def _path_dict(p, with_json: bool = False) -> dict:
|
||||
import json
|
||||
|
||||
d = {"id": p.id, "name": p.name, "created_at": p.created_at}
|
||||
if with_json:
|
||||
d["json"] = json.loads(p.path_json)
|
||||
return d
|
||||
|
||||
|
||||
@app.get("/api/paths")
|
||||
def get_paths() -> list[dict]:
|
||||
return [_path_dict(p) for p in db.get_paths()]
|
||||
|
||||
|
||||
@app.get("/api/paths/{path_id}")
|
||||
def get_path(path_id: int) -> dict:
|
||||
p = db.get_path(path_id)
|
||||
if p is None:
|
||||
raise HTTPException(status_code=404, detail=f"no path with id={path_id}")
|
||||
return _path_dict(p, with_json=True)
|
||||
|
||||
|
||||
@app.post("/api/paths")
|
||||
def create_path(payload: PathIn) -> dict:
|
||||
_validate_campath(payload.path_json)
|
||||
return _path_dict(db.add_path(payload.name, payload.path_json), with_json=True)
|
||||
|
||||
|
||||
@app.delete("/api/paths/{path_id}")
|
||||
def delete_path(path_id: int) -> dict:
|
||||
if not db.delete_path(path_id):
|
||||
raise HTTPException(status_code=404, detail=f"no path with id={path_id}")
|
||||
return {"deleted": path_id}
|
||||
|
||||
|
||||
@app.patch("/api/anchors/{anchor_id}")
|
||||
def patch_anchor(anchor_id: int, payload: AnchorPatch) -> dict:
|
||||
"""Rename / recolor an anchor in place (M13, contract #4)."""
|
||||
try:
|
||||
anchor = db.patch_anchor(anchor_id, label=payload.label, color=payload.color)
|
||||
except KeyError:
|
||||
raise HTTPException(status_code=404, detail=f"no anchor with id={anchor_id}")
|
||||
return _anchor_dict(anchor)
|
||||
|
||||
|
||||
@app.delete("/api/anchors/{anchor_id}")
|
||||
def delete_anchor(anchor_id: int) -> dict:
|
||||
"""Remove an anchor (M8 / Phase 4). Unlinks any annotations that resolved to it."""
|
||||
|
||||
25
backend/festival4d/audio_features.py
Normal file
25
backend/festival4d/audio_features.py
Normal file
@ -0,0 +1,25 @@
|
||||
"""Audio features: beats & onsets (spec M10, lane E). STUB — lane E fills the body.
|
||||
|
||||
Contract (frozen at foundation2 merge — phase-5 contract #1):
|
||||
|
||||
:func:`run_features` loads the **reference** video's ingest WAV (16 kHz mono from
|
||||
``config.AUDIO_DIR``; the reference timeline *is* ``t_global``), runs beat/tempo tracking
|
||||
and onset-strength detection (librosa is already a dependency), and writes
|
||||
``config.BEATS_JSON``::
|
||||
|
||||
{"tempo_bpm": float | null,
|
||||
"beats_s": [float, ...],
|
||||
"onsets": [{"t_global_s": float, "strength": float 0..1}, ...],
|
||||
"generated_by": "festival4d.audio_features"}
|
||||
|
||||
All times are ``t_global`` seconds. Quiet / short / beatless audio must produce a
|
||||
*valid, empty* result (``tempo_bpm: null``, empty lists) — never a crash. Returns a
|
||||
small summary dict for the CLI log. ``GET /api/beats`` serves the written file.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
|
||||
def run_features() -> dict:
|
||||
"""Analyze the reference soundtrack and write ``data/work/beats.json`` (see module doc)."""
|
||||
raise NotImplementedError("audio features (lane E / M10)")
|
||||
28
backend/festival4d/capsule.py
Normal file
28
backend/festival4d/capsule.py
Normal file
@ -0,0 +1,28 @@
|
||||
"""Memory capsule: zero-backend shareable export (spec M17, lane G). STUB — lane G fills.
|
||||
|
||||
Contract: :func:`build_capsule` writes a self-contained bundle to ``out_dir`` (default
|
||||
``config.CAPSULE_DIR``):
|
||||
|
||||
- copy ``frontend/dist`` (error clearly if missing — tell the user to ``npm run build``);
|
||||
- copy media from ``config.RAW_DIR``;
|
||||
- bake every read-only API response to files at the *same relative paths*
|
||||
(``api/manifest``, ``api/events``, ``api/anchors``, ``api/beats``,
|
||||
``api/videos/{id}/poses``, ``api/pointcloud``, ``api/splat``, ``api/audio/{id}`` —
|
||||
extracting audio via ``ingest.extract_audio_hq`` if not yet cached);
|
||||
- inject ``<script>window.__F4D_API_BASE__=""</script>`` into the copied ``index.html``;
|
||||
- set ``"capsule": true`` in the baked manifest (the frontend hides all write UI);
|
||||
- ship a stdlib ``serve.py`` in the bundle root that serves with HTTP Range support
|
||||
(video seeking needs 206s; plain ``http.server`` can't — pitfall #4).
|
||||
|
||||
Skip what doesn't exist (no splat -> no ``api/splat``) — degrade, don't block.
|
||||
Returns a summary dict (bundle path, file count, bytes) for the CLI log.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def build_capsule(out_dir: str | Path | None = None) -> dict:
|
||||
"""Build the static shareable bundle (see module doc)."""
|
||||
raise NotImplementedError("memory capsule (lane G / M17)")
|
||||
@ -4,7 +4,8 @@ FROZEN after foundation. Lanes fill in the bodies of the functions this dispatch
|
||||
to (in ``ingest.py``, ``audio_sync.py``, ``sfm.py``, ``events_ai.py``); they never
|
||||
edit this file or ``api.py``.
|
||||
|
||||
Subcommands: ``synthetic | ingest | sync | reconstruct | events | serve``.
|
||||
Subcommands: ``synthetic | ingest | sync | reconstruct | events | features | direct |
|
||||
capsule | serve``.
|
||||
|
||||
Every subcommand dispatches to a single lane entrypoint. While a lane is still a stub,
|
||||
its entrypoint raises :class:`NotImplementedError`; we catch that here and print a clear
|
||||
@ -57,6 +58,32 @@ def _cmd_events(args: argparse.Namespace) -> int:
|
||||
return 0
|
||||
|
||||
|
||||
def _cmd_features(args: argparse.Namespace) -> int:
|
||||
from festival4d import audio_features
|
||||
|
||||
result = audio_features.run_features()
|
||||
log.info("features: %s", result)
|
||||
return 0
|
||||
|
||||
|
||||
def _cmd_direct(args: argparse.Namespace) -> int:
|
||||
import json
|
||||
|
||||
from festival4d import director
|
||||
|
||||
path = director.generate_path(top_n=args.top_n, lead_s=args.lead_s)
|
||||
print(json.dumps(path, indent=2))
|
||||
return 0
|
||||
|
||||
|
||||
def _cmd_capsule(args: argparse.Namespace) -> int:
|
||||
from festival4d import capsule
|
||||
|
||||
result = capsule.build_capsule(out_dir=args.out)
|
||||
log.info("capsule: %s", result)
|
||||
return 0
|
||||
|
||||
|
||||
def _cmd_serve(args: argparse.Namespace) -> int:
|
||||
import uvicorn
|
||||
|
||||
@ -97,6 +124,20 @@ def build_parser() -> argparse.ArgumentParser:
|
||||
help="window width in seconds when --t-global-s is given")
|
||||
p_evt.set_defaults(func=_cmd_events)
|
||||
|
||||
p_feat = sub.add_parser("features", help="beat/onset analysis of the reference audio (lane E / M10)")
|
||||
p_feat.set_defaults(func=_cmd_features)
|
||||
|
||||
p_dir = sub.add_parser("direct", help="auto-director: events -> camPath JSON on stdout (lane E / M11)")
|
||||
p_dir.add_argument("--top-n", dest="top_n", type=int, default=8,
|
||||
help="number of top-confidence events to cover (default 8)")
|
||||
p_dir.add_argument("--lead-s", dest="lead_s", type=float, default=2.0,
|
||||
help="seconds of lead-in before each event (default 2.0)")
|
||||
p_dir.set_defaults(func=_cmd_direct)
|
||||
|
||||
p_cap = sub.add_parser("capsule", help="bake a zero-backend shareable bundle (lane G / M17)")
|
||||
p_cap.add_argument("--out", default=None, help="output dir (default data/capsule/)")
|
||||
p_cap.set_defaults(func=_cmd_capsule)
|
||||
|
||||
p_srv = sub.add_parser("serve", help="run the FastAPI app (M3)")
|
||||
p_srv.add_argument("--host", default=config.API_HOST)
|
||||
p_srv.add_argument("--port", type=int, default=config.API_PORT)
|
||||
|
||||
@ -40,6 +40,8 @@ FRAMES_DIR = WORK_DIR / "frames" # sampled JPEGs for SfM
|
||||
COLMAP_DIR = WORK_DIR / "colmap" # COLMAP workspace
|
||||
DB_PATH = DATA_DIR / "project.db" # SQLite (gitignored)
|
||||
|
||||
BEATS_JSON = WORK_DIR / "beats.json" # beat/onset analysis (lane E / M10, /api/beats)
|
||||
CAPSULE_DIR = DATA_DIR / "capsule" # default output of `python -m festival4d capsule` (M17)
|
||||
POINTS_PLY = WORK_DIR / "points.ply" # reconstructed / synthetic point cloud
|
||||
SPLAT_PLY = WORK_DIR / "splat.ply" # optional 3DGS splat (trained externally, e.g. via MODELBEAST — see docs/modelbeast-crossover.md); viewer prefers it over the point cloud
|
||||
SYNC_JSON = WORK_DIR / "sync.json" # exported sync solution (lane A)
|
||||
|
||||
@ -12,6 +12,7 @@ Schema (spec §2)::
|
||||
anchors(id, label, x,y,z, color)
|
||||
events(id, t_global_s, duration_s, event_type, confidence, description, source)
|
||||
annotations(id, video_id FK, t_video_s, x0,y0,x1,y1, resolved_anchor_id FK NULL)
|
||||
paths(id, name, created_at, path_json) # phase 5 (M16): saved camPath JSON, as text
|
||||
|
||||
Engine management: :func:`init_engine` (re)binds the module to a SQLite file. It defaults
|
||||
to ``config.DB_PATH`` but tests point it at a temp file. Helpers open and commit their own
|
||||
@ -129,6 +130,19 @@ class Annotation(Base):
|
||||
)
|
||||
|
||||
|
||||
class SavedPath(Base):
|
||||
"""A saved keyframed camera path (phase 5 contract #3). ``path_json`` is the frozen
|
||||
camPath JSON (``{"version": 1, "keyframes": [...]}``) stored verbatim as text —
|
||||
the DB never interprets it beyond POST-time validation in the API layer."""
|
||||
|
||||
__tablename__ = "paths"
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
||||
name: Mapped[str] = mapped_column(String, nullable=False)
|
||||
created_at: Mapped[str] = mapped_column(String, nullable=False)
|
||||
path_json: Mapped[str] = mapped_column(String, nullable=False)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Engine / session management
|
||||
# ---------------------------------------------------------------------------
|
||||
@ -350,6 +364,24 @@ def update_anchor(
|
||||
return anchor
|
||||
|
||||
|
||||
def patch_anchor(anchor_id: int, label: str | None = None, color: str | None = None) -> Anchor:
|
||||
"""Rename / recolor an anchor without moving it (phase 5 contract #4, M13).
|
||||
|
||||
Only provided fields change. Raises ``KeyError`` if the id is unknown.
|
||||
"""
|
||||
with session_scope() as s:
|
||||
anchor = s.get(Anchor, anchor_id)
|
||||
if anchor is None:
|
||||
raise KeyError(f"no anchor with id={anchor_id}")
|
||||
if label is not None:
|
||||
anchor.label = label
|
||||
if color is not None:
|
||||
anchor.color = color
|
||||
s.flush()
|
||||
s.refresh(anchor)
|
||||
return anchor
|
||||
|
||||
|
||||
def delete_anchor(anchor_id: int) -> bool:
|
||||
"""Delete an anchor, first unlinking any annotations that resolved to it.
|
||||
|
||||
@ -444,6 +476,38 @@ def update_event(
|
||||
return event
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Saved camera paths (phase 5 / M16)
|
||||
# ---------------------------------------------------------------------------
|
||||
def add_path(name: str, path_json: str) -> SavedPath:
|
||||
with session_scope() as s:
|
||||
path = SavedPath(name=name, path_json=path_json, created_at=_now_iso())
|
||||
s.add(path)
|
||||
s.flush()
|
||||
s.refresh(path)
|
||||
return path
|
||||
|
||||
|
||||
def get_paths() -> list[SavedPath]:
|
||||
with session_scope() as s:
|
||||
return list(s.scalars(select(SavedPath).order_by(SavedPath.id)))
|
||||
|
||||
|
||||
def get_path(path_id: int) -> SavedPath | None:
|
||||
with session_scope() as s:
|
||||
return s.get(SavedPath, path_id)
|
||||
|
||||
|
||||
def delete_path(path_id: int) -> bool:
|
||||
"""Delete a saved path. Returns ``True`` if a row was deleted, ``False`` if unknown."""
|
||||
with session_scope() as s:
|
||||
path = s.get(SavedPath, path_id)
|
||||
if path is None:
|
||||
return False
|
||||
s.delete(path)
|
||||
return True
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Annotations
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
25
backend/festival4d/director.py
Normal file
25
backend/festival4d/director.py
Normal file
@ -0,0 +1,25 @@
|
||||
"""Auto-director: events -> keyframed camera path (spec M11, lane E). STUB — lane E fills.
|
||||
|
||||
Contract (frozen at foundation2 merge — phase-5 contract #2): :func:`generate_path`
|
||||
returns a **camPath-compatible** dict::
|
||||
|
||||
{"version": 1,
|
||||
"keyframes": [{"t_global": float, "pos": [x, y, z], "quat": [x, y, z, w], "fov": float}]}
|
||||
|
||||
Poses are **Three.js scene space**, quaternion order **[x, y, z, w]** — convert from the
|
||||
COLMAP rows via :func:`festival4d.geometry.colmap_to_threejs`, never inline (pitfall #1).
|
||||
|
||||
Deterministic v1 rules — no ML: take the top-N events by confidence (ties -> earlier);
|
||||
for each, choose the registered camera with a pose nearest the event time; keyframes at
|
||||
``t_event - lead_s`` and ``t_event + duration``; when ``config.BEATS_JSON`` exists, snap
|
||||
keyframe times to the nearest beat (no beats file -> no snapping — degrade, don't block);
|
||||
FOV from that camera's intrinsics (``2 * atan(height / (2 * fy))`` in degrees, the same
|
||||
formula as ``scene3d.snapTo``).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
|
||||
def generate_path(top_n: int = 8, lead_s: float = 2.0) -> dict:
|
||||
"""Build a camPath JSON dict from the detected events (see module doc)."""
|
||||
raise NotImplementedError("auto-director (lane E / M11)")
|
||||
@ -34,8 +34,9 @@ def test_manifest_shape(client):
|
||||
data = client.get("/api/manifest").json()
|
||||
# Exact-set lock: additive manifest fields are fine, but must be added here consciously
|
||||
# (this assertion has already caught two silent drifts — has_splat and has_capture).
|
||||
assert set(data) == {"videos", "t_global_max", "has_poses", "has_splat", "has_capture"}
|
||||
assert set(data) == {"videos", "t_global_max", "has_poses", "has_splat", "has_capture", "capsule"}
|
||||
assert data["has_poses"] is True
|
||||
assert data["capsule"] is False # live server is never a capsule (M17 bakes true)
|
||||
assert data["has_capture"] is False # capture routes are opt-in (FESTIVAL4D_CAPTURE)
|
||||
assert len(data["videos"]) == 3
|
||||
v = data["videos"][0]
|
||||
|
||||
107
backend/tests/test_phase5_api.py
Normal file
107
backend/tests/test_phase5_api.py
Normal file
@ -0,0 +1,107 @@
|
||||
"""Phase-5 foundation2 contract tests: the new route shapes lanes E/F/G depend on
|
||||
(plan/20-phase5.md "Contracts"). Lane G deepens paths coverage in test_paths.py;
|
||||
lane E owns the real beats/director behavior tests.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import shutil
|
||||
|
||||
import pytest
|
||||
|
||||
pytestmark = pytest.mark.skipif(
|
||||
shutil.which("ffmpeg") is None or shutil.which("ffprobe") is None,
|
||||
reason="ffmpeg/ffprobe required to build the API test fixture",
|
||||
)
|
||||
|
||||
CAMPATH = {
|
||||
"version": 1,
|
||||
"keyframes": [
|
||||
{"t_global": 1.0, "pos": [0, 1, 5], "quat": [0, 0, 0, 1], "fov": 50},
|
||||
{"t_global": 4.0, "pos": [2, 1, 4], "quat": [0, 0.3826834, 0, 0.9238795], "fov": 42},
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def client():
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from festival4d import synthetic
|
||||
|
||||
synthetic.build(duration_s=1.0, run_ffmpeg=True) # into the temp data dir (conftest)
|
||||
|
||||
from festival4d import api
|
||||
|
||||
with TestClient(api.app) as c:
|
||||
yield c
|
||||
|
||||
|
||||
def test_beats_404_then_serves_file(client):
|
||||
from festival4d import config
|
||||
|
||||
config.BEATS_JSON.unlink(missing_ok=True)
|
||||
assert client.get("/api/beats").status_code == 404
|
||||
body = {"tempo_bpm": 120.0, "beats_s": [0.5, 1.0], "onsets": [], "generated_by": "festival4d.audio_features"}
|
||||
config.BEATS_JSON.parent.mkdir(parents=True, exist_ok=True)
|
||||
config.BEATS_JSON.write_text(json.dumps(body))
|
||||
try:
|
||||
assert client.get("/api/beats").json() == body
|
||||
finally:
|
||||
config.BEATS_JSON.unlink()
|
||||
|
||||
|
||||
def test_director_degrades_to_valid_empty_campath(client):
|
||||
# While lane E is stubbed the response must still round-trip through camPath.fromJSON:
|
||||
# a valid version-1 path with zero keyframes, plus a "note" marking the degradation.
|
||||
data = client.post("/api/director", json={}).json()
|
||||
assert data["version"] == 1
|
||||
assert data["keyframes"] == []
|
||||
assert "note" in data
|
||||
assert client.post("/api/director", json={"top_n": 3, "lead_s": 1.0}).status_code == 200
|
||||
|
||||
|
||||
def test_paths_crud_roundtrip(client):
|
||||
assert client.get("/api/paths").json() == []
|
||||
created = client.post(
|
||||
"/api/paths", json={"name": "sweep", "json": json.dumps(CAMPATH)}
|
||||
).json()
|
||||
assert set(created) == {"id", "name", "created_at", "json"}
|
||||
assert created["json"] == CAMPATH # deep-equal roundtrip
|
||||
pid = created["id"]
|
||||
|
||||
listing = client.get("/api/paths").json()
|
||||
assert [set(p) for p in listing] == [{"id", "name", "created_at"}]
|
||||
assert client.get(f"/api/paths/{pid}").json()["json"] == CAMPATH
|
||||
|
||||
assert client.delete(f"/api/paths/{pid}").json() == {"deleted": pid}
|
||||
assert client.get(f"/api/paths/{pid}").status_code == 404
|
||||
assert client.delete(f"/api/paths/{pid}").status_code == 404
|
||||
|
||||
|
||||
def test_paths_post_validates_campath(client):
|
||||
post = lambda body: client.post("/api/paths", json={"name": "bad", "json": body}).status_code
|
||||
assert post("not json at all {") == 422
|
||||
assert post(json.dumps({"version": 2, "keyframes": []})) == 422
|
||||
assert post(json.dumps({"version": 1})) == 422
|
||||
assert post(json.dumps({"version": 1, "keyframes": [{"t_global": 0}]})) == 422
|
||||
|
||||
|
||||
def test_anchor_patch_label_and_color(client):
|
||||
a = client.post("/api/anchors", json={"label": "F", "x": 0, "y": 0, "z": 0}).json()
|
||||
patched = client.patch(f"/api/anchors/{a['id']}", json={"label": "Fred", "color": "#4dd0ff"}).json()
|
||||
assert patched["label"] == "Fred" and patched["color"] == "#4dd0ff"
|
||||
assert (patched["x"], patched["y"], patched["z"]) == (0, 0, 0) # position untouched
|
||||
# partial patch: color-only leaves the label alone
|
||||
assert client.patch(f"/api/anchors/{a['id']}", json={"color": "#fff"}).json()["label"] == "Fred"
|
||||
assert client.patch("/api/anchors/99999", json={"label": "x"}).status_code == 404
|
||||
|
||||
|
||||
def test_cli_dispatches_stubs_gracefully():
|
||||
# The three new subcommands exist and degrade per the house pattern: exit code 2,
|
||||
# no traceback, while their lane entrypoints are stubs.
|
||||
from festival4d import cli
|
||||
|
||||
for cmd in (["features"], ["direct"], ["capsule"]):
|
||||
assert cli.main(cmd) == 2
|
||||
@ -197,6 +197,10 @@
|
||||
pointer-events: none; z-index: 6; border-radius: 2px;
|
||||
}
|
||||
|
||||
/* Capsule mode (M17, phase-5 contract #5): a static baked bundle has no write API, so
|
||||
everything marked .write-ui disappears when main.js sets body.capsule from the manifest. */
|
||||
body.capsule .write-ui { display: none !important; }
|
||||
|
||||
/* Loading / error */
|
||||
#loading {
|
||||
position: fixed; inset: 0; display: flex; align-items: center; justify-content: center;
|
||||
@ -228,13 +232,32 @@
|
||||
<button class="btn" id="btn-path-export" title="Export path as JSON">⤓</button>
|
||||
<button class="btn" id="btn-path-import" title="Import path JSON">⤒</button>
|
||||
<button class="btn" id="btn-path-clear" title="Clear all keyframes">✕</button>
|
||||
<!-- Phase 5: hidden until the owning lane's module reports ready (plan/20-phase5.md) -->
|
||||
<button class="btn" id="btn-director" style="display:none"
|
||||
title="Auto-director: build a camera path from the detected events (M11)">🎬 Auto-path</button>
|
||||
<button class="btn write-ui" id="btn-path-save" style="display:none"
|
||||
title="Save the current camera path to the project (M16)">💾</button>
|
||||
<select class="btn" id="path-load-select" style="display:none"
|
||||
title="Load a saved camera path (M16)"></select>
|
||||
<button class="btn write-ui" id="btn-path-del" style="display:none"
|
||||
title="Delete the selected saved path (M16)">🗑</button>
|
||||
<span class="ctl-sep"></span>
|
||||
<button class="btn" id="btn-spatial" style="display:none"
|
||||
title="Spatial audio: hear every camera from its position in the scene">🎧 3D audio</button>
|
||||
<button class="btn" id="btn-export" style="display:none"
|
||||
title="Export the camera path as a cinematic webm (M12)">⏺ Export</button>
|
||||
<button class="btn" id="btn-photo" style="display:none"
|
||||
title="Photo mode: hi-res still of the 3D scene (M14, P)">📷</button>
|
||||
<button class="btn" id="btn-fx" style="display:none"
|
||||
title="Moment FX: bursts and pulses on event markers (M15)">✨</button>
|
||||
<button class="btn" id="btn-anchors" style="display:none"
|
||||
title="Anchors & friend tags (M13)">⚓</button>
|
||||
<input type="file" id="path-file" accept="application/json,.json" hidden />
|
||||
</div>
|
||||
<div id="scene-hint">drag to orbit · 1–9 snap to camera · K add keyframe · Esc detach</div>
|
||||
<div id="annot-panel"></div>
|
||||
<div id="annot-panel" class="write-ui"></div>
|
||||
<!-- M13 anchor manager mounts here (lane F); hidden until anchorPanel.js is filled -->
|
||||
<div id="anchor-panel" style="display:none"></div>
|
||||
</div>
|
||||
<div id="grid-pane">
|
||||
<div id="video-grid"></div>
|
||||
|
||||
19
frontend/src/anchorPanel.js
Normal file
19
frontend/src/anchorPanel.js
Normal file
@ -0,0 +1,19 @@
|
||||
// Anchor manager & friend tags (spec M13, lane F). STUB — lane F fills THIS FILE ONLY.
|
||||
//
|
||||
// Pre-wired by foundation2: the hidden ⚓ toggle (#btn-anchors) and the empty #anchor-panel
|
||||
// container in index.html, this module's import + init() call in main.js, the hooks
|
||||
// `scene3d.focusOn(x, y, z)` (jump-to) and `PATCH ${state.apiBase}/api/anchors/{id}`
|
||||
// {label?, color?} (rename/recolor), plus DELETE /api/anchors/{id} from phase 4.
|
||||
//
|
||||
// When implementing: unhide the button, set `ready = true`; build the collapsible list
|
||||
// (color dot, label, jump-to, rename, recolor, delete) from state.anchors; after any
|
||||
// mutation update state.anchors and emit("anchors-changed") so scene3d + overlays refresh.
|
||||
// The + Tag flow reuses the M8 bbox-annotation flow WITHOUT an event (annotate.js is
|
||||
// frozen — drive it, don't edit it) and names the resulting anchor.
|
||||
// Capsule mode: give every MUTATING control the `write-ui` class — body.capsule hides that
|
||||
// class globally (contract #5); the read-only list + jump-to stay available.
|
||||
|
||||
export const anchorPanel = {
|
||||
ready: false,
|
||||
init() {}, // ponytail: stub — lane F replaces the body
|
||||
};
|
||||
@ -27,6 +27,7 @@ export class CamPath {
|
||||
if (!this.group) {
|
||||
this.group = new THREE.Group();
|
||||
scene3d.addObject(this.group);
|
||||
scene3d.registerHelper(this.group); // keyframe gizmos hide with the helpers (M14)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
16
frontend/src/director.js
Normal file
16
frontend/src/director.js
Normal file
@ -0,0 +1,16 @@
|
||||
// Auto-director UI (spec M11, lane E). STUB — lane E fills THIS FILE ONLY.
|
||||
//
|
||||
// Pre-wired by foundation2: the hidden 🎬 button (#btn-director in index.html), and this
|
||||
// module's import + init() call in main.js. main.js does NOT wire the click — you do,
|
||||
// inside init().
|
||||
//
|
||||
// When implementing: unhide the button (el.style.display = ""), set `ready = true`; on click
|
||||
// POST {top_n?, lead_s?} to `${state.apiBase}/api/director` and load the response with
|
||||
// camPath.fromJSON(...) — never edit camPath.js (frozen). The response is ALWAYS a valid
|
||||
// camPath JSON; zero keyframes means "no events yet / stubbed" — surface that in the UI,
|
||||
// don't crash (pitfall #5). Play it via the existing #btn-path flow.
|
||||
|
||||
export const director = {
|
||||
ready: false, // flips to true when the lane implements this module
|
||||
init() {}, // ponytail: stub — lane E replaces the body
|
||||
};
|
||||
18
frontend/src/exportVideo.js
Normal file
18
frontend/src/exportVideo.js
Normal file
@ -0,0 +1,18 @@
|
||||
// Cinematic export (spec M12, lane E). STUB — lane E fills THIS FILE ONLY.
|
||||
//
|
||||
// Pre-wired by foundation2: the hidden ⏺ button (#btn-export in index.html), this module's
|
||||
// import + init() call in main.js, and the transport hooks
|
||||
// `transport.captureAudioStream()` -> MediaStream tapping the live WebAudio mix (post-gain,
|
||||
// spatial included) and `transport.releaseAudioStream()` when done.
|
||||
//
|
||||
// When implementing: unhide the button, set `ready = true`; on click seek to the first
|
||||
// camPath keyframe, play the path, mux scene3d canvas `captureStream(30)` + the audio stream
|
||||
// with MediaRecorder (video/webm;codecs=vp9,opus), stop at the last keyframe, download
|
||||
// `festival4d-cut.webm`, then RESTORE all prior UI/transport state. The canvas is
|
||||
// `scene3d.renderer.domElement`. MediaRecorder needs a *focused* tab to capture real frames
|
||||
// (pitfall #2) — say so in your evidence if you couldn't.
|
||||
|
||||
export const exportVideo = {
|
||||
ready: false,
|
||||
init() {}, // ponytail: stub — lane E replaces the body
|
||||
};
|
||||
19
frontend/src/fx.js
Normal file
19
frontend/src/fx.js
Normal file
@ -0,0 +1,19 @@
|
||||
// Moment FX (spec M15, lane F). STUB — lane F fills THIS FILE ONLY.
|
||||
//
|
||||
// Pre-wired by foundation2: the hidden ✨ toggle (#btn-fx in index.html), this module's
|
||||
// import + init() call in main.js, and a per-frame `fx.update(tGlobal)` call from the master
|
||||
// loop (after transport.tick(), before scene3d.update()).
|
||||
//
|
||||
// When implementing: unhide the button, set `ready = true`. update() watches the playhead
|
||||
// crossing event markers (state.events) — detect crossings in play AND scrub (track the
|
||||
// previous tGlobal; a seek emits "seek" if you need to reset). On `pyro`/`confetti` fire a
|
||||
// short particle burst at the event's resolved anchor, falling back to the stage centroid;
|
||||
// on `bass_drop` pulse the point-cloud/splat scale for ~a beat (beats via GET /api/beats
|
||||
// when present — no beats file, use ~0.5 s; pitfall #5). Scene access ONLY through
|
||||
// scene3d.addObject/removeObject. Keep >= 30 fps on the synthetic scene.
|
||||
|
||||
export const fx = {
|
||||
ready: false,
|
||||
init() {}, // ponytail: stub — lane F replaces the body
|
||||
update(tGlobal) {}, // called every animation frame by main.js
|
||||
};
|
||||
@ -16,6 +16,15 @@ import { scene3d, videoColor } from "./scene3d.js";
|
||||
import { timeline } from "./timeline.js";
|
||||
import { annotator } from "./annotate.js";
|
||||
import { camPath } from "./camPath.js";
|
||||
// Phase-5 lane modules (stubs until lanes E/F/G fill them; each wires its own hidden button).
|
||||
import { director } from "./director.js";
|
||||
import { exportVideo } from "./exportVideo.js";
|
||||
import { anchorPanel } from "./anchorPanel.js";
|
||||
import { photoMode } from "./photoMode.js";
|
||||
import { fx } from "./fx.js";
|
||||
import { pathsStore } from "./pathsStore.js";
|
||||
|
||||
const laneModules = { director, exportVideo, anchorPanel, photoMode, fx, pathsStore };
|
||||
|
||||
async function getJSON(path) {
|
||||
const resp = await fetch(API_BASE + path);
|
||||
@ -49,6 +58,9 @@ async function boot() {
|
||||
link.style.display = "";
|
||||
}
|
||||
state.hasSplat = !!manifest.has_splat;
|
||||
// Capsule (M17): a static baked bundle has no write API — hide everything .write-ui.
|
||||
state.capsule = !!manifest.capsule;
|
||||
document.body.classList.toggle("capsule", state.capsule);
|
||||
for (const v of state.videos) state.enabled[v.id] = true;
|
||||
state.audioSourceId = referenceVideoId();
|
||||
|
||||
@ -75,8 +87,8 @@ async function boot() {
|
||||
// localStorage.setItem("f4dDebug", "1").
|
||||
if (import.meta.env?.DEV || localStorage.getItem("f4dDebug")) {
|
||||
window.__f4d = {
|
||||
state, transport, scene3d, timeline, cells: () => cells,
|
||||
poseAt, projectWorldToVideoPx,
|
||||
state, transport, scene3d, timeline, camPath, cells: () => cells,
|
||||
poseAt, projectWorldToVideoPx, ...laneModules,
|
||||
};
|
||||
}
|
||||
} catch (err) {
|
||||
@ -120,6 +132,16 @@ function buildUI() {
|
||||
wireKeyboard();
|
||||
positionVideosWhenReady();
|
||||
|
||||
// Phase-5 lane modules: each init() wires its own (hidden) controls and unhides them when
|
||||
// the lane has filled the stub. A broken module degrades to its button staying hidden.
|
||||
for (const [name, mod] of Object.entries(laneModules)) {
|
||||
try {
|
||||
mod.init();
|
||||
} catch (err) {
|
||||
console.warn(`[festival4d] lane module "${name}" init failed`, err);
|
||||
}
|
||||
}
|
||||
|
||||
// Reflect follow/roam state on the free-roam button; snapping cancels a playing path.
|
||||
on("follow", (id) => {
|
||||
el("btn-roam").classList.toggle("active", id == null);
|
||||
@ -244,6 +266,7 @@ function startLoop() {
|
||||
const frame = () => {
|
||||
transport.tick();
|
||||
for (const cell of cells) drawOverlay(cell);
|
||||
fx.update(state.tGlobal); // M15: playhead-crossing effects (no-op while stubbed)
|
||||
camPath.update(state.tGlobal); // M9: drive the viewer camera before the scene renders
|
||||
scene3d.update();
|
||||
timeline.update(state.tGlobal);
|
||||
|
||||
18
frontend/src/pathsStore.js
Normal file
18
frontend/src/pathsStore.js
Normal file
@ -0,0 +1,18 @@
|
||||
// Server-side camera paths UI (spec M16, lane G). STUB — lane G fills THIS FILE ONLY.
|
||||
//
|
||||
// Pre-wired by foundation2: the hidden controls in index.html — #btn-path-save (💾),
|
||||
// #path-load-select (dropdown), #btn-path-del (🗑) — this module's import + init() call in
|
||||
// main.js, and the API: GET /api/paths (list of {id,name,created_at}),
|
||||
// GET /api/paths/{id} ({..., json: <camPath JSON>}), POST /api/paths {name, json:<text>}
|
||||
// (422 on invalid), DELETE /api/paths/{id}.
|
||||
//
|
||||
// When implementing: unhide the controls, set `ready = true`. Save = prompt for a name,
|
||||
// POST camPath.toJSON(); load = fetch the selected path and camPath.fromJSON(json); delete =
|
||||
// DELETE the selected id. Refresh the dropdown after every mutation. #btn-path-save and
|
||||
// #btn-path-del already carry the `write-ui` class, so capsule mode (contract #5) hides
|
||||
// them for free — the load dropdown may stay if you bake paths in, else hide it too.
|
||||
|
||||
export const pathsStore = {
|
||||
ready: false,
|
||||
init() {}, // ponytail: stub — lane G replaces the body
|
||||
};
|
||||
18
frontend/src/photoMode.js
Normal file
18
frontend/src/photoMode.js
Normal file
@ -0,0 +1,18 @@
|
||||
// Photo mode (spec M14, lane F). STUB — lane F fills THIS FILE ONLY.
|
||||
//
|
||||
// Pre-wired by foundation2: the hidden 📷 button (#btn-photo in index.html), this module's
|
||||
// import + init() call in main.js, and the hook `scene3d.setHelpersVisible(false)` which
|
||||
// hides grid/axes/frusta/path-lines/keyframe-gizmos/anchor-labels in one call (restore with
|
||||
// true).
|
||||
//
|
||||
// When implementing: unhide the button, set `ready = true`, bind key "P" too; on trigger
|
||||
// pause the transport, setHelpersVisible(false), render ONE frame at 3840-wide (preserve
|
||||
// aspect) to an offscreen WebGLRenderTarget via scene3d.renderer/scene/camera, download the
|
||||
// PNG, then restore everything (helpers, renderer size, play state). Works in free-roam and
|
||||
// follow-cam; with a splat present it renders the splat (no splat -> the point cloud —
|
||||
// pitfall #5).
|
||||
|
||||
export const photoMode = {
|
||||
ready: false,
|
||||
init() {}, // ponytail: stub — lane F replaces the body
|
||||
};
|
||||
@ -65,6 +65,8 @@ export class Scene3D {
|
||||
this._raf = null;
|
||||
this.anchorGroup = null; // 3D anchor spheres + labels (M6 anchors + M8 resolved)
|
||||
this.pathMode = false; // when true, camPath (M9) drives the camera; we don't touch it
|
||||
this.helpersVisible = true; // M14 hook: grid/axes/frusta/gizmos/labels toggle
|
||||
this._helperObjs = []; // extra helper objects registered by other modules (camPath gizmos)
|
||||
}
|
||||
|
||||
init(canvas) {
|
||||
@ -96,6 +98,8 @@ export class Scene3D {
|
||||
this.scene.add(grid);
|
||||
const axes = new THREE.AxesHelper(1.5);
|
||||
this.scene.add(axes);
|
||||
this._grid = grid;
|
||||
this._axes = axes;
|
||||
|
||||
this.anchorGroup = new THREE.Group();
|
||||
this.scene.add(this.anchorGroup);
|
||||
@ -129,12 +133,13 @@ export class Scene3D {
|
||||
if (a.label) {
|
||||
const label = makeLabelSprite(a.label);
|
||||
label.position.set(a.x, a.y + 0.3, a.z);
|
||||
label.visible = this.helpersVisible; // photo mode (M14) may be hiding labels right now
|
||||
this.anchorGroup.add(label);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Expose the scene graph for auxiliary overlays (e.g. camPath gizmos, M9). */
|
||||
/** Expose the scene graph for auxiliary overlays (e.g. camPath gizmos M9, moment FX M15). */
|
||||
addObject(obj) {
|
||||
this.scene?.add(obj);
|
||||
}
|
||||
@ -142,6 +147,51 @@ export class Scene3D {
|
||||
this.scene?.remove(obj);
|
||||
}
|
||||
|
||||
/** Register an added object as a "helper" so setHelpersVisible governs it (camPath gizmos). */
|
||||
registerHelper(obj) {
|
||||
this._helperObjs.push(obj);
|
||||
obj.visible = this.helpersVisible;
|
||||
}
|
||||
|
||||
/**
|
||||
* M14 hook (phase-5 contract #6): show/hide every non-content visual in one call — grid,
|
||||
* axes, camera frusta + markers + path lines, registered gizmo groups, and anchor LABEL
|
||||
* sprites (anchor spheres stay: friend tags belong in photos). update() keeps enforcing
|
||||
* the flag on the per-video rigs, whose visibility it recomputes every frame.
|
||||
*/
|
||||
setHelpersVisible(visible) {
|
||||
this.helpersVisible = !!visible;
|
||||
if (this._grid) this._grid.visible = this.helpersVisible;
|
||||
if (this._axes) this._axes.visible = this.helpersVisible;
|
||||
for (const obj of this._helperObjs) obj.visible = this.helpersVisible;
|
||||
if (this.anchorGroup)
|
||||
for (const child of this.anchorGroup.children)
|
||||
if (child.isSprite) child.visible = this.helpersVisible;
|
||||
}
|
||||
|
||||
/**
|
||||
* M13 hook (phase-5 contract #6): fly the free-roam camera's attention to a world point
|
||||
* (anchor jump-to). Detaches follow-cam / path mode, retargets OrbitControls, and backs the
|
||||
* camera off along its current viewing direction.
|
||||
*/
|
||||
focusOn(x, y, z, dist = 6) {
|
||||
if (this.pathMode) this.exitPathMode();
|
||||
if (state.followCameraId != null) {
|
||||
state.followCameraId = null;
|
||||
this._tween = null;
|
||||
emit("follow", null);
|
||||
}
|
||||
this.camera.fov = DEFAULT_FOV;
|
||||
this.camera.updateProjectionMatrix();
|
||||
const dir = this.camera.position.clone().sub(this.controls.target);
|
||||
if (dir.lengthSq() < 1e-6) dir.set(0, 0.5, 1);
|
||||
dir.normalize().multiplyScalar(dist);
|
||||
this.controls.target.set(x, y, z);
|
||||
this.camera.position.set(x + dir.x, y + dir.y, z + dir.z);
|
||||
this.controls.enabled = true;
|
||||
this.controls.update();
|
||||
}
|
||||
|
||||
/** M9: hand camera control to camPath. update() then leaves the camera alone. */
|
||||
enterPathMode() {
|
||||
this.pathMode = true;
|
||||
@ -362,8 +412,8 @@ export class Scene3D {
|
||||
if (!rig) continue;
|
||||
const pose = this._livePose(v.id);
|
||||
const on = pose && state.enabled[v.id] !== false;
|
||||
rig.group.visible = !!on;
|
||||
if (rig.path) rig.path.visible = state.enabled[v.id] !== false;
|
||||
rig.group.visible = !!on && this.helpersVisible;
|
||||
if (rig.path) rig.path.visible = state.enabled[v.id] !== false && this.helpersVisible;
|
||||
if (!on) continue;
|
||||
const { matrixWorld } = colmapToThreejs(pose.q, pose.t);
|
||||
_m4.fromArray(matrixWorld);
|
||||
|
||||
@ -7,7 +7,11 @@
|
||||
// A hosted build sets VITE_API_BASE to a same-origin path prefix (e.g. "/festifun") so every
|
||||
// request — fetch, <video> src, PLY loader — goes through the reverse proxy under that path.
|
||||
// Empty string = same-origin at root. See deploy/DEPLOY.md.
|
||||
export const API_BASE = import.meta.env.VITE_API_BASE ?? "http://localhost:8000";
|
||||
// Phase-5 contract #5: a RUNTIME override wins over the build-time value — the memory capsule
|
||||
// (M17) injects `window.__F4D_API_BASE__ = ""` into its baked index.html so the same dist
|
||||
// bundle turns zero-backend. `??` (not `||`): empty string is a valid value (same-origin).
|
||||
export const API_BASE =
|
||||
window.__F4D_API_BASE__ ?? import.meta.env.VITE_API_BASE ?? "http://localhost:8000";
|
||||
|
||||
export const state = {
|
||||
apiBase: API_BASE,
|
||||
@ -18,6 +22,7 @@ export const state = {
|
||||
tGlobalMax: 0,
|
||||
hasPoses: false,
|
||||
hasSplat: false, // optional 3DGS splat at /api/splat (preferred over the point cloud)
|
||||
capsule: false, // manifest.capsule (M17): static baked bundle => hide ALL write UI
|
||||
poses: {}, // id -> [{frame_idx, t_video_s, q:[w,x,y,z], t:[x,y,z], intrinsics, registered}]
|
||||
anchors: [], // [{id, label, x, y, z, color}]
|
||||
events: [], // [{id, t_global_s, duration_s, event_type, confidence, description, source}]
|
||||
|
||||
@ -42,6 +42,7 @@ export class Transport {
|
||||
this._buffers = new Map(); // video id -> AudioBuffer (decoded) | null (unavailable)
|
||||
this._decoding = new Set(); // video ids with a fetch/decode in flight
|
||||
this._tracks = new Map(); // playing tracks: video id -> { src, gain, panner|null }
|
||||
this._captureDest = null; // MediaStreamAudioDestinationNode while an export taps the mix (M12)
|
||||
}
|
||||
|
||||
/** Register the video cells the transport drives and start their frame loops. */
|
||||
@ -296,11 +297,42 @@ export class Transport {
|
||||
} else {
|
||||
gain.connect(ctx.destination);
|
||||
}
|
||||
if (this._captureDest) (panner ?? gain).connect(this._captureDest); // live export keeps hearing us
|
||||
if (tv >= 0) src.start(0, tv);
|
||||
else src.start(ctx.currentTime + -tv / state.rate, 0); // camera starts later than t_global
|
||||
this._tracks.set(id, { src, gain, panner });
|
||||
}
|
||||
|
||||
/**
|
||||
* M12 hook (phase-5 contract #6): a MediaStream carrying the live WebAudio mix — tapped
|
||||
* post-gain, after the spatial panner when 🎧 is on — for MediaRecorder muxing. Tracks that
|
||||
* (re)start while the tap is live (seeks, source switches) join it automatically
|
||||
* (_startTrack above). With no decoded track (fallback <video> audio) the stream is silent —
|
||||
* degrade, don't block (pitfall #5). Call releaseAudioStream() when the export ends.
|
||||
*/
|
||||
captureAudioStream() {
|
||||
const ctx = this._ensureCtx();
|
||||
if (!this._captureDest) {
|
||||
this._captureDest = ctx.createMediaStreamDestination();
|
||||
for (const { gain, panner } of this._tracks.values())
|
||||
(panner ?? gain).connect(this._captureDest);
|
||||
}
|
||||
return this._captureDest.stream;
|
||||
}
|
||||
|
||||
/** Drop the M12 export tap (the matching half of captureAudioStream()). */
|
||||
releaseAudioStream() {
|
||||
if (!this._captureDest) return;
|
||||
for (const { gain, panner } of this._tracks.values()) {
|
||||
try {
|
||||
(panner ?? gain).disconnect(this._captureDest);
|
||||
} catch {
|
||||
/* already disconnected */
|
||||
}
|
||||
}
|
||||
this._captureDest = null;
|
||||
}
|
||||
|
||||
/**
|
||||
* 🎧 Per-frame 3D audio update (called by scene3d): the listener follows the viewer camera
|
||||
* and each playing track's panner sits at its camera's current pose. Takes plain numbers in
|
||||
|
||||
@ -172,7 +172,7 @@ structure, baked-JSON validity, and serve.py Range (206) behavior.
|
||||
|---|---|
|
||||
| Lane E | `backend/festival4d/audio_features.py`, `director.py`, `backend/tests/test_audio_features.py`, `test_director.py`, `frontend/src/director.js`, `exportVideo.js` |
|
||||
| Lane F | `frontend/src/anchorPanel.js`, `photoMode.js`, `fx.js` |
|
||||
| Lane G | `backend/festival4d/capsule.py`, `backend/tests/test_paths.py`, `test_capsule.py`, `frontend/src/pathsStore.js`, the paths-route bodies in the area foundation2 marks for lane G |
|
||||
| Lane G | `backend/festival4d/capsule.py`, `backend/tests/test_paths.py`, `test_capsule.py`, `frontend/src/pathsStore.js` — the paths-route bodies were CRUD one-liners over the db helpers, so foundation2 implemented them (basic tests in `test_phase5_api.py`); lane G owns deepening `test_paths.py` + the UI |
|
||||
| Every agent | its own `plan/status/<lane>.md` |
|
||||
| **Coordinator only** | `plan/DIRECTIVES.md` |
|
||||
| **Frozen during 5b** | contract item 7 above |
|
||||
|
||||
55
plan/status/foundation2.md
Normal file
55
plan/status/foundation2.md
Normal file
@ -0,0 +1,55 @@
|
||||
# Status — foundation2
|
||||
|
||||
## Round 1 — 2026-07-17 — STATUS: ready_to_merge
|
||||
|
||||
**Directives acknowledged:** Round 4 of plan/DIRECTIVES.md. Single-agent phase 5a — main
|
||||
working directory on branch `foundation2` (per directive 3).
|
||||
|
||||
**Acceptance checklist** (plan/20-phase5.md "foundation2 scope"):
|
||||
- [x] DB: `paths` table (contract #3) + helpers `add/get/get_one/delete_path`, plus
|
||||
`patch_anchor` (label/color only) — `db.py`
|
||||
- [x] API: `GET /api/beats` (serves `config.BEATS_JSON`, 404 when absent),
|
||||
`POST /api/director` (degrades to a VALID empty camPath + `note` while lane E is
|
||||
stubbed), `GET/POST/DELETE /api/paths*` (implemented — CRUD over the helpers was
|
||||
cheaper than stubbing; POST validates the camPath shape → 422),
|
||||
`PATCH /api/anchors/{id}` (contract #4) — `api.py`
|
||||
- [x] Manifest gained `"capsule": false` — `test_api.py` exact-set lock updated in the same
|
||||
commit (pitfall #3)
|
||||
- [x] CLI: `features`, `direct`, `capsule` subcommands dispatch into stubs; house
|
||||
degradation (exit 2, no traceback) — `cli.py`
|
||||
- [x] Backend stubs raise NotImplementedError with the frozen contract in the docstring:
|
||||
`audio_features.py`, `director.py`, `capsule.py`
|
||||
- [x] Frontend stub modules created + imported from `main.js`, each wiring its own hidden
|
||||
button in init(): `director.js`, `exportVideo.js`, `anchorPanel.js`, `photoMode.js`
|
||||
(+ per-frame `fx.update()` call in the master loop), `fx.js`, `pathsStore.js`
|
||||
- [x] index.html: ALL phase-5 buttons pre-wired hidden — #btn-director, #btn-path-save,
|
||||
#path-load-select, #btn-path-del, #btn-export, #btn-photo, #btn-fx, #btn-anchors,
|
||||
#anchor-panel container; `.write-ui` class + `body.capsule .write-ui{display:none}`
|
||||
- [x] Hooks (contract #6): `transport.captureAudioStream()/releaseAudioStream()` (post-gain
|
||||
tap, spatial included, tracks re-join across seeks), `scene3d.focusOn(x,y,z)`,
|
||||
`scene3d.setHelpersVisible(bool)` (+ `registerHelper`; camPath gizmos registered)
|
||||
- [x] `state.js`: `window.__F4D_API_BASE__` ?? `VITE_API_BASE` ?? localhost (contract #5);
|
||||
`state.capsule` from the manifest toggles `body.capsule`
|
||||
- [x] Suite ≥ 121 + new tests: **127 passed** (121 floor + 6 in `test_phase5_api.py`)
|
||||
- [x] Ownership table updated (paths-route bodies → foundation2; noted in 20-phase5.md)
|
||||
|
||||
**Done this round — evidence:**
|
||||
- `uv run pytest backend/tests` → **127 passed** (was 121).
|
||||
- `npm run build` → clean (pre-existing chunk-size warning only).
|
||||
- Live browser (Vite :5173 + uvicorn :8000, synthetic fixture): app boots (3 videos, sync
|
||||
+0 ms); all 9 new controls present and `display:none`; `setHelpersVisible(false)` hides
|
||||
grid (and restores); `focusOn(1,0.5,0)` retargets OrbitControls to exactly [1,0.5,0];
|
||||
`captureAudioStream()` returns a MediaStream with 1 audio track, release clean;
|
||||
`POST /api/director` → `{version:1, keyframes:[], note:…}` and round-trips
|
||||
`camPath.fromJSON` (0 keyframes, no throw); paths save→load→fromJSON reproduces 2
|
||||
keyframes, DELETE → 200 then 404; `PATCH /api/anchors/1 {color}` → updated dict;
|
||||
`GET /api/beats` → 404; `body.capsule` hides #annot-panel/#btn-path-save/#btn-path-del.
|
||||
|
||||
**Blockers / questions for coordinator:** none. One conscious deviation, self-adjudicated
|
||||
(coordinator == foundation2 agent this round): paths route bodies implemented here instead
|
||||
of stub-marked for lane G — they were one-line calls into the db helpers I had to write
|
||||
anyway; stubbing them would have been more code than implementing. Lane G scope shrinks to
|
||||
capsule + pathsStore.js + deepening test_paths.py.
|
||||
|
||||
**Next:** merge to `main`, then lanes E, G, F in parallel (each in its own worktree —
|
||||
directive 3 is mandatory for 5b).
|
||||
Loading…
Reference in New Issue
Block a user