foundation2 (phase 5a): contracts, stubs, hooks & UI wiring for lanes E/F/G
- DB: paths table (contract #3) + helpers; patch_anchor (label/color) - API: GET /api/beats, POST /api/director (valid-empty degradation), /api/paths CRUD (validated POST -> 422), PATCH /api/anchors/{id}; manifest gains capsule:false (test_api.py key-set lock updated consciously) - CLI: features | direct | capsule subcommands -> lane stubs (exit-2 degradation) - Backend stubs with frozen contracts in docstrings: audio_features, director, capsule - Frontend: six lane stub modules imported from main.js; all phase-5 buttons pre-wired hidden in index.html; write-ui/body.capsule read-only mode; __F4D_API_BASE__ runtime override in state.js - Hooks (contract #6): transport.captureAudioStream/releaseAudioStream, scene3d.focusOn, scene3d.setHelpersVisible (+registerHelper; camPath gizmos) - Suite 121 -> 127 passed; npm build clean; live-browser verified (status file) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
545f7a347d
commit
51650c728e
@ -12,6 +12,17 @@ Serves the synthetic project fully so lane C can treat this API as finished:
|
|||||||
POST /api/events/detect {t_global_s?, window_s?} -> runs M7 detection
|
POST /api/events/detect {t_global_s?, window_s?} -> runs M7 detection
|
||||||
POST /api/annotations {video_id, t_video_s, bbox:[x0,y0,x1,y1]} -> {anchor_id?, point?}
|
POST /api/annotations {video_id, t_video_s, bbox:[x0,y0,x1,y1]} -> {anchor_id?, point?}
|
||||||
|
|
||||||
|
Phase 5 (foundation2; shapes frozen in plan/20-phase5.md contracts):
|
||||||
|
|
||||||
|
GET /api/beats -> beats.json body (contract #1); 404 when not yet analyzed
|
||||||
|
POST /api/director {top_n?, lead_s?} -> camPath JSON (contract #2); while lane E
|
||||||
|
is stubbed, degrades to a VALID empty path + "note"
|
||||||
|
GET /api/paths -> [{id, name, created_at}]
|
||||||
|
GET /api/paths/{id} -> {id, name, created_at, json: <parsed camPath JSON>}
|
||||||
|
POST /api/paths {name, json: <camPath JSON as text>} (validated -> 422)
|
||||||
|
DELETE /api/paths/{id}
|
||||||
|
PATCH /api/anchors/{id} {label?, color?} -> updated anchor dict
|
||||||
|
|
||||||
Source videos in ``data/raw`` are mounted at ``/media`` via Starlette ``StaticFiles``,
|
Source videos in ``data/raw`` are mounted at ``/media`` via Starlette ``StaticFiles``,
|
||||||
which supports HTTP Range (required for ``<video>`` seeking; verify ``curl -H "Range:
|
which supports HTTP Range (required for ``<video>`` seeking; verify ``curl -H "Range:
|
||||||
bytes=0-100"`` -> 206). Stubbed lane entrypoints (events detection, M8 annotation
|
bytes=0-100"`` -> 206). Stubbed lane entrypoints (events detection, M8 annotation
|
||||||
@ -91,6 +102,22 @@ class EventPatch(BaseModel):
|
|||||||
confidence: float | None = None
|
confidence: float | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class AnchorPatch(BaseModel):
|
||||||
|
label: str | None = None
|
||||||
|
color: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class DirectorIn(BaseModel):
|
||||||
|
top_n: int = 8
|
||||||
|
lead_s: float = 2.0
|
||||||
|
|
||||||
|
|
||||||
|
class PathIn(BaseModel):
|
||||||
|
# Wire key is "json" (contract #3); aliased because `json` shadows a BaseModel attr.
|
||||||
|
name: str
|
||||||
|
path_json: str = Field(alias="json")
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Serialization helpers
|
# Serialization helpers
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@ -156,6 +183,9 @@ def manifest() -> dict:
|
|||||||
"has_capture": capture.capture_enabled(),
|
"has_capture": capture.capture_enabled(),
|
||||||
"has_poses": db.has_poses(),
|
"has_poses": db.has_poses(),
|
||||||
"has_splat": config.SPLAT_PLY.exists(),
|
"has_splat": config.SPLAT_PLY.exists(),
|
||||||
|
# Always False from the live server; the capsule (M17) bakes a manifest with true,
|
||||||
|
# which makes the frontend hide all write UI (phase-5 contract #5).
|
||||||
|
"capsule": False,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@ -333,6 +363,102 @@ def _existing_event_anchor_id(event_id: int) -> int | None:
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Phase 5 (foundation2). Shapes are frozen in plan/20-phase5.md "Contracts".
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
@app.get("/api/beats")
|
||||||
|
def get_beats() -> dict:
|
||||||
|
"""The beats.json analysis (contract #1). 404 until `python -m festival4d features` runs."""
|
||||||
|
import json
|
||||||
|
|
||||||
|
if not config.BEATS_JSON.exists():
|
||||||
|
raise HTTPException(status_code=404, detail="no beat analysis (run `python -m festival4d features`)")
|
||||||
|
return json.loads(config.BEATS_JSON.read_text())
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/api/director")
|
||||||
|
def run_director(payload: DirectorIn) -> dict:
|
||||||
|
"""Auto-director (M11) -> camPath JSON. Degrades to a valid EMPTY path while stubbed,
|
||||||
|
so the response always round-trips through camPath.fromJSON (contract #2)."""
|
||||||
|
from festival4d import director
|
||||||
|
|
||||||
|
try:
|
||||||
|
return director.generate_path(top_n=payload.top_n, lead_s=payload.lead_s)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
log.info("auto-director not implemented yet: %s", exc)
|
||||||
|
return {"version": 1, "keyframes": [], "note": "auto-director not implemented yet (lane E / M11)"}
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_campath(text: str) -> None:
|
||||||
|
"""422 unless ``text`` is the frozen camPath JSON (contract #2). Lane G owns hardening."""
|
||||||
|
import json
|
||||||
|
|
||||||
|
try:
|
||||||
|
data = json.loads(text)
|
||||||
|
except ValueError:
|
||||||
|
raise HTTPException(status_code=422, detail="path json is not valid JSON")
|
||||||
|
ok = (
|
||||||
|
isinstance(data, dict)
|
||||||
|
and data.get("version") == 1
|
||||||
|
and isinstance(data.get("keyframes"), list)
|
||||||
|
and all(
|
||||||
|
isinstance(k, dict)
|
||||||
|
and isinstance(k.get("t_global"), (int, float))
|
||||||
|
and isinstance(k.get("pos"), list) and len(k["pos"]) == 3
|
||||||
|
and isinstance(k.get("quat"), list) and len(k["quat"]) == 4
|
||||||
|
and isinstance(k.get("fov"), (int, float))
|
||||||
|
for k in data.get("keyframes", [])
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if not ok:
|
||||||
|
raise HTTPException(status_code=422, detail="not a camPath JSON (see plan/20-phase5.md contract #2)")
|
||||||
|
|
||||||
|
|
||||||
|
def _path_dict(p, with_json: bool = False) -> dict:
|
||||||
|
import json
|
||||||
|
|
||||||
|
d = {"id": p.id, "name": p.name, "created_at": p.created_at}
|
||||||
|
if with_json:
|
||||||
|
d["json"] = json.loads(p.path_json)
|
||||||
|
return d
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/api/paths")
|
||||||
|
def get_paths() -> list[dict]:
|
||||||
|
return [_path_dict(p) for p in db.get_paths()]
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/api/paths/{path_id}")
|
||||||
|
def get_path(path_id: int) -> dict:
|
||||||
|
p = db.get_path(path_id)
|
||||||
|
if p is None:
|
||||||
|
raise HTTPException(status_code=404, detail=f"no path with id={path_id}")
|
||||||
|
return _path_dict(p, with_json=True)
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/api/paths")
|
||||||
|
def create_path(payload: PathIn) -> dict:
|
||||||
|
_validate_campath(payload.path_json)
|
||||||
|
return _path_dict(db.add_path(payload.name, payload.path_json), with_json=True)
|
||||||
|
|
||||||
|
|
||||||
|
@app.delete("/api/paths/{path_id}")
|
||||||
|
def delete_path(path_id: int) -> dict:
|
||||||
|
if not db.delete_path(path_id):
|
||||||
|
raise HTTPException(status_code=404, detail=f"no path with id={path_id}")
|
||||||
|
return {"deleted": path_id}
|
||||||
|
|
||||||
|
|
||||||
|
@app.patch("/api/anchors/{anchor_id}")
|
||||||
|
def patch_anchor(anchor_id: int, payload: AnchorPatch) -> dict:
|
||||||
|
"""Rename / recolor an anchor in place (M13, contract #4)."""
|
||||||
|
try:
|
||||||
|
anchor = db.patch_anchor(anchor_id, label=payload.label, color=payload.color)
|
||||||
|
except KeyError:
|
||||||
|
raise HTTPException(status_code=404, detail=f"no anchor with id={anchor_id}")
|
||||||
|
return _anchor_dict(anchor)
|
||||||
|
|
||||||
|
|
||||||
@app.delete("/api/anchors/{anchor_id}")
|
@app.delete("/api/anchors/{anchor_id}")
|
||||||
def delete_anchor(anchor_id: int) -> dict:
|
def delete_anchor(anchor_id: int) -> dict:
|
||||||
"""Remove an anchor (M8 / Phase 4). Unlinks any annotations that resolved to it."""
|
"""Remove an anchor (M8 / Phase 4). Unlinks any annotations that resolved to it."""
|
||||||
|
|||||||
25
backend/festival4d/audio_features.py
Normal file
25
backend/festival4d/audio_features.py
Normal file
@ -0,0 +1,25 @@
|
|||||||
|
"""Audio features: beats & onsets (spec M10, lane E). STUB — lane E fills the body.
|
||||||
|
|
||||||
|
Contract (frozen at foundation2 merge — phase-5 contract #1):
|
||||||
|
|
||||||
|
:func:`run_features` loads the **reference** video's ingest WAV (16 kHz mono from
|
||||||
|
``config.AUDIO_DIR``; the reference timeline *is* ``t_global``), runs beat/tempo tracking
|
||||||
|
and onset-strength detection (librosa is already a dependency), and writes
|
||||||
|
``config.BEATS_JSON``::
|
||||||
|
|
||||||
|
{"tempo_bpm": float | null,
|
||||||
|
"beats_s": [float, ...],
|
||||||
|
"onsets": [{"t_global_s": float, "strength": float 0..1}, ...],
|
||||||
|
"generated_by": "festival4d.audio_features"}
|
||||||
|
|
||||||
|
All times are ``t_global`` seconds. Quiet / short / beatless audio must produce a
|
||||||
|
*valid, empty* result (``tempo_bpm: null``, empty lists) — never a crash. Returns a
|
||||||
|
small summary dict for the CLI log. ``GET /api/beats`` serves the written file.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
|
||||||
|
def run_features() -> dict:
|
||||||
|
"""Analyze the reference soundtrack and write ``data/work/beats.json`` (see module doc)."""
|
||||||
|
raise NotImplementedError("audio features (lane E / M10)")
|
||||||
28
backend/festival4d/capsule.py
Normal file
28
backend/festival4d/capsule.py
Normal file
@ -0,0 +1,28 @@
|
|||||||
|
"""Memory capsule: zero-backend shareable export (spec M17, lane G). STUB — lane G fills.
|
||||||
|
|
||||||
|
Contract: :func:`build_capsule` writes a self-contained bundle to ``out_dir`` (default
|
||||||
|
``config.CAPSULE_DIR``):
|
||||||
|
|
||||||
|
- copy ``frontend/dist`` (error clearly if missing — tell the user to ``npm run build``);
|
||||||
|
- copy media from ``config.RAW_DIR``;
|
||||||
|
- bake every read-only API response to files at the *same relative paths*
|
||||||
|
(``api/manifest``, ``api/events``, ``api/anchors``, ``api/beats``,
|
||||||
|
``api/videos/{id}/poses``, ``api/pointcloud``, ``api/splat``, ``api/audio/{id}`` —
|
||||||
|
extracting audio via ``ingest.extract_audio_hq`` if not yet cached);
|
||||||
|
- inject ``<script>window.__F4D_API_BASE__=""</script>`` into the copied ``index.html``;
|
||||||
|
- set ``"capsule": true`` in the baked manifest (the frontend hides all write UI);
|
||||||
|
- ship a stdlib ``serve.py`` in the bundle root that serves with HTTP Range support
|
||||||
|
(video seeking needs 206s; plain ``http.server`` can't — pitfall #4).
|
||||||
|
|
||||||
|
Skip what doesn't exist (no splat -> no ``api/splat``) — degrade, don't block.
|
||||||
|
Returns a summary dict (bundle path, file count, bytes) for the CLI log.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
def build_capsule(out_dir: str | Path | None = None) -> dict:
|
||||||
|
"""Build the static shareable bundle (see module doc)."""
|
||||||
|
raise NotImplementedError("memory capsule (lane G / M17)")
|
||||||
@ -4,7 +4,8 @@ FROZEN after foundation. Lanes fill in the bodies of the functions this dispatch
|
|||||||
to (in ``ingest.py``, ``audio_sync.py``, ``sfm.py``, ``events_ai.py``); they never
|
to (in ``ingest.py``, ``audio_sync.py``, ``sfm.py``, ``events_ai.py``); they never
|
||||||
edit this file or ``api.py``.
|
edit this file or ``api.py``.
|
||||||
|
|
||||||
Subcommands: ``synthetic | ingest | sync | reconstruct | events | serve``.
|
Subcommands: ``synthetic | ingest | sync | reconstruct | events | features | direct |
|
||||||
|
capsule | serve``.
|
||||||
|
|
||||||
Every subcommand dispatches to a single lane entrypoint. While a lane is still a stub,
|
Every subcommand dispatches to a single lane entrypoint. While a lane is still a stub,
|
||||||
its entrypoint raises :class:`NotImplementedError`; we catch that here and print a clear
|
its entrypoint raises :class:`NotImplementedError`; we catch that here and print a clear
|
||||||
@ -57,6 +58,32 @@ def _cmd_events(args: argparse.Namespace) -> int:
|
|||||||
return 0
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def _cmd_features(args: argparse.Namespace) -> int:
|
||||||
|
from festival4d import audio_features
|
||||||
|
|
||||||
|
result = audio_features.run_features()
|
||||||
|
log.info("features: %s", result)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def _cmd_direct(args: argparse.Namespace) -> int:
|
||||||
|
import json
|
||||||
|
|
||||||
|
from festival4d import director
|
||||||
|
|
||||||
|
path = director.generate_path(top_n=args.top_n, lead_s=args.lead_s)
|
||||||
|
print(json.dumps(path, indent=2))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def _cmd_capsule(args: argparse.Namespace) -> int:
|
||||||
|
from festival4d import capsule
|
||||||
|
|
||||||
|
result = capsule.build_capsule(out_dir=args.out)
|
||||||
|
log.info("capsule: %s", result)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
def _cmd_serve(args: argparse.Namespace) -> int:
|
def _cmd_serve(args: argparse.Namespace) -> int:
|
||||||
import uvicorn
|
import uvicorn
|
||||||
|
|
||||||
@ -97,6 +124,20 @@ def build_parser() -> argparse.ArgumentParser:
|
|||||||
help="window width in seconds when --t-global-s is given")
|
help="window width in seconds when --t-global-s is given")
|
||||||
p_evt.set_defaults(func=_cmd_events)
|
p_evt.set_defaults(func=_cmd_events)
|
||||||
|
|
||||||
|
p_feat = sub.add_parser("features", help="beat/onset analysis of the reference audio (lane E / M10)")
|
||||||
|
p_feat.set_defaults(func=_cmd_features)
|
||||||
|
|
||||||
|
p_dir = sub.add_parser("direct", help="auto-director: events -> camPath JSON on stdout (lane E / M11)")
|
||||||
|
p_dir.add_argument("--top-n", dest="top_n", type=int, default=8,
|
||||||
|
help="number of top-confidence events to cover (default 8)")
|
||||||
|
p_dir.add_argument("--lead-s", dest="lead_s", type=float, default=2.0,
|
||||||
|
help="seconds of lead-in before each event (default 2.0)")
|
||||||
|
p_dir.set_defaults(func=_cmd_direct)
|
||||||
|
|
||||||
|
p_cap = sub.add_parser("capsule", help="bake a zero-backend shareable bundle (lane G / M17)")
|
||||||
|
p_cap.add_argument("--out", default=None, help="output dir (default data/capsule/)")
|
||||||
|
p_cap.set_defaults(func=_cmd_capsule)
|
||||||
|
|
||||||
p_srv = sub.add_parser("serve", help="run the FastAPI app (M3)")
|
p_srv = sub.add_parser("serve", help="run the FastAPI app (M3)")
|
||||||
p_srv.add_argument("--host", default=config.API_HOST)
|
p_srv.add_argument("--host", default=config.API_HOST)
|
||||||
p_srv.add_argument("--port", type=int, default=config.API_PORT)
|
p_srv.add_argument("--port", type=int, default=config.API_PORT)
|
||||||
|
|||||||
@ -40,6 +40,8 @@ FRAMES_DIR = WORK_DIR / "frames" # sampled JPEGs for SfM
|
|||||||
COLMAP_DIR = WORK_DIR / "colmap" # COLMAP workspace
|
COLMAP_DIR = WORK_DIR / "colmap" # COLMAP workspace
|
||||||
DB_PATH = DATA_DIR / "project.db" # SQLite (gitignored)
|
DB_PATH = DATA_DIR / "project.db" # SQLite (gitignored)
|
||||||
|
|
||||||
|
BEATS_JSON = WORK_DIR / "beats.json" # beat/onset analysis (lane E / M10, /api/beats)
|
||||||
|
CAPSULE_DIR = DATA_DIR / "capsule" # default output of `python -m festival4d capsule` (M17)
|
||||||
POINTS_PLY = WORK_DIR / "points.ply" # reconstructed / synthetic point cloud
|
POINTS_PLY = WORK_DIR / "points.ply" # reconstructed / synthetic point cloud
|
||||||
SPLAT_PLY = WORK_DIR / "splat.ply" # optional 3DGS splat (trained externally, e.g. via MODELBEAST — see docs/modelbeast-crossover.md); viewer prefers it over the point cloud
|
SPLAT_PLY = WORK_DIR / "splat.ply" # optional 3DGS splat (trained externally, e.g. via MODELBEAST — see docs/modelbeast-crossover.md); viewer prefers it over the point cloud
|
||||||
SYNC_JSON = WORK_DIR / "sync.json" # exported sync solution (lane A)
|
SYNC_JSON = WORK_DIR / "sync.json" # exported sync solution (lane A)
|
||||||
|
|||||||
@ -12,6 +12,7 @@ Schema (spec §2)::
|
|||||||
anchors(id, label, x,y,z, color)
|
anchors(id, label, x,y,z, color)
|
||||||
events(id, t_global_s, duration_s, event_type, confidence, description, source)
|
events(id, t_global_s, duration_s, event_type, confidence, description, source)
|
||||||
annotations(id, video_id FK, t_video_s, x0,y0,x1,y1, resolved_anchor_id FK NULL)
|
annotations(id, video_id FK, t_video_s, x0,y0,x1,y1, resolved_anchor_id FK NULL)
|
||||||
|
paths(id, name, created_at, path_json) # phase 5 (M16): saved camPath JSON, as text
|
||||||
|
|
||||||
Engine management: :func:`init_engine` (re)binds the module to a SQLite file. It defaults
|
Engine management: :func:`init_engine` (re)binds the module to a SQLite file. It defaults
|
||||||
to ``config.DB_PATH`` but tests point it at a temp file. Helpers open and commit their own
|
to ``config.DB_PATH`` but tests point it at a temp file. Helpers open and commit their own
|
||||||
@ -129,6 +130,19 @@ class Annotation(Base):
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class SavedPath(Base):
|
||||||
|
"""A saved keyframed camera path (phase 5 contract #3). ``path_json`` is the frozen
|
||||||
|
camPath JSON (``{"version": 1, "keyframes": [...]}``) stored verbatim as text —
|
||||||
|
the DB never interprets it beyond POST-time validation in the API layer."""
|
||||||
|
|
||||||
|
__tablename__ = "paths"
|
||||||
|
|
||||||
|
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
||||||
|
name: Mapped[str] = mapped_column(String, nullable=False)
|
||||||
|
created_at: Mapped[str] = mapped_column(String, nullable=False)
|
||||||
|
path_json: Mapped[str] = mapped_column(String, nullable=False)
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Engine / session management
|
# Engine / session management
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@ -350,6 +364,24 @@ def update_anchor(
|
|||||||
return anchor
|
return anchor
|
||||||
|
|
||||||
|
|
||||||
|
def patch_anchor(anchor_id: int, label: str | None = None, color: str | None = None) -> Anchor:
|
||||||
|
"""Rename / recolor an anchor without moving it (phase 5 contract #4, M13).
|
||||||
|
|
||||||
|
Only provided fields change. Raises ``KeyError`` if the id is unknown.
|
||||||
|
"""
|
||||||
|
with session_scope() as s:
|
||||||
|
anchor = s.get(Anchor, anchor_id)
|
||||||
|
if anchor is None:
|
||||||
|
raise KeyError(f"no anchor with id={anchor_id}")
|
||||||
|
if label is not None:
|
||||||
|
anchor.label = label
|
||||||
|
if color is not None:
|
||||||
|
anchor.color = color
|
||||||
|
s.flush()
|
||||||
|
s.refresh(anchor)
|
||||||
|
return anchor
|
||||||
|
|
||||||
|
|
||||||
def delete_anchor(anchor_id: int) -> bool:
|
def delete_anchor(anchor_id: int) -> bool:
|
||||||
"""Delete an anchor, first unlinking any annotations that resolved to it.
|
"""Delete an anchor, first unlinking any annotations that resolved to it.
|
||||||
|
|
||||||
@ -444,6 +476,38 @@ def update_event(
|
|||||||
return event
|
return event
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Saved camera paths (phase 5 / M16)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
def add_path(name: str, path_json: str) -> SavedPath:
|
||||||
|
with session_scope() as s:
|
||||||
|
path = SavedPath(name=name, path_json=path_json, created_at=_now_iso())
|
||||||
|
s.add(path)
|
||||||
|
s.flush()
|
||||||
|
s.refresh(path)
|
||||||
|
return path
|
||||||
|
|
||||||
|
|
||||||
|
def get_paths() -> list[SavedPath]:
|
||||||
|
with session_scope() as s:
|
||||||
|
return list(s.scalars(select(SavedPath).order_by(SavedPath.id)))
|
||||||
|
|
||||||
|
|
||||||
|
def get_path(path_id: int) -> SavedPath | None:
|
||||||
|
with session_scope() as s:
|
||||||
|
return s.get(SavedPath, path_id)
|
||||||
|
|
||||||
|
|
||||||
|
def delete_path(path_id: int) -> bool:
|
||||||
|
"""Delete a saved path. Returns ``True`` if a row was deleted, ``False`` if unknown."""
|
||||||
|
with session_scope() as s:
|
||||||
|
path = s.get(SavedPath, path_id)
|
||||||
|
if path is None:
|
||||||
|
return False
|
||||||
|
s.delete(path)
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Annotations
|
# Annotations
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|||||||
25
backend/festival4d/director.py
Normal file
25
backend/festival4d/director.py
Normal file
@ -0,0 +1,25 @@
|
|||||||
|
"""Auto-director: events -> keyframed camera path (spec M11, lane E). STUB — lane E fills.
|
||||||
|
|
||||||
|
Contract (frozen at foundation2 merge — phase-5 contract #2): :func:`generate_path`
|
||||||
|
returns a **camPath-compatible** dict::
|
||||||
|
|
||||||
|
{"version": 1,
|
||||||
|
"keyframes": [{"t_global": float, "pos": [x, y, z], "quat": [x, y, z, w], "fov": float}]}
|
||||||
|
|
||||||
|
Poses are **Three.js scene space**, quaternion order **[x, y, z, w]** — convert from the
|
||||||
|
COLMAP rows via :func:`festival4d.geometry.colmap_to_threejs`, never inline (pitfall #1).
|
||||||
|
|
||||||
|
Deterministic v1 rules — no ML: take the top-N events by confidence (ties -> earlier);
|
||||||
|
for each, choose the registered camera with a pose nearest the event time; keyframes at
|
||||||
|
``t_event - lead_s`` and ``t_event + duration``; when ``config.BEATS_JSON`` exists, snap
|
||||||
|
keyframe times to the nearest beat (no beats file -> no snapping — degrade, don't block);
|
||||||
|
FOV from that camera's intrinsics (``2 * atan(height / (2 * fy))`` in degrees, the same
|
||||||
|
formula as ``scene3d.snapTo``).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
|
||||||
|
def generate_path(top_n: int = 8, lead_s: float = 2.0) -> dict:
|
||||||
|
"""Build a camPath JSON dict from the detected events (see module doc)."""
|
||||||
|
raise NotImplementedError("auto-director (lane E / M11)")
|
||||||
@ -34,8 +34,9 @@ def test_manifest_shape(client):
|
|||||||
data = client.get("/api/manifest").json()
|
data = client.get("/api/manifest").json()
|
||||||
# Exact-set lock: additive manifest fields are fine, but must be added here consciously
|
# Exact-set lock: additive manifest fields are fine, but must be added here consciously
|
||||||
# (this assertion has already caught two silent drifts — has_splat and has_capture).
|
# (this assertion has already caught two silent drifts — has_splat and has_capture).
|
||||||
assert set(data) == {"videos", "t_global_max", "has_poses", "has_splat", "has_capture"}
|
assert set(data) == {"videos", "t_global_max", "has_poses", "has_splat", "has_capture", "capsule"}
|
||||||
assert data["has_poses"] is True
|
assert data["has_poses"] is True
|
||||||
|
assert data["capsule"] is False # live server is never a capsule (M17 bakes true)
|
||||||
assert data["has_capture"] is False # capture routes are opt-in (FESTIVAL4D_CAPTURE)
|
assert data["has_capture"] is False # capture routes are opt-in (FESTIVAL4D_CAPTURE)
|
||||||
assert len(data["videos"]) == 3
|
assert len(data["videos"]) == 3
|
||||||
v = data["videos"][0]
|
v = data["videos"][0]
|
||||||
|
|||||||
107
backend/tests/test_phase5_api.py
Normal file
107
backend/tests/test_phase5_api.py
Normal file
@ -0,0 +1,107 @@
|
|||||||
|
"""Phase-5 foundation2 contract tests: the new route shapes lanes E/F/G depend on
|
||||||
|
(plan/20-phase5.md "Contracts"). Lane G deepens paths coverage in test_paths.py;
|
||||||
|
lane E owns the real beats/director behavior tests.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import shutil
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
pytestmark = pytest.mark.skipif(
|
||||||
|
shutil.which("ffmpeg") is None or shutil.which("ffprobe") is None,
|
||||||
|
reason="ffmpeg/ffprobe required to build the API test fixture",
|
||||||
|
)
|
||||||
|
|
||||||
|
CAMPATH = {
|
||||||
|
"version": 1,
|
||||||
|
"keyframes": [
|
||||||
|
{"t_global": 1.0, "pos": [0, 1, 5], "quat": [0, 0, 0, 1], "fov": 50},
|
||||||
|
{"t_global": 4.0, "pos": [2, 1, 4], "quat": [0, 0.3826834, 0, 0.9238795], "fov": 42},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(scope="module")
|
||||||
|
def client():
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from festival4d import synthetic
|
||||||
|
|
||||||
|
synthetic.build(duration_s=1.0, run_ffmpeg=True) # into the temp data dir (conftest)
|
||||||
|
|
||||||
|
from festival4d import api
|
||||||
|
|
||||||
|
with TestClient(api.app) as c:
|
||||||
|
yield c
|
||||||
|
|
||||||
|
|
||||||
|
def test_beats_404_then_serves_file(client):
|
||||||
|
from festival4d import config
|
||||||
|
|
||||||
|
config.BEATS_JSON.unlink(missing_ok=True)
|
||||||
|
assert client.get("/api/beats").status_code == 404
|
||||||
|
body = {"tempo_bpm": 120.0, "beats_s": [0.5, 1.0], "onsets": [], "generated_by": "festival4d.audio_features"}
|
||||||
|
config.BEATS_JSON.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
config.BEATS_JSON.write_text(json.dumps(body))
|
||||||
|
try:
|
||||||
|
assert client.get("/api/beats").json() == body
|
||||||
|
finally:
|
||||||
|
config.BEATS_JSON.unlink()
|
||||||
|
|
||||||
|
|
||||||
|
def test_director_degrades_to_valid_empty_campath(client):
|
||||||
|
# While lane E is stubbed the response must still round-trip through camPath.fromJSON:
|
||||||
|
# a valid version-1 path with zero keyframes, plus a "note" marking the degradation.
|
||||||
|
data = client.post("/api/director", json={}).json()
|
||||||
|
assert data["version"] == 1
|
||||||
|
assert data["keyframes"] == []
|
||||||
|
assert "note" in data
|
||||||
|
assert client.post("/api/director", json={"top_n": 3, "lead_s": 1.0}).status_code == 200
|
||||||
|
|
||||||
|
|
||||||
|
def test_paths_crud_roundtrip(client):
|
||||||
|
assert client.get("/api/paths").json() == []
|
||||||
|
created = client.post(
|
||||||
|
"/api/paths", json={"name": "sweep", "json": json.dumps(CAMPATH)}
|
||||||
|
).json()
|
||||||
|
assert set(created) == {"id", "name", "created_at", "json"}
|
||||||
|
assert created["json"] == CAMPATH # deep-equal roundtrip
|
||||||
|
pid = created["id"]
|
||||||
|
|
||||||
|
listing = client.get("/api/paths").json()
|
||||||
|
assert [set(p) for p in listing] == [{"id", "name", "created_at"}]
|
||||||
|
assert client.get(f"/api/paths/{pid}").json()["json"] == CAMPATH
|
||||||
|
|
||||||
|
assert client.delete(f"/api/paths/{pid}").json() == {"deleted": pid}
|
||||||
|
assert client.get(f"/api/paths/{pid}").status_code == 404
|
||||||
|
assert client.delete(f"/api/paths/{pid}").status_code == 404
|
||||||
|
|
||||||
|
|
||||||
|
def test_paths_post_validates_campath(client):
|
||||||
|
post = lambda body: client.post("/api/paths", json={"name": "bad", "json": body}).status_code
|
||||||
|
assert post("not json at all {") == 422
|
||||||
|
assert post(json.dumps({"version": 2, "keyframes": []})) == 422
|
||||||
|
assert post(json.dumps({"version": 1})) == 422
|
||||||
|
assert post(json.dumps({"version": 1, "keyframes": [{"t_global": 0}]})) == 422
|
||||||
|
|
||||||
|
|
||||||
|
def test_anchor_patch_label_and_color(client):
|
||||||
|
a = client.post("/api/anchors", json={"label": "F", "x": 0, "y": 0, "z": 0}).json()
|
||||||
|
patched = client.patch(f"/api/anchors/{a['id']}", json={"label": "Fred", "color": "#4dd0ff"}).json()
|
||||||
|
assert patched["label"] == "Fred" and patched["color"] == "#4dd0ff"
|
||||||
|
assert (patched["x"], patched["y"], patched["z"]) == (0, 0, 0) # position untouched
|
||||||
|
# partial patch: color-only leaves the label alone
|
||||||
|
assert client.patch(f"/api/anchors/{a['id']}", json={"color": "#fff"}).json()["label"] == "Fred"
|
||||||
|
assert client.patch("/api/anchors/99999", json={"label": "x"}).status_code == 404
|
||||||
|
|
||||||
|
|
||||||
|
def test_cli_dispatches_stubs_gracefully():
|
||||||
|
# The three new subcommands exist and degrade per the house pattern: exit code 2,
|
||||||
|
# no traceback, while their lane entrypoints are stubs.
|
||||||
|
from festival4d import cli
|
||||||
|
|
||||||
|
for cmd in (["features"], ["direct"], ["capsule"]):
|
||||||
|
assert cli.main(cmd) == 2
|
||||||
@ -197,6 +197,10 @@
|
|||||||
pointer-events: none; z-index: 6; border-radius: 2px;
|
pointer-events: none; z-index: 6; border-radius: 2px;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* Capsule mode (M17, phase-5 contract #5): a static baked bundle has no write API, so
|
||||||
|
everything marked .write-ui disappears when main.js sets body.capsule from the manifest. */
|
||||||
|
body.capsule .write-ui { display: none !important; }
|
||||||
|
|
||||||
/* Loading / error */
|
/* Loading / error */
|
||||||
#loading {
|
#loading {
|
||||||
position: fixed; inset: 0; display: flex; align-items: center; justify-content: center;
|
position: fixed; inset: 0; display: flex; align-items: center; justify-content: center;
|
||||||
@ -228,13 +232,32 @@
|
|||||||
<button class="btn" id="btn-path-export" title="Export path as JSON">⤓</button>
|
<button class="btn" id="btn-path-export" title="Export path as JSON">⤓</button>
|
||||||
<button class="btn" id="btn-path-import" title="Import path JSON">⤒</button>
|
<button class="btn" id="btn-path-import" title="Import path JSON">⤒</button>
|
||||||
<button class="btn" id="btn-path-clear" title="Clear all keyframes">✕</button>
|
<button class="btn" id="btn-path-clear" title="Clear all keyframes">✕</button>
|
||||||
|
<!-- Phase 5: hidden until the owning lane's module reports ready (plan/20-phase5.md) -->
|
||||||
|
<button class="btn" id="btn-director" style="display:none"
|
||||||
|
title="Auto-director: build a camera path from the detected events (M11)">🎬 Auto-path</button>
|
||||||
|
<button class="btn write-ui" id="btn-path-save" style="display:none"
|
||||||
|
title="Save the current camera path to the project (M16)">💾</button>
|
||||||
|
<select class="btn" id="path-load-select" style="display:none"
|
||||||
|
title="Load a saved camera path (M16)"></select>
|
||||||
|
<button class="btn write-ui" id="btn-path-del" style="display:none"
|
||||||
|
title="Delete the selected saved path (M16)">🗑</button>
|
||||||
<span class="ctl-sep"></span>
|
<span class="ctl-sep"></span>
|
||||||
<button class="btn" id="btn-spatial" style="display:none"
|
<button class="btn" id="btn-spatial" style="display:none"
|
||||||
title="Spatial audio: hear every camera from its position in the scene">🎧 3D audio</button>
|
title="Spatial audio: hear every camera from its position in the scene">🎧 3D audio</button>
|
||||||
|
<button class="btn" id="btn-export" style="display:none"
|
||||||
|
title="Export the camera path as a cinematic webm (M12)">⏺ Export</button>
|
||||||
|
<button class="btn" id="btn-photo" style="display:none"
|
||||||
|
title="Photo mode: hi-res still of the 3D scene (M14, P)">📷</button>
|
||||||
|
<button class="btn" id="btn-fx" style="display:none"
|
||||||
|
title="Moment FX: bursts and pulses on event markers (M15)">✨</button>
|
||||||
|
<button class="btn" id="btn-anchors" style="display:none"
|
||||||
|
title="Anchors & friend tags (M13)">⚓</button>
|
||||||
<input type="file" id="path-file" accept="application/json,.json" hidden />
|
<input type="file" id="path-file" accept="application/json,.json" hidden />
|
||||||
</div>
|
</div>
|
||||||
<div id="scene-hint">drag to orbit · 1–9 snap to camera · K add keyframe · Esc detach</div>
|
<div id="scene-hint">drag to orbit · 1–9 snap to camera · K add keyframe · Esc detach</div>
|
||||||
<div id="annot-panel"></div>
|
<div id="annot-panel" class="write-ui"></div>
|
||||||
|
<!-- M13 anchor manager mounts here (lane F); hidden until anchorPanel.js is filled -->
|
||||||
|
<div id="anchor-panel" style="display:none"></div>
|
||||||
</div>
|
</div>
|
||||||
<div id="grid-pane">
|
<div id="grid-pane">
|
||||||
<div id="video-grid"></div>
|
<div id="video-grid"></div>
|
||||||
|
|||||||
19
frontend/src/anchorPanel.js
Normal file
19
frontend/src/anchorPanel.js
Normal file
@ -0,0 +1,19 @@
|
|||||||
|
// Anchor manager & friend tags (spec M13, lane F). STUB — lane F fills THIS FILE ONLY.
|
||||||
|
//
|
||||||
|
// Pre-wired by foundation2: the hidden ⚓ toggle (#btn-anchors) and the empty #anchor-panel
|
||||||
|
// container in index.html, this module's import + init() call in main.js, the hooks
|
||||||
|
// `scene3d.focusOn(x, y, z)` (jump-to) and `PATCH ${state.apiBase}/api/anchors/{id}`
|
||||||
|
// {label?, color?} (rename/recolor), plus DELETE /api/anchors/{id} from phase 4.
|
||||||
|
//
|
||||||
|
// When implementing: unhide the button, set `ready = true`; build the collapsible list
|
||||||
|
// (color dot, label, jump-to, rename, recolor, delete) from state.anchors; after any
|
||||||
|
// mutation update state.anchors and emit("anchors-changed") so scene3d + overlays refresh.
|
||||||
|
// The + Tag flow reuses the M8 bbox-annotation flow WITHOUT an event (annotate.js is
|
||||||
|
// frozen — drive it, don't edit it) and names the resulting anchor.
|
||||||
|
// Capsule mode: give every MUTATING control the `write-ui` class — body.capsule hides that
|
||||||
|
// class globally (contract #5); the read-only list + jump-to stay available.
|
||||||
|
|
||||||
|
export const anchorPanel = {
|
||||||
|
ready: false,
|
||||||
|
init() {}, // ponytail: stub — lane F replaces the body
|
||||||
|
};
|
||||||
@ -27,6 +27,7 @@ export class CamPath {
|
|||||||
if (!this.group) {
|
if (!this.group) {
|
||||||
this.group = new THREE.Group();
|
this.group = new THREE.Group();
|
||||||
scene3d.addObject(this.group);
|
scene3d.addObject(this.group);
|
||||||
|
scene3d.registerHelper(this.group); // keyframe gizmos hide with the helpers (M14)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
16
frontend/src/director.js
Normal file
16
frontend/src/director.js
Normal file
@ -0,0 +1,16 @@
|
|||||||
|
// Auto-director UI (spec M11, lane E). STUB — lane E fills THIS FILE ONLY.
|
||||||
|
//
|
||||||
|
// Pre-wired by foundation2: the hidden 🎬 button (#btn-director in index.html), and this
|
||||||
|
// module's import + init() call in main.js. main.js does NOT wire the click — you do,
|
||||||
|
// inside init().
|
||||||
|
//
|
||||||
|
// When implementing: unhide the button (el.style.display = ""), set `ready = true`; on click
|
||||||
|
// POST {top_n?, lead_s?} to `${state.apiBase}/api/director` and load the response with
|
||||||
|
// camPath.fromJSON(...) — never edit camPath.js (frozen). The response is ALWAYS a valid
|
||||||
|
// camPath JSON; zero keyframes means "no events yet / stubbed" — surface that in the UI,
|
||||||
|
// don't crash (pitfall #5). Play it via the existing #btn-path flow.
|
||||||
|
|
||||||
|
export const director = {
|
||||||
|
ready: false, // flips to true when the lane implements this module
|
||||||
|
init() {}, // ponytail: stub — lane E replaces the body
|
||||||
|
};
|
||||||
18
frontend/src/exportVideo.js
Normal file
18
frontend/src/exportVideo.js
Normal file
@ -0,0 +1,18 @@
|
|||||||
|
// Cinematic export (spec M12, lane E). STUB — lane E fills THIS FILE ONLY.
|
||||||
|
//
|
||||||
|
// Pre-wired by foundation2: the hidden ⏺ button (#btn-export in index.html), this module's
|
||||||
|
// import + init() call in main.js, and the transport hooks
|
||||||
|
// `transport.captureAudioStream()` -> MediaStream tapping the live WebAudio mix (post-gain,
|
||||||
|
// spatial included) and `transport.releaseAudioStream()` when done.
|
||||||
|
//
|
||||||
|
// When implementing: unhide the button, set `ready = true`; on click seek to the first
|
||||||
|
// camPath keyframe, play the path, mux scene3d canvas `captureStream(30)` + the audio stream
|
||||||
|
// with MediaRecorder (video/webm;codecs=vp9,opus), stop at the last keyframe, download
|
||||||
|
// `festival4d-cut.webm`, then RESTORE all prior UI/transport state. The canvas is
|
||||||
|
// `scene3d.renderer.domElement`. MediaRecorder needs a *focused* tab to capture real frames
|
||||||
|
// (pitfall #2) — say so in your evidence if you couldn't.
|
||||||
|
|
||||||
|
export const exportVideo = {
|
||||||
|
ready: false,
|
||||||
|
init() {}, // ponytail: stub — lane E replaces the body
|
||||||
|
};
|
||||||
19
frontend/src/fx.js
Normal file
19
frontend/src/fx.js
Normal file
@ -0,0 +1,19 @@
|
|||||||
|
// Moment FX (spec M15, lane F). STUB — lane F fills THIS FILE ONLY.
|
||||||
|
//
|
||||||
|
// Pre-wired by foundation2: the hidden ✨ toggle (#btn-fx in index.html), this module's
|
||||||
|
// import + init() call in main.js, and a per-frame `fx.update(tGlobal)` call from the master
|
||||||
|
// loop (after transport.tick(), before scene3d.update()).
|
||||||
|
//
|
||||||
|
// When implementing: unhide the button, set `ready = true`. update() watches the playhead
|
||||||
|
// crossing event markers (state.events) — detect crossings in play AND scrub (track the
|
||||||
|
// previous tGlobal; a seek emits "seek" if you need to reset). On `pyro`/`confetti` fire a
|
||||||
|
// short particle burst at the event's resolved anchor, falling back to the stage centroid;
|
||||||
|
// on `bass_drop` pulse the point-cloud/splat scale for ~a beat (beats via GET /api/beats
|
||||||
|
// when present — no beats file, use ~0.5 s; pitfall #5). Scene access ONLY through
|
||||||
|
// scene3d.addObject/removeObject. Keep >= 30 fps on the synthetic scene.
|
||||||
|
|
||||||
|
export const fx = {
|
||||||
|
ready: false,
|
||||||
|
init() {}, // ponytail: stub — lane F replaces the body
|
||||||
|
update(tGlobal) {}, // called every animation frame by main.js
|
||||||
|
};
|
||||||
@ -16,6 +16,15 @@ import { scene3d, videoColor } from "./scene3d.js";
|
|||||||
import { timeline } from "./timeline.js";
|
import { timeline } from "./timeline.js";
|
||||||
import { annotator } from "./annotate.js";
|
import { annotator } from "./annotate.js";
|
||||||
import { camPath } from "./camPath.js";
|
import { camPath } from "./camPath.js";
|
||||||
|
// Phase-5 lane modules (stubs until lanes E/F/G fill them; each wires its own hidden button).
|
||||||
|
import { director } from "./director.js";
|
||||||
|
import { exportVideo } from "./exportVideo.js";
|
||||||
|
import { anchorPanel } from "./anchorPanel.js";
|
||||||
|
import { photoMode } from "./photoMode.js";
|
||||||
|
import { fx } from "./fx.js";
|
||||||
|
import { pathsStore } from "./pathsStore.js";
|
||||||
|
|
||||||
|
const laneModules = { director, exportVideo, anchorPanel, photoMode, fx, pathsStore };
|
||||||
|
|
||||||
async function getJSON(path) {
|
async function getJSON(path) {
|
||||||
const resp = await fetch(API_BASE + path);
|
const resp = await fetch(API_BASE + path);
|
||||||
@ -49,6 +58,9 @@ async function boot() {
|
|||||||
link.style.display = "";
|
link.style.display = "";
|
||||||
}
|
}
|
||||||
state.hasSplat = !!manifest.has_splat;
|
state.hasSplat = !!manifest.has_splat;
|
||||||
|
// Capsule (M17): a static baked bundle has no write API — hide everything .write-ui.
|
||||||
|
state.capsule = !!manifest.capsule;
|
||||||
|
document.body.classList.toggle("capsule", state.capsule);
|
||||||
for (const v of state.videos) state.enabled[v.id] = true;
|
for (const v of state.videos) state.enabled[v.id] = true;
|
||||||
state.audioSourceId = referenceVideoId();
|
state.audioSourceId = referenceVideoId();
|
||||||
|
|
||||||
@ -75,8 +87,8 @@ async function boot() {
|
|||||||
// localStorage.setItem("f4dDebug", "1").
|
// localStorage.setItem("f4dDebug", "1").
|
||||||
if (import.meta.env?.DEV || localStorage.getItem("f4dDebug")) {
|
if (import.meta.env?.DEV || localStorage.getItem("f4dDebug")) {
|
||||||
window.__f4d = {
|
window.__f4d = {
|
||||||
state, transport, scene3d, timeline, cells: () => cells,
|
state, transport, scene3d, timeline, camPath, cells: () => cells,
|
||||||
poseAt, projectWorldToVideoPx,
|
poseAt, projectWorldToVideoPx, ...laneModules,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
@ -120,6 +132,16 @@ function buildUI() {
|
|||||||
wireKeyboard();
|
wireKeyboard();
|
||||||
positionVideosWhenReady();
|
positionVideosWhenReady();
|
||||||
|
|
||||||
|
// Phase-5 lane modules: each init() wires its own (hidden) controls and unhides them when
|
||||||
|
// the lane has filled the stub. A broken module degrades to its button staying hidden.
|
||||||
|
for (const [name, mod] of Object.entries(laneModules)) {
|
||||||
|
try {
|
||||||
|
mod.init();
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(`[festival4d] lane module "${name}" init failed`, err);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// Reflect follow/roam state on the free-roam button; snapping cancels a playing path.
|
// Reflect follow/roam state on the free-roam button; snapping cancels a playing path.
|
||||||
on("follow", (id) => {
|
on("follow", (id) => {
|
||||||
el("btn-roam").classList.toggle("active", id == null);
|
el("btn-roam").classList.toggle("active", id == null);
|
||||||
@ -244,6 +266,7 @@ function startLoop() {
|
|||||||
const frame = () => {
|
const frame = () => {
|
||||||
transport.tick();
|
transport.tick();
|
||||||
for (const cell of cells) drawOverlay(cell);
|
for (const cell of cells) drawOverlay(cell);
|
||||||
|
fx.update(state.tGlobal); // M15: playhead-crossing effects (no-op while stubbed)
|
||||||
camPath.update(state.tGlobal); // M9: drive the viewer camera before the scene renders
|
camPath.update(state.tGlobal); // M9: drive the viewer camera before the scene renders
|
||||||
scene3d.update();
|
scene3d.update();
|
||||||
timeline.update(state.tGlobal);
|
timeline.update(state.tGlobal);
|
||||||
|
|||||||
18
frontend/src/pathsStore.js
Normal file
18
frontend/src/pathsStore.js
Normal file
@ -0,0 +1,18 @@
|
|||||||
|
// Server-side camera paths UI (spec M16, lane G). STUB — lane G fills THIS FILE ONLY.
|
||||||
|
//
|
||||||
|
// Pre-wired by foundation2: the hidden controls in index.html — #btn-path-save (💾),
|
||||||
|
// #path-load-select (dropdown), #btn-path-del (🗑) — this module's import + init() call in
|
||||||
|
// main.js, and the API: GET /api/paths (list of {id,name,created_at}),
|
||||||
|
// GET /api/paths/{id} ({..., json: <camPath JSON>}), POST /api/paths {name, json:<text>}
|
||||||
|
// (422 on invalid), DELETE /api/paths/{id}.
|
||||||
|
//
|
||||||
|
// When implementing: unhide the controls, set `ready = true`. Save = prompt for a name,
|
||||||
|
// POST camPath.toJSON(); load = fetch the selected path and camPath.fromJSON(json); delete =
|
||||||
|
// DELETE the selected id. Refresh the dropdown after every mutation. #btn-path-save and
|
||||||
|
// #btn-path-del already carry the `write-ui` class, so capsule mode (contract #5) hides
|
||||||
|
// them for free — the load dropdown may stay if you bake paths in, else hide it too.
|
||||||
|
|
||||||
|
export const pathsStore = {
|
||||||
|
ready: false,
|
||||||
|
init() {}, // ponytail: stub — lane G replaces the body
|
||||||
|
};
|
||||||
18
frontend/src/photoMode.js
Normal file
18
frontend/src/photoMode.js
Normal file
@ -0,0 +1,18 @@
|
|||||||
|
// Photo mode (spec M14, lane F). STUB — lane F fills THIS FILE ONLY.
|
||||||
|
//
|
||||||
|
// Pre-wired by foundation2: the hidden 📷 button (#btn-photo in index.html), this module's
|
||||||
|
// import + init() call in main.js, and the hook `scene3d.setHelpersVisible(false)` which
|
||||||
|
// hides grid/axes/frusta/path-lines/keyframe-gizmos/anchor-labels in one call (restore with
|
||||||
|
// true).
|
||||||
|
//
|
||||||
|
// When implementing: unhide the button, set `ready = true`, bind key "P" too; on trigger
|
||||||
|
// pause the transport, setHelpersVisible(false), render ONE frame at 3840-wide (preserve
|
||||||
|
// aspect) to an offscreen WebGLRenderTarget via scene3d.renderer/scene/camera, download the
|
||||||
|
// PNG, then restore everything (helpers, renderer size, play state). Works in free-roam and
|
||||||
|
// follow-cam; with a splat present it renders the splat (no splat -> the point cloud —
|
||||||
|
// pitfall #5).
|
||||||
|
|
||||||
|
export const photoMode = {
|
||||||
|
ready: false,
|
||||||
|
init() {}, // ponytail: stub — lane F replaces the body
|
||||||
|
};
|
||||||
@ -65,6 +65,8 @@ export class Scene3D {
|
|||||||
this._raf = null;
|
this._raf = null;
|
||||||
this.anchorGroup = null; // 3D anchor spheres + labels (M6 anchors + M8 resolved)
|
this.anchorGroup = null; // 3D anchor spheres + labels (M6 anchors + M8 resolved)
|
||||||
this.pathMode = false; // when true, camPath (M9) drives the camera; we don't touch it
|
this.pathMode = false; // when true, camPath (M9) drives the camera; we don't touch it
|
||||||
|
this.helpersVisible = true; // M14 hook: grid/axes/frusta/gizmos/labels toggle
|
||||||
|
this._helperObjs = []; // extra helper objects registered by other modules (camPath gizmos)
|
||||||
}
|
}
|
||||||
|
|
||||||
init(canvas) {
|
init(canvas) {
|
||||||
@ -96,6 +98,8 @@ export class Scene3D {
|
|||||||
this.scene.add(grid);
|
this.scene.add(grid);
|
||||||
const axes = new THREE.AxesHelper(1.5);
|
const axes = new THREE.AxesHelper(1.5);
|
||||||
this.scene.add(axes);
|
this.scene.add(axes);
|
||||||
|
this._grid = grid;
|
||||||
|
this._axes = axes;
|
||||||
|
|
||||||
this.anchorGroup = new THREE.Group();
|
this.anchorGroup = new THREE.Group();
|
||||||
this.scene.add(this.anchorGroup);
|
this.scene.add(this.anchorGroup);
|
||||||
@ -129,12 +133,13 @@ export class Scene3D {
|
|||||||
if (a.label) {
|
if (a.label) {
|
||||||
const label = makeLabelSprite(a.label);
|
const label = makeLabelSprite(a.label);
|
||||||
label.position.set(a.x, a.y + 0.3, a.z);
|
label.position.set(a.x, a.y + 0.3, a.z);
|
||||||
|
label.visible = this.helpersVisible; // photo mode (M14) may be hiding labels right now
|
||||||
this.anchorGroup.add(label);
|
this.anchorGroup.add(label);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Expose the scene graph for auxiliary overlays (e.g. camPath gizmos, M9). */
|
/** Expose the scene graph for auxiliary overlays (e.g. camPath gizmos M9, moment FX M15). */
|
||||||
addObject(obj) {
|
addObject(obj) {
|
||||||
this.scene?.add(obj);
|
this.scene?.add(obj);
|
||||||
}
|
}
|
||||||
@ -142,6 +147,51 @@ export class Scene3D {
|
|||||||
this.scene?.remove(obj);
|
this.scene?.remove(obj);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Register an added object as a "helper" so setHelpersVisible governs it (camPath gizmos). */
|
||||||
|
registerHelper(obj) {
|
||||||
|
this._helperObjs.push(obj);
|
||||||
|
obj.visible = this.helpersVisible;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* M14 hook (phase-5 contract #6): show/hide every non-content visual in one call — grid,
|
||||||
|
* axes, camera frusta + markers + path lines, registered gizmo groups, and anchor LABEL
|
||||||
|
* sprites (anchor spheres stay: friend tags belong in photos). update() keeps enforcing
|
||||||
|
* the flag on the per-video rigs, whose visibility it recomputes every frame.
|
||||||
|
*/
|
||||||
|
setHelpersVisible(visible) {
|
||||||
|
this.helpersVisible = !!visible;
|
||||||
|
if (this._grid) this._grid.visible = this.helpersVisible;
|
||||||
|
if (this._axes) this._axes.visible = this.helpersVisible;
|
||||||
|
for (const obj of this._helperObjs) obj.visible = this.helpersVisible;
|
||||||
|
if (this.anchorGroup)
|
||||||
|
for (const child of this.anchorGroup.children)
|
||||||
|
if (child.isSprite) child.visible = this.helpersVisible;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* M13 hook (phase-5 contract #6): fly the free-roam camera's attention to a world point
|
||||||
|
* (anchor jump-to). Detaches follow-cam / path mode, retargets OrbitControls, and backs the
|
||||||
|
* camera off along its current viewing direction.
|
||||||
|
*/
|
||||||
|
focusOn(x, y, z, dist = 6) {
|
||||||
|
if (this.pathMode) this.exitPathMode();
|
||||||
|
if (state.followCameraId != null) {
|
||||||
|
state.followCameraId = null;
|
||||||
|
this._tween = null;
|
||||||
|
emit("follow", null);
|
||||||
|
}
|
||||||
|
this.camera.fov = DEFAULT_FOV;
|
||||||
|
this.camera.updateProjectionMatrix();
|
||||||
|
const dir = this.camera.position.clone().sub(this.controls.target);
|
||||||
|
if (dir.lengthSq() < 1e-6) dir.set(0, 0.5, 1);
|
||||||
|
dir.normalize().multiplyScalar(dist);
|
||||||
|
this.controls.target.set(x, y, z);
|
||||||
|
this.camera.position.set(x + dir.x, y + dir.y, z + dir.z);
|
||||||
|
this.controls.enabled = true;
|
||||||
|
this.controls.update();
|
||||||
|
}
|
||||||
|
|
||||||
/** M9: hand camera control to camPath. update() then leaves the camera alone. */
|
/** M9: hand camera control to camPath. update() then leaves the camera alone. */
|
||||||
enterPathMode() {
|
enterPathMode() {
|
||||||
this.pathMode = true;
|
this.pathMode = true;
|
||||||
@ -362,8 +412,8 @@ export class Scene3D {
|
|||||||
if (!rig) continue;
|
if (!rig) continue;
|
||||||
const pose = this._livePose(v.id);
|
const pose = this._livePose(v.id);
|
||||||
const on = pose && state.enabled[v.id] !== false;
|
const on = pose && state.enabled[v.id] !== false;
|
||||||
rig.group.visible = !!on;
|
rig.group.visible = !!on && this.helpersVisible;
|
||||||
if (rig.path) rig.path.visible = state.enabled[v.id] !== false;
|
if (rig.path) rig.path.visible = state.enabled[v.id] !== false && this.helpersVisible;
|
||||||
if (!on) continue;
|
if (!on) continue;
|
||||||
const { matrixWorld } = colmapToThreejs(pose.q, pose.t);
|
const { matrixWorld } = colmapToThreejs(pose.q, pose.t);
|
||||||
_m4.fromArray(matrixWorld);
|
_m4.fromArray(matrixWorld);
|
||||||
|
|||||||
@ -7,7 +7,11 @@
|
|||||||
// A hosted build sets VITE_API_BASE to a same-origin path prefix (e.g. "/festifun") so every
|
// A hosted build sets VITE_API_BASE to a same-origin path prefix (e.g. "/festifun") so every
|
||||||
// request — fetch, <video> src, PLY loader — goes through the reverse proxy under that path.
|
// request — fetch, <video> src, PLY loader — goes through the reverse proxy under that path.
|
||||||
// Empty string = same-origin at root. See deploy/DEPLOY.md.
|
// Empty string = same-origin at root. See deploy/DEPLOY.md.
|
||||||
export const API_BASE = import.meta.env.VITE_API_BASE ?? "http://localhost:8000";
|
// Phase-5 contract #5: a RUNTIME override wins over the build-time value — the memory capsule
|
||||||
|
// (M17) injects `window.__F4D_API_BASE__ = ""` into its baked index.html so the same dist
|
||||||
|
// bundle turns zero-backend. `??` (not `||`): empty string is a valid value (same-origin).
|
||||||
|
export const API_BASE =
|
||||||
|
window.__F4D_API_BASE__ ?? import.meta.env.VITE_API_BASE ?? "http://localhost:8000";
|
||||||
|
|
||||||
export const state = {
|
export const state = {
|
||||||
apiBase: API_BASE,
|
apiBase: API_BASE,
|
||||||
@ -18,6 +22,7 @@ export const state = {
|
|||||||
tGlobalMax: 0,
|
tGlobalMax: 0,
|
||||||
hasPoses: false,
|
hasPoses: false,
|
||||||
hasSplat: false, // optional 3DGS splat at /api/splat (preferred over the point cloud)
|
hasSplat: false, // optional 3DGS splat at /api/splat (preferred over the point cloud)
|
||||||
|
capsule: false, // manifest.capsule (M17): static baked bundle => hide ALL write UI
|
||||||
poses: {}, // id -> [{frame_idx, t_video_s, q:[w,x,y,z], t:[x,y,z], intrinsics, registered}]
|
poses: {}, // id -> [{frame_idx, t_video_s, q:[w,x,y,z], t:[x,y,z], intrinsics, registered}]
|
||||||
anchors: [], // [{id, label, x, y, z, color}]
|
anchors: [], // [{id, label, x, y, z, color}]
|
||||||
events: [], // [{id, t_global_s, duration_s, event_type, confidence, description, source}]
|
events: [], // [{id, t_global_s, duration_s, event_type, confidence, description, source}]
|
||||||
|
|||||||
@ -42,6 +42,7 @@ export class Transport {
|
|||||||
this._buffers = new Map(); // video id -> AudioBuffer (decoded) | null (unavailable)
|
this._buffers = new Map(); // video id -> AudioBuffer (decoded) | null (unavailable)
|
||||||
this._decoding = new Set(); // video ids with a fetch/decode in flight
|
this._decoding = new Set(); // video ids with a fetch/decode in flight
|
||||||
this._tracks = new Map(); // playing tracks: video id -> { src, gain, panner|null }
|
this._tracks = new Map(); // playing tracks: video id -> { src, gain, panner|null }
|
||||||
|
this._captureDest = null; // MediaStreamAudioDestinationNode while an export taps the mix (M12)
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Register the video cells the transport drives and start their frame loops. */
|
/** Register the video cells the transport drives and start their frame loops. */
|
||||||
@ -296,11 +297,42 @@ export class Transport {
|
|||||||
} else {
|
} else {
|
||||||
gain.connect(ctx.destination);
|
gain.connect(ctx.destination);
|
||||||
}
|
}
|
||||||
|
if (this._captureDest) (panner ?? gain).connect(this._captureDest); // live export keeps hearing us
|
||||||
if (tv >= 0) src.start(0, tv);
|
if (tv >= 0) src.start(0, tv);
|
||||||
else src.start(ctx.currentTime + -tv / state.rate, 0); // camera starts later than t_global
|
else src.start(ctx.currentTime + -tv / state.rate, 0); // camera starts later than t_global
|
||||||
this._tracks.set(id, { src, gain, panner });
|
this._tracks.set(id, { src, gain, panner });
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* M12 hook (phase-5 contract #6): a MediaStream carrying the live WebAudio mix — tapped
|
||||||
|
* post-gain, after the spatial panner when 🎧 is on — for MediaRecorder muxing. Tracks that
|
||||||
|
* (re)start while the tap is live (seeks, source switches) join it automatically
|
||||||
|
* (_startTrack above). With no decoded track (fallback <video> audio) the stream is silent —
|
||||||
|
* degrade, don't block (pitfall #5). Call releaseAudioStream() when the export ends.
|
||||||
|
*/
|
||||||
|
captureAudioStream() {
|
||||||
|
const ctx = this._ensureCtx();
|
||||||
|
if (!this._captureDest) {
|
||||||
|
this._captureDest = ctx.createMediaStreamDestination();
|
||||||
|
for (const { gain, panner } of this._tracks.values())
|
||||||
|
(panner ?? gain).connect(this._captureDest);
|
||||||
|
}
|
||||||
|
return this._captureDest.stream;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Drop the M12 export tap (the matching half of captureAudioStream()). */
|
||||||
|
releaseAudioStream() {
|
||||||
|
if (!this._captureDest) return;
|
||||||
|
for (const { gain, panner } of this._tracks.values()) {
|
||||||
|
try {
|
||||||
|
(panner ?? gain).disconnect(this._captureDest);
|
||||||
|
} catch {
|
||||||
|
/* already disconnected */
|
||||||
|
}
|
||||||
|
}
|
||||||
|
this._captureDest = null;
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 🎧 Per-frame 3D audio update (called by scene3d): the listener follows the viewer camera
|
* 🎧 Per-frame 3D audio update (called by scene3d): the listener follows the viewer camera
|
||||||
* and each playing track's panner sits at its camera's current pose. Takes plain numbers in
|
* and each playing track's panner sits at its camera's current pose. Takes plain numbers in
|
||||||
|
|||||||
@ -172,7 +172,7 @@ structure, baked-JSON validity, and serve.py Range (206) behavior.
|
|||||||
|---|---|
|
|---|---|
|
||||||
| Lane E | `backend/festival4d/audio_features.py`, `director.py`, `backend/tests/test_audio_features.py`, `test_director.py`, `frontend/src/director.js`, `exportVideo.js` |
|
| Lane E | `backend/festival4d/audio_features.py`, `director.py`, `backend/tests/test_audio_features.py`, `test_director.py`, `frontend/src/director.js`, `exportVideo.js` |
|
||||||
| Lane F | `frontend/src/anchorPanel.js`, `photoMode.js`, `fx.js` |
|
| Lane F | `frontend/src/anchorPanel.js`, `photoMode.js`, `fx.js` |
|
||||||
| Lane G | `backend/festival4d/capsule.py`, `backend/tests/test_paths.py`, `test_capsule.py`, `frontend/src/pathsStore.js`, the paths-route bodies in the area foundation2 marks for lane G |
|
| Lane G | `backend/festival4d/capsule.py`, `backend/tests/test_paths.py`, `test_capsule.py`, `frontend/src/pathsStore.js` — the paths-route bodies were CRUD one-liners over the db helpers, so foundation2 implemented them (basic tests in `test_phase5_api.py`); lane G owns deepening `test_paths.py` + the UI |
|
||||||
| Every agent | its own `plan/status/<lane>.md` |
|
| Every agent | its own `plan/status/<lane>.md` |
|
||||||
| **Coordinator only** | `plan/DIRECTIVES.md` |
|
| **Coordinator only** | `plan/DIRECTIVES.md` |
|
||||||
| **Frozen during 5b** | contract item 7 above |
|
| **Frozen during 5b** | contract item 7 above |
|
||||||
|
|||||||
55
plan/status/foundation2.md
Normal file
55
plan/status/foundation2.md
Normal file
@ -0,0 +1,55 @@
|
|||||||
|
# Status — foundation2
|
||||||
|
|
||||||
|
## Round 1 — 2026-07-17 — STATUS: ready_to_merge
|
||||||
|
|
||||||
|
**Directives acknowledged:** Round 4 of plan/DIRECTIVES.md. Single-agent phase 5a — main
|
||||||
|
working directory on branch `foundation2` (per directive 3).
|
||||||
|
|
||||||
|
**Acceptance checklist** (plan/20-phase5.md "foundation2 scope"):
|
||||||
|
- [x] DB: `paths` table (contract #3) + helpers `add/get/get_one/delete_path`, plus
|
||||||
|
`patch_anchor` (label/color only) — `db.py`
|
||||||
|
- [x] API: `GET /api/beats` (serves `config.BEATS_JSON`, 404 when absent),
|
||||||
|
`POST /api/director` (degrades to a VALID empty camPath + `note` while lane E is
|
||||||
|
stubbed), `GET/POST/DELETE /api/paths*` (implemented — CRUD over the helpers was
|
||||||
|
cheaper than stubbing; POST validates the camPath shape → 422),
|
||||||
|
`PATCH /api/anchors/{id}` (contract #4) — `api.py`
|
||||||
|
- [x] Manifest gained `"capsule": false` — `test_api.py` exact-set lock updated in the same
|
||||||
|
commit (pitfall #3)
|
||||||
|
- [x] CLI: `features`, `direct`, `capsule` subcommands dispatch into stubs; house
|
||||||
|
degradation (exit 2, no traceback) — `cli.py`
|
||||||
|
- [x] Backend stubs raise NotImplementedError with the frozen contract in the docstring:
|
||||||
|
`audio_features.py`, `director.py`, `capsule.py`
|
||||||
|
- [x] Frontend stub modules created + imported from `main.js`, each wiring its own hidden
|
||||||
|
button in init(): `director.js`, `exportVideo.js`, `anchorPanel.js`, `photoMode.js`
|
||||||
|
(+ per-frame `fx.update()` call in the master loop), `fx.js`, `pathsStore.js`
|
||||||
|
- [x] index.html: ALL phase-5 buttons pre-wired hidden — #btn-director, #btn-path-save,
|
||||||
|
#path-load-select, #btn-path-del, #btn-export, #btn-photo, #btn-fx, #btn-anchors,
|
||||||
|
#anchor-panel container; `.write-ui` class + `body.capsule .write-ui{display:none}`
|
||||||
|
- [x] Hooks (contract #6): `transport.captureAudioStream()/releaseAudioStream()` (post-gain
|
||||||
|
tap, spatial included, tracks re-join across seeks), `scene3d.focusOn(x,y,z)`,
|
||||||
|
`scene3d.setHelpersVisible(bool)` (+ `registerHelper`; camPath gizmos registered)
|
||||||
|
- [x] `state.js`: `window.__F4D_API_BASE__` ?? `VITE_API_BASE` ?? localhost (contract #5);
|
||||||
|
`state.capsule` from the manifest toggles `body.capsule`
|
||||||
|
- [x] Suite ≥ 121 + new tests: **127 passed** (121 floor + 6 in `test_phase5_api.py`)
|
||||||
|
- [x] Ownership table updated (paths-route bodies → foundation2; noted in 20-phase5.md)
|
||||||
|
|
||||||
|
**Done this round — evidence:**
|
||||||
|
- `uv run pytest backend/tests` → **127 passed** (was 121).
|
||||||
|
- `npm run build` → clean (pre-existing chunk-size warning only).
|
||||||
|
- Live browser (Vite :5173 + uvicorn :8000, synthetic fixture): app boots (3 videos, sync
|
||||||
|
+0 ms); all 9 new controls present and `display:none`; `setHelpersVisible(false)` hides
|
||||||
|
grid (and restores); `focusOn(1,0.5,0)` retargets OrbitControls to exactly [1,0.5,0];
|
||||||
|
`captureAudioStream()` returns a MediaStream with 1 audio track, release clean;
|
||||||
|
`POST /api/director` → `{version:1, keyframes:[], note:…}` and round-trips
|
||||||
|
`camPath.fromJSON` (0 keyframes, no throw); paths save→load→fromJSON reproduces 2
|
||||||
|
keyframes, DELETE → 200 then 404; `PATCH /api/anchors/1 {color}` → updated dict;
|
||||||
|
`GET /api/beats` → 404; `body.capsule` hides #annot-panel/#btn-path-save/#btn-path-del.
|
||||||
|
|
||||||
|
**Blockers / questions for coordinator:** none. One conscious deviation, self-adjudicated
|
||||||
|
(coordinator == foundation2 agent this round): paths route bodies implemented here instead
|
||||||
|
of stub-marked for lane G — they were one-line calls into the db helpers I had to write
|
||||||
|
anyway; stubbing them would have been more code than implementing. Lane G scope shrinks to
|
||||||
|
capsule + pathsStore.js + deepening test_paths.py.
|
||||||
|
|
||||||
|
**Next:** merge to `main`, then lanes E, G, F in parallel (each in its own worktree —
|
||||||
|
directive 3 is mandatory for 5b).
|
||||||
Loading…
Reference in New Issue
Block a user