Conflicts resolved: (a) test_phase5_api.py — with lane G's capsule AND
lane E's features/direct all landed, the CLI stub loop is empty, so
test_cli_dispatches_stubs_gracefully is retired with a pointer comment to
the real dispatch coverage (test_capsule.py / test_audio_features.py /
test_director.py); (b) CHANGE_REQUESTS.md — lane G keeps the ratified
CR-4; lane E's entry renumbered to CR-5 and updated for the merged
reality. Merged tree: 189 passed (169 main + 21 lane E - 1 retired),
frontend build clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Coordinator-verified: ownership clean (3 owned modules + status only);
lane self-verified live in isolated servers (evidence in plan/status/lane-F.md).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
✨ toggle: fx.update(tGlobal) (already called each frame by main.js) tracks
the previous tGlobal and fires on marker crossings in play AND scrub;
backward jumps and >2s forward jumps just reset the tracker. pyro/confetti ->
~1s additive particle burst at the event's resolved anchor (mapped by the
backend's label convention, stage-centroid fallback); bass_drop -> point
cloud/splat scale pulse for ~a beat (60/tempo via GET /api/beats, else 0.5s),
exact scale restored. Scene access only via scene3d.addObject/removeObject.
Hot path: 6 preallocated burst slots (160 particles each), zero per-frame
allocation, whole body behind a breaker so a throw can never kill the app
loop (it disables FX instead).
Live-verified: pyro fired at t=10.00 in play, confetti on scrub crossing 17.0,
no retro-fire on backward jump, pulse 1.16x -> exact 1.0 restore, worst-case
frame body 0.101 ms CPU (~330x inside the 30fps budget), clean toggle-off
(plan/status/lane-F.md).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
📷 / P: pause if playing, scene3d.setHelpersVisible(false) + one scene3d
update so the per-video rigs honor the flag, then render ONE frame at 3840px
wide (aspect preserved, pixelRatio 1) by temporarily resizing the live
renderer, snapshot via canvas.toBlob in the same task, download
festival4d-photo.png, and restore pixelRatio/size/camera.aspect/helpers/play
state (try/finally). Splat: DropInViewer is a scene child and renders with
the same call — no branch; live splat verification deferred to coordinator.
Live-verified: PNG 3840x3093 (aspect 1.2416 preserved), zero helpers in the
captured image, full restore incl. play state (plan/status/lane-F.md).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Collapsible #anchor-panel (toggle ⚓ #btn-anchors): every anchor gets a color
dot (recolor via inline color input), label (rename via prompt), one-click
jump-to (scene3d.focusOn), and delete — PATCH/DELETE /api/anchors/{id}, then
state.anchors update + emit("anchors-changed") so scene3d + overlays refresh.
＋ Tag drives the FROZEN M8 annotator without an event (annotate.js is
null-event safe by construction; the backend keeps event-less annotations as
independent anchors): prompt a name -> startAnnotateMode -> user drags a bbox
on any video -> the resolved anchor is PATCHed to the name + a palette color.
Esc / ＋ Tag again cancels. All mutating controls carry .write-ui (capsule
mode keeps the read-only list + jump). CSS injected from the module —
index.html is frozen.
Live-verified: create->rename->recolor->jump->delete roundtrip, labels in 3D
+ both video overlays, reload survival, capsule hiding (plan/status/lane-F.md).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
frontend/src/exportVideo.js fills the foundation2 stub: the ⏺ button
records the scene3d canvas captureStream(30) muxed with the live WebAudio
mix via the transport.captureAudioStream() hook (spatial included, silent
tap when no track — degrade, don't block), MediaRecorder vp9/opus webm
(vp8/webm fallbacks), spanning first -> last camPath keyframe. Playback is
driven through the existing #btn-path flow at forced rate 1 (file duration
must equal the path span), stops at the last keyframe (or early on a
second click / anything that halts the ride), downloads
festival4d-cut.webm, then restores ALL prior transport/UI state (playhead,
rate, play state, path state, followed camera) and releases the audio tap
+ canvas tracks. Button disabled under 2 keyframes by chaining main.js's
camPath.onChange (frozen files untouched).
Requires live-browser verification (focused tab — captureStream delivers
no frames hidden): see plan/status/lane-E.md. Frontend build clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
generate_path() fills the foundation2 stub with the deterministic v1 rules:
top-N events by confidence (ties -> earlier, then shot chronologically);
per event, the registered camera whose pose track is nearest the event
time; keyframes at t_event - lead_s (clamped >= 0) and t_event + duration;
nearest-beat snapping when beats.json exists (missing/corrupt file -> no
snapping); FOV = 2*atan(h/(2*fy)) like scene3d.snapTo. Every pose goes
through the frozen geometry.colmap_to_threejs and the camPath quaternion
is emitted in Three.js [x,y,z,w] order (pitfall #1 — locked by test).
Degradation: no events / no registered poses -> valid empty camPath with a
note; the response always round-trips through camPath.fromJSON.
frontend/src/director.js wires the pre-wired hidden 🎬 button: POST
/api/director, camPath.fromJSON(response), auto-play via the existing
#btn-path flow; empty path or fetch failure surfaces on the button and
recovers — never a crash.
Acceptance (test_director.py, 14 tests): structure-locked against the
frozen camPath shape (exact key sets, unit quats, monotone times);
top-3 grid == [1,4,8,11,14] incl. shared-keyframe dedupe; conversion
matches colmap_to_threejs + mat_to_quat exactly; snapping puts every key
on the beat grid; unregistered poses excluded; API route + CLI dispatch
real output. Suite 148 passed; frontend build clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Unhides #btn-path-save / #path-load-select / #btn-path-del; save prompts a name and
POSTs camPath.toJSON(), load fetches + camPath.fromJSON, delete confirms; dropdown
refreshes after every mutation. Capsule mode: write buttons hide via write-ui, the
load dropdown keeps working against baked api/paths (hides itself if unavailable).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Master clock: the selected soundtrack (new GET /api/audio/{id} —
listening-quality stereo AAC, lazily ffmpeg-extracted and cached in
data/work/audio_hq/) plays through WebAudio, and t_global derives from
AudioContext.currentTime. Every <video> is muted picture-only, so hard
seeks and playbackRate corrections are inaudible (the unmuted video
used to warble on every nudge). Graceful fallback to the old
performance.now() clock + <video> audio when a track can't load, with
an in-place upgrade if the decode lands mid-play. Drift is compensated
in the track's playbackRate; _resync() now recomputes from the live
clock since the soundtrack keeps playing while rAF is throttled.
Spatial audio (🎧 3D audio, shown when poses exist): every enabled
camera's track plays through an HRTF PannerNode at its live
reconstructed pose, gains normalized 1/sqrt(N); the listener follows
the viewer camera each frame (free roam, follow-cam, and M9 paths).
Tracks that finish decoding mid-play join the mix in place; toggling,
seeking, rate and enable changes rebuild the graph at the current
t_global with no timeline jump.
The window.__f4d debug hook is now also available in production builds
via localStorage f4dDebug=1 (headless panes throttle rAF; the pump is
the only way to drive the loop there).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Records rather than streams: each device records a clip WITH AUDIO and uploads
it; the normal offline pipeline takes over. Fits the design almost for free —
MediaRecorder's webm/mp4 are already ingestible, and audio-based sync means
devices need no clock sync and needn't start together.
- capture.py: POST /api/capture/upload (streamed, sanitized name, size cap,
partial cleanup), heartbeat/devices registry, GET /capture page.
- static/capture.html: standalone page (no build step) — device picker, live
preview, record->upload, 1fps snapshot heartbeat, 'who's shooting what'
monitor, and secure-context detection that tells you to use
(getUserMedia needs HTTPS — the #1 gotcha for phones).
- OPT-IN via FESTIVAL4D_CAPTURE=1; manifest reports has_capture and the viewer
only then shows a Capture link. Public deploy leaves it off (open upload
endpoint would be unsafe) — noted in DEPLOY.md.
- Adds python-multipart dep. Suite 103 -> 119.
Also fixes test_manifest_shape, which was already RED on main: f12b6e0 added
has_splat to the manifest without updating the exact-set lock. Updated to the
true set (now also has_capture); the lock is what caught both drifts.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- GET /api/splat serves data/work/splat.ply; manifest reports has_splat
- scene3d: lazy-load @mkkellogg/gaussian-splats-3d DropInViewer when a splat
exists (rigs/anchors/paths overlay as before); point-cloud fallback intact
- scripts/splat_via_modelbeast.sh: train the splat on the MODELBEAST fleet
(colmap_poses -> brush_train via the mb queue) and drop it in place
- docs/modelbeast-crossover.md: full crossover map (fleet compute, volumetric
capture w/ CorridorKey + audio sync, per-moment splats)
Verified in-browser: 32MB room splat rendered via /api/splat with overlays live.
Frontend: API_BASE env-driven (VITE_API_BASE, default localhost:8000 so dev is
unchanged); vite.config.js base from FESTIVAL4D_BASE. Prod build
'FESTIVAL4D_BASE=/festifun/ VITE_API_BASE=/festifun' serves same-origin under
the prefix. Validated end-to-end via a local proxy mirroring the nginx config:
app boots, media Range 206, 3D + overlays + timeline render under /festifun.
deploy/: nginx location blocks (prefix strip + Range), systemd unit
(uvicorn 127.0.0.1:8000, no keys on the public box), DEPLOY.md. Labels baked
into the shipped DB locally so detect can't spend credits from an anonymous
public endpoint. Backend suite 103 passed; default build unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fixes the sole Round-3 wart: annotating one object from two views now yields
ONE anchor (the single-view fallback is superseded in place by the
triangulation, same id) instead of a sibling pair. Adds DELETE /api/anchors/{id}
(unlinks annotations, 404 on unknown) + a deletable anchor list in the
correction panel. db gains get/update/delete_anchor + get_annotations(event_id).
CR-3 filed (additive). Suite 101 -> 103; frontend build clean; delete UI
verified live (state + DOM + 3D scene stay consistent).
Also: docs/ideas.md (future extensions), plan/ISSUES.md (seeded with the
gemini-2.5-flash retirement, FIXED), plan/status/phase4.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Full frontend built against the synthetic API (no dependence on lanes A/B/D):
M4 — synchronized playback: master clock from performance.now() (never a <video>);
per-video correction via requestVideoFrameCallback (exact mediaTime) with a
currentTime+half-frame fallback — hard-seek >150ms / nudge playbackRate +/-5% 20-150ms /
lock <20ms; out-of-range videos pause+dim; single audio source; dev sync-error overlay.
Timebase mirrors config.py (lib/timebase.js). Verified: seek is exact to 0ms; continuous
inter-video desync 13ms mean / 46ms max.
M5 — 3D viewer: PLY point cloud, per-video camera paths + current-pose frusta, OrbitControls,
snap-to-camera (intrinsics->PerspectiveCamera fov) + free roam. All COLMAP->Three.js via the
frozen lib/pose.js; pose interpolation in lib/poseTrack.js (slerp+lerp of contract inputs).
M6 — anchor overlays: letterbox-correct per-video canvas; anchors projected via pose.js;
behind-camera cull. Projection agrees with an independent pinhole to 1.45e-13 px.
Timeline — colored event markers + legend, hover tooltip, click-to-jump; draggable scrubber.
Phase-3 seams (annotate.js M8, camPath.js M9) left as stubs. No frozen files edited; no new
deps; no change requests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
