frontend/src/exportVideo.js fills the foundation2 stub: the ⏺ button
records the scene3d canvas captureStream(30) muxed with the live WebAudio
mix via the transport.captureAudioStream() hook (spatial included, silent
tap when no track — degrade, don't block), MediaRecorder vp9/opus webm
(vp8/webm fallbacks), spanning first -> last camPath keyframe. Playback is
driven through the existing #btn-path flow at forced rate 1 (file duration
must equal the path span), stops at the last keyframe (or early on a
second click / anything that halts the ride), downloads
festival4d-cut.webm, then restores ALL prior transport/UI state (playhead,
rate, play state, path state, followed camera) and releases the audio tap
+ canvas tracks. Button disabled under 2 keyframes by chaining main.js's
camPath.onChange (frozen files untouched).
Requires live-browser verification (focused tab — captureStream delivers
no frames hidden): see plan/status/lane-E.md. Frontend build clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
generate_path() fills the foundation2 stub with the deterministic v1 rules:
top-N events by confidence (ties -> earlier, then shot chronologically);
per event, the registered camera whose pose track is nearest the event
time; keyframes at t_event - lead_s (clamped >= 0) and t_event + duration;
nearest-beat snapping when beats.json exists (missing/corrupt file -> no
snapping); FOV = 2*atan(h/(2*fy)) like scene3d.snapTo. Every pose goes
through the frozen geometry.colmap_to_threejs and the camPath quaternion
is emitted in Three.js [x,y,z,w] order (pitfall #1 — locked by test).
Degradation: no events / no registered poses -> valid empty camPath with a
note; the response always round-trips through camPath.fromJSON.
frontend/src/director.js wires the pre-wired hidden 🎬 button: POST
/api/director, camPath.fromJSON(response), auto-play via the existing
#btn-path flow; empty path or fetch failure surfaces on the button and
recovers — never a crash.
Acceptance (test_director.py, 14 tests): structure-locked against the
frozen camPath shape (exact key sets, unit quats, monotone times);
top-3 grid == [1,4,8,11,14] incl. shared-keyframe dedupe; conversion
matches colmap_to_threejs + mat_to_quat exactly; snapping puts every key
on the beat grid; unregistered poses excluded; API route + CLI dispatch
real output. Suite 148 passed; frontend build clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Master clock: the selected soundtrack (new GET /api/audio/{id} —
listening-quality stereo AAC, lazily ffmpeg-extracted and cached in
data/work/audio_hq/) plays through WebAudio, and t_global derives from
AudioContext.currentTime. Every <video> is muted picture-only, so hard
seeks and playbackRate corrections are inaudible (the unmuted video
used to warble on every nudge). Graceful fallback to the old
performance.now() clock + <video> audio when a track can't load, with
an in-place upgrade if the decode lands mid-play. Drift is compensated
in the track's playbackRate; _resync() now recomputes from the live
clock since the soundtrack keeps playing while rAF is throttled.
Spatial audio (🎧 3D audio, shown when poses exist): every enabled
camera's track plays through an HRTF PannerNode at its live
reconstructed pose, gains normalized 1/sqrt(N); the listener follows
the viewer camera each frame (free roam, follow-cam, and M9 paths).
Tracks that finish decoding mid-play join the mix in place; toggling,
seeking, rate and enable changes rebuild the graph at the current
t_global with no timeline jump.
The window.__f4d debug hook is now also available in production builds
via localStorage f4dDebug=1 (headless panes throttle rAF; the pump is
the only way to drive the loop there).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Records rather than streams: each device records a clip WITH AUDIO and uploads
it; the normal offline pipeline takes over. Fits the design almost for free —
MediaRecorder's webm/mp4 are already ingestible, and audio-based sync means
devices need no clock sync and needn't start together.
- capture.py: POST /api/capture/upload (streamed, sanitized name, size cap,
partial cleanup), heartbeat/devices registry, GET /capture page.
- static/capture.html: standalone page (no build step) — device picker, live
preview, record->upload, 1fps snapshot heartbeat, 'who's shooting what'
monitor, and secure-context detection that tells you to use
(getUserMedia needs HTTPS — the #1 gotcha for phones).
- OPT-IN via FESTIVAL4D_CAPTURE=1; manifest reports has_capture and the viewer
only then shows a Capture link. Public deploy leaves it off (open upload
endpoint would be unsafe) — noted in DEPLOY.md.
- Adds python-multipart dep. Suite 103 -> 119.
Also fixes test_manifest_shape, which was already RED on main: f12b6e0 added
has_splat to the manifest without updating the exact-set lock. Updated to the
true set (now also has_capture); the lock is what caught both drifts.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- GET /api/splat serves data/work/splat.ply; manifest reports has_splat
- scene3d: lazy-load @mkkellogg/gaussian-splats-3d DropInViewer when a splat
exists (rigs/anchors/paths overlay as before); point-cloud fallback intact
- scripts/splat_via_modelbeast.sh: train the splat on the MODELBEAST fleet
(colmap_poses -> brush_train via the mb queue) and drop it in place
- docs/modelbeast-crossover.md: full crossover map (fleet compute, volumetric
capture w/ CorridorKey + audio sync, per-moment splats)
Verified in-browser: 32MB room splat rendered via /api/splat with overlays live.
Frontend: API_BASE env-driven (VITE_API_BASE, default localhost:8000 so dev is
unchanged); vite.config.js base from FESTIVAL4D_BASE. Prod build
'FESTIVAL4D_BASE=/festifun/ VITE_API_BASE=/festifun' serves same-origin under
the prefix. Validated end-to-end via a local proxy mirroring the nginx config:
app boots, media Range 206, 3D + overlays + timeline render under /festifun.
deploy/: nginx location blocks (prefix strip + Range), systemd unit
(uvicorn 127.0.0.1:8000, no keys on the public box), DEPLOY.md. Labels baked
into the shipped DB locally so detect can't spend credits from an anonymous
public endpoint. Backend suite 103 passed; default build unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fixes the sole Round-3 wart: annotating one object from two views now yields
ONE anchor (the single-view fallback is superseded in place by the
triangulation, same id) instead of a sibling pair. Adds DELETE /api/anchors/{id}
(unlinks annotations, 404 on unknown) + a deletable anchor list in the
correction panel. db gains get/update/delete_anchor + get_annotations(event_id).
CR-3 filed (additive). Suite 101 -> 103; frontend build clean; delete UI
verified live (state + DOM + 3D scene stay consistent).
Also: docs/ideas.md (future extensions), plan/ISSUES.md (seeded with the
gemini-2.5-flash retirement, FIXED), plan/status/phase4.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Full frontend built against the synthetic API (no dependence on lanes A/B/D):
M4 — synchronized playback: master clock from performance.now() (never a <video>);
per-video correction via requestVideoFrameCallback (exact mediaTime) with a
currentTime+half-frame fallback — hard-seek >150ms / nudge playbackRate +/-5% 20-150ms /
lock <20ms; out-of-range videos pause+dim; single audio source; dev sync-error overlay.
Timebase mirrors config.py (lib/timebase.js). Verified: seek is exact to 0ms; continuous
inter-video desync 13ms mean / 46ms max.
M5 — 3D viewer: PLY point cloud, per-video camera paths + current-pose frusta, OrbitControls,
snap-to-camera (intrinsics->PerspectiveCamera fov) + free roam. All COLMAP->Three.js via the
frozen lib/pose.js; pose interpolation in lib/poseTrack.js (slerp+lerp of contract inputs).
M6 — anchor overlays: letterbox-correct per-video canvas; anchors projected via pose.js;
behind-camera cull. Projection agrees with an independent pinhole to 1.45e-13 px.
Timeline — colored event markers + legend, hover tooltip, click-to-jump; draggable scrubber.
Phase-3 seams (annotate.js M8, camPath.js M9) left as stubs. No frozen files edited; no new
deps; no change requests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
