From bf9e41be7f8498ea6a3779726a5c98bf4ab84505 Mon Sep 17 00:00:00 2001
From: type-two <jingkohn@gmail.com>
Date: Fri, 17 Jul 2026 18:20:19 +1000
Subject: [PATCH] ideas: stereo/spatial-video ingest, depth seeding,
 microcontroller rig kit; mark phase-5 items shipped

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---
 docs/ideas.md | 37 ++++++++++++++++++++++++++++++++-----
 1 file changed, 32 insertions(+), 5 deletions(-)

diff --git a/docs/ideas.md b/docs/ideas.md
index 7588e5e..595b2f3 100644
--- a/docs/ideas.md
+++ b/docs/ideas.md
@@ -3,11 +3,12 @@
 Parking lot for post-v0.1.0 extensions (spec §5 future-extensions). Not committed work — a place
 to capture direction so the prototype's scope stays honest. Roughly ordered by payoff-to-effort.
 
-> **Status 2026-07-17:** splatting ✅ shipped (see `modelbeast-crossover.md`); spatial audio ✅
-> shipped (🎧, WebAudio master clock). Cinematic export → **M12**, multi-modal audio analysis →
-> **M10/M11**, anchor manager → **M13**, path persistence → **M16** — all now specced in
-> `plan/20-phase5.md`. Still parked here: realtime ingest, object/artist tracking,
-> per-project workspaces, sync-graph visualization, per-moment splats (crossover doc §3).
+> **Status 2026-07-17 (evening):** splatting ✅ shipped (see `modelbeast-crossover.md`);
+> spatial audio ✅ shipped (🎧, WebAudio master clock). Cinematic export ✅ **M12**,
+> multi-modal audio analysis ✅ **M10/M11**, anchor manager ✅ **M13**, path persistence ✅
+> **M16** — all SHIPPED in phase 5 (`plan/20-phase5.md`, suite floor 189). Still parked here:
+> realtime ingest, object/artist tracking, per-project workspaces, sync-graph visualization,
+> per-moment splats (crossover doc §3), and the new hardware ideas below.
 
 ## Neural rendering — Gaussian Splatting
 Replace the sparse COLMAP point cloud with a **3D Gaussian Splatting** model for photorealistic
@@ -40,6 +41,32 @@ Extend M8 (manual bbox → 3D anchor) with automatic per-frame tracking (e.g. a
 tracker), so an anchor follows the lead performer across time instead of being static. Enables
 a "follow artist" auto-camera mode in the 3D viewer.
 
+## Stereo / spatial-video ingest (split the eyes, pin the scale)
+Spatial video (iPhone 15 Pro+ MV-HEVC, QooCam, Canon dual-fisheye) is two synced views with a
+**factory-known baseline**. Ingest tweak: detect multi-view files and split L/R into two
+pipeline videos (ffmpeg can demux the streams; identical offset, shared audio). Payoff beyond
+one extra viewpoint: COLMAP reconstructions are scale-free, and a known stereo baseline anchors
+**metric scale** for the whole scene — real-meter units for anchors, paths, and spatial audio
+distances. Cheap ingest change, high leverage.
+
+## Depth-camera seeding (small venues only)
+RGB-D cams (ZED 2i, OAK-D, LiDAR phones) can seed/regularize splat training (depth-supervised
+3DGS → fewer floaters, faster convergence) and densify the point cloud without SfM guessing.
+Honest constraint: consumer depth dies past ~10–15 m (LiDAR ~5 m), so this only pays at club /
+small-venue range, not a festival main stage. Ingest: accept a per-video depth track or
+per-frame depth PNGs alongside the RGB.
+
+## Microcontroller rig kit (Arduino as crew, not camera)
+An ESP32-CAM is not concert-grade — but a microcontroller on the rig earns its place three ways:
+- **Sync/calibration beacon:** an LED blinking a known pattern visible to all cameras =
+  visual ground truth to validate the audio sync (and a free calibration target).
+- **Servo pan/tilt or slider mount:** one slowly-sweeping fixed camera contributes rich
+  parallax from a single device; the per-frame pose track already handles moving cameras.
+- **Record start/stop trigger** for the whole rig.
+DIY camera *nodes* should be Raspberry Pi + **Global Shutter** camera module (kills
+rolling-shutter wobble under fast stage lights) pushing into the existing `/capture` endpoint
+(`FESTIVAL4D_CAPTURE=1`) — which was built for exactly this.
+
 ## Quality-of-life
 - **Standalone anchor manager** — the delete UI currently lives in the event correction panel;
   a dedicated always-available anchor list would decouple anchor management from events.