SCENEGOD/lanes/B-timeline.md

Lane B — Timeline: master clock, tracks, keyframes, evaluation

You own: scenegod/web/timeline.js, tlui.js (+ your marked block at the bottom of style.css, + your dev-only stagestub.js until SYNC 1). Milestone: M1. You develop against the Stage API in PLAN §4.2 — code to the contract, not to Lane A's implementation.

B1. stagestub.js (first commit, deleted at SYNC 1)

Implements the full Stage API from PLAN §4.2 with plain objects: entities are {id, kind, transform, params}, setTransform/setParam record into a stub.applied array and console.debug. prepareClip returns a fake {duration: 2.4} clip. This lets every timeline feature be built and tested headless before Lane A lands.

B2. timeline.js — the model + clock (no DOM in this file)

• Owns the authoritative scene fields: fps, duration, entities[].tracks, cameraCuts. load(sceneJson) / toJSON() round-trip losslessly — unknown fields on entities are preserved (Lane A owns them).
• Master clock: play() uses requestAnimationFrame deltas; seek(t) clamps to [0, duration]; every advance calls evaluate(t) then onTick listeners. step(frame) = seek(frame/fps) with NO raf — pure and deterministic (Lane C's final render calls it in a loop).
• evaluate(t) per entity:
  1. Transform track: binary-search the bracketing keys, interpolate — pos/scale lerp, rot via Quaternion.slerp (convert stored eulers once at load, cache quats on the key objects), easing (linear|in|out|inout = cubic, step = hold) applied to the segment's normalized u. 0 keys → rest transform; 1 key → hold. Result → stage.setTransform(id, …).
  2. Params track: same interpolation per key name (numbers lerp, colors lerp in sRGB — fine for v1) → stage.setParam.
  3. Clip blocks (characters): a block covers [start, start + (out-in)*loop]. If t inside: local clip time = in + ((t-start) % (out-in)); drive via the entity's mixer — one AnimationAction per block, action.time = localTime; action.weight = w; mixer.update(0) (set-time style, NOT free-running — scrubbing must be exact). Crossfade: in the last fade seconds of a block that has a successor, ramp weight 1→0 while successor ramps 0→1. Outside any block: weight 0 (character holds rest / last transform).
  4. Camera cuts: last cut with cut.t <= t → stage.setActiveCamera.
• Keyframe ops API (tlui + Lane A's event use these): addKey(id, track, key), moveKey, deleteKey, addClipBlock(id, block), moveClipBlock, trimClipBlock, addCut(t, cameraId) — every mutator keeps arrays sorted by t and pushes an inverse op onto a simple undo stack (M2: wire to Ctrl+Z; the stack itself costs nothing now).
• Listen for scenegod:capturekey (see Lane A dock): on it, addKey(id, 'transform', {t: this.time, ...stage.entityTransform(id), ease:'inout'}).

B3. tlui.js — the panel (mounts into #timeline)

• Left column: track names (one row per entity, auto-synced to stage.entities via onChange/poll; + Cameras row + a per-character clip sub-row). Right: time ruler + canvas-drawn track lanes (a <canvas> per lane or one big one — your call, log the decision).
• Interactions (M1): click ruler = seek; drag playhead; space = play/pause; double-click a transform lane = add key at that t from current stage state; drag key = move (snap to frame, hold Alt = free); right-click key = delete; drag a clip block horizontally; drag block edges = trim. Home/End = 0/duration.
• Scene bar: name field, duration field, Save/Load buttons → POST/GET /scenes/{name} (Lane C; until then, localStorage fallback — keep the fetch code, catch and fall back).
• Keep ALL DOM here; timeline.js must run headless under node for tests.

B4. self-check — web/timeline_test.mjs

Runs under plain node (import timeline.js + stagestub): build a scene with 2 transform keys + 1 clip block + 2 camera cuts, step() through 90 frames, assert: interpolated midpoint position, ease-step hold, clip local time at 3 sample frames, active camera flips at the cut, toJSON() deep-equals a re-loaded copy. node web/timeline_test.mjs green = M1 code-done.

M1 acceptance (log it)

• timeline_test.mjs passes.
• In the browser against the stub: scrub bar moves entities in the stub log, keys draggable, play/pause works, save/load round-trips via localStorage.
• Do NOT integrate with the real Stage yourself — that's SYNC 1, the orchestrator runs it (your code should make it a one-line swap).

M2 preview (locked until orchestrator flips it)

Clip crossfades polish, params lanes UI, camera-cut lane UI, snapping config, undo wiring, box-select + multi-drag of keys.

ORCHESTRATOR UPDATES

2026-07-18 — M1 ACCEPTED, SYNC 1 done. M2 is UNLOCKED.

Your model survived integration with the real Stage; transform interp and clip retarget playback verified frame-exact via step(). Notes:

1. Fix applied by orchestrator (in your file, keep it): _evalClips now early-returns when the entity has no clip blocks or no mixer — the real Stage returns null from entityMixer for backdrops/props/cameras; your stub always returned an object, which hid the crash. Lesson for the stub: mirror the contract's nullable returns.
2. stagestub.js was NOT deleted — timeline_test.mjs imports it. It's now a permanent test fixture, not dev scaffolding. tldev.html is gone.
3. SYNC findings, fix first in M2: (a) after load() + applyState, your track-name rows never appeared (only "Cameras") — rebuild rows on load() and on stage.onChange; (b) the scene-bar name/duration fields don't refresh after load() — bind them. Verified repro: load a 2-entity scene programmatically, rows stay empty.
4. New event: Lane A will dispatch scenegod:clipdrop {detail:{id, path, clipIndex}} when a clip is dropped on a character — add a clip block on that entity at the playhead (default out = clip duration after preload, fade 0.25 when it abuts a neighbor).
5. rAF note: in the embedded preview the tab can be document.hidden → rAF fully suspended → play() appears frozen. Not a bug in your clock (verified); don't "fix" it. step() is the deterministic path and works.
6. Injected <style id=laneB-tl-css> is fine permanently — keep it. Then per plan: crossfade fade-IN polish, params + camera-cut lane UI, snapping config, box-select multi-drag (undo already wired).

2026-07-18 — M2 ACCEPTED, SYNC 2 PASSED. Session 3 (polish) granted.

Your flagged dock-entity gap is closed: orchestrator added _mirror to timeline.js (ctor subscribes stage.onChange; upserts an entity with empty tracks, removes when gone from stage — removal detected via stage.getEntity(id) === null). Verified live: 6 dock-added entities entered the model, clipdrop landed a 4.47s block, sunset param lerp + camera cuts all frame-exact. Session 3 items:

1. Absorb _mirror — review it, then add a timeline_test.mjs case (stub add → mutate → remove; assert entity+tracks lifecycle). The stub needs getEntity to return null after removal for this to test honestly.
2. Programmatic scene.duration = x doesn't refresh the scene bar (only load() fires onLoad). Add a setDuration(x) mutator that clamps existing keys' visibility and notifies the UI; wire the dur field to it.
3. Your listed polish: snapping increment config, keyable-param add UI.

2026-07-18 — session 3 ACCEPTED. M4-B unlocked: audio on the timeline.

Good catch fixing the stub's removeEntity-onChange payload — that's the honest-stub lesson applied. All three items verified (setDuration bar refresh + snap select seen live; tests green). M4-B scope (PLAN §5):

1. Audio rows in tlui from scene.audio[]: one lane per clip, draggable offset (start), gain in the inspector-side of the row, add via a 🎵 button reading /assets/tree audio category; delete.
2. Web Audio playback synced to the master clock: on play() schedule AudioBufferSourceNodes at start - time offsets; on seek/pause stop
   • reschedule. step() NEVER touches audio (deterministic render path — the server muxes from scene.audio, already working end-to-end).
3. Viseme keyframes (pairs with Lane A's M4-A): a morphs track kind — {t, key: visemeName, value: weight} evaluated via stage.setMorph(id, name, weight) with lerp between keys; plus importRhubarb(id, rhubarbJson, t0) mapping mouthShape cues → morph keys. Guard: only offer when stage.visemeTargets(id).length > 0.

2026-07-18 — session 4 ACCEPTED. v1 COMPLETE — no required B work.

Audio rows + Web Audio + morphs track all verified live (🎵 picker, decoded 8s block at 0.5s offset, gain input, param/cut/sun lanes, zero console errors). Your ownership note is acted on: CLAUDE.md now mandates staging own files explicitly — thanks for flagging it instead of letting it slide. Optional session-5 backlog, in order of value: (1) "lip-sync from audio" button — POST the audio path to Lane C's /rhubarb, feed the JSON to importRhubarb (503 → toast the install hint); (2) audio block trim; (3) UX nit found live: mouse wheel over the track canvas zooms, so users may not discover the panel scrolls — consider wheel = scroll, Cmd/Ctrl+wheel = zoom.

2026-07-19 — M5 DIRECTOR MODE unlocked (spec in PLAN §5 M5). Your part:

1. Listen for scenegod:direct (contract in lanes/A §M5.3): every preset op becomes NORMAL keys/cuts at the playhead through your existing mutators — shot → transform+fov keys on the camera + a cut to it; light → param keys on sun/ambient (+bg); mark → transform key; walk_to_mark → key at playhead-with-current-pos + key at playhead+2s at the mark. All undo-able (one undo entry per op — group the batch, don't leave 5 half-undoable keys).
2. Wire the "lip-sync from audio" button now that /rhubarb is LIVE and verified (GET /rhubarb?path=audio/test/vo.wav returns real mouthCues — vo.wav is in assets): on an audio row, 🗣 button → fetch cues → importRhubarb(characterId, json, audioStart); character picker if >1 viseme-bearing char; toast the 503 install hint if absent.
3. Session-5 backlog if time remains: audio trim, wheel-scroll fix (above).

2026-07-19 — M6 FLOW VIDEO PLATES queued behind M5 (full spec PLAN §5 M6).

Do M5 first; M6 is unlocked the moment your M5 acceptance is logged. Test media will be arriving in assets/backdrops/video/ via scripts/flow_intake.py.

2026-07-20 — M5+M6 ACCEPTED at SYNC 6 (all lanes ran as orchestrator subagents).

Verified live: DIRECT bar CU/low/frame → camera+keys+cut, ONE undo reverts; golden_hour keys; video plate follows the clock frame-exact; finalRender shows the Flow plate behind an animating character. Orchestrator seam fixes to absorb: (1) server: VIDEO_EXTS in tree/media-types, video beats poster as primary; (2) stage.entityVideo(id) added (Lane B seam); (3) syncVideos forces videoTex.needsUpdate — r160 VideoTexture rides requestVideoFrameCallback which never fires in hidden tabs → black plates in renders. Backlog carried: honor audio in/out at mux (C), walk-to-mark prevPos pre-apply (A), live viseme asset still owed by John.

2026-07-25 — M7-B ACCEPTED at SYNC 9. M7 COMPLETE.

Verified live: scene bar reads ? · New… · Save · Load; picker lists all three templates (2 with thumbs, golden-hour first); NO confirm on an empty scene, confirm DOES fire on a dirty one; the pick loads 7 entities + 2 cuts and refreshes name/dur; toJSON() carries no template key. Help overlay: opens on ?, 24 real bindings across STAGE/TIMELINE/SCENE BAR, Escape closes, and it correctly does NOT hijack ? while the name field has focus. One orchestrator fix absorbed: .tl-help dt was auto-width + white-space:nowrap, so long terms overflowed and the three columns overlapped on screen — now a capped, wrapping term column (tlui.js CSS). Your call to strip the template key inside applyScene rather than lean on round-trip behaviour was the right one; PLAN §4.1 stays honest. Carried forward (yours or Lane C's): audio in/out still ignored at ffmpeg mux — draft playback honours the trim, the render doesn't. Worth closing before anyone cuts to music.

2026-07-26 — M9-B ACCEPTED at SYNC 12. Beat snapping + FILM panel both verified live.

Grid: 128.02 BPM / 35 beats / 8 downbeats / conf 0.865 off house128.wav, snap select auto-flips to beat, and snapped positions land 8–10 ms from the TRUE grid (0.468735s period) at t=0.3 through t=12.4 — i.e. all residual is Lane C's detector, none is yours. Bar snap hits a downbeat. beatGrid survives toJSON→load. FILM panel lists both sequences, demo-two-shot shows 3 editable shot rows and the server's own 11.50s arithmetic. Two calls I want to endorse, because both were judgement rather than instruction-following:

1. Storing ONE top-level scene.beatGrid rather than per-clip — "which grid am I snapping to" needs exactly one answer.
2. Keeping beats in FILE seconds and re-deriving the offset from the audio clip, so dragging the audio drags the grid and a head-trim leaves it put. That's the behaviour a musician expects and it falls out of the data shape instead of special-casing. And you CHECKED validate_scene before inventing a field (and round-tripped it live) rather than guessing — exactly right; that's the difference between a schema change and a bug. NEXT: nothing queued. Open ideas if you want them: ?from=&to= windowing on /beats for DJ mixes (logged for Lane C), and auto-cut — given a grid and a shot list, distribute cuts onto downbeats automatically. That last one is small now and would be the whole music-video workflow in one button.