SCENEGOD/lanes/B-timeline.md

# Lane B — Timeline: master clock, tracks, keyframes, evaluation

You own: `scenegod/web/timeline.js`, `tlui.js` (+ your marked block at the
bottom of `style.css`, + your dev-only `stagestub.js` until SYNC 1).
Milestone: **M1**. You develop against the Stage API in PLAN §4.2 — code to
the contract, not to Lane A's implementation.

## B1. stagestub.js (first commit, deleted at SYNC 1)
Implements the full Stage API from PLAN §4.2 with plain objects: entities
are `{id, kind, transform, params}`, `setTransform`/`setParam` record into a
`stub.applied` array and `console.debug`. `prepareClip` returns a fake
`{duration: 2.4}` clip. This lets every timeline feature be built and tested
headless before Lane A lands.

## B2. timeline.js — the model + clock (no DOM in this file)
- Owns the authoritative scene fields: `fps`, `duration`, `entities[].tracks`,
  `cameraCuts`. `load(sceneJson)` / `toJSON()` round-trip **losslessly** —
  unknown fields on entities are preserved (Lane A owns them).
- Master clock: `play()` uses `requestAnimationFrame` deltas; `seek(t)`
  clamps to `[0, duration]`; every advance calls `evaluate(t)` then
  `onTick` listeners. `step(frame)` = `seek(frame/fps)` with NO raf — pure
  and deterministic (Lane C's final render calls it in a loop).
- `evaluate(t)` per entity:
  1. **Transform track**: binary-search the bracketing keys, interpolate —
     pos/scale lerp, rot via Quaternion.slerp (convert stored eulers once at
     load, cache quats on the key objects), easing (`linear|in|out|inout` =
     cubic, `step` = hold) applied to the segment's normalized u. 0 keys →
     rest transform; 1 key → hold. Result → `stage.setTransform(id, …)`.
  2. **Params track**: same interpolation per `key` name (numbers lerp,
     colors lerp in sRGB — fine for v1) → `stage.setParam`.
  3. **Clip blocks** (characters): a block covers
     `[start, start + (out-in)*loop]`. If t inside: local clip time =
     `in + ((t-start) % (out-in))`; drive via the entity's mixer — one
     `AnimationAction` per block, `action.time = localTime;
     action.weight = w; mixer.update(0)` (set-time style, NOT free-running —
     scrubbing must be exact). Crossfade: in the last `fade` seconds of a
     block that has a successor, ramp weight 1→0 while successor ramps 0→1.
     Outside any block: weight 0 (character holds rest / last transform).
  4. **Camera cuts**: last cut with `cut.t <= t` → `stage.setActiveCamera`.
- Keyframe ops API (tlui + Lane A's event use these): `addKey(id, track, key)`,
  `moveKey`, `deleteKey`, `addClipBlock(id, block)`, `moveClipBlock`,
  `trimClipBlock`, `addCut(t, cameraId)` — every mutator keeps arrays sorted
  by `t` and pushes an inverse op onto a simple undo stack (M2: wire to
  Ctrl+Z; the stack itself costs nothing now).
- Listen for `scenegod:capturekey` (see Lane A dock): on it,
  `addKey(id, 'transform', {t: this.time, ...stage.entityTransform(id), ease:'inout'})`.

## B3. tlui.js — the panel (mounts into `#timeline`)
- Left column: track names (one row per entity, auto-synced to
  stage.entities via `onChange`/poll; + Cameras row + a per-character clip
  sub-row). Right: time ruler + canvas-drawn track lanes (a `<canvas>` per
  lane or one big one — your call, log the decision).
- Interactions (M1): click ruler = seek; drag playhead; space = play/pause;
  double-click a transform lane = add key at that t from current stage state;
  drag key = move (snap to frame, hold Alt = free); right-click key = delete;
  drag a clip block horizontally; drag block edges = trim. Home/End = 0/duration.
- Scene bar: name field, duration field, Save/Load buttons →
  `POST/GET /scenes/{name}` (Lane C; until then, localStorage fallback —
  keep the fetch code, catch and fall back).
- Keep ALL DOM here; timeline.js must run headless under node for tests.

## B4. self-check — `web/timeline_test.mjs`
Runs under plain `node` (import timeline.js + stagestub): build a scene with
2 transform keys + 1 clip block + 2 camera cuts, `step()` through 90 frames,
assert: interpolated midpoint position, ease-step hold, clip local time at
3 sample frames, active camera flips at the cut, `toJSON()` deep-equals a
re-`load`ed copy. `node web/timeline_test.mjs` green = M1 code-done.

## M1 acceptance (log it)
- timeline_test.mjs passes.
- In the browser against the stub: scrub bar moves entities in the stub log,
  keys draggable, play/pause works, save/load round-trips via localStorage.
- Do NOT integrate with the real Stage yourself — that's SYNC 1, the
  orchestrator runs it (your code should make it a one-line swap).

## M2 preview (locked until orchestrator flips it)
Clip crossfades polish, params lanes UI, camera-cut lane UI, snapping
config, undo wiring, box-select + multi-drag of keys.

## ORCHESTRATOR UPDATES

### 2026-07-18 — M1 ACCEPTED, SYNC 1 done. **M2 is UNLOCKED.**
Your model survived integration with the real Stage; transform interp and
clip retarget playback verified frame-exact via `step()`. Notes:

1. **Fix applied by orchestrator** (in your file, keep it): `_evalClips` now
   early-returns when the entity has no clip blocks or no mixer — the real
   Stage returns `null` from `entityMixer` for backdrops/props/cameras; your
   stub always returned an object, which hid the crash. Lesson for the stub:
   mirror the contract's nullable returns.
2. **stagestub.js was NOT deleted** — `timeline_test.mjs` imports it. It's
   now a permanent test fixture, not dev scaffolding. tldev.html is gone.
3. **SYNC findings, fix first in M2:** (a) after `load()` + `applyState`,
   your track-name rows never appeared (only "Cameras") — rebuild rows on
   `load()` and on `stage.onChange`; (b) the scene-bar name/duration fields
   don't refresh after `load()` — bind them. Verified repro: load a 2-entity
   scene programmatically, rows stay empty.
4. **New event:** Lane A will dispatch `scenegod:clipdrop`
   `{detail:{id, path, clipIndex}}` when a clip is dropped on a character —
   add a clip block on that entity at the playhead (default out = clip
   duration after preload, fade 0.25 when it abuts a neighbor).
5. rAF note: in the embedded preview the tab can be `document.hidden` →
   rAF fully suspended → `play()` appears frozen. Not a bug in your clock
   (verified); don't "fix" it. `step()` is the deterministic path and works.
6. Injected `<style id=laneB-tl-css>` is fine permanently — keep it.
Then per plan: crossfade fade-IN polish, params + camera-cut lane UI,
snapping config, box-select multi-drag (undo already wired).

### 2026-07-18 — M2 ACCEPTED, SYNC 2 PASSED. **Session 3 (polish) granted.**
Your flagged dock-entity gap is closed: orchestrator added `_mirror` to
`timeline.js` (ctor subscribes `stage.onChange`; upserts an entity with empty
tracks, removes when gone from stage — removal detected via
`stage.getEntity(id) === null`). Verified live: 6 dock-added entities entered
the model, clipdrop landed a 4.47s block, sunset param lerp + camera cuts all
frame-exact. Session 3 items:
1. **Absorb `_mirror`** — review it, then add a timeline_test.mjs case
   (stub add → mutate → remove; assert entity+tracks lifecycle). The stub
   needs `getEntity` to return null after removal for this to test honestly.
2. Programmatic `scene.duration = x` doesn't refresh the scene bar (only
   `load()` fires onLoad). Add a `setDuration(x)` mutator that clamps
   existing keys' visibility and notifies the UI; wire the dur field to it.
3. Your listed polish: snapping increment config, keyable-param add UI.

### 2026-07-18 — session 3 ACCEPTED. **M4-B unlocked: audio on the timeline.**
Good catch fixing the stub's removeEntity-onChange payload — that's the
honest-stub lesson applied. All three items verified (setDuration bar
refresh + snap select seen live; tests green). M4-B scope (PLAN §5):
1. Audio rows in tlui from `scene.audio[]`: one lane per clip, draggable
   offset (`start`), gain in the inspector-side of the row, add via a 🎵
   button reading `/assets/tree` audio category; delete.
2. Web Audio playback synced to the master clock: on `play()` schedule
   `AudioBufferSourceNode`s at `start - time` offsets; on `seek/pause` stop
   + reschedule. `step()` NEVER touches audio (deterministic render path —
   the server muxes from scene.audio, already working end-to-end).
3. Viseme keyframes (pairs with Lane A's M4-A): a `morphs` track kind —
   `{t, key: visemeName, value: weight}` evaluated via
   `stage.setMorph(id, name, weight)` with lerp between keys; plus
   `importRhubarb(id, rhubarbJson, t0)` mapping mouthShape cues → morph keys.
   Guard: only offer when `stage.visemeTargets(id).length > 0`.

### 2026-07-18 — session 4 ACCEPTED. **v1 COMPLETE — no required B work.**
Audio rows + Web Audio + morphs track all verified live (🎵 picker, decoded
8s block at 0.5s offset, gain input, param/cut/sun lanes, zero console
errors). Your ownership note is acted on: CLAUDE.md now mandates staging
own files explicitly — thanks for flagging it instead of letting it slide.
Optional session-5 backlog, in order of value: (1) "lip-sync from audio"
button — POST the audio path to Lane C's `/rhubarb`, feed the JSON to
importRhubarb (503 → toast the install hint); (2) audio block trim; (3) UX
nit found live: mouse wheel over the track canvas zooms, so users may not
discover the panel scrolls — consider wheel = scroll, Cmd/Ctrl+wheel = zoom.

### 2026-07-19 — **M5 DIRECTOR MODE unlocked** (spec in PLAN §5 M5). Your part:
1. Listen for `scenegod:direct` (contract in lanes/A §M5.3): every preset op
   becomes NORMAL keys/cuts at the playhead through your existing mutators —
   `shot` → transform+fov keys on the camera + a cut to it; `light` → param
   keys on sun/ambient (+bg); `mark` → transform key; `walk_to_mark` → key at
   playhead-with-current-pos + key at playhead+2s at the mark. All undo-able
   (one undo entry per op — group the batch, don't leave 5 half-undoable keys).
2. Wire the "lip-sync from audio" button now that `/rhubarb` is LIVE and
   verified (`GET /rhubarb?path=audio/test/vo.wav` returns real mouthCues —
   vo.wav is in assets): on an audio row, 🗣 button → fetch cues →
   `importRhubarb(characterId, json, audioStart)`; character picker if >1
   viseme-bearing char; toast the 503 install hint if absent.
3. Session-5 backlog if time remains: audio trim, wheel-scroll fix (above).

### 2026-07-19 — M6 FLOW VIDEO PLATES queued behind M5 (full spec PLAN §5 M6).
Do M5 first; M6 is unlocked the moment your M5 acceptance is logged. Test
media will be arriving in assets/backdrops/video/ via scripts/flow_intake.py.

### 2026-07-20 — M5+M6 ACCEPTED at SYNC 6 (all lanes ran as orchestrator subagents).
Verified live: DIRECT bar CU/low/frame → camera+keys+cut, ONE undo reverts;
golden_hour keys; video plate follows the clock frame-exact; finalRender shows
the Flow plate behind an animating character. Orchestrator seam fixes to absorb:
(1) server: VIDEO_EXTS in tree/media-types, video beats poster as primary;
(2) stage.entityVideo(id) added (Lane B seam); (3) syncVideos forces
videoTex.needsUpdate — r160 VideoTexture rides requestVideoFrameCallback which
never fires in hidden tabs → black plates in renders. Backlog carried: honor
audio in/out at mux (C), walk-to-mark prevPos pre-apply (A), live viseme asset
still owed by John.

### 2026-07-25 — M7-B ACCEPTED at SYNC 9. **M7 COMPLETE.**
Verified live: scene bar reads `? · New… · Save · Load`; picker lists all three templates
(2 with thumbs, golden-hour first); NO confirm on an empty scene, confirm DOES fire on a
dirty one; the pick loads 7 entities + 2 cuts and refreshes name/dur; `toJSON()` carries no
`template` key. Help overlay: opens on `?`, 24 real bindings across STAGE/TIMELINE/SCENE BAR,
Escape closes, and it correctly does NOT hijack `?` while the name field has focus.
One orchestrator fix absorbed: `.tl-help dt` was `auto`-width + `white-space:nowrap`, so long
terms overflowed and the three columns overlapped on screen — now a capped, wrapping term
column (tlui.js CSS). Your call to strip the `template` key inside `applyScene` rather than
lean on round-trip behaviour was the right one; PLAN §4.1 stays honest.
Carried forward (yours or Lane C's): audio `in`/`out` still ignored at ffmpeg mux — draft
playback honours the trim, the render doesn't. Worth closing before anyone cuts to music.

### 2026-07-26 — M9-B ACCEPTED at SYNC 12. Beat snapping + FILM panel both verified live.
Grid: 128.02 BPM / 35 beats / 8 downbeats / conf 0.865 off house128.wav, snap select
auto-flips to beat, and snapped positions land 8–10 ms from the TRUE grid (0.468735s
period) at t=0.3 through t=12.4 — i.e. all residual is Lane C's detector, none is yours.
Bar snap hits a downbeat. beatGrid survives toJSON→load. FILM panel lists both sequences,
demo-two-shot shows 3 editable shot rows and the server's own 11.50s arithmetic.
Two calls I want to endorse, because both were judgement rather than instruction-following:
1. Storing ONE top-level `scene.beatGrid` rather than per-clip — "which grid am I snapping
   to" needs exactly one answer.
2. Keeping beats in FILE seconds and re-deriving the offset from the audio clip, so
   dragging the audio drags the grid and a head-trim leaves it put. That's the behaviour a
   musician expects and it falls out of the data shape instead of special-casing.
And you CHECKED validate_scene before inventing a field (and round-tripped it live) rather
than guessing — exactly right; that's the difference between a schema change and a bug.
NEXT: nothing queued. Open ideas if you want them: `?from=&to=` windowing on /beats for DJ
mixes (logged for Lane C), and auto-cut — given a grid and a shot list, distribute cuts
onto downbeats automatically. That last one is small now and would be the whole music-video
workflow in one button.

### 2026-07-26 — M10-B ACCEPTED at SYNC 13. Auto-cut verified on real material.
On `scenes/music-video` (4 cameras, saved 128.02 BPM grid): 4 cuts at 1.4161 / 5.1654 /
8.9146 / 12.6639 — each an exact `gridTimes('bar')` member (asserted by ===, not proximity),
3.749s apart = 2 bars, cycling cam2→cam3→cam4→cam1 with no consecutive repeat, and ONE undo
restored the prior array byte-for-byte. Refusal paths (no grid / one camera) leave both
cameraCuts and the undo stack untouched.
The phase-offset decision — never open the cycle on the camera already live — is the single
best call in this session; without it "from the playhead" would land the same angle twice,
which is the one thing a cut must never do.
One amendment requested (not a blocker, sent separately): keep user cuts that genuinely
change the angle, but DROP a kept cut whose camera equals the effective camera before it.
Real case: a hand cut at 4.50→cam2 survived immediately after an auto 1.42→cam2, i.e. a
literal no-op cut that renders as nothing but reads as an edit in the file. Report dropped
ones in the popover alongside `kept`.
NEXT: nothing queued for Lane B.