169 lines
10 KiB
Markdown
169 lines
10 KiB
Markdown
# Lane B — Timeline: master clock, tracks, keyframes, evaluation
|
|
|
|
You own: `scenegod/web/timeline.js`, `tlui.js` (+ your marked block at the
|
|
bottom of `style.css`, + your dev-only `stagestub.js` until SYNC 1).
|
|
Milestone: **M1**. You develop against the Stage API in PLAN §4.2 — code to
|
|
the contract, not to Lane A's implementation.
|
|
|
|
## B1. stagestub.js (first commit, deleted at SYNC 1)
|
|
Implements the full Stage API from PLAN §4.2 with plain objects: entities
|
|
are `{id, kind, transform, params}`, `setTransform`/`setParam` record into a
|
|
`stub.applied` array and `console.debug`. `prepareClip` returns a fake
|
|
`{duration: 2.4}` clip. This lets every timeline feature be built and tested
|
|
headless before Lane A lands.
|
|
|
|
## B2. timeline.js — the model + clock (no DOM in this file)
|
|
- Owns the authoritative scene fields: `fps`, `duration`, `entities[].tracks`,
|
|
`cameraCuts`. `load(sceneJson)` / `toJSON()` round-trip **losslessly** —
|
|
unknown fields on entities are preserved (Lane A owns them).
|
|
- Master clock: `play()` uses `requestAnimationFrame` deltas; `seek(t)`
|
|
clamps to `[0, duration]`; every advance calls `evaluate(t)` then
|
|
`onTick` listeners. `step(frame)` = `seek(frame/fps)` with NO raf — pure
|
|
and deterministic (Lane C's final render calls it in a loop).
|
|
- `evaluate(t)` per entity:
|
|
1. **Transform track**: binary-search the bracketing keys, interpolate —
|
|
pos/scale lerp, rot via Quaternion.slerp (convert stored eulers once at
|
|
load, cache quats on the key objects), easing (`linear|in|out|inout` =
|
|
cubic, `step` = hold) applied to the segment's normalized u. 0 keys →
|
|
rest transform; 1 key → hold. Result → `stage.setTransform(id, …)`.
|
|
2. **Params track**: same interpolation per `key` name (numbers lerp,
|
|
colors lerp in sRGB — fine for v1) → `stage.setParam`.
|
|
3. **Clip blocks** (characters): a block covers
|
|
`[start, start + (out-in)*loop]`. If t inside: local clip time =
|
|
`in + ((t-start) % (out-in))`; drive via the entity's mixer — one
|
|
`AnimationAction` per block, `action.time = localTime;
|
|
action.weight = w; mixer.update(0)` (set-time style, NOT free-running —
|
|
scrubbing must be exact). Crossfade: in the last `fade` seconds of a
|
|
block that has a successor, ramp weight 1→0 while successor ramps 0→1.
|
|
Outside any block: weight 0 (character holds rest / last transform).
|
|
4. **Camera cuts**: last cut with `cut.t <= t` → `stage.setActiveCamera`.
|
|
- Keyframe ops API (tlui + Lane A's event use these): `addKey(id, track, key)`,
|
|
`moveKey`, `deleteKey`, `addClipBlock(id, block)`, `moveClipBlock`,
|
|
`trimClipBlock`, `addCut(t, cameraId)` — every mutator keeps arrays sorted
|
|
by `t` and pushes an inverse op onto a simple undo stack (M2: wire to
|
|
Ctrl+Z; the stack itself costs nothing now).
|
|
- Listen for `scenegod:capturekey` (see Lane A dock): on it,
|
|
`addKey(id, 'transform', {t: this.time, ...stage.entityTransform(id), ease:'inout'})`.
|
|
|
|
## B3. tlui.js — the panel (mounts into `#timeline`)
|
|
- Left column: track names (one row per entity, auto-synced to
|
|
stage.entities via `onChange`/poll; + Cameras row + a per-character clip
|
|
sub-row). Right: time ruler + canvas-drawn track lanes (a `<canvas>` per
|
|
lane or one big one — your call, log the decision).
|
|
- Interactions (M1): click ruler = seek; drag playhead; space = play/pause;
|
|
double-click a transform lane = add key at that t from current stage state;
|
|
drag key = move (snap to frame, hold Alt = free); right-click key = delete;
|
|
drag a clip block horizontally; drag block edges = trim. Home/End = 0/duration.
|
|
- Scene bar: name field, duration field, Save/Load buttons →
|
|
`POST/GET /scenes/{name}` (Lane C; until then, localStorage fallback —
|
|
keep the fetch code, catch and fall back).
|
|
- Keep ALL DOM here; timeline.js must run headless under node for tests.
|
|
|
|
## B4. self-check — `web/timeline_test.mjs`
|
|
Runs under plain `node` (import timeline.js + stagestub): build a scene with
|
|
2 transform keys + 1 clip block + 2 camera cuts, `step()` through 90 frames,
|
|
assert: interpolated midpoint position, ease-step hold, clip local time at
|
|
3 sample frames, active camera flips at the cut, `toJSON()` deep-equals a
|
|
re-`load`ed copy. `node web/timeline_test.mjs` green = M1 code-done.
|
|
|
|
## M1 acceptance (log it)
|
|
- timeline_test.mjs passes.
|
|
- In the browser against the stub: scrub bar moves entities in the stub log,
|
|
keys draggable, play/pause works, save/load round-trips via localStorage.
|
|
- Do NOT integrate with the real Stage yourself — that's SYNC 1, the
|
|
orchestrator runs it (your code should make it a one-line swap).
|
|
|
|
## M2 preview (locked until orchestrator flips it)
|
|
Clip crossfades polish, params lanes UI, camera-cut lane UI, snapping
|
|
config, undo wiring, box-select + multi-drag of keys.
|
|
|
|
## ORCHESTRATOR UPDATES
|
|
|
|
### 2026-07-18 — M1 ACCEPTED, SYNC 1 done. **M2 is UNLOCKED.**
|
|
Your model survived integration with the real Stage; transform interp and
|
|
clip retarget playback verified frame-exact via `step()`. Notes:
|
|
|
|
1. **Fix applied by orchestrator** (in your file, keep it): `_evalClips` now
|
|
early-returns when the entity has no clip blocks or no mixer — the real
|
|
Stage returns `null` from `entityMixer` for backdrops/props/cameras; your
|
|
stub always returned an object, which hid the crash. Lesson for the stub:
|
|
mirror the contract's nullable returns.
|
|
2. **stagestub.js was NOT deleted** — `timeline_test.mjs` imports it. It's
|
|
now a permanent test fixture, not dev scaffolding. tldev.html is gone.
|
|
3. **SYNC findings, fix first in M2:** (a) after `load()` + `applyState`,
|
|
your track-name rows never appeared (only "Cameras") — rebuild rows on
|
|
`load()` and on `stage.onChange`; (b) the scene-bar name/duration fields
|
|
don't refresh after `load()` — bind them. Verified repro: load a 2-entity
|
|
scene programmatically, rows stay empty.
|
|
4. **New event:** Lane A will dispatch `scenegod:clipdrop`
|
|
`{detail:{id, path, clipIndex}}` when a clip is dropped on a character —
|
|
add a clip block on that entity at the playhead (default out = clip
|
|
duration after preload, fade 0.25 when it abuts a neighbor).
|
|
5. rAF note: in the embedded preview the tab can be `document.hidden` →
|
|
rAF fully suspended → `play()` appears frozen. Not a bug in your clock
|
|
(verified); don't "fix" it. `step()` is the deterministic path and works.
|
|
6. Injected `<style id=laneB-tl-css>` is fine permanently — keep it.
|
|
Then per plan: crossfade fade-IN polish, params + camera-cut lane UI,
|
|
snapping config, box-select multi-drag (undo already wired).
|
|
|
|
### 2026-07-18 — M2 ACCEPTED, SYNC 2 PASSED. **Session 3 (polish) granted.**
|
|
Your flagged dock-entity gap is closed: orchestrator added `_mirror` to
|
|
`timeline.js` (ctor subscribes `stage.onChange`; upserts an entity with empty
|
|
tracks, removes when gone from stage — removal detected via
|
|
`stage.getEntity(id) === null`). Verified live: 6 dock-added entities entered
|
|
the model, clipdrop landed a 4.47s block, sunset param lerp + camera cuts all
|
|
frame-exact. Session 3 items:
|
|
1. **Absorb `_mirror`** — review it, then add a timeline_test.mjs case
|
|
(stub add → mutate → remove; assert entity+tracks lifecycle). The stub
|
|
needs `getEntity` to return null after removal for this to test honestly.
|
|
2. Programmatic `scene.duration = x` doesn't refresh the scene bar (only
|
|
`load()` fires onLoad). Add a `setDuration(x)` mutator that clamps
|
|
existing keys' visibility and notifies the UI; wire the dur field to it.
|
|
3. Your listed polish: snapping increment config, keyable-param add UI.
|
|
|
|
### 2026-07-18 — session 3 ACCEPTED. **M4-B unlocked: audio on the timeline.**
|
|
Good catch fixing the stub's removeEntity-onChange payload — that's the
|
|
honest-stub lesson applied. All three items verified (setDuration bar
|
|
refresh + snap select seen live; tests green). M4-B scope (PLAN §5):
|
|
1. Audio rows in tlui from `scene.audio[]`: one lane per clip, draggable
|
|
offset (`start`), gain in the inspector-side of the row, add via a 🎵
|
|
button reading `/assets/tree` audio category; delete.
|
|
2. Web Audio playback synced to the master clock: on `play()` schedule
|
|
`AudioBufferSourceNode`s at `start - time` offsets; on `seek/pause` stop
|
|
+ reschedule. `step()` NEVER touches audio (deterministic render path —
|
|
the server muxes from scene.audio, already working end-to-end).
|
|
3. Viseme keyframes (pairs with Lane A's M4-A): a `morphs` track kind —
|
|
`{t, key: visemeName, value: weight}` evaluated via
|
|
`stage.setMorph(id, name, weight)` with lerp between keys; plus
|
|
`importRhubarb(id, rhubarbJson, t0)` mapping mouthShape cues → morph keys.
|
|
Guard: only offer when `stage.visemeTargets(id).length > 0`.
|
|
|
|
### 2026-07-18 — session 4 ACCEPTED. **v1 COMPLETE — no required B work.**
|
|
Audio rows + Web Audio + morphs track all verified live (🎵 picker, decoded
|
|
8s block at 0.5s offset, gain input, param/cut/sun lanes, zero console
|
|
errors). Your ownership note is acted on: CLAUDE.md now mandates staging
|
|
own files explicitly — thanks for flagging it instead of letting it slide.
|
|
Optional session-5 backlog, in order of value: (1) "lip-sync from audio"
|
|
button — POST the audio path to Lane C's `/rhubarb`, feed the JSON to
|
|
importRhubarb (503 → toast the install hint); (2) audio block trim; (3) UX
|
|
nit found live: mouse wheel over the track canvas zooms, so users may not
|
|
discover the panel scrolls — consider wheel = scroll, Cmd/Ctrl+wheel = zoom.
|
|
|
|
### 2026-07-19 — **M5 DIRECTOR MODE unlocked** (spec in PLAN §5 M5). Your part:
|
|
1. Listen for `scenegod:direct` (contract in lanes/A §M5.3): every preset op
|
|
becomes NORMAL keys/cuts at the playhead through your existing mutators —
|
|
`shot` → transform+fov keys on the camera + a cut to it; `light` → param
|
|
keys on sun/ambient (+bg); `mark` → transform key; `walk_to_mark` → key at
|
|
playhead-with-current-pos + key at playhead+2s at the mark. All undo-able
|
|
(one undo entry per op — group the batch, don't leave 5 half-undoable keys).
|
|
2. Wire the "lip-sync from audio" button now that `/rhubarb` is LIVE and
|
|
verified (`GET /rhubarb?path=audio/test/vo.wav` returns real mouthCues —
|
|
vo.wav is in assets): on an audio row, 🗣 button → fetch cues →
|
|
`importRhubarb(characterId, json, audioStart)`; character picker if >1
|
|
viseme-bearing char; toast the 503 install hint if absent.
|
|
3. Session-5 backlog if time remains: audio trim, wheel-scroll fix (above).
|
|
|
|
### 2026-07-19 — M6 FLOW VIDEO PLATES queued behind M5 (full spec PLAN §5 M6).
|
|
Do M5 first; M6 is unlocked the moment your M5 acceptance is logged. Test
|
|
media will be arriving in assets/backdrops/video/ via scripts/flow_intake.py.
|