838edd0e1f
A hand-placed cut is kept only if it genuinely changes the angle: the plan now merges surviving cuts with the placed ones, walks the merged list tracking the EFFECTIVE camera, and drops in-range user cuts whose camera equals it. Real case: 4.50->cam2 became a no-op once auto-cut placed cam2 at 1.4161 - it rendered as nothing but read as an edit in the saved file. Removed in the same group() so it stays one undo; popover and toast report kept vs dropped; cuts outside the placed range are still never touched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
568 lines
38 KiB
Markdown
568 lines
38 KiB
Markdown
# Lane B log — append-only, newest at bottom
|
||
|
||
Format per session:
|
||
|
||
## <date> session N
|
||
DONE: …
|
||
DECISIONS: …
|
||
BLOCKED/REQUESTS: …
|
||
NEXT: …
|
||
|
||
## 2026-07-18 session 1
|
||
DONE: M1 complete. All files under `scenegod/web/`.
|
||
- `timeline.js` — model + clock, no DOM, no three import (self-contained
|
||
quat/euler/slerp math so `node` can run it). `load/toJSON` lossless
|
||
round-trip (unknown entity fields preserved via structuredClone + a
|
||
WeakMap for cached quats). `evaluate(t)`: transform (pos/scale lerp, rot
|
||
slerp→euler, easing incl. step-hold), params (num lerp, color sRGB lerp),
|
||
clip blocks (set-time exact scrubbing, fade-out ramp), camera cuts (fires
|
||
only on change). Clock: `play()` rAF deltas, `seek` clamp, `step(frame)`
|
||
pure/deterministic for Lane C render. Mutators addKey/moveKey/deleteKey/
|
||
addClipBlock/moveClipBlock/trimClipBlock/addCut + undo stack (`undo()`
|
||
wired; Ctrl+Z hooked in tlui). `capturekey` window listener present.
|
||
Clips need `await tl.preload()` before evaluate (prepareClip is async).
|
||
- `tlui.js` — canvas panel into `#timeline`. Ruler scrub, drag playhead,
|
||
space=play/pause, dblclick lane=add key from stage state, drag key (snap
|
||
to frame, Alt=free), right-click=delete, drag clip block/edge, Home/End,
|
||
Ctrl+Z. Scene bar name/dur + Save/Load (POST/GET `/scenes/{name}`,
|
||
localStorage fallback). Names column auto-syncs to `stage.entities()` via
|
||
`onChange`.
|
||
- `stagestub.js` — full §4.2 Stage as plain objects; records effects into
|
||
`stage.applied[]`. Dev-only, delete at SYNC 1.
|
||
- `timeline_test.mjs` — `node scenegod/web/timeline_test.mjs` → green
|
||
(midpoint interp, step-ease hold, clip local time @3 frames, camera flip,
|
||
lossless round-trip, unknown-field preservation).
|
||
- `tldev.html` — dev-only browser harness. Verified in browser: scrub moves
|
||
entities (interp live in log), play runs to duration + auto-stops, camera
|
||
cut flips camA→camB at t=3, keys/clips/cuts render. Delete at SYNC 1.
|
||
DECISIONS:
|
||
- Timeline imports NO three.js — inlined euler↔quat + slerp (~30 lines) so
|
||
the node self-check needs zero deps (there is no node_modules/three in
|
||
this stack). setTransform rot is euler (matches entityTransform format).
|
||
- One `<canvas>` for all lanes (not canvas-per-lane); hit-testing via a
|
||
hitbox list rebuilt each draw. Simpler, fast enough at this scale.
|
||
- CSS injected from tlui.js via a `<style id=laneB-tl-css>` rather than
|
||
creating `web/style.css` — that file is Lane A's and didn't exist yet.
|
||
Move into a `/* === LANE B === */` block at SYNC if John wants it
|
||
centralized. (See REQUEST below.)
|
||
- M1 crossfade = fade-OUT only; cross-block fade-IN is M2 "crossfade polish"
|
||
per the milestone list. Marked with a ponytail comment in timeline.js.
|
||
BLOCKED/REQUESTS:
|
||
- None blocking. Note for orchestrator/Lane A: my timeline styles live in
|
||
an injected `<style>` in tlui.js. If you'd rather they sit in style.css,
|
||
say so and I'll emit a `/* === LANE B === */` block for Lane A to paste
|
||
(I won't edit style.css directly — ownership).
|
||
- Lane A's files (index.html, stage.js, room3d.js, dock.js, style.css) are
|
||
present untracked in the shared working dir but uncommitted — I staged
|
||
ONLY my 5 files, left theirs alone.
|
||
NEXT: Hold at M1 boundary. At SYNC 1 the stub swaps for the real Stage
|
||
(one-line change in the page import) and tldev.html/stagestub.js get
|
||
deleted. M2 (locked): clip crossfade polish, params/cut lane UI, snapping
|
||
config, undo Ctrl+Z already stubbed, box-select multi-drag.
|
||
|
||
## 2026-07-18 session 2 (M2)
|
||
DONE: M2 per orchestrator update. SYNC findings fixed first, then polish.
|
||
- SYNC1 #3a/#3b: added `Timeline.onLoad(cb)`; `load()` fires it →
|
||
TimelineUI rebuilds rows + refreshes name/duration fields. Verified with
|
||
the exact repro (UI built against empty stage, then `load()` a 2-entity
|
||
scene → rows now appear, scene bar updates).
|
||
- SYNC1 #4 clipdrop: `scenegod:clipdrop {id,path,clipIndex}` → `addClipDrop`
|
||
drops a block at the playhead, `out = clip.duration` (from preload), auto
|
||
`fade:0.25` where it abuts a neighbour. Verified via dispatched event.
|
||
- Crossfade fade-IN (was fade-OUT only): a predecessor's fade pre-rolls the
|
||
successor's playback start earlier by `pred.fade` so local time is
|
||
continuous across the boundary; weights ramp 1→0 / 0→1 and sum to 1
|
||
(asserted in test).
|
||
- Params lanes (per-key hue), camera-cut lane add via dblclick on Cameras
|
||
row (uses active cam / first camera), snap toggle in scene bar (Alt still
|
||
= free), box-select + multi-drag + Delete of keys, selection highlight,
|
||
fade wedge drawn on clip blocks, panel grows to content height (was fixed
|
||
200px → clipped the Cameras row once params rows were added).
|
||
- timeline_test.mjs extended: onLoad fires once, addClipDrop out=duration,
|
||
crossfade weights sum to 1. All green.
|
||
DECISIONS / BUG:
|
||
- **tlui reads tracks from the TIMELINE, not the stage entity.** Found mid-
|
||
verify: `load()` structuredClones the scene, so timeline.scene.entities and
|
||
stage entities are SEPARATE objects. Drawing from stage meant clipdrop/
|
||
addKey (which mutate the timeline copy) never appeared. Repointed all draw
|
||
methods to `_te(id) = tl.scene.entities.find(id)`. Rows now built from the
|
||
timeline list merged with any stage-only (dock-added, not-yet-keyframed)
|
||
entities. Per PLAN the timeline owns tracks — this is the correct source.
|
||
- Grow-to-content height, no scroll yet (ponytail: add max-height+scroll only
|
||
if a scene ever has enough entities to dominate the viewport).
|
||
BLOCKED/REQUESTS:
|
||
- Gap for the orchestrator/Lane A to note: there's no bridge for a
|
||
dock-added stage entity to enter `timeline.scene.entities`, so a freshly
|
||
dropped character can't be keyframed until a scene load. tlui shows it as a
|
||
row (empty tracks) but mutators need the entity present. If Lane A's dock
|
||
should register new entities with the timeline, we need a
|
||
`timeline.addEntity(desc)` (or the dock calls `stage.captureState()` →
|
||
`timeline.load`). Flagging for SYNC 2 — not building it uninvited.
|
||
- Verified with a throwaway harness (`_check.html`) served on :8099; deleted,
|
||
NOT committed (orchestrator removed the M1 tldev.html deliberately).
|
||
NEXT: Hold at M2. Remaining M2 polish if a session 3 is granted: undo already
|
||
wired to Ctrl+Z; could add snapping increment config + keyable-param add UI
|
||
(currently params keys come from Lane A capture/inspector). Await SYNC 2.
|
||
|
||
## 2026-07-18 session 3 (M2 polish)
|
||
DONE: All three orchestrator session-3 items.
|
||
1. Absorbed `_mirror` (orchestrator's seam) — reviewed; it upserts dock-added
|
||
stage entities into `scene.entities` with empty tracks and drops them when
|
||
`stage.getEntity(id)===null`. Fixed the STUB to make removal testable:
|
||
`removeEntity` now fires `onChange` WITH the removed entity object (was
|
||
firing `null`, so `_mirror(null)` early-returned and never removed).
|
||
Added timeline_test.mjs lifecycle case: dock-add → keyframe → remove,
|
||
asserting entity+empty-tracks appear then vanish.
|
||
2. `setDuration(x)` mutator: clamps the playhead into range, KEEPS keys past
|
||
the new end (lossless), notifies the UI (reuses onLoad → syncRows +
|
||
refreshBar). Wired the dur field to it — plain `scene.duration = x` skipped
|
||
the bar refresh. Test asserts clamp + key retention + notify.
|
||
3. Snap-increment select (frame / 0.1 / 0.25 / 1s) in the scene bar; `_snap`
|
||
rounds to the chosen step (Alt still = free). Keyable-param add: dblclick a
|
||
params lane captures the entity's current param values as keys (skips
|
||
`type`/boolean descriptors) — mirrors the transform-capture dblclick.
|
||
DECISIONS:
|
||
- params lane now shows only when the entity has ≥1 param key OR kind is
|
||
camera/light. `_mirror` seeds `tracks.params=[]` (truthy), which was
|
||
giving every mirrored character an empty params lane; tightened to
|
||
`tracks?.params?.length > 0 || camera|light`.
|
||
- setDuration reuses the onLoad callback list as its "refresh UI chrome" hook
|
||
rather than adding a second callback channel (lazy; onLoad handler already
|
||
does syncRows+refreshBar+draw).
|
||
BLOCKED/REQUESTS: none. Verified with a throwaway harness (`_check.html`,
|
||
served :8099, deleted, NOT committed). timeline_test.mjs green.
|
||
NEXT: M2 polish complete. Undo already wired (Ctrl+Z). Open for M4 timeline
|
||
work when unlocked (audio tracks on the timeline UI + Web Audio sync, per
|
||
PLAN §5 M4) or any SYNC 3 fixes. Holding until orchestrator flips M4.
|
||
|
||
## 2026-07-18 session 4 (M4-B: audio + visemes)
|
||
DONE: M4-B all three items.
|
||
1. Audio rows in tlui from scene.audio[]: one lane per clip below Cameras,
|
||
green block at the DECODED buffer width (fallback width until decoded),
|
||
gain <input> in the name column, drag = offset (moveAudio), right-click =
|
||
delete. 🎵 button reads /assets/tree audio category into a small floating
|
||
menu → addAudio at the playhead.
|
||
2. Web Audio draft playback synced to the master clock: lazy AudioContext,
|
||
decode+cache buffers (/assets/file), _audioStart schedules
|
||
AudioBufferSourceNodes at (start - time) with per-clip gain, _audioStop on
|
||
pause/natural-end, user seeks reschedule (_seek helper). step()/render
|
||
never touch audio — server muxes from scene.audio.
|
||
3. morphs track kind: _evalMorphs lerps per viseme name → stage.setMorph;
|
||
importRhubarb(id, json, t0) maps mouthCues → morph keys; tlui shows a
|
||
"↳ visemes" lane guarded by stage.visemeTargets(id).length > 0.
|
||
Model (timeline.js, stagestub.js, timeline_test.mjs) committed ba29d0c;
|
||
tests cover morph lerp, rhubarb keys, audio mutators+undo. All green.
|
||
VERIFIED (real server :8020, real assets): 🎵 picker listed + added
|
||
audio/test/tone.wav, buffer decoded (8s), green block drawn, Web Audio
|
||
scheduled/rescheduled/stopped (ctx running), viseme lane rendered
|
||
importRhubarb keys, setMorph fired while scrubbing. Contract match confirmed:
|
||
real stage.js ships setMorph(id,name,weight) + visemeTargets(id) (returns
|
||
Object.keys(visemes)) — my `.length>0` guard works against both.
|
||
DECISIONS:
|
||
- Audio duration is a client draw concern (decode+cache in tlui), NOT stored
|
||
in the {path,start,gain} schema — server already knows durations at mux.
|
||
- importRhubarb = simple per-cue ramp (shape 1 at cue start → 0 at end);
|
||
hold/crossfade tuning deferred (ponytail marker in timeline.js) until it
|
||
looks wrong on a real blendshape character.
|
||
- Audio blocks can extend past scene.duration (tone.wav is 8s in a 6s scene)
|
||
— left as-is; the ruler just clips visually. Trim is a later nicety.
|
||
BLOCKED/REQUESTS:
|
||
- **Ownership note for orchestrator**: my tlui.js M4-B changes (+146 lines)
|
||
were committed inside Lane A's `df4e06f [laneA] M4-A ...`, not a [laneB]
|
||
commit — Lane A's commit swept my then-uncommitted tlui.js working changes
|
||
along with their dock.js/stage.js. Code is intact + verified; only the
|
||
attribution is off. Heads-up that our git hygiene let a cross-lane file
|
||
ride in on another lane's commit; a `git add <own files>` (not `-A`) avoids
|
||
it. Nothing for me to fix.
|
||
NEXT: M4-B complete. Open items if a session 5 lands: audio block trim/clip to
|
||
duration, viseme import UI button (currently importRhubarb is API-only; Lane
|
||
C has a /rhubarb endpoint — could wire a "lip-sync from audio" button that
|
||
POSTs the clip and feeds the result to importRhubarb). Holding for SYNC/next
|
||
orchestrator update.
|
||
|
||
## 2026-07-20 session 5 (M5 director-mode B + backlog, M6 video-follow)
|
||
DONE: All three M5 items + the M6 item, headless-verified.
|
||
1. `scenegod:direct` listener (timeline.js ctor) → `applyDirect(detail)`:
|
||
shot → transform+fov keys on the camera at playhead + addCut; light →
|
||
param keys; mark → transform key (walkTo → current/prevPos key at
|
||
playhead + mark key at playhead+2s). New `group(fn)` collapses every
|
||
mutator push inside one op into ONE undo entry — one Ctrl+Z reverts a
|
||
whole preset. `importRhubarb` is now grouped too.
|
||
**Payload coordination**: presets.js landed mid-session (untracked, Lane
|
||
A in flight), so I read it — Lane A's REAL payloads differ from the
|
||
PLAN-stated contract: values are spread at TOP level (`{op:'shot', camId,
|
||
pos, rot, fov,…}` — no `transform:{}` wrapper), and light is
|
||
`{sun:{color,intensity}, ambient:{…}, bg, sunId, ambientId}` (per-entity
|
||
values, not a flat `params` map; bg keys onto the ambient entity;
|
||
applyDirect ALSO captures the sun-boom transform from the stage since the
|
||
payload lacks sunPos rot). applyDirect accepts BOTH shapes (contract +
|
||
live); tests cover both.
|
||
2. 🗣 button on every audio row (tlui) → `GET /rhubarb?path=` →
|
||
`importRhubarb(charId, json, clip.start)`. Character picker (`_choose`
|
||
promise-menu) when >1 viseme-bearing character, guard toast when 0,
|
||
`_toast` shows the 503 install hint from FastAPI `detail`. Viseme lane
|
||
resyncs after import.
|
||
3. Backlog: (a) wheel fix — plain wheel is NOT captured (page scrolls);
|
||
Cmd/Ctrl+wheel (incl. trackpad pinch) zooms 1–64× around the cursor,
|
||
horizontal wheel pans when zoomed; all geometry through _t2x/_x2t so
|
||
keys/blocks/playhead/hitboxes just work. (b) audio trim: drag the block's
|
||
right edge → `trimAudio(clip, {in,out})` (source-relative, like clip
|
||
blocks; absent = untrimmed so the {path,start,gain} schema is unchanged
|
||
for untouched clips); draft Web Audio playback honors in/out; ✂ badge on
|
||
trimmed blocks; undo restores pristine (fields deleted, not undefined).
|
||
4. M6: `_evalVideo` in evaluate for backdrop/screen entities via new nullable
|
||
stage seam `entityVideo(id)`: `currentTime = max(0, t-videoStart) %
|
||
duration` ONLY when drift >50ms; play/pause follows the clock (pause()
|
||
now re-evaluates so videos freeze); play() promise .catch()ed, never
|
||
awaited — step() stays sync (render determinism is stage.syncVideos,
|
||
Lane A/C).
|
||
Files: timeline.js, tlui.js, stagestub.js (StubVideo records currentTime
|
||
sets; entityVideo null for image backdrops — honest-nullable),
|
||
timeline_test.mjs. `node scenegod/web/timeline_test.mjs` → all green
|
||
(shot-via-real-EventTarget-dispatch + one-undo revert, generic + Lane A
|
||
light shapes, walkTo 2 keys, Lane A mark shape, rhubarb group undo, trim
|
||
undo, video drift-gate/modulo/videoStart-clamp/play-pause, nullable video).
|
||
DECISIONS:
|
||
- group() splices the undo stack rather than a begin/end transaction API —
|
||
mutators stay dumb, nesting works, zero cost when fn pushes ≤1 entry.
|
||
- Sun transform key on light presets comes from stage.entityTransform at
|
||
event time (Lane A setTransforms before firing) — payload has sunPos but
|
||
no rot; reading the stage gets both without a payload change.
|
||
- Video seek target clamps negative (t < videoStart → hold frame 0); wrap
|
||
uses plain % (t-videoStart ≥ 0 after clamp).
|
||
- StubVideo lives in a stage-side Map, NOT on the entity object, so
|
||
captureState/clone stays clean.
|
||
BLOCKED/REQUESTS:
|
||
- **Lane A**: walk-to-mark can't work from the current payload — applyMark
|
||
setTransforms the subject BEFORE firing, so "current pos" at event time is
|
||
already the mark. When you add a walk mark, either include `prevPos` (+
|
||
`walkTo:true`) in the detail (applyDirect already honors both) or skip the
|
||
pre-apply for walk marks. Also: no walk mark exists in grammar.js MARKS yet.
|
||
- **Lane C**: audio trim adds optional `in`/`out` (source-relative seconds)
|
||
to scene.audio clips — mux should honor them (ffmpeg -ss/-t per clip);
|
||
absent = whole file (schema unchanged for untrimmed clips). Client draft
|
||
playback already honors them.
|
||
- **Orchestrator**: PLAN §4.1 audio schema + §4.2 Stage API could gain the
|
||
two seams now in live use: audio {in?, out?} and `entityVideo(id) ->
|
||
HTMLVideoElement|null` (Lane A implements on real Stage for M6-A).
|
||
NEXT: SYNC — browser-verify DIRECT panel button → keys+cut land + single
|
||
undo; 🗣 on real vo.wav → viseme keys on a blendshape character (and the
|
||
503 toast with rhubarb absent); Cmd+wheel zoom feel; video backdrop
|
||
scrub-follow once Lane A ships entityVideo. Holding at M6-B boundary.
|
||
|
||
## 2026-07-25 session 6 (M7-B: the way in — templates + shortcuts overlay)
|
||
DONE: both required items + one backlog item, all headless-verified.
|
||
1. **New from template** (scene bar, left of Save/Load — `tlui.js:122`).
|
||
`_pickTemplate()` (tlui.js:585) — `GET templates` → floating `.tl-menu.tmpl`
|
||
(same pattern as the 🎵 picker) with title + description + thumb
|
||
(`assets/file?path=`, `img.onerror` → text-only row). All URLs RELATIVE.
|
||
`newFromTemplate(name)` (tlui.js:616): warn-before-clobber via
|
||
`tl.hasContent()` → `confirm()`, then `GET templates/{name}` →
|
||
`tl.applyScene(json)`, `_decodeAll()`, `draw()`. Any failure (fetch, !ok,
|
||
apply) toasts through `_toast` (window.sgToast still points at it).
|
||
2. **`?` shortcuts overlay** (tlui.js:633 `_toggleHelp` / `_helpClose`,
|
||
CSS `.tl-help` at tlui.js:48). Opens with `?` or the `?` button
|
||
(tlui.js:121), closes on Escape / click-outside / ✕. Three columns —
|
||
STAGE, TIMELINE, SCENE BAR — written by READING the code, not from memory:
|
||
stage.js:118-122 (W/E/R gizmo, selection-gated), dock.js drops, and every
|
||
binding in my `_key/_down/_dbl/_ctx/_wheel`. Deliberately NOT listed:
|
||
right-click on a clip block (only keys/cuts/audio blocks delete), left-edge
|
||
trim on animation clip blocks (only the right edge is a hit target).
|
||
3. Backlog: **audio HEAD trim** — dragging an audio block's LEFT edge now eats
|
||
into the source (`in` and `start` move together so the audible part stays
|
||
put) instead of being dead space. `timeline.js:trimAudioIn` (grouped: one
|
||
undo for the pair, clamps at the file head, at t=0, and 0.05s before `out`);
|
||
tlui `aedge` hits now carry `side` ('in' | 'out').
|
||
Model seams added to timeline.js: `applyScene(json)` (timeline.js:173) —
|
||
stage.applyState → load → preload, the ONE apply path now shared by Load and
|
||
New-from-template; and `hasContent()` (timeline.js:185).
|
||
TEST: `node scenegod/web/timeline_test.mjs` → green, +5 new blocks:
|
||
hasContent (empty / cuts-only / audio-only / dock-added entity), applyScene
|
||
happy path (entities on stage, tracks in the model, onLoad once, camera live,
|
||
clip preloaded and playing at t=1, `template` key stripped, caller's json not
|
||
mutated), a loop that applies EVERY shipped `templates/*.json` through
|
||
applyScene against the stub ("3 shipped templates applied clean"), and
|
||
head-trim (pair + single undo + both clamps). `node --check` clean on
|
||
timeline.js, tlui.js, stagestub.js, timeline_test.mjs.
|
||
DECISIONS:
|
||
- `tlui.load()` now routes through `tl.applyScene` too, so Load and templates
|
||
cannot drift apart (the ask was "apply it the same way Load does" — made it
|
||
literally the same code rather than a copy).
|
||
- `applyScene` deletes the `template` key before loading. server.py:264 claims
|
||
that metadata "never survives a Timeline round-trip" — it WOULD have, since
|
||
load() structuredClones unknown fields on purpose (lossless round-trip is a
|
||
tested feature). Stripping it in one place makes Lane C's comment true and
|
||
keeps saved scenes plain scene JSON.
|
||
- `hasContent()` counts ANY entity, cut or audio clip — a dock-added,
|
||
not-yet-keyframed character is still work worth a confirm. A fresh page has
|
||
zero entities (Stage creates none in its ctor), so the first New… never
|
||
nags. Used plain `confirm()` — a modal I'd have to build is not more honest
|
||
than the browser's.
|
||
- `_key` now also ignores SELECT + contentEditable targets, so `?` (and Space)
|
||
can't be hijacked while the snap dropdown or any text field has focus.
|
||
- The `?` button is excluded from the overlay's click-outside handler,
|
||
otherwise mousedown closed it and the click reopened it.
|
||
BLOCKED/REQUESTS:
|
||
- **Lane C / orchestrator (nit)**: `GET /templates` returns thumbs that are
|
||
ASSET-relative (`backdrops/video/scenegod_plates/*.jpg`) — I render them via
|
||
`assets/file?path=`. If a future template ever ships a thumb that lives
|
||
outside SCENEGOD_ASSETS, the picker will show a broken (auto-removed) image;
|
||
keep thumbs inside the assets root.
|
||
- **Lane C (still open from session 5)**: audio `in`/`out` at ffmpeg mux.
|
||
Head trim makes it much easier to hit — a head-trimmed clip now renders from
|
||
the file start server-side while the draft playback starts at `in`.
|
||
NEXT: nothing required. Remaining polish if a session 7 lands: box-select
|
||
multi-drag refinements (marquee currently only selects keys — clip/audio
|
||
blocks and cuts are not box-selectable), and a per-drag undo coalesce (a
|
||
mousemove drag pushes one undo entry per frame; Ctrl+Z walks it back a pixel
|
||
at a time). Holding at the M7-B boundary.
|
||
|
||
## 2026-07-26 session 7 (M9-B: snap to the beat + the FILM panel)
|
||
DONE: both features, headless-verified. No git run this session (orchestrator's
|
||
instruction) — `timeline.js`, `tlui.js`, `timeline_test.mjs` and this log are
|
||
edited in place, uncommitted. `stagestub.js` needed no change this time.
|
||
|
||
**M9-B1 — SNAP TO THE BEAT.**
|
||
- Model (`timeline.js:578-673`): `setBeatGrid/clearBeatGrid` (`:593`),
|
||
`beatOffset()` (`:600`), `gridTimes(kind)` (`:616`), `snapToGrid(t, kind)`
|
||
(`:635`), plus the pure `beatGridFrom(res, offset)` export (`:678`) that maps
|
||
Lane C's `/beats` response into the stored shape.
|
||
- UI (`tlui.js`): `♪` button on every audio row next to `🗣` (`:~250`,
|
||
`_detectBeats` at `:775`) — `GET beats?path=` → cache per path
|
||
(`this._beatCache`) → `setBeatGrid` → snap flips to `beat` and the toast says
|
||
the BPM, the beat count and the confidence. **beat** + **bar** added to the
|
||
snap-increment select (`:156`); `_snap()` (`:477`) routes those two through
|
||
`tl.snapToGrid` and falls back to frames when no grid exists (plus a toast if
|
||
you pick beat/bar with nothing detected). Grid drawing: `_drawBeatGrid`
|
||
(`:365`) — beats `rgba(146,180,232,0.13)`, downbeats `rgba(224,175,104,0.34)`,
|
||
over the lanes, suppressed below ~7px spacing (beats drop out first, then bars,
|
||
then nothing — never a solid wash).
|
||
- Because `_snap` was already the ONE snapping funnel, keys, clip blocks, audio
|
||
blocks, both trim edges, camera cuts and the playhead all became musical for
|
||
free. That was the whole bet in the M2 design and it paid.
|
||
|
||
**M9-B2 — FILM panel** (`tlui.js:848-1130`): collapsible `.tl-film` section
|
||
mounted INSIDE `#timeline` (`_buildFilm` `:854`, toggled by `🎬` in the scene
|
||
bar `:158`). Sequence dropdown + New…/Rename/Delete, computed runtime, size
|
||
picker, Save, Render film. Shot rows (`_filmRow` `:960`): scene `<select>` from
|
||
`GET scenes` (with `/Ns` scene length shown), in/out numbers, transition
|
||
cut|dissolve (+ seconds, only where a next shot exists), `↑ ↓ ✕`. All server
|
||
talk goes through Lane C's `film.js` helpers (`listSequences/getSequence/
|
||
saveSequence/deleteSequence/renderSequence`) — no hand-rolled URLs.
|
||
`_filmRender` (`:1100`): `hasContent()` + a confirm that says plainly that every
|
||
shot loads its own scene and the open scene will be REPLACED; then it SAVES
|
||
first (the server renders the saved shot list, not the editor), disables the
|
||
whole panel for the duration (`_filmBusySet` `:1045`, so no second render), shows
|
||
`shot 2/3 — sync2-sunset — frame 45/120` live, and ends with a link to
|
||
`films/{id}/out.mp4` + a note that the stage now holds the last shot's scene.
|
||
422s from `plan_sequence` are unwrapped by `_errText` and shown in red.
|
||
|
||
TEST: `node scenegod/web/timeline_test.mjs` → green, +2 blocks (`:394`, `:482`).
|
||
Beat block runs against the REAL captured `/beats` payload for
|
||
`audio/test/house128.wav` (bpm 128.02, 35 beats, 8 downbeats, conf 0.865), and
|
||
asserts: no grid → `snapToGrid` null + nothing to draw; nearest beat over 9 time
|
||
points and nearest downbeat over 6; the 0.24/0.25s midpoint flip; bars use Lane
|
||
C's downbeat COSET (first downbeat is beats[3], not beats[0]); extrapolation past
|
||
the last beat; never snapping below 0; grid follows `moveAudio` and survives
|
||
`trimAudioIn` unmoved (start and `in` move together); `gridTimes` covers a longer
|
||
scene and stays sorted; bar lines are a subset of beat lines; a 0.12-confidence
|
||
grid still snaps and a grid alone does NOT make `hasContent()` true; `toJSON` →
|
||
`load` → identical snapping; `clearBeatGrid` leaves plain scene JSON. Film block
|
||
asserts `filmRuntime` equals the server's arithmetic on the shipped
|
||
`demo-two-shot` (11.50s), plus the last-shot-dissolve and cut-transitionDur
|
||
edge cases. `node --check` clean on all four files.
|
||
SERVER-SIDE CHECKS (read-only against the running :8020, nothing restarted):
|
||
- `validate_scene` (server.py:235) inspects only version/entities/tracks/
|
||
cameraCuts — verified by calling it directly with a `beatGrid` scene
|
||
(`violations: []`), so the new top-level key is legal, not merely tolerated.
|
||
- Full round-trip proven live: POST a scene carrying `beatGrid` → 200, GET it
|
||
back → grid intact byte-for-byte. Test scene file deleted again (scenes/ holds
|
||
the same four it did before).
|
||
DECISIONS:
|
||
- **Field shape — ONE top-level `scene.beatGrid`, not per-audio-clip**:
|
||
`{path, bpm, confidence, meter, period, offset, beats[], downbeats[]}`.
|
||
A music video has one tempo, and "which grid am I snapping to" must have
|
||
exactly one answer; per-clip grids would need a picker for a question nobody
|
||
asks. `beats`/`downbeats` stay in FILE seconds exactly as Lane C returns them
|
||
(so the stored grid is comparable with a re-fetch) and `offset` records where
|
||
the file's t=0 sat. The EFFECTIVE offset is re-derived from the audio clip
|
||
with the same `path` whenever one is still on the timeline — drag the audio
|
||
block and the grid comes with it; delete the clip and the stored `offset`
|
||
keeps it working.
|
||
- **The grid extrapolates at constant tempo outside the analysed range**, in
|
||
BOTH `gridTimes` (drawing) and `snapToGrid`, so the lines you see and the
|
||
snap you get agree everywhere. Without it, an 8s track under a 30s scene drags
|
||
every late key back onto the last real beat — a trap, not a feature. Lane C's
|
||
grid is explicitly constant-tempo, so this invents nothing.
|
||
- **Low confidence warns, never refuses** (Lane C: >0.5 solid, <0.2 mush). The
|
||
UI toasts "LOW confidence (0.12) — this grid may be mush" at <0.35 and snaps
|
||
anyway; the model has no opinion about confidence at all.
|
||
- The fetch + per-path cache live in **tlui**, like every other server call in
|
||
this lane (`/rhubarb`, `/templates`, `/assets/tree`); `timeline.js` got the
|
||
pure `beatGridFrom` instead, so the response→storage contract is testable
|
||
headless without stubbing `fetch` for a three-line call.
|
||
- `hasContent()` deliberately ignores `beatGrid`: it is one button press to
|
||
rebuild, so it is not work worth a confirm.
|
||
- **film.js is imported lazily** (`_filmMod()`, first 🎬 press) rather than at
|
||
the top of tlui.js: the timeline must still mount if Lane C's client files are
|
||
missing or broken, and nobody who never opens FILM should pay for render.js.
|
||
- `filmRuntime(shots)` is exported from tlui.js as a pure function so the number
|
||
on screen can be pinned to the server's own formula in the node test — tlui.js
|
||
touches no DOM at module scope, and the test's `import('./tlui.js')` now fails
|
||
loudly the day that changes.
|
||
- Rename is save-then-delete (a failed save leaves the original untouched); the
|
||
slug the server returns becomes the working name.
|
||
BLOCKED/REQUESTS:
|
||
- **Lane C / orchestrator (nice-to-have, not blocking)**: `/beats` on a DJ mix
|
||
assumes one tempo for the whole hour — Lane C already has `?from=&to=`
|
||
windowing on its own backlog. When it lands I'd store one grid per window, or
|
||
at minimum re-detect from the playhead; the current shape can grow a
|
||
`range:[from,to]` field without breaking anything.
|
||
- **Lane C (still open, third session running)**: nothing new — audio `in`/`out`
|
||
at the mux is CLOSED as of C session 7, thanks.
|
||
- **Orchestrator**: PLAN §4.1 could gain the two fields now in live use —
|
||
`scene.beatGrid` (shape above) and the already-shipped audio `{in?, out?}`.
|
||
NEXT: nothing required. Backlog if a session 8 lands: (1) a "nudge to grid" action
|
||
that re-snaps EXISTING keys/cuts to the nearest beat (today only new drags snap);
|
||
(2) box-select still only grabs keys, not clip/audio blocks or cuts; (3) per-drag
|
||
undo coalesce (a mousemove drag still pushes one undo entry per frame);
|
||
(4) film-level `audio[]` has no UI — a sequence's soundtrack must still be edited
|
||
by hand, and it is the obvious partner to beat-snapping.
|
||
|
||
## 2026-07-26 session 8 (M10-B: AUTO-CUT + align a film to the beat)
|
||
DONE: both, headless-verified. No git run (standing instruction) — `timeline.js`,
|
||
`tlui.js`, `timeline_test.mjs` and this log edited in place. `stagestub.js`
|
||
needed no change again. The running :8020 already serves the edits (checked with
|
||
a read-only curl: `/web/timeline.js` carries `autoCutPlan`, nothing restarted).
|
||
|
||
**M10-B1 — AUTO-CUT** (`timeline.js:635-700`, UI `tlui.js:823-910`).
|
||
- `cameraIds()` (`:635`) — cameras in scene order, merged with any stage camera
|
||
not yet mirrored (the same merge tlui's `_rows` uses).
|
||
- `autoCutPlan(opts)` (`:647`) — PURE preview: `{ok, cuts, replaced, kept,
|
||
cameras, cadence, barPeriod, confidence, bpm, from, to}` or `{ok:false,error}`.
|
||
Cuts come straight out of `gridTimes('bar')` (so a placed t IS a grid downbeat,
|
||
not a rounded neighbour); `bars.filter((_,i)=>i%cadence===0)`; cameras cycle
|
||
`cams[(i+phase)%n]`.
|
||
- `autoCut(opts)` (`:687`) — applies inside `group()` → ONE undo; `throw`s the
|
||
plan's error rather than half-applying; `seek(time)` so the new cut goes live.
|
||
- `removeCut(cut)` (`:459`) — new undo-able mutator (replacements had to be
|
||
undoable too; tlui's right-click delete still splices directly, unchanged).
|
||
- UI: `✂` in the scene bar → popover with cadence (1/2/4/8 bars, default 2),
|
||
range (whole scene | from the playhead), a checkbox per camera, and a live
|
||
"will place N cuts · one every 3.75s" line that is literally `autoCutPlan()`
|
||
— the previewed count and the applied count are the same call. Refusals
|
||
render in that line in red with Apply disabled. Escape / click-outside close.
|
||
The count re-runs on every tick, so scrubbing with "from the playhead"
|
||
selected keeps it honest.
|
||
TEST: `node scenegod/web/timeline_test.mjs` → green, +2 blocks (`:498`, `:646`).
|
||
Auto-cut block asserts against the same real house128.wav payload: 8 downbeats
|
||
in a 16s scene → cadence 1/2/4/8 gives 8/4/2/1 cuts; every placed `t` is
|
||
`===` a member of `gridTimes('bar')`; 2 cameras strictly alternate A/B/A/B;
|
||
3 cameras cycle A/B/C/A/B/C/A/B; no camera lands twice in a row; ONE undo
|
||
restores the exact prior `cameraCuts` (deepEqual against a pre-snapshot) and
|
||
the stack grows by exactly 1; re-running replaces its own cuts (4 cuts stay 4,
|
||
all `t` unique, same positions); a hand cut ON a downbeat is replaced, one
|
||
BETWEEN downbeats is kept untouched and reported; no-grid and one-camera both
|
||
throw with a clear message AND leave `cameraCuts` and the undo stack empty;
|
||
an empty range refuses; the plan follows the audio when the block is dragged.
|
||
Also run against the REAL `scenes/music-video.json` (4 cams, saved 128.02 BPM
|
||
grid): cadence 2 → 1.416:cam2 5.165:cam3 8.915:cam4 12.664:cam1, undo restores
|
||
the file's original two cuts exactly.
|
||
|
||
**M10-B2 — align a film's shots to the beat** (`timeline.js:772-793` pure,
|
||
`tlui.js:1213-1252` panel). `♪ align` button in the FILM head row.
|
||
- WHICH GRID: the open scene's if it has one, else the first shot scene (in
|
||
shot order) with a saved `beatGrid`, fetched via `scenes/{name}`. The status
|
||
line always names the source ("from the open scene \"music-video\"" /
|
||
"from shot scene \"x\"") — "why did it snap there" must have a visible answer.
|
||
- `alignShotsToGrid(shots, grid, off, durations)` snaps each `in`/`out` to the
|
||
nearest downbeat, never reorders, and returns `{shots, skipped}`. A shot that
|
||
would collapse is returned BYTE-IDENTICAL and listed in `skipped`; the panel
|
||
prints "left alone: #3 (would collapse)". Runtime redraws via `_filmRows()`;
|
||
nothing is sent to the server until the user presses Save.
|
||
TEST: order preserved, both boundaries land on real grid lines, `out > in` for
|
||
every surviving shot, the collapsing shot is untouched + reported, the caller's
|
||
array is not mutated, transitions ride along, aligning twice is a no-op, and an
|
||
offset grid shifts the whole alignment. Verified against both shipped
|
||
sequences with the music-video grid: demo-two-shot 0-4.5|4.5-8|0-4 becomes
|
||
1.4161-5.1654|5.1654-7.04|1.4161-3.2907, 0 skipped.
|
||
DECISIONS:
|
||
- **Pure grid helpers extracted** (`beatOffsetOf`, `beatPeriodOf`, `gridTimesOf`,
|
||
`snapToGridWith`, `timeline.js:716-770`); `beatOffset/gridTimes/snapToGrid`
|
||
are now one-line delegates. B2 aligns shots of scenes that are NOT loaded —
|
||
it has a grid and a shot list, no Timeline — and it must get bit-identical
|
||
arithmetic to what the user saw while cutting. Behaviour unchanged: the whole
|
||
M9 beat block still passes untouched.
|
||
- **Replace ON a position, keep everything else.** An existing cut within half
|
||
a frame of a placed time is replaced (a duplicate 15ms away is a flash frame);
|
||
cuts elsewhere in the range are the user's edit and are NOT deleted. But they
|
||
interleave and one can repeat a camera, so the plan returns `kept` and the
|
||
popover says "1 of your own cuts left in place between them". Disclosure
|
||
beats either silently stuttering or silently deleting. Flip to "clear the
|
||
range" if you'd rather, it's a one-line change — but I'd want John to ask.
|
||
- **Phase offset so the run doesn't open on the camera already live.** With
|
||
"from the playhead", the last cut before the range is checked; if it holds
|
||
`cams[0]` the cycle starts at `cams[1]`. Otherwise a beat-perfect run can
|
||
still open with the same angle twice, which is the one thing a cut must
|
||
never do. Verified live on music-video.json (cut at 0 = cam1 → run opens cam2).
|
||
- **No cleverness, as instructed**: no random order, no energy analysis, no
|
||
variation. Every knob is on the popover and the output is ordinary
|
||
`cameraCuts` the user hand-edits after.
|
||
- **B2 clamps by walking back a whole bar and RE-SNAPPING** (`timeline.js:786`),
|
||
not by subtracting the bar period: Lane C rounds beats to 4dp and the period
|
||
to 6dp, so `b - bar` lands 48µs off a real downbeat. "Lands on a downbeat"
|
||
has to mean the real one, and the test asserts membership at 1e-6.
|
||
- `autoCut` THROWS on refusal instead of returning a flag: the caller already
|
||
has `autoCutPlan()` to check, so a thrown error can't be ignored into a
|
||
half-applied edit. The popover never lets you press Apply on a bad plan anyway.
|
||
- FILM align has no undo (the panel has none by design — the server doc is the
|
||
state and Save is the gate); the status line says "Save to keep it".
|
||
BLOCKED/REQUESTS:
|
||
- **Orchestrator**: PLAN §4.1 still doesn't list `scene.beatGrid` or audio
|
||
`{in?, out?}`, both in live use since M9 (carried from last session).
|
||
- **Lane C (nice-to-have)**: `GET scenes` returns `{name, duration}` but not
|
||
whether a scene HAS a beatGrid. B2 has to fetch whole scene JSONs one at a
|
||
time to find one. A `hasGrid`/`bpm` field on the list would make the FILM
|
||
panel's grid hunt one request instead of N.
|
||
NEXT: nothing required. Backlog if a session 9 lands, in order: (1) "nudge to
|
||
grid" for EXISTING keys/cuts (today only new drags and auto-cut are musical);
|
||
(2) box-select still only grabs keys, not clip/audio blocks or cuts; (3)
|
||
per-drag undo coalesce (a mousemove drag still pushes one entry per frame);
|
||
(4) film-level `audio[]` has no UI — the obvious partner to ♪ align.
|
||
|
||
## 2026-07-26 session 8b (SYNC13 follow-up: drop no-op kept cuts)
|
||
DONE: orchestrator's M10-B acceptance note actioned — a kept cut that selects the
|
||
camera already live is now DROPPED, not preserved. `autoCutPlan` (`timeline.js:668-690`)
|
||
no longer filters `kept` positionally; it merges the surviving existing cuts with
|
||
the about-to-be-placed ones, sorts by t, walks the merged list tracking the
|
||
EFFECTIVE camera, and splits in-range user cuts into `kept` (changes the angle)
|
||
vs `dropped` (`camera === eff` — cuts cam2→cam2, i.e. nothing). `autoCut`
|
||
(`:704`) removes `dropped` alongside `replaced`, inside the same `group()`, so it
|
||
is still ONE undo. Popover line now reads "… · 2 of your cuts kept · 1 dropped
|
||
(no camera change)"; the apply toast says which and why.
|
||
Walking the MERGED list (not comparing to the previous array element) is the
|
||
point: the duplicate only exists once the auto cuts are in — 4.5:cam2 is a
|
||
perfectly good cut in the file as saved, and only becomes a no-op because
|
||
auto-cut puts cam2 at 1.4161.
|
||
TEST: +3 cases (`timeline_test.mjs:566-598`): a cut that changes the angle is kept
|
||
(unchanged behaviour, still asserted), a cut duplicating the incoming auto camera
|
||
is dropped and ONE undo restores it with everything else, and a mixed pair
|
||
(2.0:camA redundant after the 1.4161 auto camA / 6.0:camA real after the 5.1654
|
||
auto camB) drops exactly one and keeps exactly one. Also asserts the final list
|
||
has no consecutive camera repeat AT ALL, which is the property that was missing.
|
||
Replayed the orchestrator's own case on `scenes/music-video.json`: file cuts
|
||
`0:cam1, 4.5:cam2` → auto-cut cadence 2 → `0:cam1 1.42:cam2 5.17:cam3 8.91:cam4
|
||
12.66:cam1`, the 4.5 no-op gone, no repeats, one undo byte-identical to the file.
|
||
`node scenegod/web/timeline_test.mjs` green; `node --check` clean on all four.
|
||
DECISIONS:
|
||
- The drop is limited to cuts INSIDE the placed range and only when the camera is
|
||
genuinely unchanged. A user cut before the first or after the last placed cut is
|
||
still never touched — auto-cut has no business outside its own range.
|
||
- Noted for whoever picks up `♪ align`: the "name the grid source in the status
|
||
line" pattern is endorsed by the orchestrator; shot lists can in principle span
|
||
scenes with different grids, and saying which one was used is the honest answer
|
||
rather than silently picking. Keep that pattern for anything similar.
|
||
BLOCKED/REQUESTS: unchanged from session 8.
|
||
NEXT: unchanged from session 8 (nudge-to-grid, box-select beyond keys, per-drag undo
|
||
coalesce, film-level audio[] UI).
|