[laneB] log: M4-B session 4 (audio + visemes)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
df4e06f047
commit
252a6f666c
@ -134,3 +134,48 @@ BLOCKED/REQUESTS: none. Verified with a throwaway harness (`_check.html`,
|
||||
NEXT: M2 polish complete. Undo already wired (Ctrl+Z). Open for M4 timeline
|
||||
work when unlocked (audio tracks on the timeline UI + Web Audio sync, per
|
||||
PLAN §5 M4) or any SYNC 3 fixes. Holding until orchestrator flips M4.
|
||||
|
||||
## 2026-07-18 session 4 (M4-B: audio + visemes)
|
||||
DONE: M4-B all three items.
|
||||
1. Audio rows in tlui from scene.audio[]: one lane per clip below Cameras,
|
||||
green block at the DECODED buffer width (fallback width until decoded),
|
||||
gain <input> in the name column, drag = offset (moveAudio), right-click =
|
||||
delete. 🎵 button reads /assets/tree audio category into a small floating
|
||||
menu → addAudio at the playhead.
|
||||
2. Web Audio draft playback synced to the master clock: lazy AudioContext,
|
||||
decode+cache buffers (/assets/file), _audioStart schedules
|
||||
AudioBufferSourceNodes at (start - time) with per-clip gain, _audioStop on
|
||||
pause/natural-end, user seeks reschedule (_seek helper). step()/render
|
||||
never touch audio — server muxes from scene.audio.
|
||||
3. morphs track kind: _evalMorphs lerps per viseme name → stage.setMorph;
|
||||
importRhubarb(id, json, t0) maps mouthCues → morph keys; tlui shows a
|
||||
"↳ visemes" lane guarded by stage.visemeTargets(id).length > 0.
|
||||
Model (timeline.js, stagestub.js, timeline_test.mjs) committed ba29d0c;
|
||||
tests cover morph lerp, rhubarb keys, audio mutators+undo. All green.
|
||||
VERIFIED (real server :8020, real assets): 🎵 picker listed + added
|
||||
audio/test/tone.wav, buffer decoded (8s), green block drawn, Web Audio
|
||||
scheduled/rescheduled/stopped (ctx running), viseme lane rendered
|
||||
importRhubarb keys, setMorph fired while scrubbing. Contract match confirmed:
|
||||
real stage.js ships setMorph(id,name,weight) + visemeTargets(id) (returns
|
||||
Object.keys(visemes)) — my `.length>0` guard works against both.
|
||||
DECISIONS:
|
||||
- Audio duration is a client draw concern (decode+cache in tlui), NOT stored
|
||||
in the {path,start,gain} schema — server already knows durations at mux.
|
||||
- importRhubarb = simple per-cue ramp (shape 1 at cue start → 0 at end);
|
||||
hold/crossfade tuning deferred (ponytail marker in timeline.js) until it
|
||||
looks wrong on a real blendshape character.
|
||||
- Audio blocks can extend past scene.duration (tone.wav is 8s in a 6s scene)
|
||||
— left as-is; the ruler just clips visually. Trim is a later nicety.
|
||||
BLOCKED/REQUESTS:
|
||||
- **Ownership note for orchestrator**: my tlui.js M4-B changes (+146 lines)
|
||||
were committed inside Lane A's `df4e06f [laneA] M4-A ...`, not a [laneB]
|
||||
commit — Lane A's commit swept my then-uncommitted tlui.js working changes
|
||||
along with their dock.js/stage.js. Code is intact + verified; only the
|
||||
attribution is off. Heads-up that our git hygiene let a cross-lane file
|
||||
ride in on another lane's commit; a `git add <own files>` (not `-A`) avoids
|
||||
it. Nothing for me to fix.
|
||||
NEXT: M4-B complete. Open items if a session 5 lands: audio block trim/clip to
|
||||
duration, viseme import UI button (currently importRhubarb is API-only; Lane
|
||||
C has a /rhubarb endpoint — could wire a "lip-sync from audio" button that
|
||||
POSTs the clip and feeds the result to importRhubarb). Holding for SYNC/next
|
||||
orchestrator update.
|
||||
|
||||
Loading…
Reference in New Issue
Block a user