[laneB] log: M4-B session 4 (audio + visemes)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
df4e06f047
commit
252a6f666c
@ -134,3 +134,48 @@ BLOCKED/REQUESTS: none. Verified with a throwaway harness (`_check.html`,
|
|||||||
NEXT: M2 polish complete. Undo already wired (Ctrl+Z). Open for M4 timeline
|
NEXT: M2 polish complete. Undo already wired (Ctrl+Z). Open for M4 timeline
|
||||||
work when unlocked (audio tracks on the timeline UI + Web Audio sync, per
|
work when unlocked (audio tracks on the timeline UI + Web Audio sync, per
|
||||||
PLAN §5 M4) or any SYNC 3 fixes. Holding until orchestrator flips M4.
|
PLAN §5 M4) or any SYNC 3 fixes. Holding until orchestrator flips M4.
|
||||||
|
|
||||||
|
## 2026-07-18 session 4 (M4-B: audio + visemes)
|
||||||
|
DONE: M4-B all three items.
|
||||||
|
1. Audio rows in tlui from scene.audio[]: one lane per clip below Cameras,
|
||||||
|
green block at the DECODED buffer width (fallback width until decoded),
|
||||||
|
gain <input> in the name column, drag = offset (moveAudio), right-click =
|
||||||
|
delete. 🎵 button reads /assets/tree audio category into a small floating
|
||||||
|
menu → addAudio at the playhead.
|
||||||
|
2. Web Audio draft playback synced to the master clock: lazy AudioContext,
|
||||||
|
decode+cache buffers (/assets/file), _audioStart schedules
|
||||||
|
AudioBufferSourceNodes at (start - time) with per-clip gain, _audioStop on
|
||||||
|
pause/natural-end, user seeks reschedule (_seek helper). step()/render
|
||||||
|
never touch audio — server muxes from scene.audio.
|
||||||
|
3. morphs track kind: _evalMorphs lerps per viseme name → stage.setMorph;
|
||||||
|
importRhubarb(id, json, t0) maps mouthCues → morph keys; tlui shows a
|
||||||
|
"↳ visemes" lane guarded by stage.visemeTargets(id).length > 0.
|
||||||
|
Model (timeline.js, stagestub.js, timeline_test.mjs) committed ba29d0c;
|
||||||
|
tests cover morph lerp, rhubarb keys, audio mutators+undo. All green.
|
||||||
|
VERIFIED (real server :8020, real assets): 🎵 picker listed + added
|
||||||
|
audio/test/tone.wav, buffer decoded (8s), green block drawn, Web Audio
|
||||||
|
scheduled/rescheduled/stopped (ctx running), viseme lane rendered
|
||||||
|
importRhubarb keys, setMorph fired while scrubbing. Contract match confirmed:
|
||||||
|
real stage.js ships setMorph(id,name,weight) + visemeTargets(id) (returns
|
||||||
|
Object.keys(visemes)) — my `.length>0` guard works against both.
|
||||||
|
DECISIONS:
|
||||||
|
- Audio duration is a client draw concern (decode+cache in tlui), NOT stored
|
||||||
|
in the {path,start,gain} schema — server already knows durations at mux.
|
||||||
|
- importRhubarb = simple per-cue ramp (shape 1 at cue start → 0 at end);
|
||||||
|
hold/crossfade tuning deferred (ponytail marker in timeline.js) until it
|
||||||
|
looks wrong on a real blendshape character.
|
||||||
|
- Audio blocks can extend past scene.duration (tone.wav is 8s in a 6s scene)
|
||||||
|
— left as-is; the ruler just clips visually. Trim is a later nicety.
|
||||||
|
BLOCKED/REQUESTS:
|
||||||
|
- **Ownership note for orchestrator**: my tlui.js M4-B changes (+146 lines)
|
||||||
|
were committed inside Lane A's `df4e06f [laneA] M4-A ...`, not a [laneB]
|
||||||
|
commit — Lane A's commit swept my then-uncommitted tlui.js working changes
|
||||||
|
along with their dock.js/stage.js. Code is intact + verified; only the
|
||||||
|
attribution is off. Heads-up that our git hygiene let a cross-lane file
|
||||||
|
ride in on another lane's commit; a `git add <own files>` (not `-A`) avoids
|
||||||
|
it. Nothing for me to fix.
|
||||||
|
NEXT: M4-B complete. Open items if a session 5 lands: audio block trim/clip to
|
||||||
|
duration, viseme import UI button (currently importRhubarb is API-only; Lane
|
||||||
|
C has a /rhubarb endpoint — could wire a "lip-sync from audio" button that
|
||||||
|
POSTs the clip and feeds the result to importRhubarb). Holding for SYNC/next
|
||||||
|
orchestrator update.
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user