SCENEGOD/logs/laneA.md

# Lane A log — append-only, newest at bottom

Format per session:

    ## <date> session N
    DONE: …
    DECISIONS: …
    BLOCKED/REQUESTS: …
    NEXT: …

(no sessions yet)

## 2026-07-18 session 1
DONE: M1 shipped and verified end-to-end in a browser.
- `web/room3d.js` — ported the rigroom retarget as a pure ES module (no DOM/globals).
  Exports parseAny/canon/boneMap/captureRest/bakeRetarget/disposeRoot/stats + `_selftest`.
  bakeRetarget is now `(clip, srcRoot, srcRest, tgtRoot, tgtRest, opts?)` — target is a
  parameter instead of the closed-over `char`. Algorithm kept byte-for-byte (30fps world-delta
  bake, hips-height ratio, quat sign continuity, dup-hierarchy dedupe). Added `.bvh` to parseAny.
- `web/stage.js` — `class Stage` per PLAN §4.2. IBL (RoomEnvironment+PMREM), grid+circle floor,
  OrbitControls director cam, TransformControls gizmo (W/E/R), raycast select, shadow-casting.
  addEntity handles character/prop/backdrop(plane|corner|dome)/camera/light; wrapper-Group holds
  all transforms, loaded root normalized+grounded underneath. prepareClip caches parse-per-path and
  bake-per-(entity,path,index). captureState/applyState round-trip entities.
- `web/dock.js` — asset dock (Characters/Props/Backdrops/Animations tabs from `/assets/tree`,
  offline→file-drop) + inspector (transform/params editors, ⏺key dispatches `scenegod:capturekey`,
  delete). Local drop: image→backdrop, mesh→character, clip-onto-selected-character→bake+play.
- `web/index.html` + `web/style.css` — CSS-grid layout (dock/view/inspector/timeline), rigroom
  importmap (three 0.160.0), `?selftest=1` badge. Timeline loaded via guarded dynamic import so the
  page never breaks when Lane B's file is absent.

VERIFICATION (browser, in-app):
- `?selftest=1`: PASS — two 3-bone skeletons, different rests+namespaces, baked 1s clip,
  world-delta match maxErr=6.4e-12, srcMoved=0.224rad (guard against a silently-frozen source).
- Real end-to-end (Female_mixamo.fbx char + "Running.fbx" clip + a jpg backdrop, served from a
  throwaway scratchpad harness): char loaded (65 bones, 18.6k tris, normalized), gizmo-moved,
  clip retargeted (66 tracks) and animating live. Screenshot: `logs/shots/m1-lady-running.png`.
- Page boots with zero console errors with BOTH timeline.js and the server absent.

DECISIONS:
- selftest uses underscore namespaces (`mixamorig1_`/`mixamorig4_`), not colon — a colon in a bone
  name breaks three's PropertyBinding so the source clip wouldn't bind (silent pass). canon() still
  collapses both to the same key, which is the thing under test.
- Added `Stage.prepareClipUpload(id, buf, name, idx)` (bake from in-memory bytes) so the offline
  file-drop path can retarget without a server. `Stage.setMixerAuto(false)` lets Lane B take over
  mixer stepping at SYNC 1 (default true so dropped clips preview standalone now).
- backdrop param edits (mode/image/width) don't rebuild the mesh yet — marked M2. fov/light
  color/intensity are live.

BLOCKED/REQUESTS:
- @Lane C: please confirm the item shape inside `/assets/tree` arrays. dock.js tolerates both a
  bare path string and `{path,name,files:[{path,ext}]}` (rigroom-style) for now — tell me the real
  one and I'll drop the fallback. Also confirm the page is served such that absolute `/web/*.js`
  imports resolve (I used `/web/...` to be URL-independent).

NEXT (M2, do not start until orchestrator flips the milestone line):
- multiple cameras + PiP overlay via renderActiveCamera, light entity param plumbing polish,
  backdrop rebuild-on-param-change, drag-from-dock onto viewport (currently click-to-add).

## 2026-07-18 session 2 (M2)
DONE: M2 Lane-A scope shipped and verified in the *integrated* app (Lane C server :8020 +
Lane B timeline/tlui mounted). Worked the orchestrator's priority order:
- **P0 /assets/tree shape** — dropped the bare-string fallback; dock.js now reads the confirmed
  `{name, path, formats, thumb}` and renders `thumb` (→ `/assets/file?path=`) when present.
- **P1 eventize clip drops** — `dock._applyClip(id,path,idx)`: when `window.timeline` exists it
  dispatches `scenegod:clipdrop {detail:{id,path,clipIndex}}` (Lane B places the block); only
  bakes+previews standalone when no timeline. Uploaded/local clip files can't hand off a server
  path, so with a timeline up they toast a "use the dock" note instead of a dead preview.
- **P2 onChange on every entity** — `Stage.addEntity`/`removeEntity` now fire `_fireChange(en)`,
  so tlui's `onChange→_syncRows` builds a row per entity, incl. programmatic `applyState` loads.
  Fixed the empty-names bug on both add and remove. Also fixed a latent bug: the gizmo `mouseUp`
  fired `_fireChange` with the *id* not the entity, so the inspector never refreshed on drag.
- **P4 cameras/lights/PiP/backdrop/drag**:
  - `Stage` PiP: overlay `<canvas class="pip">` in `#view`, rendered each frame from the active
    cam (render active→main canvas, blit to pip, then render director last so the main view is
    correct). `setActiveCamera` shows/hides it. Verified 16,495 lit pixels (real content).
  - `renderActiveCamera(canvas|null)` hardened as the M3 final-render target; added
    `pause()/resume()` so Lane C can freeze the director loop and drive frames deterministically.
  - dock "add rig" toolbar: **+ camera / + key light / + ambient** (these have no asset file);
    inspector gains **◉ set active camera** for cameras; light `castShadow` param plumbed.
  - `setParam` rebuilds the backdrop mesh on mode/image/width change (`_rebuildBackdrop`).
  - drag-from-dock onto the viewport: cards are `draggable`; `#view` drop places chars/props at
    the cursor (`screenToFloor`) and bakes clips onto the character under the cursor (`entityAt`).

VERIFICATION (in-app, :8020, zero console errors throughout):
- Added man + key light + ambient + camera via toolbar → all 4 appear as tlui rows immediately
  (P2 proven). Man lit by the scene lights. Screenshot: `logs/shots/m2-cameras-lights-dome.png`.
- Clicked ANIMATIONS→hiphop on the selected man → `scenegod:clipdrop` fired with
  `{id:'e1', path:'animations/test/hiphop.fbx', clipIndex:0}` (P1 proven). Lane B DOES consume it
  (timeline.js:98) but the block doesn't land — see the integration gap below.
- capturekey button → `scenegod:capturekey {id:'e1'}`. backdrop plane→dome rebuild swapped the
  root mesh with no error.

DECISIONS:
- PiP renders the active cam into the *main* WebGL canvas then blits to the 2D pip canvas (one
  shared context); director view is rendered last each frame so it always wins the main canvas.
  Same primitive is the M3 render path: `pause()` the loop, `renderActiveCamera(null)`, read pixels.
- Local/upload clip files are a no-op-with-note when a timeline is mounted (no server path to place
  a block). ponytail: revisit only if John wants drag-in clip files to become timeline blocks.

BLOCKED/REQUESTS — INTEGRATION GAP (orchestrator, please route):
- Live dock/stage-added entities never reach the timeline's model. `Timeline.scene.entities`
  starts `[]` (timeline.js:80) and is only filled by `timeline.load(sceneJson)`. Both
  `scenegod:clipdrop`→`addClipDrop` and `scenegod:capturekey`→`addKey` call `_tracks(id)`, which
  throws `no entity <id>` for anything added through the dock. Verified live at :8020:
  `manOnStage=true, inTimelineModel=false, addClipDropErr="no entity e1"`. So on today's build,
  keyframing/clip-dropping a dock-added entity is dead — it only works after a scene `load()`.
- This is a cross-lane seam, not a Lane-A-file fix (timeline.js is Lane B's). The natural hook is
  already in place on my side: **`stage.onChange(en)` now fires on every add AND remove**
  (my P2 fix). Suggested resolution (Lane B, ~4 lines): in the Timeline ctor,
  `stage.onChange(en => this._mirror(en))` where `_mirror` upserts/removes a `scene.entities`
  entry from `{id,kind,label,source,params, transform: stage.entityTransform(id), tracks:{}}`.
  I can instead dispatch a dedicated `scenegod:entityadd/entityremove` event from the dock if the
  orchestrator prefers an explicit contract over reusing onChange — say the word and I'll add it.

NEXT: once the entity-mirror seam is decided, the "lady + man + street + 2 cameras + sunset" SYNC-2
demo should play. Holding at the M2 boundary until the orchestrator flips the milestone line.

## 2026-07-18 session 3 (M3-polish)
DONE: both M3-polish items the orchestrator flagged, verified live at :8020.
- **PiP cap + swap.** Was fixed 256×144 and (with the CSS `aspect-ratio`/`%` combo) blew up to
  340×413 portrait on a `<canvas>` — CSS aspect-ratio is unreliable on replaced elements. Now sized
  in `_resize()`: `clamp(180, view.width*0.24, 340)` wide × 16:9, bottom-right. Click the inset to
  swap main↔active-cam (`_pipSwap`). Refactored `renderActiveCamera`→ private `_render(cam,canvas)`
  that restores the cam's aspect after a pip blit, so swapping never distorts either camera
  (verified: dirCam.aspect stays 1.79 while it's shown in the inset). `renderActiveCamera(canvas)`
  is now a one-line wrapper over `_render(this._activeCam(), canvas)` — M3 contract unchanged;
  `pause()/resume()` untouched. Verified 180×101 @1.78 ratio, swap flips the views cleanly.
- **lady.glb flat-white diagnosed + fixed.** It's the ASSET: a Reallusion/Character-Creator export
  with `extensionsRequired:["KHR_materials_pbrSpecularGlossiness"]` — diffuse lives in that
  extension, which three r160 dropped, so all 26 materials fell back to white MeshStandard. (rigroom
  would show the same — no port regression.) Since SCENEGOD must eat any dropped character, added a
  minimal `SpecGlossPlugin` registered on GLTFLoader in room3d.js: spec-gloss → MeshStandard
  (diffuse→map/color, roughness≈1−glossiness, metalness 0). Only touches materials carrying the
  extension; metallic-rough GLBs and FBX untouched. Verified: lady loads with withMap=26/26 and
  renders fully textured (skin/clothes/face). Shot: `logs/shots/m3-pip-swap.png`.

DECISIONS:
- Runtime spec-gloss shim over an offline gltf-transform convert: keeps CC characters "just work"
  in the drop-on-stage flow and in the render pipeline, no per-asset conversion step. Approximation
  (ignores the specular map) is fine for machinima lighting; upgrade path = full spec→metal convert
  if a hero asset needs it.
- PiP size lives in JS, not CSS — `<canvas>` + `aspect-ratio` is the trap that produced the giant
  inset. CSS keeps only position/border.

BLOCKED/REQUESTS: none.

NEXT: hold at M3-polish boundary. Available for SYNC-3 findings (finalRender drives
`pause`/`resume`/`renderActiveCamera` — kept stable per the hard contract).

## 2026-07-18 session 4 (SYNC4 fix + M4-A visemes)
DONE:
- **applyState/source clean fix.** Absorbed the orchestrator's SYNC4 hotfix and did the schema-correct
  version: `addEntity` defaults camera/light `source` to `{type:'none'}` (PLAN §4.1); `captureState`
  emits it; `applyState` skips only entities that need bytes it lacks (`type:'upload'` AND
  kind∈{character,prop,backdrop}), so `'none'` always rebuilds and legacy `'upload'` cams still
  rebuild. Verified live: save→load of {character, camera, light} → srcTypes
  `[character:assets, camera:none, light:none]`, all 3 come back (rig no longer lost).
- **M4-A viseme lane** (my deliverable = detection + setMorph hook + badge; Rhubarb→keys is Lane B):
  - `detectVisemes(root)` (stage.js): scans every skinned mesh's `morphTargetDictionary`, loosely
    maps names → canonical visemes A/E/I/O/U/MBP/FV/L (covers CC `V_Open/V_Explosive/V_Dental_Lip/
    V_Tongue_*`, ARKit `jawOpen/mouthPucker/mouthFunnel/mouthClose`, plain vowels). Built once at
    character load; stored on the entity as `{morphs, visemes}` (head/teeth/tongue meshes sharing a
    morph name are grouped so one call drives them all).
  - `stage.visemeTargets(id)` → canonical names present; `stage.setMorph(id, name, weight)` drives a
    canonical viseme OR a raw morph name across all mapped meshes.
  - inspector 🗣 badge (dock.js): lists detected visemes, greyed `🗣 no visemes` when the rig has none.
  - `visemeSelfCheck()` exported: synthetic dict of CC+ARKit+blink names → asserts A/U/MBP/FV/L
    detected, E/I/O NOT (no blink/brow false positives), and setMorph('A') drives the mapped indices
    only. Verified in-browser: `ok=true, found=[A,FV,L,MBP,U]`.

VERIFICATION (:8020, zero console errors): viseme self-check ok; man/lady show greyed 🗣 badge
(neither rig carries mouth morphs — lady.glb is a CC body export with 0 morph targets, confirmed by
reading the GLB); save/load round-trip keeps cameras+lights.

DECISIONS:
- Viseme matcher is intentionally loose (orchestrator asked "match loosely") and detection-only; the
  authoritative Rhubarb-letter→canonical mapping + keyframing is Lane B's. `setMorph` also accepts
  raw morph names so B can drive anything the matcher missed.

BLOCKED/REQUESTS:
- @orchestrator: no viseme-bearing test character in `assets/` (man.fbx and lady.glb both have zero
  mouth morphs), so the LIT badge + a visible mouth move are proven only by the synthetic self-check,
  not on a real asset. If you can drop a CC/ARKit head with visemes into `assets/characters/test/`,
  I'll confirm live and Lane B can validate Rhubarb end-to-end.

NEXT: hold at M4-A boundary. Available for a live viseme asset check and any SYNC findings.

## 2026-07-20 session 5 (M5 DIRECTOR MODE + M6 FLOW VIDEO PLATES)
DONE — both milestones, headless-verified (no browser/servers touched per orchestrator order):

M5:
- `web/grammar.js` (NEW, pure data, zero imports): SHOTS (exact spans/centers from the lane
  spec), ANGLES (eye/low −18°/high +22°/dutch 8° roll/ots_L/ots_R), FOCALS [18,24,35,50,85],
  LIGHTS (day/golden_hour/night/noir/horror-underlight/neon/overcast — {sun{color,intensity,
  elev,azim}, ambient{color,intensity}, bg}), MARKS (center/two_shot_L/R/face_camera/face_other).
- `web/presets.js` (NEW, NO three.js import — node-runnable): fovForFocal (presets.js:11),
  lookAtEuler w/ dutch roll + three-matching XYZ euler extraction (:14), solveShot bbox→
  pos/rot/fov/dist deterministic solver incl. OTS over-the-shoulder placement (:37),
  frameSubject (:62, bbox via new stage.entityBBox), applyLight (:75 — ensures/reuses one
  sun+ambient pair, writes grammar values verbatim, aims a 20m sun boom via lookAtEuler,
  background rides the AMBIENT entity as param `bg` so it's keyable AND survives save/load),
  applyMark (:99, pos marks keep y; face marks yaw toward active cam / nearest character),
  applyOps (:139) — ONE op runner shared by panel buttons and /director NL, fires
  `scenegod:direct {op, ...values, entityIds}` per action (B keys them; I never touch the
  timeline), mountDirectPanel (:174) — shot/angle/lens segments + 🎥 frame, light buttons,
  mark buttons, NL box.
- NL box: POST /director {text, selected, entities:[{id,kind,label}]} → expects {ops:[...]},
  applied via applyOps. Hidden by default; probe `GET /director` shows it on 405/200 (FastAPI
  405 = POST route exists); hides itself again when a POST returns 503.
- index.html:14 `#direct` bar + guarded dynamic import (:49) — page survives presets.js absent.
  style.css:85 DIRECT styles (before the LANE B marker).
- stage.js additions for M5: entityBBox (stage.js:490, plain arrays — keeps presets three-free),
  cameraWorldPos (:497), light param `bg` → scene.background live (:393) + restored at
  _buildLight (:335) so a loaded scene gets its sky back.

M6:
- `_buildBackdrop` rework (stage.js:225): params.mode gains "video" (renders as plane), and
  plane/corner/dome accept .mp4/.webm/.mov sources; THREE.VideoTexture muted+loop+playsInline,
  SRGB; poster jpg (server /assets/thumb same-stem sidecar) shown until first play/seek then
  swapped in via _swapOnPlay (:309). Standalone page (no timeline → _mixerAuto) auto-plays as
  ambient preview; with a timeline mounted videos stay paused at poster for B's clock.
- NEW kind `screen` (addEntity :165, _buildScreen :272): video plane, params {video,width,
  emissive}, MeshStandard + emissiveMap boost (default 0.6) so club screens read "on" under
  noir/night. ponytail: CRT curve skipped until a real TV asset asks.
- `stage.syncVideos(t)` (:506): seeks every video to ((t−videoStart) mod duration), pauses,
  awaits 'seeked' per video (1.5s timeout guard so a stuck decoder can't hang a render), skips
  sub-4ms deltas. render.js already awaits it per frame (Lane C landed the hook). Plus
  videos() (:501) + setVideoPlaying(on) (:503) for Lane B's clock-follow, and params.videoStart
  plumbed (inspector fields, dock.js:214/220).
- dock.js: _videoPath (:86, derives <stem>.<videoext> from formats), 🎞 badge + 📺
  add-as-screen button on video backdrop cards (:106), click/drag adds mode:'video' plates
  (:132 and _viewDrop), local .mp4 drop → upload video plate, screen/backdrop inspector fields
  (width/emissive/vid start; mode select gains 'video').
- Lifecycle: removeEntity/_rebuild release the <video> (_dropVideo :316); applyState treats
  upload screens as needs-bytes (skip+warn) like backdrops.

VERIFICATION (headless):
- `web/presets_test.mjs` (NEW): fov-from-focal exact for 18/35/85mm (+35mm≈37.849°); CU dist <
  MS dist on a 1.7m subject + exact span-formula match + CU aims at .87H; dutch sets rot z =
  8° with zero stray pitch/yaw; low angle puts camera below aim pitched up;
  applyLight('golden_hour') returns grammar values verbatim (deepEqual) + applies
  intensity/bg to a stub stage + reuses the sun/ambient pair on a second preset.
  → `node scenegod/web/presets_test.mjs` = ALL PASS.
- Scratchpad ops smoke (not committed): applyOps shot-by-LABEL auto-creates 'shotcam', sets
  85mm fov 16.07°, activates cam, low-angle below aim; ots_R auto-resolves the other
  character; two_shot_L pos; face_camera/face_other yaw; unresolvable subject reports error
  without throwing. ALL PASS.
- ESM syntax-check (node --check via scratchpad .mjs copies): stage/dock/presets/grammar/room3d
  all clean. Retarget code and every existing Stage API untouched.

DECISIONS:
- presets.js takes bboxes as plain arrays (new stage.entityBBox) instead of importing three —
  the whole solver runs under node, same trick as Lane B's timeline.
- ONE op path: panel buttons call applyOps exactly like /director responses do, so button and
  LLM behavior can never drift; every applied op fires one scenegod:direct event.
- Background color is a param (`bg`) on the ambient light entity rather than new scene-level
  schema — keyable through B's existing params track, persists via captureState, zero schema
  change. Orchestrator: if you'd rather have scene-level bg in PLAN §4.1, say so and I'll move it.
- Poster-until-play = swap material maps on first 'playing' OR 'seeked' — so a final render
  that never plays (only seeks) still gets real video frames, and an idle stage shows the jpg.
- OTS shoulder offset is a fixed 0.35m slide + 0.1m rise along the partner line — deterministic,
  no LLM; eyeball-tune at SYNC if the shoulder isn't reading.

BLOCKED/REQUESTS:
- **@Lane C (blocking M6 dock visibility): mp4 is invisible to the server.** server.py
  ASSET_EXTS/CATEGORIES: backdrops primary = IMG_EXTS only and mp4 ∉ ASSET_EXTS, so
  assets/backdrops/video/scenegod_plates/suburb_strip_goldenhour.mp4 never appears in
  /assets/tree (its poster jpg shows up as a plain image backdrop instead). Please add
  VIDEO_EXTS = {"mp4","webm","mov"} to ASSET_EXTS and to the backdrops primary set. My dock
  already handles the resulting shape (finds the video ext in `formats`, derives
  <stem>.<ext>). Also add MEDIA_TYPES entries ("mp4":"video/mp4","webm":"video/webm",
  "mov":"video/quicktime") — /assets/file currently serves octet-stream and <video> can be
  picky about that (and note Range support matters for long plates).
  Until then, video plates ARE testable via console: `stage.addEntity({kind:'backdrop',
  label:'plate', source:{type:'assets', path:'backdrops/video/scenegod_plates/
  suburb_strip_goldenhour.mp4'}, params:{mode:'video', image:'backdrops/video/
  scenegod_plates/suburb_strip_goldenhour.mp4', width:12}})` (file streams fine, just untyped).
- @Lane C: /director endpoint per PLAN M5 — my probe expects GET→405 (or 200) when it exists,
  and I hide the box on POST 503. Request contract above; response {ops:[{op:'shot'|'light'|
  'mark', ...}]} — shapes documented at presets.js:136.
- @Lane B: your M5/M6 hooks are ready on my side — `scenegod:direct` events carry full values
  for keying; for video clock-follow use stage.videos() [{id,video,start}] +
  stage.setVideoPlaying(on) + drift-check video.currentTime yourself per PLAN (>50ms), or just
  await stage.syncVideos(t) on hard seeks.

NEXT / orchestrator SYNC checklist (browser):
1. Select man → CU low 35mm → 🎥 frame: camera frames his face from below in one click; PiP
   shows it (frame auto-activates the shot cam).
2. golden_hour button: sun+ambient recolor + orange sky in one click; scrub proves B keyed it
   (needs B's scenegod:direct consumer — not landed yet as of this session).
3. Suburb plate: console addEntity above → poster shows, then video motion (auto-play only
   when timeline absent; with timeline mounted it waits for B's clock). 📺 dock button appears
   once Lane C ships the tree change.
4. finalRender a 2s scene with the plate — frames must show advancing video (syncVideos is
   awaited by render.js already).
5. NL box stays hidden until /director exists (probe is GET → expect 405 to reveal).