f20a8d5d4e
MOVES grammar + solveMove(): push in/pull out (one rung along the shot ladder,
along the aim ray), truck L/R (lateral, parallel, no re-aim), crane up/down
(vertical then re-aim, floor-clamped), orbit L/R (constant distance, +/-25deg),
zoom in/out (focal only - pos and rot returned identical). Amounts are
frame-relative so a move reads the same on a CU as a WS. Emits a new 'move' op
on scenegod:direct with explicit from/to states, pre-clamped duration and ease;
the START is applied to the live camera immediately.
'+ angle' creates a NEW camera from the current shot settings instead of
reframing the active one - the friction hit while building a 4-camera scene by
hand. 🎥 frame now prefers the live camera so it adjusts what you just made
rather than silently editing camera 1. fitAnchor exposed in the backdrop
inspector (it was inert from the UI).
Verified live: push_in 1.975->1.588m fov fixed; zoom_in travel 0.000 with
pos/rot identical, fov 37.85->26.99; orbit_L constant distance; truck_R
parallel. 4 suites green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
544 lines
41 KiB
Markdown
544 lines
41 KiB
Markdown
# Lane A log — append-only, newest at bottom
|
||
|
||
Format per session:
|
||
|
||
## <date> session N
|
||
DONE: …
|
||
DECISIONS: …
|
||
BLOCKED/REQUESTS: …
|
||
NEXT: …
|
||
|
||
(no sessions yet)
|
||
|
||
## 2026-07-18 session 1
|
||
DONE: M1 shipped and verified end-to-end in a browser.
|
||
- `web/room3d.js` — ported the rigroom retarget as a pure ES module (no DOM/globals).
|
||
Exports parseAny/canon/boneMap/captureRest/bakeRetarget/disposeRoot/stats + `_selftest`.
|
||
bakeRetarget is now `(clip, srcRoot, srcRest, tgtRoot, tgtRest, opts?)` — target is a
|
||
parameter instead of the closed-over `char`. Algorithm kept byte-for-byte (30fps world-delta
|
||
bake, hips-height ratio, quat sign continuity, dup-hierarchy dedupe). Added `.bvh` to parseAny.
|
||
- `web/stage.js` — `class Stage` per PLAN §4.2. IBL (RoomEnvironment+PMREM), grid+circle floor,
|
||
OrbitControls director cam, TransformControls gizmo (W/E/R), raycast select, shadow-casting.
|
||
addEntity handles character/prop/backdrop(plane|corner|dome)/camera/light; wrapper-Group holds
|
||
all transforms, loaded root normalized+grounded underneath. prepareClip caches parse-per-path and
|
||
bake-per-(entity,path,index). captureState/applyState round-trip entities.
|
||
- `web/dock.js` — asset dock (Characters/Props/Backdrops/Animations tabs from `/assets/tree`,
|
||
offline→file-drop) + inspector (transform/params editors, ⏺key dispatches `scenegod:capturekey`,
|
||
delete). Local drop: image→backdrop, mesh→character, clip-onto-selected-character→bake+play.
|
||
- `web/index.html` + `web/style.css` — CSS-grid layout (dock/view/inspector/timeline), rigroom
|
||
importmap (three 0.160.0), `?selftest=1` badge. Timeline loaded via guarded dynamic import so the
|
||
page never breaks when Lane B's file is absent.
|
||
|
||
VERIFICATION (browser, in-app):
|
||
- `?selftest=1`: PASS — two 3-bone skeletons, different rests+namespaces, baked 1s clip,
|
||
world-delta match maxErr=6.4e-12, srcMoved=0.224rad (guard against a silently-frozen source).
|
||
- Real end-to-end (Female_mixamo.fbx char + "Running.fbx" clip + a jpg backdrop, served from a
|
||
throwaway scratchpad harness): char loaded (65 bones, 18.6k tris, normalized), gizmo-moved,
|
||
clip retargeted (66 tracks) and animating live. Screenshot: `logs/shots/m1-lady-running.png`.
|
||
- Page boots with zero console errors with BOTH timeline.js and the server absent.
|
||
|
||
DECISIONS:
|
||
- selftest uses underscore namespaces (`mixamorig1_`/`mixamorig4_`), not colon — a colon in a bone
|
||
name breaks three's PropertyBinding so the source clip wouldn't bind (silent pass). canon() still
|
||
collapses both to the same key, which is the thing under test.
|
||
- Added `Stage.prepareClipUpload(id, buf, name, idx)` (bake from in-memory bytes) so the offline
|
||
file-drop path can retarget without a server. `Stage.setMixerAuto(false)` lets Lane B take over
|
||
mixer stepping at SYNC 1 (default true so dropped clips preview standalone now).
|
||
- backdrop param edits (mode/image/width) don't rebuild the mesh yet — marked M2. fov/light
|
||
color/intensity are live.
|
||
|
||
BLOCKED/REQUESTS:
|
||
- @Lane C: please confirm the item shape inside `/assets/tree` arrays. dock.js tolerates both a
|
||
bare path string and `{path,name,files:[{path,ext}]}` (rigroom-style) for now — tell me the real
|
||
one and I'll drop the fallback. Also confirm the page is served such that absolute `/web/*.js`
|
||
imports resolve (I used `/web/...` to be URL-independent).
|
||
|
||
NEXT (M2, do not start until orchestrator flips the milestone line):
|
||
- multiple cameras + PiP overlay via renderActiveCamera, light entity param plumbing polish,
|
||
backdrop rebuild-on-param-change, drag-from-dock onto viewport (currently click-to-add).
|
||
|
||
## 2026-07-18 session 2 (M2)
|
||
DONE: M2 Lane-A scope shipped and verified in the *integrated* app (Lane C server :8020 +
|
||
Lane B timeline/tlui mounted). Worked the orchestrator's priority order:
|
||
- **P0 /assets/tree shape** — dropped the bare-string fallback; dock.js now reads the confirmed
|
||
`{name, path, formats, thumb}` and renders `thumb` (→ `/assets/file?path=`) when present.
|
||
- **P1 eventize clip drops** — `dock._applyClip(id,path,idx)`: when `window.timeline` exists it
|
||
dispatches `scenegod:clipdrop {detail:{id,path,clipIndex}}` (Lane B places the block); only
|
||
bakes+previews standalone when no timeline. Uploaded/local clip files can't hand off a server
|
||
path, so with a timeline up they toast a "use the dock" note instead of a dead preview.
|
||
- **P2 onChange on every entity** — `Stage.addEntity`/`removeEntity` now fire `_fireChange(en)`,
|
||
so tlui's `onChange→_syncRows` builds a row per entity, incl. programmatic `applyState` loads.
|
||
Fixed the empty-names bug on both add and remove. Also fixed a latent bug: the gizmo `mouseUp`
|
||
fired `_fireChange` with the *id* not the entity, so the inspector never refreshed on drag.
|
||
- **P4 cameras/lights/PiP/backdrop/drag**:
|
||
- `Stage` PiP: overlay `<canvas class="pip">` in `#view`, rendered each frame from the active
|
||
cam (render active→main canvas, blit to pip, then render director last so the main view is
|
||
correct). `setActiveCamera` shows/hides it. Verified 16,495 lit pixels (real content).
|
||
- `renderActiveCamera(canvas|null)` hardened as the M3 final-render target; added
|
||
`pause()/resume()` so Lane C can freeze the director loop and drive frames deterministically.
|
||
- dock "add rig" toolbar: **+ camera / + key light / + ambient** (these have no asset file);
|
||
inspector gains **◉ set active camera** for cameras; light `castShadow` param plumbed.
|
||
- `setParam` rebuilds the backdrop mesh on mode/image/width change (`_rebuildBackdrop`).
|
||
- drag-from-dock onto the viewport: cards are `draggable`; `#view` drop places chars/props at
|
||
the cursor (`screenToFloor`) and bakes clips onto the character under the cursor (`entityAt`).
|
||
|
||
VERIFICATION (in-app, :8020, zero console errors throughout):
|
||
- Added man + key light + ambient + camera via toolbar → all 4 appear as tlui rows immediately
|
||
(P2 proven). Man lit by the scene lights. Screenshot: `logs/shots/m2-cameras-lights-dome.png`.
|
||
- Clicked ANIMATIONS→hiphop on the selected man → `scenegod:clipdrop` fired with
|
||
`{id:'e1', path:'animations/test/hiphop.fbx', clipIndex:0}` (P1 proven). Lane B DOES consume it
|
||
(timeline.js:98) but the block doesn't land — see the integration gap below.
|
||
- capturekey button → `scenegod:capturekey {id:'e1'}`. backdrop plane→dome rebuild swapped the
|
||
root mesh with no error.
|
||
|
||
DECISIONS:
|
||
- PiP renders the active cam into the *main* WebGL canvas then blits to the 2D pip canvas (one
|
||
shared context); director view is rendered last each frame so it always wins the main canvas.
|
||
Same primitive is the M3 render path: `pause()` the loop, `renderActiveCamera(null)`, read pixels.
|
||
- Local/upload clip files are a no-op-with-note when a timeline is mounted (no server path to place
|
||
a block). ponytail: revisit only if John wants drag-in clip files to become timeline blocks.
|
||
|
||
BLOCKED/REQUESTS — INTEGRATION GAP (orchestrator, please route):
|
||
- Live dock/stage-added entities never reach the timeline's model. `Timeline.scene.entities`
|
||
starts `[]` (timeline.js:80) and is only filled by `timeline.load(sceneJson)`. Both
|
||
`scenegod:clipdrop`→`addClipDrop` and `scenegod:capturekey`→`addKey` call `_tracks(id)`, which
|
||
throws `no entity <id>` for anything added through the dock. Verified live at :8020:
|
||
`manOnStage=true, inTimelineModel=false, addClipDropErr="no entity e1"`. So on today's build,
|
||
keyframing/clip-dropping a dock-added entity is dead — it only works after a scene `load()`.
|
||
- This is a cross-lane seam, not a Lane-A-file fix (timeline.js is Lane B's). The natural hook is
|
||
already in place on my side: **`stage.onChange(en)` now fires on every add AND remove**
|
||
(my P2 fix). Suggested resolution (Lane B, ~4 lines): in the Timeline ctor,
|
||
`stage.onChange(en => this._mirror(en))` where `_mirror` upserts/removes a `scene.entities`
|
||
entry from `{id,kind,label,source,params, transform: stage.entityTransform(id), tracks:{}}`.
|
||
I can instead dispatch a dedicated `scenegod:entityadd/entityremove` event from the dock if the
|
||
orchestrator prefers an explicit contract over reusing onChange — say the word and I'll add it.
|
||
|
||
NEXT: once the entity-mirror seam is decided, the "lady + man + street + 2 cameras + sunset" SYNC-2
|
||
demo should play. Holding at the M2 boundary until the orchestrator flips the milestone line.
|
||
|
||
## 2026-07-18 session 3 (M3-polish)
|
||
DONE: both M3-polish items the orchestrator flagged, verified live at :8020.
|
||
- **PiP cap + swap.** Was fixed 256×144 and (with the CSS `aspect-ratio`/`%` combo) blew up to
|
||
340×413 portrait on a `<canvas>` — CSS aspect-ratio is unreliable on replaced elements. Now sized
|
||
in `_resize()`: `clamp(180, view.width*0.24, 340)` wide × 16:9, bottom-right. Click the inset to
|
||
swap main↔active-cam (`_pipSwap`). Refactored `renderActiveCamera`→ private `_render(cam,canvas)`
|
||
that restores the cam's aspect after a pip blit, so swapping never distorts either camera
|
||
(verified: dirCam.aspect stays 1.79 while it's shown in the inset). `renderActiveCamera(canvas)`
|
||
is now a one-line wrapper over `_render(this._activeCam(), canvas)` — M3 contract unchanged;
|
||
`pause()/resume()` untouched. Verified 180×101 @1.78 ratio, swap flips the views cleanly.
|
||
- **lady.glb flat-white diagnosed + fixed.** It's the ASSET: a Reallusion/Character-Creator export
|
||
with `extensionsRequired:["KHR_materials_pbrSpecularGlossiness"]` — diffuse lives in that
|
||
extension, which three r160 dropped, so all 26 materials fell back to white MeshStandard. (rigroom
|
||
would show the same — no port regression.) Since SCENEGOD must eat any dropped character, added a
|
||
minimal `SpecGlossPlugin` registered on GLTFLoader in room3d.js: spec-gloss → MeshStandard
|
||
(diffuse→map/color, roughness≈1−glossiness, metalness 0). Only touches materials carrying the
|
||
extension; metallic-rough GLBs and FBX untouched. Verified: lady loads with withMap=26/26 and
|
||
renders fully textured (skin/clothes/face). Shot: `logs/shots/m3-pip-swap.png`.
|
||
|
||
DECISIONS:
|
||
- Runtime spec-gloss shim over an offline gltf-transform convert: keeps CC characters "just work"
|
||
in the drop-on-stage flow and in the render pipeline, no per-asset conversion step. Approximation
|
||
(ignores the specular map) is fine for machinima lighting; upgrade path = full spec→metal convert
|
||
if a hero asset needs it.
|
||
- PiP size lives in JS, not CSS — `<canvas>` + `aspect-ratio` is the trap that produced the giant
|
||
inset. CSS keeps only position/border.
|
||
|
||
BLOCKED/REQUESTS: none.
|
||
|
||
NEXT: hold at M3-polish boundary. Available for SYNC-3 findings (finalRender drives
|
||
`pause`/`resume`/`renderActiveCamera` — kept stable per the hard contract).
|
||
|
||
## 2026-07-18 session 4 (SYNC4 fix + M4-A visemes)
|
||
DONE:
|
||
- **applyState/source clean fix.** Absorbed the orchestrator's SYNC4 hotfix and did the schema-correct
|
||
version: `addEntity` defaults camera/light `source` to `{type:'none'}` (PLAN §4.1); `captureState`
|
||
emits it; `applyState` skips only entities that need bytes it lacks (`type:'upload'` AND
|
||
kind∈{character,prop,backdrop}), so `'none'` always rebuilds and legacy `'upload'` cams still
|
||
rebuild. Verified live: save→load of {character, camera, light} → srcTypes
|
||
`[character:assets, camera:none, light:none]`, all 3 come back (rig no longer lost).
|
||
- **M4-A viseme lane** (my deliverable = detection + setMorph hook + badge; Rhubarb→keys is Lane B):
|
||
- `detectVisemes(root)` (stage.js): scans every skinned mesh's `morphTargetDictionary`, loosely
|
||
maps names → canonical visemes A/E/I/O/U/MBP/FV/L (covers CC `V_Open/V_Explosive/V_Dental_Lip/
|
||
V_Tongue_*`, ARKit `jawOpen/mouthPucker/mouthFunnel/mouthClose`, plain vowels). Built once at
|
||
character load; stored on the entity as `{morphs, visemes}` (head/teeth/tongue meshes sharing a
|
||
morph name are grouped so one call drives them all).
|
||
- `stage.visemeTargets(id)` → canonical names present; `stage.setMorph(id, name, weight)` drives a
|
||
canonical viseme OR a raw morph name across all mapped meshes.
|
||
- inspector 🗣 badge (dock.js): lists detected visemes, greyed `🗣 no visemes` when the rig has none.
|
||
- `visemeSelfCheck()` exported: synthetic dict of CC+ARKit+blink names → asserts A/U/MBP/FV/L
|
||
detected, E/I/O NOT (no blink/brow false positives), and setMorph('A') drives the mapped indices
|
||
only. Verified in-browser: `ok=true, found=[A,FV,L,MBP,U]`.
|
||
|
||
VERIFICATION (:8020, zero console errors): viseme self-check ok; man/lady show greyed 🗣 badge
|
||
(neither rig carries mouth morphs — lady.glb is a CC body export with 0 morph targets, confirmed by
|
||
reading the GLB); save/load round-trip keeps cameras+lights.
|
||
|
||
DECISIONS:
|
||
- Viseme matcher is intentionally loose (orchestrator asked "match loosely") and detection-only; the
|
||
authoritative Rhubarb-letter→canonical mapping + keyframing is Lane B's. `setMorph` also accepts
|
||
raw morph names so B can drive anything the matcher missed.
|
||
|
||
BLOCKED/REQUESTS:
|
||
- @orchestrator: no viseme-bearing test character in `assets/` (man.fbx and lady.glb both have zero
|
||
mouth morphs), so the LIT badge + a visible mouth move are proven only by the synthetic self-check,
|
||
not on a real asset. If you can drop a CC/ARKit head with visemes into `assets/characters/test/`,
|
||
I'll confirm live and Lane B can validate Rhubarb end-to-end.
|
||
|
||
NEXT: hold at M4-A boundary. Available for a live viseme asset check and any SYNC findings.
|
||
|
||
## 2026-07-20 session 5 (M5 DIRECTOR MODE + M6 FLOW VIDEO PLATES)
|
||
DONE — both milestones, headless-verified (no browser/servers touched per orchestrator order):
|
||
|
||
M5:
|
||
- `web/grammar.js` (NEW, pure data, zero imports): SHOTS (exact spans/centers from the lane
|
||
spec), ANGLES (eye/low −18°/high +22°/dutch 8° roll/ots_L/ots_R), FOCALS [18,24,35,50,85],
|
||
LIGHTS (day/golden_hour/night/noir/horror-underlight/neon/overcast — {sun{color,intensity,
|
||
elev,azim}, ambient{color,intensity}, bg}), MARKS (center/two_shot_L/R/face_camera/face_other).
|
||
- `web/presets.js` (NEW, NO three.js import — node-runnable): fovForFocal (presets.js:11),
|
||
lookAtEuler w/ dutch roll + three-matching XYZ euler extraction (:14), solveShot bbox→
|
||
pos/rot/fov/dist deterministic solver incl. OTS over-the-shoulder placement (:37),
|
||
frameSubject (:62, bbox via new stage.entityBBox), applyLight (:75 — ensures/reuses one
|
||
sun+ambient pair, writes grammar values verbatim, aims a 20m sun boom via lookAtEuler,
|
||
background rides the AMBIENT entity as param `bg` so it's keyable AND survives save/load),
|
||
applyMark (:99, pos marks keep y; face marks yaw toward active cam / nearest character),
|
||
applyOps (:139) — ONE op runner shared by panel buttons and /director NL, fires
|
||
`scenegod:direct {op, ...values, entityIds}` per action (B keys them; I never touch the
|
||
timeline), mountDirectPanel (:174) — shot/angle/lens segments + 🎥 frame, light buttons,
|
||
mark buttons, NL box.
|
||
- NL box: POST /director {text, selected, entities:[{id,kind,label}]} → expects {ops:[...]},
|
||
applied via applyOps. Hidden by default; probe `GET /director` shows it on 405/200 (FastAPI
|
||
405 = POST route exists); hides itself again when a POST returns 503.
|
||
- index.html:14 `#direct` bar + guarded dynamic import (:49) — page survives presets.js absent.
|
||
style.css:85 DIRECT styles (before the LANE B marker).
|
||
- stage.js additions for M5: entityBBox (stage.js:490, plain arrays — keeps presets three-free),
|
||
cameraWorldPos (:497), light param `bg` → scene.background live (:393) + restored at
|
||
_buildLight (:335) so a loaded scene gets its sky back.
|
||
|
||
M6:
|
||
- `_buildBackdrop` rework (stage.js:225): params.mode gains "video" (renders as plane), and
|
||
plane/corner/dome accept .mp4/.webm/.mov sources; THREE.VideoTexture muted+loop+playsInline,
|
||
SRGB; poster jpg (server /assets/thumb same-stem sidecar) shown until first play/seek then
|
||
swapped in via _swapOnPlay (:309). Standalone page (no timeline → _mixerAuto) auto-plays as
|
||
ambient preview; with a timeline mounted videos stay paused at poster for B's clock.
|
||
- NEW kind `screen` (addEntity :165, _buildScreen :272): video plane, params {video,width,
|
||
emissive}, MeshStandard + emissiveMap boost (default 0.6) so club screens read "on" under
|
||
noir/night. ponytail: CRT curve skipped until a real TV asset asks.
|
||
- `stage.syncVideos(t)` (:506): seeks every video to ((t−videoStart) mod duration), pauses,
|
||
awaits 'seeked' per video (1.5s timeout guard so a stuck decoder can't hang a render), skips
|
||
sub-4ms deltas. render.js already awaits it per frame (Lane C landed the hook). Plus
|
||
videos() (:501) + setVideoPlaying(on) (:503) for Lane B's clock-follow, and params.videoStart
|
||
plumbed (inspector fields, dock.js:214/220).
|
||
- dock.js: _videoPath (:86, derives <stem>.<videoext> from formats), 🎞 badge + 📺
|
||
add-as-screen button on video backdrop cards (:106), click/drag adds mode:'video' plates
|
||
(:132 and _viewDrop), local .mp4 drop → upload video plate, screen/backdrop inspector fields
|
||
(width/emissive/vid start; mode select gains 'video').
|
||
- Lifecycle: removeEntity/_rebuild release the <video> (_dropVideo :316); applyState treats
|
||
upload screens as needs-bytes (skip+warn) like backdrops.
|
||
|
||
VERIFICATION (headless):
|
||
- `web/presets_test.mjs` (NEW): fov-from-focal exact for 18/35/85mm (+35mm≈37.849°); CU dist <
|
||
MS dist on a 1.7m subject + exact span-formula match + CU aims at .87H; dutch sets rot z =
|
||
8° with zero stray pitch/yaw; low angle puts camera below aim pitched up;
|
||
applyLight('golden_hour') returns grammar values verbatim (deepEqual) + applies
|
||
intensity/bg to a stub stage + reuses the sun/ambient pair on a second preset.
|
||
→ `node scenegod/web/presets_test.mjs` = ALL PASS.
|
||
- Scratchpad ops smoke (not committed): applyOps shot-by-LABEL auto-creates 'shotcam', sets
|
||
85mm fov 16.07°, activates cam, low-angle below aim; ots_R auto-resolves the other
|
||
character; two_shot_L pos; face_camera/face_other yaw; unresolvable subject reports error
|
||
without throwing. ALL PASS.
|
||
- ESM syntax-check (node --check via scratchpad .mjs copies): stage/dock/presets/grammar/room3d
|
||
all clean. Retarget code and every existing Stage API untouched.
|
||
|
||
DECISIONS:
|
||
- presets.js takes bboxes as plain arrays (new stage.entityBBox) instead of importing three —
|
||
the whole solver runs under node, same trick as Lane B's timeline.
|
||
- ONE op path: panel buttons call applyOps exactly like /director responses do, so button and
|
||
LLM behavior can never drift; every applied op fires one scenegod:direct event.
|
||
- Background color is a param (`bg`) on the ambient light entity rather than new scene-level
|
||
schema — keyable through B's existing params track, persists via captureState, zero schema
|
||
change. Orchestrator: if you'd rather have scene-level bg in PLAN §4.1, say so and I'll move it.
|
||
- Poster-until-play = swap material maps on first 'playing' OR 'seeked' — so a final render
|
||
that never plays (only seeks) still gets real video frames, and an idle stage shows the jpg.
|
||
- OTS shoulder offset is a fixed 0.35m slide + 0.1m rise along the partner line — deterministic,
|
||
no LLM; eyeball-tune at SYNC if the shoulder isn't reading.
|
||
|
||
BLOCKED/REQUESTS:
|
||
- **@Lane C (blocking M6 dock visibility): mp4 is invisible to the server.** server.py
|
||
ASSET_EXTS/CATEGORIES: backdrops primary = IMG_EXTS only and mp4 ∉ ASSET_EXTS, so
|
||
assets/backdrops/video/scenegod_plates/suburb_strip_goldenhour.mp4 never appears in
|
||
/assets/tree (its poster jpg shows up as a plain image backdrop instead). Please add
|
||
VIDEO_EXTS = {"mp4","webm","mov"} to ASSET_EXTS and to the backdrops primary set. My dock
|
||
already handles the resulting shape (finds the video ext in `formats`, derives
|
||
<stem>.<ext>). Also add MEDIA_TYPES entries ("mp4":"video/mp4","webm":"video/webm",
|
||
"mov":"video/quicktime") — /assets/file currently serves octet-stream and <video> can be
|
||
picky about that (and note Range support matters for long plates).
|
||
Until then, video plates ARE testable via console: `stage.addEntity({kind:'backdrop',
|
||
label:'plate', source:{type:'assets', path:'backdrops/video/scenegod_plates/
|
||
suburb_strip_goldenhour.mp4'}, params:{mode:'video', image:'backdrops/video/
|
||
scenegod_plates/suburb_strip_goldenhour.mp4', width:12}})` (file streams fine, just untyped).
|
||
- @Lane C: /director endpoint per PLAN M5 — my probe expects GET→405 (or 200) when it exists,
|
||
and I hide the box on POST 503. Request contract above; response {ops:[{op:'shot'|'light'|
|
||
'mark', ...}]} — shapes documented at presets.js:136.
|
||
- @Lane B: your M5/M6 hooks are ready on my side — `scenegod:direct` events carry full values
|
||
for keying; for video clock-follow use stage.videos() [{id,video,start}] +
|
||
stage.setVideoPlaying(on) + drift-check video.currentTime yourself per PLAN (>50ms), or just
|
||
await stage.syncVideos(t) on hard seeks.
|
||
|
||
NEXT / orchestrator SYNC checklist (browser):
|
||
1. Select man → CU low 35mm → 🎥 frame: camera frames his face from below in one click; PiP
|
||
shows it (frame auto-activates the shot cam).
|
||
2. golden_hour button: sun+ambient recolor + orange sky in one click; scrub proves B keyed it
|
||
(needs B's scenegod:direct consumer — not landed yet as of this session).
|
||
3. Suburb plate: console addEntity above → poster shows, then video motion (auto-play only
|
||
when timeline absent; with timeline mounted it waits for B's clock). 📺 dock button appears
|
||
once Lane C ships the tree change.
|
||
4. finalRender a 2s scene with the plate — frames must show advancing video (syncVideos is
|
||
awaited by render.js already).
|
||
5. NL box stays hidden until /director exists (probe is GET → expect 405 to reveal).
|
||
|
||
## 2026-07-25 session 6 (M7-A1 non-mixamo rigs + M7-A2 world-integrated backdrops)
|
||
DONE — both M7 items, headless-verified (no browser, no servers touched, per orchestrator order):
|
||
|
||
**M7-A1 — bone aliasing.** `canon()` used to strip a mixamorig prefix and nothing else, so every
|
||
non-mixamo skeleton paired 0 bones and died at "missing hips" (lady.glb has never been animatable).
|
||
- `room3d.js:70` legacy strip is now `rawCanon`; `room3d.js:136` `canon = raw ∈ CANON ? raw : alias(n) || raw`.
|
||
The CANON short-circuit is what makes mixamo→mixamo **bit-identical** — a name that already canons
|
||
to a mixamo bone never touches the alias tables (test asserts all 65 man.fbx bones × 5 namespace
|
||
spellings against an inlined copy of the old one-liner).
|
||
- Tables (room3d.js:77–125): `CANON` (the 54 mixamo names, generated), `EXACT` side-less bones,
|
||
`STEMS` sided stems + generated finger stems, `SIDES` side decompositions. `alias()` walks 4
|
||
progressively-normalised candidates — raw → prefix-stripped → index-stripped → both — and takes the
|
||
first hit, so **`CC_Base_Spine01` (2nd spine bone) beats the generic `spine_01` (1st) reading**.
|
||
Structure handled: namespaces (`mixamorig4:`, `Armature|`), rig prefixes (`CC_Base_`, `DEF-`/`ORG-`/
|
||
`MCH-`, `Bip01 `), CC's per-export numeric tail (`_02`, `_0100`), and side spellings
|
||
`L_`/`_l`/`.L`/`Left`. A decomposition only counts when the leftover stem is known, so `lowerarm`
|
||
is never read as left-"owerarm" and `shoulder` is never read as "shoulde"+right.
|
||
- Covered: Character Creator (bone names read out of the REAL lady.glb JSON chunk, not guessed),
|
||
Unreal/HumanIK, Blender Rigify, Unity-humanoid/VRM/Rokoko long names, plain lowercase, 3ds Max Biped.
|
||
Twist / share / toe-detail / facial / breast / eye bones deliberately stay UNMAPPED — the pair map
|
||
already tolerates missing pairs, and a guessed mapping is worse than an unbaked bone.
|
||
- Guardrail: pairing moved out of `bakeRetarget` into three-free `pairBones()` (room3d.js:146). Under
|
||
`min(8, boneCount)` matches it throws e.g.
|
||
`retarget: only 1/54 bones matched (target looks like 'CC_Base', source looks like 'mixamo'; first
|
||
unmatched bone 'CC_Base_BoneRoot_01'); unsupported skeleton`. `rigStyle()` (room3d.js:127) does the
|
||
detection. The missing-hips error (room3d.js:176) now names both styles too. `min(8, …)` keeps
|
||
`_selftest`'s 3-bone synthetic skeletons legal.
|
||
- **Result on the real asset: lady.glb pairs 52/102 bones** — core 6 + both arms/legs + all 30 finger
|
||
bones, `CC_Base_Hip` wins `hips` and `CC_Base_Pelvis` folds onto the same key and is left unbaked
|
||
(dedupe does the right thing). Reverse direction (CC as the clip SOURCE onto a mixamo body) is 52/52.
|
||
|
||
**M7-A2 — world-integrated backdrops.** A plate read as a billboard in the void: gap under it, alien
|
||
floor, no air. Three fixes, all recomputed on load/param/camera change, never per frame (stage.js:340–415).
|
||
- `params.fit:'frustum'` — `_fitBackdrop` scales `en.root` to COVER the active camera's frustum at
|
||
`params.fitDist` (default 14m) with 6% bleed, drops the plate's centre onto the wrapper origin, parks
|
||
the wrapper on the camera axis and `lookAt`s the camera. Because the plate now extends BELOW y=0 the
|
||
floor cuts it at a clean horizon — the gap is gone. `fitScale()` (stage.js:62) is a pure export so
|
||
node can assert the algebra. Recompute triggers: `addEntity`, `_rebuild`, `setParam(fov)` on the
|
||
ACTIVE cam, `setActiveCamera`, `_resize`, end of `applyState`, and the inspector's `⤢ refit`.
|
||
Absent `fit` = old behaviour (saved scenes unchanged); dock-added plates default to `'frustum'`.
|
||
- `params.groundTint` (default on) — `_plateGround` draws the plate's map (image, poster, or video
|
||
frame) into a 24×24 2D canvas and averages the bottom 7 rows, once per load/param change; the stage
|
||
floor takes that colour ×0.85 and the grid helper fades to 0.12 opacity while a plate is present.
|
||
- Atmosphere — `FogExp2` coloured from the same sample, density from a **keyable `fog` param on the
|
||
ambient light entity** (same trick as `bg`: Lane B's params track drives it with zero schema change).
|
||
Default 0.02, off with no plate. Plate/dome materials get `fog:false` so atmosphere sits between the
|
||
camera and the plate instead of washing it out. Fog object is mutated, not reallocated, when keyed.
|
||
- dock.js: inspector gains `fit` select, `fit dist`, `⤢ refit to camera`, `ground tint` checkbox
|
||
(new `_chk` helper, dock.js:250) and `fog` on ambient lights; new `_chk` styling in style.css:52.
|
||
|
||
VERIFICATION (headless only):
|
||
- `node scenegod/web/room3d_test.mjs` → `ALL PASS (26 CC bones paired against mixamo)`. NEW file; it
|
||
loads the REAL room3d.js with the three imports stripped and imported as a data: URL, so the shipped
|
||
tables are what's asserted (no drift-prone copy). Covers: 65 mixamo bones × 5 namespaces identical to
|
||
the legacy canon; 20 CC name→canonical assertions; 8 CC twist/share/facial bones must stay unmapped;
|
||
36 Unreal/Rigify/Unity/Biped assertions; side-less limbs must stay unmapped; CC↔mixamo pairing = 26
|
||
on the sample; mixamo→mixamo pairing 65/65 unchanged; the guardrail throws with style + unmatched
|
||
root in the message; 3-bone `_selftest` skeletons still legal; `rigStyle` on 5 rigs.
|
||
- `node scenegod/web/stage_test.mjs` → `ALL PASS (fit s=2.186 → plate 21.9×12.3m over a 20.6×11.6m
|
||
frame at 14m)`. NEW file, same import trick on stage.js: frustum size, cover-on-both-axes, bleed
|
||
factor, portrait/letterbox plates scaling on the tight axis, distance-invariance of framing, longer
|
||
lens → smaller plate.
|
||
- `node scenegod/web/presets_test.mjs` (M5) and `timeline_test.mjs` (Lane B) still pass unchanged.
|
||
- `node --check` clean on room3d.js / stage.js / dock.js / presets.js / grammar.js and on index.html's
|
||
inline module.
|
||
|
||
DECISIONS:
|
||
- Aliasing lives in `canon()` behind a CANON short-circuit rather than a new module — Lane A owns 5
|
||
files and the baker is the only consumer. `pairBones`/`rigStyle`/`fitScale` are exported THREE-free
|
||
so node can test the parts that actually encode judgement.
|
||
- Ambiguity resolved by qualification, not by guessing: CC's 3-bone torso (Waist/Spine01/Spine02) maps
|
||
to mixamo Spine/Spine1/Spine2 via `ccbase*` keys; UE's `spine_01/02/03` maps to the same three via
|
||
generic keys. Chains longer than mixamo's 3 (UE `spine_04/05`, rigify `spine.003+`) stay unmapped.
|
||
- **Known limit, marked `ponytail:` at room3d.js:190**: lady.glb hangs its hip height off LOCAL Z
|
||
(Z-up CC export), so the hips-travel `ratio` lands 0 and clips play IN PLACE. That is the safe read —
|
||
the source's travel delta is in the SOURCE's axes and scaling it into a Z-up hip would fly the
|
||
character. Rotations are unaffected (they transfer through world space), so she dances correctly.
|
||
Proper fix = rotate the travel delta through both rests' parent world quats; deliberately not done,
|
||
since it would perturb the verified mixamo→mixamo path for a case nobody has asked for yet.
|
||
- Auto-fit OWNS the backdrop's wrapper transform (marked `ponytail:` at stage.js:355) — keyframe a
|
||
plate only with `fit:'free'`. Fit is NOT recomputed on camera *movement* (only fov/cut/resize):
|
||
following a dolly would glue the plate to the lens and kill parallax.
|
||
- **Absorbed bug fix you should know about (visual delta):** `_buildLight` ran `_refreshDefaults()`
|
||
BEFORE the entity was registered in `_entities`, so adding a scene light never dimmed the built-in
|
||
hemi (2.2) / key — every M5 lighting preset was being washed out by studio defaults. Now refreshed
|
||
after registration (stage.js:197). Expect noir/night/golden_hour to read much stronger than in the
|
||
SYNC-6 screenshots; RoomEnvironment IBL still keeps nothing pitch black.
|
||
|
||
BLOCKED/REQUESTS:
|
||
- @orchestrator (PLAN §4.1 amendment, no code impact): backdrop `params` gains `fit:"frustum"|"free"`,
|
||
`fitDist` (m, default 14) and `groundTint` (bool, default true); `light` params gain `fog` (FogExp2
|
||
density, keyable, lives on the ambient entity next to `bg`). All round-trip through
|
||
captureState/applyState today.
|
||
- @Lane B (free win, not blocking): `fog` behaves exactly like `bg` — a params-track key on the ambient
|
||
light entity animates the atmosphere. A "haze up over 2s" key is now a one-liner for the DIRECT panel.
|
||
- Still owed by John: a viseme-bearing test character (carried from session 4).
|
||
|
||
NEXT / orchestrator SYNC checklist (browser, :8020):
|
||
1. `?selftest=1` → header badge must read `selftest: PASS (maxErr=…, alias ok)` — the alias check is
|
||
new (index.html:33), it canons CC + Unreal + namespaced-mixamo names in-page.
|
||
2. **Lady dances (the headline):** CHARACTERS → lady → ANIMATIONS → hiphop.fbx (drag onto her, or
|
||
select her and click the card) → block lands on the timeline → press play. Before today this threw
|
||
`retarget: missing hips (mixamorig skeleton?)`. Console proof if you want it:
|
||
`stage.prepareClip('e1','animations/test/hiphop.fbx',0).then(c=>console.log(c.tracks.length))` →
|
||
expect **53** tracks (52 bone quats + 1 hips position). She dances in place by design (see the
|
||
ponytail note above); man.fbx is unchanged and still travels.
|
||
3. **Plate fills frame + ground + fog:** BACKDROPS → the golden-hour video plate (or street.jpg) →
|
||
it should land already filling the view with no gap under it, the floor should go warm/orange, the
|
||
grid should nearly vanish. Inspector shows `fit: frustum`, `fit dist 14`, `ground tint ✓`. Move the
|
||
director cam and hit `⤢ refit to camera` to re-fill. Add an ambient light → its inspector `fog`
|
||
field (0.02 default) — crank to 0.08 to see the haze eat the far floor, 0 to kill it.
|
||
4. Cut to a scene camera (◉ set active camera) — the plate refits to THAT lens; change its fov and it
|
||
refits again. PiP should show the same filled frame.
|
||
5. Regression watch: existing saved scenes (`sync2-sunset`) have no `fit` param → plates must load at
|
||
their old size/place. Lighting will look stronger than before (see the `_refreshDefaults` note).
|
||
|
||
## 2026-07-26 session 7 (M11-A CAMERA MOVEMENT + "+ angle" + fitAnchor)
|
||
DONE — all three items, headless-verified (no browser, no git, :8020 untouched per orders):
|
||
|
||
**M11-A1 — move presets.** Every shot the app makes was locked off; now the solver places a camera
|
||
AND moves it. `grammar.js:60` MOVES (10, pure data) + `SHOT_ORDER` (:74) + `EASES` (:77) +
|
||
`MOVE_MAX_DUR` 6s (:78) + `DOLLY_CLAMP` (:79). `presets.js:87` `solveMove()` returns the same shape
|
||
`solveShot` does, twice: `{move, from:{pos,rot,fov}, to:{pos,rot,fov}, aim, dist, shot, angle, focal,
|
||
endShot, endFocal}`. START is byte-for-byte what 🎥 frame would solve for the panel's
|
||
{shot,angle,focal}; END is that state displaced. Per kind:
|
||
- **dolly** (push_in/pull_out): distance of one rung along SHOT_ORDER (MS→MCU 1.61m→1.12m on a 1.7m
|
||
subject at 35mm), travelling ALONG the start→aim ray — `rot` and `fov` untouched. At ECU/EWS the
|
||
rung runs out and it clamps by DOLLY_CLAMP (÷1.4 / ×1.4) so the button always does something.
|
||
- **truck** (L/R): lateral along level camera-right, `rot` COPIED (a truck stays parallel — it is not
|
||
a pan and not an orbit). Amount = 0.55 × the half-frame WIDTH, so the subject slides to ~the
|
||
outer third and stays in frame on any lens, and distance-to-subject shifts <6%.
|
||
- **crane** (up/down): vertical only, then re-aimed (`lookAtEuler`) so the subject stays framed;
|
||
amount = 1.2 × the half-frame HEIGHT. Floor-clamped at y=0.05, and the clamp re-aims too.
|
||
- **orbit** (L/R): rotates the horizontal offset about the aim, y held → distance constant to 1e-9,
|
||
sweep exactly ±25° (`degrees` overridable).
|
||
- **zoom** (in/out): one rung along FOCALS (35→50 / 35→24; past the ends ×1.6) — `pos` and `rot`
|
||
returned as identical arrays. A zoom is not a dolly and the code makes that structural, not a
|
||
comment.
|
||
Amounts are FRAME-RELATIVE (multiples of span·H/2) rather than metres, so a move reads the same on a
|
||
CU and a WS, an 18mm and an 85mm — asserted, not assumed. `moveCamera()` (`presets.js:143`) is the
|
||
applier: solves, then puts the camera on the START state immediately (viewport shows frame 1 of the
|
||
move) and returns the pair. `moveDuration()` (`:79`) = scene remaining from the playhead, min 0.5s,
|
||
capped at 6s; no timeline / past the end → the cap. UI: a MOVE segment in the DIRECT bar
|
||
(`presets.js:339`) — move select · dur box (blank = auto) · ease select · **🎬 move**.
|
||
|
||
**M11-A2 — "+ angle"** (`presets.js:331`, next to 🎥 frame). New op flag `newCam:true` on the
|
||
EXISTING `shot` op, so panel and /director still share one code path: `_camFor()` (`:235`) makes a
|
||
fresh camera labelled by `_camLabel()` (`:225` → cam2, cam3, … skipping taken labels), solves the
|
||
current shot/angle/lens onto it, activates it, touches nothing else. Emits an ordinary `shot` event,
|
||
so Lane B keys + cuts it exactly as today — coverage now takes one click instead of a console call.
|
||
|
||
**M11-A3 — fitAnchor** exposed (`dock.js:228`): 0–1 slider + live readout (new `_range` helper,
|
||
`dock.js:262`; styles `style.css:53`), shown next to `fit dist` when `fit:'frustum'`.
|
||
`stage.js:507` now refits on `fitAnchor` too (it was inert from the inspector before).
|
||
|
||
VERIFICATION (headless):
|
||
- `node scenegod/web/presets_test.mjs` → **ALL PASS (10 moves; push in 1.61m → 1.12m, orbit ±25°,
|
||
zoom 35→50mm at a fixed 1.61m)**. New blocks assert the geometry, not the plumbing: an
|
||
independent `aimsAt()` built from three's own `Euler('XYZ')` third column
|
||
(`[sin y, −sin x·cos y, cos x·cos y]`) — NOT from `lookAtEuler`'s internals — plus a
|
||
`assert.throws` proving that checker fails a camera turned 17° off its subject. Then: push-in ends
|
||
closer / pull-out farther with both states still aimed; the push-in end distance EQUALS the MCU
|
||
solve; travel is parallel to the view axis; ECU push-in still pushes; truck distance within 10%,
|
||
displacement ⟂ to the view axis, level, rot identical, inside the half-frame width, L/R exact
|
||
mirrors, same metres at 85mm as at 35mm, farther on a WS; crane changes only y, re-aims, pitch
|
||
crosses from ~0 to negative (up) / positive (down); orbit distance equal to 1e-9, sweep exactly
|
||
±25°, a 90° orbit lands square to the side, orbit-left agrees with truck-left; zoom `pos`/`rot`
|
||
deepEqual and distance unchanged while fov moves — with the explicit push-in-vs-zoom-in pair
|
||
("position, not lens" / "lens, not position"). Plus the ladders, `moveDuration` in 5 states, and
|
||
an applyOps block (stub stage + shimmed `dispatchEvent`/`window.timeline`) asserting the exact
|
||
`move` event keys, t0 = playhead 2s, dur = 6 (8s remaining capped), explicit dur/ease honoured,
|
||
unknown ease → `inout`, the START state actually landing on the camera, no camera spam, and the
|
||
"+ angle" sequence cam2/cam3 with the other cameras' transforms byte-identical.
|
||
- `node scenegod/web/stage_test.mjs` and `room3d_test.mjs` unchanged → ALL PASS. Lane B's
|
||
`timeline_test.mjs` also still green (read-only run; their `applyDirect` ignores an unknown op, so
|
||
today's build no-ops on `move` instead of throwing).
|
||
- `node --check` clean: grammar.js, presets.js, stage.js, dock.js, presets_test.mjs.
|
||
|
||
**THE `move` OP CONTRACT (Lane B, next session) — emitted once per 🎬 move:**
|
||
```js
|
||
CustomEvent('scenegod:direct', { detail: {
|
||
op: 'move', camId, t0, dur, ease, // ease ∈ 'inout' | 'linear'
|
||
from: { pos:[x,y,z], rot:[x,y,z], fov }, // ALREADY applied to the live camera by Lane A
|
||
to: { pos:[x,y,z], rot:[x,y,z], fov },
|
||
entityIds: [camId],
|
||
// informational, safe to ignore: move, subjectId, shot, angle, focal
|
||
}})
|
||
```
|
||
Expected handling: inside one `group()` → `addKey(camId,'transform',{t:t0, ...from, ease})` +
|
||
`addKey(camId,'transform',{t:t0+dur, ...to, ease})`, and the same pair on `params` `fov` (fov is the
|
||
ONLY thing that changes on a zoom — key it always, it costs one key and keeps every move uniform).
|
||
A cut is your call: `addCut(t0, camId)` matches what `shot` does and is what I'd expect, but the
|
||
payload doesn't presume it. `t0` is my read of your playhead (`window.timeline.time`, 0 without a
|
||
timeline) — prefer your own if they ever disagree. `dur` is already clamped to the scene remainder,
|
||
so `t0+dur` never lands past `scene.duration`.
|
||
|
||
DECISIONS:
|
||
- **Push-in is a pure axis dolly** (rot + fov fixed), not "solve the tighter shot and interpolate".
|
||
Re-solving would also raise the aim (MS centre .65H → MCU .75H) — a dolly plus a covert tilt. The
|
||
rung is used for the DISTANCE only. It keeps the dolly/zoom pair a controlled experiment: same
|
||
intent, one variable each. A director who wants MCU composition picks MCU.
|
||
- **Frame-relative amounts, not metres.** Truck/crane amounts multiply span·H/2 (and ×aspect for the
|
||
truck), which is the framed half-height by construction. A fixed 1.5m truck is fine on a WS and
|
||
throws the subject out of frame on an 85mm CU; this can't.
|
||
- **The truck does not re-aim.** The lane brief says both "stays parallel" and "keeps the subject
|
||
framed"; those only co-exist if the lateral is bounded by the frame, so it is (0.55 of half-width)
|
||
and the test asserts the subject is still inside. Re-aiming would make it a tracking pan, which is
|
||
a different move and should be a different button if John wants one.
|
||
- **🎥 frame now prefers the LIVE camera** (`_camFor`, `presets.js:235`), falling back to
|
||
`shotcam`→first→create, i.e. the old order whenever no scene camera is active. Without this, "+
|
||
angle" was a trap: build cam2, adjust the size, and the edit silently lands on camera 1. Behaviour
|
||
is unchanged in every scene that existed before (🎥 frame activates the camera it frames).
|
||
- Both new Stage getters are one-liners and read-only: `activeCamera()` (`stage.js:587`) and
|
||
`viewAspect()` (`:634`, so the truck is bounded by the REAL frame, not an assumed 16:9). No
|
||
existing contract touched — Stage API, `syncVideos`/`entityVideo`, the shot/light/mark payloads,
|
||
scene-JSON round-trip and relative URLs are all as they were.
|
||
- `newCam` rides the existing `shot` op instead of a new `angle` op: one code path for buttons and
|
||
/director, and Lane B needs no change to key coverage cameras.
|
||
- Coverage cameras are numbered (cam2/cam3), not named `wide`/`close`: the label outlives the shot
|
||
size that produced it, and a lying label is worse than a boring one.
|
||
|
||
BLOCKED/REQUESTS:
|
||
- **@Lane B (the one thing that makes this real)**: implement `applyDirect` for `op:'move'` per the
|
||
contract above. Until then the camera snaps to the move's START and the END exists only in the
|
||
event — nothing animates.
|
||
- @orchestrator: no PLAN §4.1 schema change needed — moves land as ordinary transform/params keys.
|
||
Worth adding MOVES/EASES to the M5 grammar bullet in §5 when you next touch it.
|
||
- Still owed by John: a viseme-bearing test character (carried from session 4).
|
||
|
||
NEXT / orchestrator SYNC checklist (browser, :8020):
|
||
1. Select man → shot MS, lens 35mm → **🎬 move** with `push in`, dur blank, ease inout. Camera jumps
|
||
to the MS start and the PiP shows it. Console proof of the payload:
|
||
`addEventListener('scenegod:direct', e => console.log(e.detail), {once:true})` before pressing —
|
||
expect `op:'move'`, `t0` = the playhead, `dur` = min(6, scene remaining), `from`/`to` with `to`
|
||
closer. Once Lane B lands its handler, play should push in over the duration.
|
||
2. `zoom in` on the same subject: `from.pos` and `to.pos` must be IDENTICAL in that console line and
|
||
only `fov` differs (35→50mm ⇒ 37.85°→27.0°). That is the dolly-vs-zoom distinction on screen.
|
||
3. `orbit left` 25° then `crane up`: the subject stays centred through both ends (apply the END by
|
||
hand if B isn't ready: `stage.setTransform(camId,{pos:d.to.pos,rot:d.to.rot})`).
|
||
4. **+ angle**: with man selected, MS → 🎥 frame (cam becomes `shotcam`), then CU low 85mm →
|
||
**+ angle** → a NEW `cam2` appears in the dock/timeline rows, goes live, and `shotcam` has not
|
||
moved. Press 🎥 frame again → it now adjusts cam2 (the live one), creating nothing.
|
||
`scenes/music-video.json` (4 cams) is the scene to sanity-check that nothing else shifts.
|
||
5. Load a plate with `fit: frustum` → inspector shows **anchor** under `fit dist`; drag it 0.75→0.4
|
||
and the plate slides down live (horizon drops); 1.0 puts its top edge on the lens axis.
|