7fea880286
Lane A: bone-alias layer so CC_Base_*/Unreal-HumanIK/Rigify/Rokoko/Biped rigs fold onto the canonical mixamo names — lady.glb pairs 52/102 bones and retargets (53 tracks) where she previously threw 'missing hips'; mixamo→mixamo pairing asserted bit-identical. Backdrops gain frustum auto-fit, floor tint sampled from the plate, and FogExp2 in the plate's colour via a keyable param. Also caught: _buildLight ran _refreshDefaults before registration, so scene lights never dimmed the built-in hemi (every preset was washed out). Orchestrator SYNC-7 fixes (found only by rendering actual frames): - editor chrome (gizmo/grid/helpers/cam proxies) was baked into output mp4s - timeline's per-tick rest transform dragged fitted plates out of frame - fit used the viewport aspect, output renders at its own → letterboxed plates - plate offset put its top edge on the lens axis → void above centre-frame; now params.fitAnchor (default 0.75) picks the row that sits at eye level Verified: 5 suites green; 1280x720 render shows a CC character dancing a mixamo clip against an edge-to-edge Flow plate with a horizon and matched ground. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
414 lines
31 KiB
Markdown
414 lines
31 KiB
Markdown
# Lane A log — append-only, newest at bottom
|
||
|
||
Format per session:
|
||
|
||
## <date> session N
|
||
DONE: …
|
||
DECISIONS: …
|
||
BLOCKED/REQUESTS: …
|
||
NEXT: …
|
||
|
||
(no sessions yet)
|
||
|
||
## 2026-07-18 session 1
|
||
DONE: M1 shipped and verified end-to-end in a browser.
|
||
- `web/room3d.js` — ported the rigroom retarget as a pure ES module (no DOM/globals).
|
||
Exports parseAny/canon/boneMap/captureRest/bakeRetarget/disposeRoot/stats + `_selftest`.
|
||
bakeRetarget is now `(clip, srcRoot, srcRest, tgtRoot, tgtRest, opts?)` — target is a
|
||
parameter instead of the closed-over `char`. Algorithm kept byte-for-byte (30fps world-delta
|
||
bake, hips-height ratio, quat sign continuity, dup-hierarchy dedupe). Added `.bvh` to parseAny.
|
||
- `web/stage.js` — `class Stage` per PLAN §4.2. IBL (RoomEnvironment+PMREM), grid+circle floor,
|
||
OrbitControls director cam, TransformControls gizmo (W/E/R), raycast select, shadow-casting.
|
||
addEntity handles character/prop/backdrop(plane|corner|dome)/camera/light; wrapper-Group holds
|
||
all transforms, loaded root normalized+grounded underneath. prepareClip caches parse-per-path and
|
||
bake-per-(entity,path,index). captureState/applyState round-trip entities.
|
||
- `web/dock.js` — asset dock (Characters/Props/Backdrops/Animations tabs from `/assets/tree`,
|
||
offline→file-drop) + inspector (transform/params editors, ⏺key dispatches `scenegod:capturekey`,
|
||
delete). Local drop: image→backdrop, mesh→character, clip-onto-selected-character→bake+play.
|
||
- `web/index.html` + `web/style.css` — CSS-grid layout (dock/view/inspector/timeline), rigroom
|
||
importmap (three 0.160.0), `?selftest=1` badge. Timeline loaded via guarded dynamic import so the
|
||
page never breaks when Lane B's file is absent.
|
||
|
||
VERIFICATION (browser, in-app):
|
||
- `?selftest=1`: PASS — two 3-bone skeletons, different rests+namespaces, baked 1s clip,
|
||
world-delta match maxErr=6.4e-12, srcMoved=0.224rad (guard against a silently-frozen source).
|
||
- Real end-to-end (Female_mixamo.fbx char + "Running.fbx" clip + a jpg backdrop, served from a
|
||
throwaway scratchpad harness): char loaded (65 bones, 18.6k tris, normalized), gizmo-moved,
|
||
clip retargeted (66 tracks) and animating live. Screenshot: `logs/shots/m1-lady-running.png`.
|
||
- Page boots with zero console errors with BOTH timeline.js and the server absent.
|
||
|
||
DECISIONS:
|
||
- selftest uses underscore namespaces (`mixamorig1_`/`mixamorig4_`), not colon — a colon in a bone
|
||
name breaks three's PropertyBinding so the source clip wouldn't bind (silent pass). canon() still
|
||
collapses both to the same key, which is the thing under test.
|
||
- Added `Stage.prepareClipUpload(id, buf, name, idx)` (bake from in-memory bytes) so the offline
|
||
file-drop path can retarget without a server. `Stage.setMixerAuto(false)` lets Lane B take over
|
||
mixer stepping at SYNC 1 (default true so dropped clips preview standalone now).
|
||
- backdrop param edits (mode/image/width) don't rebuild the mesh yet — marked M2. fov/light
|
||
color/intensity are live.
|
||
|
||
BLOCKED/REQUESTS:
|
||
- @Lane C: please confirm the item shape inside `/assets/tree` arrays. dock.js tolerates both a
|
||
bare path string and `{path,name,files:[{path,ext}]}` (rigroom-style) for now — tell me the real
|
||
one and I'll drop the fallback. Also confirm the page is served such that absolute `/web/*.js`
|
||
imports resolve (I used `/web/...` to be URL-independent).
|
||
|
||
NEXT (M2, do not start until orchestrator flips the milestone line):
|
||
- multiple cameras + PiP overlay via renderActiveCamera, light entity param plumbing polish,
|
||
backdrop rebuild-on-param-change, drag-from-dock onto viewport (currently click-to-add).
|
||
|
||
## 2026-07-18 session 2 (M2)
|
||
DONE: M2 Lane-A scope shipped and verified in the *integrated* app (Lane C server :8020 +
|
||
Lane B timeline/tlui mounted). Worked the orchestrator's priority order:
|
||
- **P0 /assets/tree shape** — dropped the bare-string fallback; dock.js now reads the confirmed
|
||
`{name, path, formats, thumb}` and renders `thumb` (→ `/assets/file?path=`) when present.
|
||
- **P1 eventize clip drops** — `dock._applyClip(id,path,idx)`: when `window.timeline` exists it
|
||
dispatches `scenegod:clipdrop {detail:{id,path,clipIndex}}` (Lane B places the block); only
|
||
bakes+previews standalone when no timeline. Uploaded/local clip files can't hand off a server
|
||
path, so with a timeline up they toast a "use the dock" note instead of a dead preview.
|
||
- **P2 onChange on every entity** — `Stage.addEntity`/`removeEntity` now fire `_fireChange(en)`,
|
||
so tlui's `onChange→_syncRows` builds a row per entity, incl. programmatic `applyState` loads.
|
||
Fixed the empty-names bug on both add and remove. Also fixed a latent bug: the gizmo `mouseUp`
|
||
fired `_fireChange` with the *id* not the entity, so the inspector never refreshed on drag.
|
||
- **P4 cameras/lights/PiP/backdrop/drag**:
|
||
- `Stage` PiP: overlay `<canvas class="pip">` in `#view`, rendered each frame from the active
|
||
cam (render active→main canvas, blit to pip, then render director last so the main view is
|
||
correct). `setActiveCamera` shows/hides it. Verified 16,495 lit pixels (real content).
|
||
- `renderActiveCamera(canvas|null)` hardened as the M3 final-render target; added
|
||
`pause()/resume()` so Lane C can freeze the director loop and drive frames deterministically.
|
||
- dock "add rig" toolbar: **+ camera / + key light / + ambient** (these have no asset file);
|
||
inspector gains **◉ set active camera** for cameras; light `castShadow` param plumbed.
|
||
- `setParam` rebuilds the backdrop mesh on mode/image/width change (`_rebuildBackdrop`).
|
||
- drag-from-dock onto the viewport: cards are `draggable`; `#view` drop places chars/props at
|
||
the cursor (`screenToFloor`) and bakes clips onto the character under the cursor (`entityAt`).
|
||
|
||
VERIFICATION (in-app, :8020, zero console errors throughout):
|
||
- Added man + key light + ambient + camera via toolbar → all 4 appear as tlui rows immediately
|
||
(P2 proven). Man lit by the scene lights. Screenshot: `logs/shots/m2-cameras-lights-dome.png`.
|
||
- Clicked ANIMATIONS→hiphop on the selected man → `scenegod:clipdrop` fired with
|
||
`{id:'e1', path:'animations/test/hiphop.fbx', clipIndex:0}` (P1 proven). Lane B DOES consume it
|
||
(timeline.js:98) but the block doesn't land — see the integration gap below.
|
||
- capturekey button → `scenegod:capturekey {id:'e1'}`. backdrop plane→dome rebuild swapped the
|
||
root mesh with no error.
|
||
|
||
DECISIONS:
|
||
- PiP renders the active cam into the *main* WebGL canvas then blits to the 2D pip canvas (one
|
||
shared context); director view is rendered last each frame so it always wins the main canvas.
|
||
Same primitive is the M3 render path: `pause()` the loop, `renderActiveCamera(null)`, read pixels.
|
||
- Local/upload clip files are a no-op-with-note when a timeline is mounted (no server path to place
|
||
a block). ponytail: revisit only if John wants drag-in clip files to become timeline blocks.
|
||
|
||
BLOCKED/REQUESTS — INTEGRATION GAP (orchestrator, please route):
|
||
- Live dock/stage-added entities never reach the timeline's model. `Timeline.scene.entities`
|
||
starts `[]` (timeline.js:80) and is only filled by `timeline.load(sceneJson)`. Both
|
||
`scenegod:clipdrop`→`addClipDrop` and `scenegod:capturekey`→`addKey` call `_tracks(id)`, which
|
||
throws `no entity <id>` for anything added through the dock. Verified live at :8020:
|
||
`manOnStage=true, inTimelineModel=false, addClipDropErr="no entity e1"`. So on today's build,
|
||
keyframing/clip-dropping a dock-added entity is dead — it only works after a scene `load()`.
|
||
- This is a cross-lane seam, not a Lane-A-file fix (timeline.js is Lane B's). The natural hook is
|
||
already in place on my side: **`stage.onChange(en)` now fires on every add AND remove**
|
||
(my P2 fix). Suggested resolution (Lane B, ~4 lines): in the Timeline ctor,
|
||
`stage.onChange(en => this._mirror(en))` where `_mirror` upserts/removes a `scene.entities`
|
||
entry from `{id,kind,label,source,params, transform: stage.entityTransform(id), tracks:{}}`.
|
||
I can instead dispatch a dedicated `scenegod:entityadd/entityremove` event from the dock if the
|
||
orchestrator prefers an explicit contract over reusing onChange — say the word and I'll add it.
|
||
|
||
NEXT: once the entity-mirror seam is decided, the "lady + man + street + 2 cameras + sunset" SYNC-2
|
||
demo should play. Holding at the M2 boundary until the orchestrator flips the milestone line.
|
||
|
||
## 2026-07-18 session 3 (M3-polish)
|
||
DONE: both M3-polish items the orchestrator flagged, verified live at :8020.
|
||
- **PiP cap + swap.** Was fixed 256×144 and (with the CSS `aspect-ratio`/`%` combo) blew up to
|
||
340×413 portrait on a `<canvas>` — CSS aspect-ratio is unreliable on replaced elements. Now sized
|
||
in `_resize()`: `clamp(180, view.width*0.24, 340)` wide × 16:9, bottom-right. Click the inset to
|
||
swap main↔active-cam (`_pipSwap`). Refactored `renderActiveCamera`→ private `_render(cam,canvas)`
|
||
that restores the cam's aspect after a pip blit, so swapping never distorts either camera
|
||
(verified: dirCam.aspect stays 1.79 while it's shown in the inset). `renderActiveCamera(canvas)`
|
||
is now a one-line wrapper over `_render(this._activeCam(), canvas)` — M3 contract unchanged;
|
||
`pause()/resume()` untouched. Verified 180×101 @1.78 ratio, swap flips the views cleanly.
|
||
- **lady.glb flat-white diagnosed + fixed.** It's the ASSET: a Reallusion/Character-Creator export
|
||
with `extensionsRequired:["KHR_materials_pbrSpecularGlossiness"]` — diffuse lives in that
|
||
extension, which three r160 dropped, so all 26 materials fell back to white MeshStandard. (rigroom
|
||
would show the same — no port regression.) Since SCENEGOD must eat any dropped character, added a
|
||
minimal `SpecGlossPlugin` registered on GLTFLoader in room3d.js: spec-gloss → MeshStandard
|
||
(diffuse→map/color, roughness≈1−glossiness, metalness 0). Only touches materials carrying the
|
||
extension; metallic-rough GLBs and FBX untouched. Verified: lady loads with withMap=26/26 and
|
||
renders fully textured (skin/clothes/face). Shot: `logs/shots/m3-pip-swap.png`.
|
||
|
||
DECISIONS:
|
||
- Runtime spec-gloss shim over an offline gltf-transform convert: keeps CC characters "just work"
|
||
in the drop-on-stage flow and in the render pipeline, no per-asset conversion step. Approximation
|
||
(ignores the specular map) is fine for machinima lighting; upgrade path = full spec→metal convert
|
||
if a hero asset needs it.
|
||
- PiP size lives in JS, not CSS — `<canvas>` + `aspect-ratio` is the trap that produced the giant
|
||
inset. CSS keeps only position/border.
|
||
|
||
BLOCKED/REQUESTS: none.
|
||
|
||
NEXT: hold at M3-polish boundary. Available for SYNC-3 findings (finalRender drives
|
||
`pause`/`resume`/`renderActiveCamera` — kept stable per the hard contract).
|
||
|
||
## 2026-07-18 session 4 (SYNC4 fix + M4-A visemes)
|
||
DONE:
|
||
- **applyState/source clean fix.** Absorbed the orchestrator's SYNC4 hotfix and did the schema-correct
|
||
version: `addEntity` defaults camera/light `source` to `{type:'none'}` (PLAN §4.1); `captureState`
|
||
emits it; `applyState` skips only entities that need bytes it lacks (`type:'upload'` AND
|
||
kind∈{character,prop,backdrop}), so `'none'` always rebuilds and legacy `'upload'` cams still
|
||
rebuild. Verified live: save→load of {character, camera, light} → srcTypes
|
||
`[character:assets, camera:none, light:none]`, all 3 come back (rig no longer lost).
|
||
- **M4-A viseme lane** (my deliverable = detection + setMorph hook + badge; Rhubarb→keys is Lane B):
|
||
- `detectVisemes(root)` (stage.js): scans every skinned mesh's `morphTargetDictionary`, loosely
|
||
maps names → canonical visemes A/E/I/O/U/MBP/FV/L (covers CC `V_Open/V_Explosive/V_Dental_Lip/
|
||
V_Tongue_*`, ARKit `jawOpen/mouthPucker/mouthFunnel/mouthClose`, plain vowels). Built once at
|
||
character load; stored on the entity as `{morphs, visemes}` (head/teeth/tongue meshes sharing a
|
||
morph name are grouped so one call drives them all).
|
||
- `stage.visemeTargets(id)` → canonical names present; `stage.setMorph(id, name, weight)` drives a
|
||
canonical viseme OR a raw morph name across all mapped meshes.
|
||
- inspector 🗣 badge (dock.js): lists detected visemes, greyed `🗣 no visemes` when the rig has none.
|
||
- `visemeSelfCheck()` exported: synthetic dict of CC+ARKit+blink names → asserts A/U/MBP/FV/L
|
||
detected, E/I/O NOT (no blink/brow false positives), and setMorph('A') drives the mapped indices
|
||
only. Verified in-browser: `ok=true, found=[A,FV,L,MBP,U]`.
|
||
|
||
VERIFICATION (:8020, zero console errors): viseme self-check ok; man/lady show greyed 🗣 badge
|
||
(neither rig carries mouth morphs — lady.glb is a CC body export with 0 morph targets, confirmed by
|
||
reading the GLB); save/load round-trip keeps cameras+lights.
|
||
|
||
DECISIONS:
|
||
- Viseme matcher is intentionally loose (orchestrator asked "match loosely") and detection-only; the
|
||
authoritative Rhubarb-letter→canonical mapping + keyframing is Lane B's. `setMorph` also accepts
|
||
raw morph names so B can drive anything the matcher missed.
|
||
|
||
BLOCKED/REQUESTS:
|
||
- @orchestrator: no viseme-bearing test character in `assets/` (man.fbx and lady.glb both have zero
|
||
mouth morphs), so the LIT badge + a visible mouth move are proven only by the synthetic self-check,
|
||
not on a real asset. If you can drop a CC/ARKit head with visemes into `assets/characters/test/`,
|
||
I'll confirm live and Lane B can validate Rhubarb end-to-end.
|
||
|
||
NEXT: hold at M4-A boundary. Available for a live viseme asset check and any SYNC findings.
|
||
|
||
## 2026-07-20 session 5 (M5 DIRECTOR MODE + M6 FLOW VIDEO PLATES)
|
||
DONE — both milestones, headless-verified (no browser/servers touched per orchestrator order):
|
||
|
||
M5:
|
||
- `web/grammar.js` (NEW, pure data, zero imports): SHOTS (exact spans/centers from the lane
|
||
spec), ANGLES (eye/low −18°/high +22°/dutch 8° roll/ots_L/ots_R), FOCALS [18,24,35,50,85],
|
||
LIGHTS (day/golden_hour/night/noir/horror-underlight/neon/overcast — {sun{color,intensity,
|
||
elev,azim}, ambient{color,intensity}, bg}), MARKS (center/two_shot_L/R/face_camera/face_other).
|
||
- `web/presets.js` (NEW, NO three.js import — node-runnable): fovForFocal (presets.js:11),
|
||
lookAtEuler w/ dutch roll + three-matching XYZ euler extraction (:14), solveShot bbox→
|
||
pos/rot/fov/dist deterministic solver incl. OTS over-the-shoulder placement (:37),
|
||
frameSubject (:62, bbox via new stage.entityBBox), applyLight (:75 — ensures/reuses one
|
||
sun+ambient pair, writes grammar values verbatim, aims a 20m sun boom via lookAtEuler,
|
||
background rides the AMBIENT entity as param `bg` so it's keyable AND survives save/load),
|
||
applyMark (:99, pos marks keep y; face marks yaw toward active cam / nearest character),
|
||
applyOps (:139) — ONE op runner shared by panel buttons and /director NL, fires
|
||
`scenegod:direct {op, ...values, entityIds}` per action (B keys them; I never touch the
|
||
timeline), mountDirectPanel (:174) — shot/angle/lens segments + 🎥 frame, light buttons,
|
||
mark buttons, NL box.
|
||
- NL box: POST /director {text, selected, entities:[{id,kind,label}]} → expects {ops:[...]},
|
||
applied via applyOps. Hidden by default; probe `GET /director` shows it on 405/200 (FastAPI
|
||
405 = POST route exists); hides itself again when a POST returns 503.
|
||
- index.html:14 `#direct` bar + guarded dynamic import (:49) — page survives presets.js absent.
|
||
style.css:85 DIRECT styles (before the LANE B marker).
|
||
- stage.js additions for M5: entityBBox (stage.js:490, plain arrays — keeps presets three-free),
|
||
cameraWorldPos (:497), light param `bg` → scene.background live (:393) + restored at
|
||
_buildLight (:335) so a loaded scene gets its sky back.
|
||
|
||
M6:
|
||
- `_buildBackdrop` rework (stage.js:225): params.mode gains "video" (renders as plane), and
|
||
plane/corner/dome accept .mp4/.webm/.mov sources; THREE.VideoTexture muted+loop+playsInline,
|
||
SRGB; poster jpg (server /assets/thumb same-stem sidecar) shown until first play/seek then
|
||
swapped in via _swapOnPlay (:309). Standalone page (no timeline → _mixerAuto) auto-plays as
|
||
ambient preview; with a timeline mounted videos stay paused at poster for B's clock.
|
||
- NEW kind `screen` (addEntity :165, _buildScreen :272): video plane, params {video,width,
|
||
emissive}, MeshStandard + emissiveMap boost (default 0.6) so club screens read "on" under
|
||
noir/night. ponytail: CRT curve skipped until a real TV asset asks.
|
||
- `stage.syncVideos(t)` (:506): seeks every video to ((t−videoStart) mod duration), pauses,
|
||
awaits 'seeked' per video (1.5s timeout guard so a stuck decoder can't hang a render), skips
|
||
sub-4ms deltas. render.js already awaits it per frame (Lane C landed the hook). Plus
|
||
videos() (:501) + setVideoPlaying(on) (:503) for Lane B's clock-follow, and params.videoStart
|
||
plumbed (inspector fields, dock.js:214/220).
|
||
- dock.js: _videoPath (:86, derives <stem>.<videoext> from formats), 🎞 badge + 📺
|
||
add-as-screen button on video backdrop cards (:106), click/drag adds mode:'video' plates
|
||
(:132 and _viewDrop), local .mp4 drop → upload video plate, screen/backdrop inspector fields
|
||
(width/emissive/vid start; mode select gains 'video').
|
||
- Lifecycle: removeEntity/_rebuild release the <video> (_dropVideo :316); applyState treats
|
||
upload screens as needs-bytes (skip+warn) like backdrops.
|
||
|
||
VERIFICATION (headless):
|
||
- `web/presets_test.mjs` (NEW): fov-from-focal exact for 18/35/85mm (+35mm≈37.849°); CU dist <
|
||
MS dist on a 1.7m subject + exact span-formula match + CU aims at .87H; dutch sets rot z =
|
||
8° with zero stray pitch/yaw; low angle puts camera below aim pitched up;
|
||
applyLight('golden_hour') returns grammar values verbatim (deepEqual) + applies
|
||
intensity/bg to a stub stage + reuses the sun/ambient pair on a second preset.
|
||
→ `node scenegod/web/presets_test.mjs` = ALL PASS.
|
||
- Scratchpad ops smoke (not committed): applyOps shot-by-LABEL auto-creates 'shotcam', sets
|
||
85mm fov 16.07°, activates cam, low-angle below aim; ots_R auto-resolves the other
|
||
character; two_shot_L pos; face_camera/face_other yaw; unresolvable subject reports error
|
||
without throwing. ALL PASS.
|
||
- ESM syntax-check (node --check via scratchpad .mjs copies): stage/dock/presets/grammar/room3d
|
||
all clean. Retarget code and every existing Stage API untouched.
|
||
|
||
DECISIONS:
|
||
- presets.js takes bboxes as plain arrays (new stage.entityBBox) instead of importing three —
|
||
the whole solver runs under node, same trick as Lane B's timeline.
|
||
- ONE op path: panel buttons call applyOps exactly like /director responses do, so button and
|
||
LLM behavior can never drift; every applied op fires one scenegod:direct event.
|
||
- Background color is a param (`bg`) on the ambient light entity rather than new scene-level
|
||
schema — keyable through B's existing params track, persists via captureState, zero schema
|
||
change. Orchestrator: if you'd rather have scene-level bg in PLAN §4.1, say so and I'll move it.
|
||
- Poster-until-play = swap material maps on first 'playing' OR 'seeked' — so a final render
|
||
that never plays (only seeks) still gets real video frames, and an idle stage shows the jpg.
|
||
- OTS shoulder offset is a fixed 0.35m slide + 0.1m rise along the partner line — deterministic,
|
||
no LLM; eyeball-tune at SYNC if the shoulder isn't reading.
|
||
|
||
BLOCKED/REQUESTS:
|
||
- **@Lane C (blocking M6 dock visibility): mp4 is invisible to the server.** server.py
|
||
ASSET_EXTS/CATEGORIES: backdrops primary = IMG_EXTS only and mp4 ∉ ASSET_EXTS, so
|
||
assets/backdrops/video/scenegod_plates/suburb_strip_goldenhour.mp4 never appears in
|
||
/assets/tree (its poster jpg shows up as a plain image backdrop instead). Please add
|
||
VIDEO_EXTS = {"mp4","webm","mov"} to ASSET_EXTS and to the backdrops primary set. My dock
|
||
already handles the resulting shape (finds the video ext in `formats`, derives
|
||
<stem>.<ext>). Also add MEDIA_TYPES entries ("mp4":"video/mp4","webm":"video/webm",
|
||
"mov":"video/quicktime") — /assets/file currently serves octet-stream and <video> can be
|
||
picky about that (and note Range support matters for long plates).
|
||
Until then, video plates ARE testable via console: `stage.addEntity({kind:'backdrop',
|
||
label:'plate', source:{type:'assets', path:'backdrops/video/scenegod_plates/
|
||
suburb_strip_goldenhour.mp4'}, params:{mode:'video', image:'backdrops/video/
|
||
scenegod_plates/suburb_strip_goldenhour.mp4', width:12}})` (file streams fine, just untyped).
|
||
- @Lane C: /director endpoint per PLAN M5 — my probe expects GET→405 (or 200) when it exists,
|
||
and I hide the box on POST 503. Request contract above; response {ops:[{op:'shot'|'light'|
|
||
'mark', ...}]} — shapes documented at presets.js:136.
|
||
- @Lane B: your M5/M6 hooks are ready on my side — `scenegod:direct` events carry full values
|
||
for keying; for video clock-follow use stage.videos() [{id,video,start}] +
|
||
stage.setVideoPlaying(on) + drift-check video.currentTime yourself per PLAN (>50ms), or just
|
||
await stage.syncVideos(t) on hard seeks.
|
||
|
||
NEXT / orchestrator SYNC checklist (browser):
|
||
1. Select man → CU low 35mm → 🎥 frame: camera frames his face from below in one click; PiP
|
||
shows it (frame auto-activates the shot cam).
|
||
2. golden_hour button: sun+ambient recolor + orange sky in one click; scrub proves B keyed it
|
||
(needs B's scenegod:direct consumer — not landed yet as of this session).
|
||
3. Suburb plate: console addEntity above → poster shows, then video motion (auto-play only
|
||
when timeline absent; with timeline mounted it waits for B's clock). 📺 dock button appears
|
||
once Lane C ships the tree change.
|
||
4. finalRender a 2s scene with the plate — frames must show advancing video (syncVideos is
|
||
awaited by render.js already).
|
||
5. NL box stays hidden until /director exists (probe is GET → expect 405 to reveal).
|
||
|
||
## 2026-07-25 session 6 (M7-A1 non-mixamo rigs + M7-A2 world-integrated backdrops)
|
||
DONE — both M7 items, headless-verified (no browser, no servers touched, per orchestrator order):
|
||
|
||
**M7-A1 — bone aliasing.** `canon()` used to strip a mixamorig prefix and nothing else, so every
|
||
non-mixamo skeleton paired 0 bones and died at "missing hips" (lady.glb has never been animatable).
|
||
- `room3d.js:70` legacy strip is now `rawCanon`; `room3d.js:136` `canon = raw ∈ CANON ? raw : alias(n) || raw`.
|
||
The CANON short-circuit is what makes mixamo→mixamo **bit-identical** — a name that already canons
|
||
to a mixamo bone never touches the alias tables (test asserts all 65 man.fbx bones × 5 namespace
|
||
spellings against an inlined copy of the old one-liner).
|
||
- Tables (room3d.js:77–125): `CANON` (the 54 mixamo names, generated), `EXACT` side-less bones,
|
||
`STEMS` sided stems + generated finger stems, `SIDES` side decompositions. `alias()` walks 4
|
||
progressively-normalised candidates — raw → prefix-stripped → index-stripped → both — and takes the
|
||
first hit, so **`CC_Base_Spine01` (2nd spine bone) beats the generic `spine_01` (1st) reading**.
|
||
Structure handled: namespaces (`mixamorig4:`, `Armature|`), rig prefixes (`CC_Base_`, `DEF-`/`ORG-`/
|
||
`MCH-`, `Bip01 `), CC's per-export numeric tail (`_02`, `_0100`), and side spellings
|
||
`L_`/`_l`/`.L`/`Left`. A decomposition only counts when the leftover stem is known, so `lowerarm`
|
||
is never read as left-"owerarm" and `shoulder` is never read as "shoulde"+right.
|
||
- Covered: Character Creator (bone names read out of the REAL lady.glb JSON chunk, not guessed),
|
||
Unreal/HumanIK, Blender Rigify, Unity-humanoid/VRM/Rokoko long names, plain lowercase, 3ds Max Biped.
|
||
Twist / share / toe-detail / facial / breast / eye bones deliberately stay UNMAPPED — the pair map
|
||
already tolerates missing pairs, and a guessed mapping is worse than an unbaked bone.
|
||
- Guardrail: pairing moved out of `bakeRetarget` into three-free `pairBones()` (room3d.js:146). Under
|
||
`min(8, boneCount)` matches it throws e.g.
|
||
`retarget: only 1/54 bones matched (target looks like 'CC_Base', source looks like 'mixamo'; first
|
||
unmatched bone 'CC_Base_BoneRoot_01'); unsupported skeleton`. `rigStyle()` (room3d.js:127) does the
|
||
detection. The missing-hips error (room3d.js:176) now names both styles too. `min(8, …)` keeps
|
||
`_selftest`'s 3-bone synthetic skeletons legal.
|
||
- **Result on the real asset: lady.glb pairs 52/102 bones** — core 6 + both arms/legs + all 30 finger
|
||
bones, `CC_Base_Hip` wins `hips` and `CC_Base_Pelvis` folds onto the same key and is left unbaked
|
||
(dedupe does the right thing). Reverse direction (CC as the clip SOURCE onto a mixamo body) is 52/52.
|
||
|
||
**M7-A2 — world-integrated backdrops.** A plate read as a billboard in the void: gap under it, alien
|
||
floor, no air. Three fixes, all recomputed on load/param/camera change, never per frame (stage.js:340–415).
|
||
- `params.fit:'frustum'` — `_fitBackdrop` scales `en.root` to COVER the active camera's frustum at
|
||
`params.fitDist` (default 14m) with 6% bleed, drops the plate's centre onto the wrapper origin, parks
|
||
the wrapper on the camera axis and `lookAt`s the camera. Because the plate now extends BELOW y=0 the
|
||
floor cuts it at a clean horizon — the gap is gone. `fitScale()` (stage.js:62) is a pure export so
|
||
node can assert the algebra. Recompute triggers: `addEntity`, `_rebuild`, `setParam(fov)` on the
|
||
ACTIVE cam, `setActiveCamera`, `_resize`, end of `applyState`, and the inspector's `⤢ refit`.
|
||
Absent `fit` = old behaviour (saved scenes unchanged); dock-added plates default to `'frustum'`.
|
||
- `params.groundTint` (default on) — `_plateGround` draws the plate's map (image, poster, or video
|
||
frame) into a 24×24 2D canvas and averages the bottom 7 rows, once per load/param change; the stage
|
||
floor takes that colour ×0.85 and the grid helper fades to 0.12 opacity while a plate is present.
|
||
- Atmosphere — `FogExp2` coloured from the same sample, density from a **keyable `fog` param on the
|
||
ambient light entity** (same trick as `bg`: Lane B's params track drives it with zero schema change).
|
||
Default 0.02, off with no plate. Plate/dome materials get `fog:false` so atmosphere sits between the
|
||
camera and the plate instead of washing it out. Fog object is mutated, not reallocated, when keyed.
|
||
- dock.js: inspector gains `fit` select, `fit dist`, `⤢ refit to camera`, `ground tint` checkbox
|
||
(new `_chk` helper, dock.js:250) and `fog` on ambient lights; new `_chk` styling in style.css:52.
|
||
|
||
VERIFICATION (headless only):
|
||
- `node scenegod/web/room3d_test.mjs` → `ALL PASS (26 CC bones paired against mixamo)`. NEW file; it
|
||
loads the REAL room3d.js with the three imports stripped and imported as a data: URL, so the shipped
|
||
tables are what's asserted (no drift-prone copy). Covers: 65 mixamo bones × 5 namespaces identical to
|
||
the legacy canon; 20 CC name→canonical assertions; 8 CC twist/share/facial bones must stay unmapped;
|
||
36 Unreal/Rigify/Unity/Biped assertions; side-less limbs must stay unmapped; CC↔mixamo pairing = 26
|
||
on the sample; mixamo→mixamo pairing 65/65 unchanged; the guardrail throws with style + unmatched
|
||
root in the message; 3-bone `_selftest` skeletons still legal; `rigStyle` on 5 rigs.
|
||
- `node scenegod/web/stage_test.mjs` → `ALL PASS (fit s=2.186 → plate 21.9×12.3m over a 20.6×11.6m
|
||
frame at 14m)`. NEW file, same import trick on stage.js: frustum size, cover-on-both-axes, bleed
|
||
factor, portrait/letterbox plates scaling on the tight axis, distance-invariance of framing, longer
|
||
lens → smaller plate.
|
||
- `node scenegod/web/presets_test.mjs` (M5) and `timeline_test.mjs` (Lane B) still pass unchanged.
|
||
- `node --check` clean on room3d.js / stage.js / dock.js / presets.js / grammar.js and on index.html's
|
||
inline module.
|
||
|
||
DECISIONS:
|
||
- Aliasing lives in `canon()` behind a CANON short-circuit rather than a new module — Lane A owns 5
|
||
files and the baker is the only consumer. `pairBones`/`rigStyle`/`fitScale` are exported THREE-free
|
||
so node can test the parts that actually encode judgement.
|
||
- Ambiguity resolved by qualification, not by guessing: CC's 3-bone torso (Waist/Spine01/Spine02) maps
|
||
to mixamo Spine/Spine1/Spine2 via `ccbase*` keys; UE's `spine_01/02/03` maps to the same three via
|
||
generic keys. Chains longer than mixamo's 3 (UE `spine_04/05`, rigify `spine.003+`) stay unmapped.
|
||
- **Known limit, marked `ponytail:` at room3d.js:190**: lady.glb hangs its hip height off LOCAL Z
|
||
(Z-up CC export), so the hips-travel `ratio` lands 0 and clips play IN PLACE. That is the safe read —
|
||
the source's travel delta is in the SOURCE's axes and scaling it into a Z-up hip would fly the
|
||
character. Rotations are unaffected (they transfer through world space), so she dances correctly.
|
||
Proper fix = rotate the travel delta through both rests' parent world quats; deliberately not done,
|
||
since it would perturb the verified mixamo→mixamo path for a case nobody has asked for yet.
|
||
- Auto-fit OWNS the backdrop's wrapper transform (marked `ponytail:` at stage.js:355) — keyframe a
|
||
plate only with `fit:'free'`. Fit is NOT recomputed on camera *movement* (only fov/cut/resize):
|
||
following a dolly would glue the plate to the lens and kill parallax.
|
||
- **Absorbed bug fix you should know about (visual delta):** `_buildLight` ran `_refreshDefaults()`
|
||
BEFORE the entity was registered in `_entities`, so adding a scene light never dimmed the built-in
|
||
hemi (2.2) / key — every M5 lighting preset was being washed out by studio defaults. Now refreshed
|
||
after registration (stage.js:197). Expect noir/night/golden_hour to read much stronger than in the
|
||
SYNC-6 screenshots; RoomEnvironment IBL still keeps nothing pitch black.
|
||
|
||
BLOCKED/REQUESTS:
|
||
- @orchestrator (PLAN §4.1 amendment, no code impact): backdrop `params` gains `fit:"frustum"|"free"`,
|
||
`fitDist` (m, default 14) and `groundTint` (bool, default true); `light` params gain `fog` (FogExp2
|
||
density, keyable, lives on the ambient entity next to `bg`). All round-trip through
|
||
captureState/applyState today.
|
||
- @Lane B (free win, not blocking): `fog` behaves exactly like `bg` — a params-track key on the ambient
|
||
light entity animates the atmosphere. A "haze up over 2s" key is now a one-liner for the DIRECT panel.
|
||
- Still owed by John: a viseme-bearing test character (carried from session 4).
|
||
|
||
NEXT / orchestrator SYNC checklist (browser, :8020):
|
||
1. `?selftest=1` → header badge must read `selftest: PASS (maxErr=…, alias ok)` — the alias check is
|
||
new (index.html:33), it canons CC + Unreal + namespaced-mixamo names in-page.
|
||
2. **Lady dances (the headline):** CHARACTERS → lady → ANIMATIONS → hiphop.fbx (drag onto her, or
|
||
select her and click the card) → block lands on the timeline → press play. Before today this threw
|
||
`retarget: missing hips (mixamorig skeleton?)`. Console proof if you want it:
|
||
`stage.prepareClip('e1','animations/test/hiphop.fbx',0).then(c=>console.log(c.tracks.length))` →
|
||
expect **53** tracks (52 bone quats + 1 hips position). She dances in place by design (see the
|
||
ponytail note above); man.fbx is unchanged and still travels.
|
||
3. **Plate fills frame + ground + fog:** BACKDROPS → the golden-hour video plate (or street.jpg) →
|
||
it should land already filling the view with no gap under it, the floor should go warm/orange, the
|
||
grid should nearly vanish. Inspector shows `fit: frustum`, `fit dist 14`, `ground tint ✓`. Move the
|
||
director cam and hit `⤢ refit to camera` to re-fill. Add an ambient light → its inspector `fog`
|
||
field (0.02 default) — crank to 0.08 to see the haze eat the far floor, 0 to kill it.
|
||
4. Cut to a scene camera (◉ set active camera) — the plate refits to THAT lens; change its fov and it
|
||
refits again. PiP should show the same filled frame.
|
||
5. Regression watch: existing saved scenes (`sync2-sunset`) have no `fit` param → plates must load at
|
||
their old size/place. Lighting will look stronger than before (see the `_refreshDefaults` note).
|