From dfd4cebd2bd26093bd99a458d84ffb3ce986d21a Mon Sep 17 00:00:00 2001 From: type-two Date: Sat, 18 Jul 2026 20:14:52 +1000 Subject: [PATCH] [laneA] M2: PiP + cameras/lights toolbar, eventized clip drops, onChange-on-add - dock: /assets/tree {name,path,formats,thumb}; +camera/+key/+ambient toolbar; clip apply dispatches scenegod:clipdrop when timeline present (else preview); drag-from-dock places onto the viewport (screenToFloor / entityAt) - stage: PiP overlay from active camera; renderActiveCamera hardened as M3 target + pause()/resume(); backdrop rebuild-on-param; fire onChange on add+remove (fixes tlui empty-names + a gizmo mouseUp id-vs-entity bug); set-active-camera - verified in the integrated app (:8020, Lane B timeline mounted), zero console errors. Found + logged the entity-mirror integration gap (timeline model isn't seeded from live stage adds) for the orchestrator. Co-Authored-By: Claude Opus 4.8 --- logs/laneA.md | 59 ++++++++++++++++++++++++ scenegod/web/dock.js | 101 ++++++++++++++++++++++++++++++++++------- scenegod/web/stage.js | 56 +++++++++++++++++++++-- scenegod/web/style.css | 15 ++++++ 4 files changed, 209 insertions(+), 22 deletions(-) diff --git a/logs/laneA.md b/logs/laneA.md index 07ba5f4..f9de794 100644 --- a/logs/laneA.md +++ b/logs/laneA.md @@ -56,3 +56,62 @@ BLOCKED/REQUESTS: NEXT (M2, do not start until orchestrator flips the milestone line): - multiple cameras + PiP overlay via renderActiveCamera, light entity param plumbing polish, backdrop rebuild-on-param-change, drag-from-dock onto viewport (currently click-to-add). + +## 2026-07-18 session 2 (M2) +DONE: M2 Lane-A scope shipped and verified in the *integrated* app (Lane C server :8020 + +Lane B timeline/tlui mounted). Worked the orchestrator's priority order: +- **P0 /assets/tree shape** — dropped the bare-string fallback; dock.js now reads the confirmed + `{name, path, formats, thumb}` and renders `thumb` (→ `/assets/file?path=`) when present. +- **P1 eventize clip drops** — `dock._applyClip(id,path,idx)`: when `window.timeline` exists it + dispatches `scenegod:clipdrop {detail:{id,path,clipIndex}}` (Lane B places the block); only + bakes+previews standalone when no timeline. Uploaded/local clip files can't hand off a server + path, so with a timeline up they toast a "use the dock" note instead of a dead preview. +- **P2 onChange on every entity** — `Stage.addEntity`/`removeEntity` now fire `_fireChange(en)`, + so tlui's `onChange→_syncRows` builds a row per entity, incl. programmatic `applyState` loads. + Fixed the empty-names bug on both add and remove. Also fixed a latent bug: the gizmo `mouseUp` + fired `_fireChange` with the *id* not the entity, so the inspector never refreshed on drag. +- **P4 cameras/lights/PiP/backdrop/drag**: + - `Stage` PiP: overlay `` in `#view`, rendered each frame from the active + cam (render active→main canvas, blit to pip, then render director last so the main view is + correct). `setActiveCamera` shows/hides it. Verified 16,495 lit pixels (real content). + - `renderActiveCamera(canvas|null)` hardened as the M3 final-render target; added + `pause()/resume()` so Lane C can freeze the director loop and drive frames deterministically. + - dock "add rig" toolbar: **+ camera / + key light / + ambient** (these have no asset file); + inspector gains **◉ set active camera** for cameras; light `castShadow` param plumbed. + - `setParam` rebuilds the backdrop mesh on mode/image/width change (`_rebuildBackdrop`). + - drag-from-dock onto the viewport: cards are `draggable`; `#view` drop places chars/props at + the cursor (`screenToFloor`) and bakes clips onto the character under the cursor (`entityAt`). + +VERIFICATION (in-app, :8020, zero console errors throughout): +- Added man + key light + ambient + camera via toolbar → all 4 appear as tlui rows immediately + (P2 proven). Man lit by the scene lights. Screenshot: `logs/shots/m2-cameras-lights-dome.png`. +- Clicked ANIMATIONS→hiphop on the selected man → `scenegod:clipdrop` fired with + `{id:'e1', path:'animations/test/hiphop.fbx', clipIndex:0}` (P1 proven). Lane B DOES consume it + (timeline.js:98) but the block doesn't land — see the integration gap below. +- capturekey button → `scenegod:capturekey {id:'e1'}`. backdrop plane→dome rebuild swapped the + root mesh with no error. + +DECISIONS: +- PiP renders the active cam into the *main* WebGL canvas then blits to the 2D pip canvas (one + shared context); director view is rendered last each frame so it always wins the main canvas. + Same primitive is the M3 render path: `pause()` the loop, `renderActiveCamera(null)`, read pixels. +- Local/upload clip files are a no-op-with-note when a timeline is mounted (no server path to place + a block). ponytail: revisit only if John wants drag-in clip files to become timeline blocks. + +BLOCKED/REQUESTS — INTEGRATION GAP (orchestrator, please route): +- Live dock/stage-added entities never reach the timeline's model. `Timeline.scene.entities` + starts `[]` (timeline.js:80) and is only filled by `timeline.load(sceneJson)`. Both + `scenegod:clipdrop`→`addClipDrop` and `scenegod:capturekey`→`addKey` call `_tracks(id)`, which + throws `no entity ` for anything added through the dock. Verified live at :8020: + `manOnStage=true, inTimelineModel=false, addClipDropErr="no entity e1"`. So on today's build, + keyframing/clip-dropping a dock-added entity is dead — it only works after a scene `load()`. +- This is a cross-lane seam, not a Lane-A-file fix (timeline.js is Lane B's). The natural hook is + already in place on my side: **`stage.onChange(en)` now fires on every add AND remove** + (my P2 fix). Suggested resolution (Lane B, ~4 lines): in the Timeline ctor, + `stage.onChange(en => this._mirror(en))` where `_mirror` upserts/removes a `scene.entities` + entry from `{id,kind,label,source,params, transform: stage.entityTransform(id), tracks:{}}`. + I can instead dispatch a dedicated `scenegod:entityadd/entityremove` event from the dock if the + orchestrator prefers an explicit contract over reusing onChange — say the word and I'll add it. + +NEXT: once the entity-mirror seam is decided, the "lady + man + street + 2 cameras + sunset" SYNC-2 +demo should play. Holding at the M2 boundary until the orchestrator flips the milestone line. diff --git a/scenegod/web/dock.js b/scenegod/web/dock.js index c036984..c6891fe 100644 --- a/scenegod/web/dock.js +++ b/scenegod/web/dock.js @@ -14,6 +14,7 @@ export class Dock { stage.onChange(en => { if(en && en.id === this._selId) this.renderInspector(en); }); this._selId = null; this._buildTabs(); + this._viewDrop(); this._dropZone(); this.load(); } @@ -31,6 +32,14 @@ export class Dock { } _buildTabs(){ + const add = document.createElement('div'); add.className = 'addbar'; + for(const [txt, what] of [['+ camera','camera'], ['+ key light','keylight'], ['+ ambient','ambient']]){ + const b = document.createElement('button'); b.textContent = txt; + b.onclick = () => this._addRig(what).catch(e => this._toast('failed: ' + String(e).slice(0,80))); + add.appendChild(b); + } + this.dock.appendChild(add); + const bar = document.createElement('div'); bar.className = 'tabs'; for(const t of TABS){ const b = document.createElement('button'); b.textContent = t.toUpperCase(); @@ -43,11 +52,35 @@ export class Dock { this.dock.appendChild(this.list); } - // tolerate string paths or {path,name,files:[{path,ext}]} until Lane C nails the shape (logged REQUEST) - _path(it){ return typeof it === 'string' ? it - : (it.path || (it.files && it.files[0] && it.files[0].path) || it.file || ''); } - _name(it){ return typeof it === 'string' ? it.split('/').pop() - : (it.name || this._path(it).split('/').pop()); } + // drag a dock card onto the viewport → place at the drop point (chars/props) or bake onto the + // character under the cursor (animations). Library items only; local files use the window drop. + _viewDrop(){ + this.view.addEventListener('dragover', e => { + if([...e.dataTransfer.types].includes('application/scenegod')) e.preventDefault(); }); + this.view.addEventListener('drop', async e => { + const raw = e.dataTransfer.getData('application/scenegod'); + if(!raw) return; // a local file drop — let the window handler take it + e.preventDefault(); e.stopPropagation(); + const { tab, path, name } = JSON.parse(raw); + try { + if(tab === 'animations'){ + const id = this.stage.entityAt(e.clientX, e.clientY, 'character'); + if(!id) return this._toast('drop the clip onto a character'); + return this._applyClip(id, path, 0); + } + const kind = tab === 'characters' ? 'character' : tab === 'props' ? 'prop' : 'backdrop'; + const desc = { kind, label: name, source:{ type:'assets', path } }; + if(kind === 'backdrop') desc.params = { mode:'plane', image:path, width:10 }; + else desc.transform = { pos: this.stage.screenToFloor(e.clientX, e.clientY), rot:[0,0,0], scale:1 }; + const en = await this.stage.addEntity(desc); + this.stage.select(en.id); + } catch(err){ this._toast('drop failed: ' + String(err).slice(0,90)); } + }); + } + + // /assets/tree item shape (confirmed by Lane C): {name, path, formats:[...], thumb: relpath|null} + _path(it){ return it.path; } + _name(it){ return it.name || (it.path || '').split('/').pop(); } renderBrowser(){ this.list.innerHTML = ''; @@ -60,9 +93,14 @@ export class Dock { } const icon = { characters:'🕺', props:'🪑', backdrops:'🖼', animations:'🏃' }[this.tab]; for(const it of items){ - const c = document.createElement('div'); c.className = 'card'; - c.innerHTML = `
${icon}
${this._name(it)}
`; + const c = document.createElement('div'); c.className = 'card'; c.draggable = true; + const th = it.thumb + ? `` + : `
${icon}
`; + c.innerHTML = `${th}
${this._name(it)}
`; c.onclick = () => this._addFromLibrary(it); + c.ondragstart = ev => ev.dataTransfer.setData('application/scenegod', + JSON.stringify({ tab: this.tab, path: this._path(it), name: this._name(it) })); this.list.appendChild(c); } } @@ -73,11 +111,7 @@ export class Dock { if(this.tab === 'animations'){ const en = this.stage.getEntity(this._selId); if(!en || en.kind !== 'character') return this._toast('select a character first'); - this._toast('baking clip…'); - const clip = await this.stage.prepareClip(en.id, path, 0); - this.stage.playClip(en.id, clip); - this._toast('▶ ' + this._name(it)); - return; + return this._applyClip(en.id, path, 0); } const kind = this.tab === 'characters' ? 'character' : this.tab === 'props' ? 'prop' : 'backdrop'; const desc = { kind, label: this._name(it), source: { type:'assets', path } }; @@ -89,6 +123,31 @@ export class Dock { } catch(e){ this._toast('failed: ' + String(e).slice(0, 100)); } } + // clips go to the timeline once it's mounted; standalone preview only when it isn't + async _applyClip(id, path, clipIndex){ + if(window.timeline){ + dispatchEvent(new CustomEvent('scenegod:clipdrop', { detail:{ id, path, clipIndex } })); + this._toast('+ ' + path.split('/').pop() + ' → timeline'); + return; + } + this._toast('baking clip…'); + const clip = await this.stage.prepareClip(id, path, clipIndex); + this.stage.playClip(id, clip); + this._toast('▶ ' + path.split('/').pop()); + } + + // cameras and lights have no asset file — spawn them from the toolbar + async _addRig(what){ + const D = { + camera: { kind:'camera', label:'camera', params:{ fov:45 }, transform:{ pos:[0,1.6,4], rot:[0,0,0], scale:1 } }, + keylight: { kind:'light', label:'key light', params:{ type:'key', color:'#ffffff', intensity:2.2, castShadow:true }, transform:{ pos:[3,5,2], rot:[-0.9,0.5,0], scale:1 } }, + ambient: { kind:'light', label:'ambient', params:{ type:'ambient', color:'#ffffff', intensity:1.0 } }, + }[what]; + const en = await this.stage.addEntity(D); + this.stage.select(en.id); + if(en.kind === 'camera') this.stage.setActiveCamera(en.id); // preview through the new cam + } + // ---- inspector ---- renderInspector(en){ this._selId = en ? en.id : null; @@ -104,15 +163,19 @@ export class Dock { el.appendChild(this._vec3('rot', t.rot, v => this.stage.setTransform(en.id, { rot:v }))); el.appendChild(this._num('scale', t.scale, v => this.stage.setTransform(en.id, { scale:v }))); - if(en.kind === 'camera') + if(en.kind === 'camera'){ el.appendChild(this._num('fov', en.params.fov ?? 45, v => this.stage.setParam(en.id, 'fov', v))); + const act = document.createElement('button'); act.textContent = '◉ set active camera'; act.className = 'wide'; + act.onclick = () => this.stage.setActiveCamera(en.id); + el.appendChild(act); + } if(en.kind === 'light'){ el.appendChild(this._num('intensity', en.params.intensity ?? 1.5, v => this.stage.setParam(en.id, 'intensity', v))); el.appendChild(this._color('color', en.params.color || '#ffffff', v => this.stage.setParam(en.id, 'color', v))); } if(en.kind === 'backdrop') el.appendChild(this._sel('mode', ['plane','corner','dome'], en.params.mode || 'plane', - v => this.stage.setParam(en.id, 'mode', v))); // rebuild is M2 + v => this.stage.setParam(en.id, 'mode', v))); // stage rebuilds the mesh on change const row = document.createElement('div'); row.className = 'ibtns'; const key = document.createElement('button'); key.textContent = '⏺ key'; key.title = 'capture keyframe'; @@ -158,9 +221,13 @@ export class Dock { const sel = this.stage.getEntity(this._selId); const probe = await this._probe(buf, f.name); if(sel && sel.kind === 'character' && probe.anims && !probe.meshes){ - this._toast('baking ' + f.name + '…'); - const clip = await this.stage.prepareClipUpload(sel.id, buf, f.name, 0); - this.stage.playClip(sel.id, clip); this._toast('▶ ' + f.name); + // uploaded clip has no server path → the timeline can't place a block for it + if(window.timeline){ this._toast('add clips from the dock, not local files, once the timeline is up'); } + else { + this._toast('baking ' + f.name + '…'); + const clip = await this.stage.prepareClipUpload(sel.id, buf, f.name, 0); + this.stage.playClip(sel.id, clip); this._toast('▶ ' + f.name); + } } else { const en = await this.stage.addEntity({ kind:'character', label:f.name, source:{ type:'upload', path:f.name }, _buf:buf }); diff --git a/scenegod/web/stage.js b/scenegod/web/stage.js index 5d6a07e..f89150c 100644 --- a/scenegod/web/stage.js +++ b/scenegod/web/stage.js @@ -24,6 +24,12 @@ export class Stage { r.setPixelRatio(devicePixelRatio); r.shadowMap.enabled = true; view.appendChild(r.domElement); + this._paused = false; // M3 final render freezes the director loop, then drives renderActiveCamera + + // PiP overlay — the active camera's view, shown top-right when a camera is active + const pip = this._pip = document.createElement('canvas'); + pip.width = 256; pip.height = 144; pip.className = 'pip'; pip.style.display = 'none'; + view.appendChild(pip); const scene = this.scene = new THREE.Scene(); const cam = this._dirCam = new THREE.PerspectiveCamera(45, 1, 0.01, 3000); @@ -45,7 +51,7 @@ export class Stage { // gizmo const tc = this._tc = new TransformControls(cam, r.domElement); tc.addEventListener('dragging-changed', e => { ctr.enabled = !e.value; }); - tc.addEventListener('mouseUp', () => { if(this._selected) this._fireChange(this._selected); }); + tc.addEventListener('mouseUp', () => { if(this._selected) this._fireChange(this.getEntity(this._selected)); }); scene.add(tc); addEventListener('keydown', e => { if(!this._selected) return; @@ -58,9 +64,12 @@ export class Stage { this.clock = new THREE.Clock(); this._resize(); addEventListener('resize', () => this._resize()); const loop = () => { requestAnimationFrame(loop); - ctr.update(); const dt = this.clock.getDelta(); + if(this._paused) return; // M3: caller drives rendering deterministically + ctr.update(); if(this._mixerAuto) for(const en of this._entities.values()) en.mixer && en.mixer.update(dt); + // PiP first (it scribbles the active cam onto the main canvas), then the director view last + if(this._activeCamId && pip.style.display !== 'none') this.renderActiveCamera(pip); r.render(scene, this._dirCam); }; loop(); @@ -109,6 +118,7 @@ export class Stage { this.scene.add(wrapper); this._entities.set(id, en); if(desc.transform) this.setTransform(id, desc.transform); + this._fireChange(en); // tlui/dock build a row per entity off onChange (incl. applyState loads) return en; } @@ -121,6 +131,7 @@ export class Stage { this._entities.delete(id); for(const k of [...this._baked.keys()]) if(k.startsWith(id+'|')) this._baked.delete(k); if(en.kind === 'light') this._refreshDefaults(); + this._fireChange(en); // trigger a row resync so the deleted entity drops out } async _loadAsset(desc){ @@ -250,8 +261,16 @@ export class Stage { if(en.light){ if(key === 'intensity') en.light.intensity = value; if(key === 'color') en.light.color.set(value); + if(key === 'castShadow' && en.light.isDirectionalLight) en.light.castShadow = !!value; } - // backdrop mode/image/width changes rebuild lazily — M2; ponytail: rebuild-on-change, add when keyable. + if(en.kind === 'backdrop' && (key === 'mode' || key === 'image' || key === 'width')) + this._rebuildBackdrop(en); // geometry depends on these — rebuild the mesh + } + async _rebuildBackdrop(en){ + const old = en.root; + en.root = await this._buildBackdrop(en); + en.wrapper.add(en.root); + if(old){ en.wrapper.remove(old); disposeRoot(old); } } // ---- clips (bake is expensive — cache per path parse and per (entity,path,index) bake) ---- @@ -297,16 +316,43 @@ export class Stage { setMixerAuto(on){ this._mixerAuto = on; } // Timeline calls setMixerAuto(false) at integration // ---- cameras ---- - setActiveCamera(id){ this._activeCamId = id; } + setActiveCamera(id){ + this._activeCamId = id; + this._pip.style.display = (id && this._entities.get(id)?.cam) ? 'block' : 'none'; + } _activeCam(){ const en = this._activeCamId && this._entities.get(this._activeCamId); return en && en.cam ? en.cam : this._dirCam; } + // PiP: pass the small overlay canvas. Final render (M3): pass null after pause() — the active cam + // lands on the main WebGL canvas at its current size for readPixels/toBlob. Hard contract, keep it. renderActiveCamera(canvas){ const cam = this._activeCam(); - if(cam.isPerspectiveCamera && canvas){ cam.aspect = canvas.width/canvas.height; cam.updateProjectionMatrix(); } + const aspect = canvas ? canvas.width/canvas.height + : this.renderer.domElement.width/this.renderer.domElement.height; + if(cam.isPerspectiveCamera){ cam.aspect = aspect; cam.updateProjectionMatrix(); } this.renderer.render(this.scene, cam); if(canvas){ const ctx = canvas.getContext('2d'); ctx.drawImage(this.renderer.domElement, 0, 0, canvas.width, canvas.height); } } + pause(){ this._paused = true; } + resume(){ this._paused = false; this.clock.getDelta(); } // swallow the paused gap so mixers don't jump + + // ---- pointer→world helpers (drag-from-dock placement) ---- + screenToFloor(clientX, clientY){ + const rect = this.renderer.domElement.getBoundingClientRect(); + const ndc = new THREE.Vector2(((clientX-rect.left)/rect.width)*2-1, -((clientY-rect.top)/rect.height)*2+1); + const rc = new THREE.Raycaster(); rc.setFromCamera(ndc, this._dirCam); + const hit = new THREE.Vector3(); + return rc.ray.intersectPlane(new THREE.Plane(new THREE.Vector3(0,1,0), 0), hit) ? [hit.x, 0, hit.z] : [0,0,0]; + } + entityAt(clientX, clientY, kind){ + const rect = this.renderer.domElement.getBoundingClientRect(); + const ndc = new THREE.Vector2(((clientX-rect.left)/rect.width)*2-1, -((clientY-rect.top)/rect.height)*2+1); + const rc = new THREE.Raycaster(); rc.setFromCamera(ndc, this._dirCam); + const hits = rc.intersectObjects(this.entities().map(e=>e.wrapper), true); + for(const h of hits){ let o=h.object; while(o && o.userData.entityId==null) o=o.parent; + if(o){ const en=this._entities.get(o.userData.entityId); if(en && (!kind || en.kind===kind)) return en.id; } } + return null; + } // ---- scene JSON round-trip (entities only; Timeline owns tracks/duration/cuts) ---- captureState(){ diff --git a/scenegod/web/style.css b/scenegod/web/style.css index 11726d8..be3c8d5 100644 --- a/scenegod/web/style.css +++ b/scenegod/web/style.css @@ -56,6 +56,9 @@ header .tag{font-family:var(--mono);font-size:10px;color:var(--teal);letter-spac padding:8px;font-family:var(--mono);font-size:12px;cursor:pointer} .ibtns button:hover{border-color:var(--teal)} .ibtns button.danger:hover{border-color:var(--bad);color:var(--bad)} +button.wide{width:100%;margin:8px 0 0;background:var(--panel);border:1px solid var(--line2);color:var(--ink); + border-radius:7px;padding:8px;font-family:var(--mono);font-size:11px;cursor:pointer} +button.wide:hover{border-color:var(--teal)} /* drop + toast */ body.hot::after{content:'drop to load';position:fixed;inset:0;display:flex;align-items:center;justify-content:center; @@ -63,5 +66,17 @@ body.hot::after{content:'drop to load';position:fixed;inset:0;display:flex;align #toast{position:fixed;bottom:24px;left:50%;transform:translateX(-50%);background:var(--panel); border:1px solid var(--teal);border-radius:9px;padding:9px 16px;font-size:13px;display:none;z-index:50;max-width:80vw} +/* PiP — active camera preview overlay */ +.pip{position:absolute;top:12px;right:12px;width:256px;height:144px;border:1px solid var(--teal); + border-radius:6px;background:#000;box-shadow:0 4px 18px rgba(0,0,0,.55);z-index:5} +.pip::after{content:'CAM'} + +/* dock "add rig" toolbar (cameras + lights have no asset file) */ +.addbar{display:flex;gap:6px;margin-bottom:10px} +.addbar button{flex:1;background:var(--panel);border:1px solid var(--line2);color:var(--ink);border-radius:7px; + padding:6px 4px;font-family:var(--mono);font-size:10.5px;cursor:pointer} +.addbar button:hover{border-color:var(--teal)} +.card[draggable]{cursor:grab} + /* === LANE B === */ /* Lane B appends timeline styling below this marker */