[laneA] M2: PiP + cameras/lights toolbar, eventized clip drops, onChange-on-add

- dock: /assets/tree {name,path,formats,thumb}; +camera/+key/+ambient toolbar;
  clip apply dispatches scenegod:clipdrop when timeline present (else preview);
  drag-from-dock places onto the viewport (screenToFloor / entityAt)
- stage: PiP overlay from active camera; renderActiveCamera hardened as M3 target
  + pause()/resume(); backdrop rebuild-on-param; fire onChange on add+remove
  (fixes tlui empty-names + a gizmo mouseUp id-vs-entity bug); set-active-camera
- verified in the integrated app (:8020, Lane B timeline mounted), zero console
  errors. Found + logged the entity-mirror integration gap (timeline model isn't
  seeded from live stage adds) for the orchestrator.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
type-two 2026-07-18 20:14:52 +10:00
parent bbb3f99016
commit dfd4cebd2b
4 changed files with 209 additions and 22 deletions

View File

@ -56,3 +56,62 @@ BLOCKED/REQUESTS:
NEXT (M2, do not start until orchestrator flips the milestone line):
- multiple cameras + PiP overlay via renderActiveCamera, light entity param plumbing polish,
backdrop rebuild-on-param-change, drag-from-dock onto viewport (currently click-to-add).
## 2026-07-18 session 2 (M2)
DONE: M2 Lane-A scope shipped and verified in the *integrated* app (Lane C server :8020 +
Lane B timeline/tlui mounted). Worked the orchestrator's priority order:
- **P0 /assets/tree shape** — dropped the bare-string fallback; dock.js now reads the confirmed
`{name, path, formats, thumb}` and renders `thumb` (→ `/assets/file?path=`) when present.
- **P1 eventize clip drops**`dock._applyClip(id,path,idx)`: when `window.timeline` exists it
dispatches `scenegod:clipdrop {detail:{id,path,clipIndex}}` (Lane B places the block); only
bakes+previews standalone when no timeline. Uploaded/local clip files can't hand off a server
path, so with a timeline up they toast a "use the dock" note instead of a dead preview.
- **P2 onChange on every entity**`Stage.addEntity`/`removeEntity` now fire `_fireChange(en)`,
so tlui's `onChange→_syncRows` builds a row per entity, incl. programmatic `applyState` loads.
Fixed the empty-names bug on both add and remove. Also fixed a latent bug: the gizmo `mouseUp`
fired `_fireChange` with the *id* not the entity, so the inspector never refreshed on drag.
- **P4 cameras/lights/PiP/backdrop/drag**:
- `Stage` PiP: overlay `<canvas class="pip">` in `#view`, rendered each frame from the active
cam (render active→main canvas, blit to pip, then render director last so the main view is
correct). `setActiveCamera` shows/hides it. Verified 16,495 lit pixels (real content).
- `renderActiveCamera(canvas|null)` hardened as the M3 final-render target; added
`pause()/resume()` so Lane C can freeze the director loop and drive frames deterministically.
- dock "add rig" toolbar: **+ camera / + key light / + ambient** (these have no asset file);
inspector gains **◉ set active camera** for cameras; light `castShadow` param plumbed.
- `setParam` rebuilds the backdrop mesh on mode/image/width change (`_rebuildBackdrop`).
- drag-from-dock onto the viewport: cards are `draggable`; `#view` drop places chars/props at
the cursor (`screenToFloor`) and bakes clips onto the character under the cursor (`entityAt`).
VERIFICATION (in-app, :8020, zero console errors throughout):
- Added man + key light + ambient + camera via toolbar → all 4 appear as tlui rows immediately
(P2 proven). Man lit by the scene lights. Screenshot: `logs/shots/m2-cameras-lights-dome.png`.
- Clicked ANIMATIONS→hiphop on the selected man → `scenegod:clipdrop` fired with
`{id:'e1', path:'animations/test/hiphop.fbx', clipIndex:0}` (P1 proven). Lane B DOES consume it
(timeline.js:98) but the block doesn't land — see the integration gap below.
- capturekey button → `scenegod:capturekey {id:'e1'}`. backdrop plane→dome rebuild swapped the
root mesh with no error.
DECISIONS:
- PiP renders the active cam into the *main* WebGL canvas then blits to the 2D pip canvas (one
shared context); director view is rendered last each frame so it always wins the main canvas.
Same primitive is the M3 render path: `pause()` the loop, `renderActiveCamera(null)`, read pixels.
- Local/upload clip files are a no-op-with-note when a timeline is mounted (no server path to place
a block). ponytail: revisit only if John wants drag-in clip files to become timeline blocks.
BLOCKED/REQUESTS — INTEGRATION GAP (orchestrator, please route):
- Live dock/stage-added entities never reach the timeline's model. `Timeline.scene.entities`
starts `[]` (timeline.js:80) and is only filled by `timeline.load(sceneJson)`. Both
`scenegod:clipdrop`→`addClipDrop` and `scenegod:capturekey`→`addKey` call `_tracks(id)`, which
throws `no entity <id>` for anything added through the dock. Verified live at :8020:
`manOnStage=true, inTimelineModel=false, addClipDropErr="no entity e1"`. So on today's build,
keyframing/clip-dropping a dock-added entity is dead — it only works after a scene `load()`.
- This is a cross-lane seam, not a Lane-A-file fix (timeline.js is Lane B's). The natural hook is
already in place on my side: **`stage.onChange(en)` now fires on every add AND remove**
(my P2 fix). Suggested resolution (Lane B, ~4 lines): in the Timeline ctor,
`stage.onChange(en => this._mirror(en))` where `_mirror` upserts/removes a `scene.entities`
entry from `{id,kind,label,source,params, transform: stage.entityTransform(id), tracks:{}}`.
I can instead dispatch a dedicated `scenegod:entityadd/entityremove` event from the dock if the
orchestrator prefers an explicit contract over reusing onChange — say the word and I'll add it.
NEXT: once the entity-mirror seam is decided, the "lady + man + street + 2 cameras + sunset" SYNC-2
demo should play. Holding at the M2 boundary until the orchestrator flips the milestone line.

View File

@ -14,6 +14,7 @@ export class Dock {
stage.onChange(en => { if(en && en.id === this._selId) this.renderInspector(en); });
this._selId = null;
this._buildTabs();
this._viewDrop();
this._dropZone();
this.load();
}
@ -31,6 +32,14 @@ export class Dock {
}
_buildTabs(){
const add = document.createElement('div'); add.className = 'addbar';
for(const [txt, what] of [['+ camera','camera'], ['+ key light','keylight'], ['+ ambient','ambient']]){
const b = document.createElement('button'); b.textContent = txt;
b.onclick = () => this._addRig(what).catch(e => this._toast('failed: ' + String(e).slice(0,80)));
add.appendChild(b);
}
this.dock.appendChild(add);
const bar = document.createElement('div'); bar.className = 'tabs';
for(const t of TABS){
const b = document.createElement('button'); b.textContent = t.toUpperCase();
@ -43,11 +52,35 @@ export class Dock {
this.dock.appendChild(this.list);
}
// tolerate string paths or {path,name,files:[{path,ext}]} until Lane C nails the shape (logged REQUEST)
_path(it){ return typeof it === 'string' ? it
: (it.path || (it.files && it.files[0] && it.files[0].path) || it.file || ''); }
_name(it){ return typeof it === 'string' ? it.split('/').pop()
: (it.name || this._path(it).split('/').pop()); }
// drag a dock card onto the viewport → place at the drop point (chars/props) or bake onto the
// character under the cursor (animations). Library items only; local files use the window drop.
_viewDrop(){
this.view.addEventListener('dragover', e => {
if([...e.dataTransfer.types].includes('application/scenegod')) e.preventDefault(); });
this.view.addEventListener('drop', async e => {
const raw = e.dataTransfer.getData('application/scenegod');
if(!raw) return; // a local file drop — let the window handler take it
e.preventDefault(); e.stopPropagation();
const { tab, path, name } = JSON.parse(raw);
try {
if(tab === 'animations'){
const id = this.stage.entityAt(e.clientX, e.clientY, 'character');
if(!id) return this._toast('drop the clip onto a character');
return this._applyClip(id, path, 0);
}
const kind = tab === 'characters' ? 'character' : tab === 'props' ? 'prop' : 'backdrop';
const desc = { kind, label: name, source:{ type:'assets', path } };
if(kind === 'backdrop') desc.params = { mode:'plane', image:path, width:10 };
else desc.transform = { pos: this.stage.screenToFloor(e.clientX, e.clientY), rot:[0,0,0], scale:1 };
const en = await this.stage.addEntity(desc);
this.stage.select(en.id);
} catch(err){ this._toast('drop failed: ' + String(err).slice(0,90)); }
});
}
// /assets/tree item shape (confirmed by Lane C): {name, path, formats:[...], thumb: relpath|null}
_path(it){ return it.path; }
_name(it){ return it.name || (it.path || '').split('/').pop(); }
renderBrowser(){
this.list.innerHTML = '';
@ -60,9 +93,14 @@ export class Dock {
}
const icon = { characters:'🕺', props:'🪑', backdrops:'🖼', animations:'🏃' }[this.tab];
for(const it of items){
const c = document.createElement('div'); c.className = 'card';
c.innerHTML = `<div class="th">${icon}</div><div class="nm">${this._name(it)}</div>`;
const c = document.createElement('div'); c.className = 'card'; c.draggable = true;
const th = it.thumb
? `<img class="th" loading="lazy" src="/assets/file?path=${encodeURIComponent(it.thumb)}">`
: `<div class="th">${icon}</div>`;
c.innerHTML = `${th}<div class="nm">${this._name(it)}</div>`;
c.onclick = () => this._addFromLibrary(it);
c.ondragstart = ev => ev.dataTransfer.setData('application/scenegod',
JSON.stringify({ tab: this.tab, path: this._path(it), name: this._name(it) }));
this.list.appendChild(c);
}
}
@ -73,11 +111,7 @@ export class Dock {
if(this.tab === 'animations'){
const en = this.stage.getEntity(this._selId);
if(!en || en.kind !== 'character') return this._toast('select a character first');
this._toast('baking clip…');
const clip = await this.stage.prepareClip(en.id, path, 0);
this.stage.playClip(en.id, clip);
this._toast('▶ ' + this._name(it));
return;
return this._applyClip(en.id, path, 0);
}
const kind = this.tab === 'characters' ? 'character' : this.tab === 'props' ? 'prop' : 'backdrop';
const desc = { kind, label: this._name(it), source: { type:'assets', path } };
@ -89,6 +123,31 @@ export class Dock {
} catch(e){ this._toast('failed: ' + String(e).slice(0, 100)); }
}
// clips go to the timeline once it's mounted; standalone preview only when it isn't
async _applyClip(id, path, clipIndex){
if(window.timeline){
dispatchEvent(new CustomEvent('scenegod:clipdrop', { detail:{ id, path, clipIndex } }));
this._toast('+ ' + path.split('/').pop() + ' → timeline');
return;
}
this._toast('baking clip…');
const clip = await this.stage.prepareClip(id, path, clipIndex);
this.stage.playClip(id, clip);
this._toast('▶ ' + path.split('/').pop());
}
// cameras and lights have no asset file — spawn them from the toolbar
async _addRig(what){
const D = {
camera: { kind:'camera', label:'camera', params:{ fov:45 }, transform:{ pos:[0,1.6,4], rot:[0,0,0], scale:1 } },
keylight: { kind:'light', label:'key light', params:{ type:'key', color:'#ffffff', intensity:2.2, castShadow:true }, transform:{ pos:[3,5,2], rot:[-0.9,0.5,0], scale:1 } },
ambient: { kind:'light', label:'ambient', params:{ type:'ambient', color:'#ffffff', intensity:1.0 } },
}[what];
const en = await this.stage.addEntity(D);
this.stage.select(en.id);
if(en.kind === 'camera') this.stage.setActiveCamera(en.id); // preview through the new cam
}
// ---- inspector ----
renderInspector(en){
this._selId = en ? en.id : null;
@ -104,15 +163,19 @@ export class Dock {
el.appendChild(this._vec3('rot', t.rot, v => this.stage.setTransform(en.id, { rot:v })));
el.appendChild(this._num('scale', t.scale, v => this.stage.setTransform(en.id, { scale:v })));
if(en.kind === 'camera')
if(en.kind === 'camera'){
el.appendChild(this._num('fov', en.params.fov ?? 45, v => this.stage.setParam(en.id, 'fov', v)));
const act = document.createElement('button'); act.textContent = '◉ set active camera'; act.className = 'wide';
act.onclick = () => this.stage.setActiveCamera(en.id);
el.appendChild(act);
}
if(en.kind === 'light'){
el.appendChild(this._num('intensity', en.params.intensity ?? 1.5, v => this.stage.setParam(en.id, 'intensity', v)));
el.appendChild(this._color('color', en.params.color || '#ffffff', v => this.stage.setParam(en.id, 'color', v)));
}
if(en.kind === 'backdrop')
el.appendChild(this._sel('mode', ['plane','corner','dome'], en.params.mode || 'plane',
v => this.stage.setParam(en.id, 'mode', v))); // rebuild is M2
v => this.stage.setParam(en.id, 'mode', v))); // stage rebuilds the mesh on change
const row = document.createElement('div'); row.className = 'ibtns';
const key = document.createElement('button'); key.textContent = '⏺ key'; key.title = 'capture keyframe';
@ -158,9 +221,13 @@ export class Dock {
const sel = this.stage.getEntity(this._selId);
const probe = await this._probe(buf, f.name);
if(sel && sel.kind === 'character' && probe.anims && !probe.meshes){
this._toast('baking ' + f.name + '…');
const clip = await this.stage.prepareClipUpload(sel.id, buf, f.name, 0);
this.stage.playClip(sel.id, clip); this._toast('▶ ' + f.name);
// uploaded clip has no server path → the timeline can't place a block for it
if(window.timeline){ this._toast('add clips from the dock, not local files, once the timeline is up'); }
else {
this._toast('baking ' + f.name + '…');
const clip = await this.stage.prepareClipUpload(sel.id, buf, f.name, 0);
this.stage.playClip(sel.id, clip); this._toast('▶ ' + f.name);
}
} else {
const en = await this.stage.addEntity({ kind:'character', label:f.name,
source:{ type:'upload', path:f.name }, _buf:buf });

View File

@ -24,6 +24,12 @@ export class Stage {
r.setPixelRatio(devicePixelRatio);
r.shadowMap.enabled = true;
view.appendChild(r.domElement);
this._paused = false; // M3 final render freezes the director loop, then drives renderActiveCamera
// PiP overlay — the active camera's view, shown top-right when a camera is active
const pip = this._pip = document.createElement('canvas');
pip.width = 256; pip.height = 144; pip.className = 'pip'; pip.style.display = 'none';
view.appendChild(pip);
const scene = this.scene = new THREE.Scene();
const cam = this._dirCam = new THREE.PerspectiveCamera(45, 1, 0.01, 3000);
@ -45,7 +51,7 @@ export class Stage {
// gizmo
const tc = this._tc = new TransformControls(cam, r.domElement);
tc.addEventListener('dragging-changed', e => { ctr.enabled = !e.value; });
tc.addEventListener('mouseUp', () => { if(this._selected) this._fireChange(this._selected); });
tc.addEventListener('mouseUp', () => { if(this._selected) this._fireChange(this.getEntity(this._selected)); });
scene.add(tc);
addEventListener('keydown', e => {
if(!this._selected) return;
@ -58,9 +64,12 @@ export class Stage {
this.clock = new THREE.Clock();
this._resize(); addEventListener('resize', () => this._resize());
const loop = () => { requestAnimationFrame(loop);
ctr.update();
const dt = this.clock.getDelta();
if(this._paused) return; // M3: caller drives rendering deterministically
ctr.update();
if(this._mixerAuto) for(const en of this._entities.values()) en.mixer && en.mixer.update(dt);
// PiP first (it scribbles the active cam onto the main canvas), then the director view last
if(this._activeCamId && pip.style.display !== 'none') this.renderActiveCamera(pip);
r.render(scene, this._dirCam);
};
loop();
@ -109,6 +118,7 @@ export class Stage {
this.scene.add(wrapper);
this._entities.set(id, en);
if(desc.transform) this.setTransform(id, desc.transform);
this._fireChange(en); // tlui/dock build a row per entity off onChange (incl. applyState loads)
return en;
}
@ -121,6 +131,7 @@ export class Stage {
this._entities.delete(id);
for(const k of [...this._baked.keys()]) if(k.startsWith(id+'|')) this._baked.delete(k);
if(en.kind === 'light') this._refreshDefaults();
this._fireChange(en); // trigger a row resync so the deleted entity drops out
}
async _loadAsset(desc){
@ -250,8 +261,16 @@ export class Stage {
if(en.light){
if(key === 'intensity') en.light.intensity = value;
if(key === 'color') en.light.color.set(value);
if(key === 'castShadow' && en.light.isDirectionalLight) en.light.castShadow = !!value;
}
// backdrop mode/image/width changes rebuild lazily — M2; ponytail: rebuild-on-change, add when keyable.
if(en.kind === 'backdrop' && (key === 'mode' || key === 'image' || key === 'width'))
this._rebuildBackdrop(en); // geometry depends on these — rebuild the mesh
}
async _rebuildBackdrop(en){
const old = en.root;
en.root = await this._buildBackdrop(en);
en.wrapper.add(en.root);
if(old){ en.wrapper.remove(old); disposeRoot(old); }
}
// ---- clips (bake is expensive — cache per path parse and per (entity,path,index) bake) ----
@ -297,16 +316,43 @@ export class Stage {
setMixerAuto(on){ this._mixerAuto = on; } // Timeline calls setMixerAuto(false) at integration
// ---- cameras ----
setActiveCamera(id){ this._activeCamId = id; }
setActiveCamera(id){
this._activeCamId = id;
this._pip.style.display = (id && this._entities.get(id)?.cam) ? 'block' : 'none';
}
_activeCam(){ const en = this._activeCamId && this._entities.get(this._activeCamId);
return en && en.cam ? en.cam : this._dirCam; }
// PiP: pass the small overlay canvas. Final render (M3): pass null after pause() — the active cam
// lands on the main WebGL canvas at its current size for readPixels/toBlob. Hard contract, keep it.
renderActiveCamera(canvas){
const cam = this._activeCam();
if(cam.isPerspectiveCamera && canvas){ cam.aspect = canvas.width/canvas.height; cam.updateProjectionMatrix(); }
const aspect = canvas ? canvas.width/canvas.height
: this.renderer.domElement.width/this.renderer.domElement.height;
if(cam.isPerspectiveCamera){ cam.aspect = aspect; cam.updateProjectionMatrix(); }
this.renderer.render(this.scene, cam);
if(canvas){ const ctx = canvas.getContext('2d');
ctx.drawImage(this.renderer.domElement, 0, 0, canvas.width, canvas.height); }
}
pause(){ this._paused = true; }
resume(){ this._paused = false; this.clock.getDelta(); } // swallow the paused gap so mixers don't jump
// ---- pointer→world helpers (drag-from-dock placement) ----
screenToFloor(clientX, clientY){
const rect = this.renderer.domElement.getBoundingClientRect();
const ndc = new THREE.Vector2(((clientX-rect.left)/rect.width)*2-1, -((clientY-rect.top)/rect.height)*2+1);
const rc = new THREE.Raycaster(); rc.setFromCamera(ndc, this._dirCam);
const hit = new THREE.Vector3();
return rc.ray.intersectPlane(new THREE.Plane(new THREE.Vector3(0,1,0), 0), hit) ? [hit.x, 0, hit.z] : [0,0,0];
}
entityAt(clientX, clientY, kind){
const rect = this.renderer.domElement.getBoundingClientRect();
const ndc = new THREE.Vector2(((clientX-rect.left)/rect.width)*2-1, -((clientY-rect.top)/rect.height)*2+1);
const rc = new THREE.Raycaster(); rc.setFromCamera(ndc, this._dirCam);
const hits = rc.intersectObjects(this.entities().map(e=>e.wrapper), true);
for(const h of hits){ let o=h.object; while(o && o.userData.entityId==null) o=o.parent;
if(o){ const en=this._entities.get(o.userData.entityId); if(en && (!kind || en.kind===kind)) return en.id; } }
return null;
}
// ---- scene JSON round-trip (entities only; Timeline owns tracks/duration/cuts) ----
captureState(){

View File

@ -56,6 +56,9 @@ header .tag{font-family:var(--mono);font-size:10px;color:var(--teal);letter-spac
padding:8px;font-family:var(--mono);font-size:12px;cursor:pointer}
.ibtns button:hover{border-color:var(--teal)}
.ibtns button.danger:hover{border-color:var(--bad);color:var(--bad)}
button.wide{width:100%;margin:8px 0 0;background:var(--panel);border:1px solid var(--line2);color:var(--ink);
border-radius:7px;padding:8px;font-family:var(--mono);font-size:11px;cursor:pointer}
button.wide:hover{border-color:var(--teal)}
/* drop + toast */
body.hot::after{content:'drop to load';position:fixed;inset:0;display:flex;align-items:center;justify-content:center;
@ -63,5 +66,17 @@ body.hot::after{content:'drop to load';position:fixed;inset:0;display:flex;align
#toast{position:fixed;bottom:24px;left:50%;transform:translateX(-50%);background:var(--panel);
border:1px solid var(--teal);border-radius:9px;padding:9px 16px;font-size:13px;display:none;z-index:50;max-width:80vw}
/* PiP — active camera preview overlay */
.pip{position:absolute;top:12px;right:12px;width:256px;height:144px;border:1px solid var(--teal);
border-radius:6px;background:#000;box-shadow:0 4px 18px rgba(0,0,0,.55);z-index:5}
.pip::after{content:'CAM'}
/* dock "add rig" toolbar (cameras + lights have no asset file) */
.addbar{display:flex;gap:6px;margin-bottom:10px}
.addbar button{flex:1;background:var(--panel);border:1px solid var(--line2);color:var(--ink);border-radius:7px;
padding:6px 4px;font-family:var(--mono);font-size:10.5px;cursor:pointer}
.addbar button:hover{border-color:var(--teal)}
.card[draggable]{cursor:grab}
/* === LANE B === */
/* Lane B appends timeline styling below this marker */