Commit Graph

26 Commits

Author SHA1 Message Date
type-two
a8273127ba [m5] DIRECTOR MODE round: cinema-grammar presets spec + /director LLM lane; rhubarb live
- PLAN §5 M5: shot sizes/angles/focals/light presets/blocking marks as
  one-click deterministic presets (grammar.js data + presets.js solver,
  Lane A), keyed at the playhead via scenegod:direct events (Lane B),
  optional NL box -> POST /director -> tailnet Ollama with server-side op
  validation (Lane C; m4pro serves qwen2.5:7b)
- rhubarb 1.14.0 built from John's source zip (JING5), installed arm64 at
  ~/.local/opt/rhubarb, symlinked /opt/homebrew/bin; happy path verified:
  /rhubarb on a real spoken wav (assets/audio/test/vo.wav) returns mouthCues
- lane B unlocked to wire the lip-sync-from-audio button now

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 01:09:30 +10:00
type-two
866551f266 [sync5] viseme matcher side-suffix fix, v1 complete: README, git-hygiene rule
- stage.js detectVisemes: NONMOUTH blocklist + side-stripped matching —
  /(^|_)l($|_)/ was swallowing every ARKit left-side morph (eyeblink_l,
  browdown_l), so setMorph('L') would blink the eye; self-check hardened to
  assert group contents (L === tongue only, smile lands on E)
- CLAUDE.md: lanes stage own files explicitly, never git add -A (laneB's
  attribution finding)
- README.md: run + feature summary; lane files: session 4 accepted for all,
  v1 declared complete, optional backlog recorded

Verified: node + 9 server test groups green, /rhubarb 503 gate live,
audio lane (tone.wav block, gain) + cut markers + sun keys on screen,
zero console errors.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 21:33:53 +10:00
type-two
252a6f666c [laneB] log: M4-B session 4 (audio + visemes)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 21:23:57 +10:00
type-two
df4e06f047 [laneA] M4-A visemes + applyState source:none fix
- stage: detectVisemes() maps CC/ARKit/plain morph names -> canonical
  A/E/I/O/U/MBP/FV/L at character load; visemeTargets(id) + setMorph(id,name,w)
  for the timeline to drive lip sync; visemeSelfCheck() asserts the matcher
- dock: inspector 🗣 badge (greyed when a rig has no mouth morphs)
- stage: camera/light default source:{type:'none'} (PLAN §4.1); applyState
  skips only upload mesh/backdrop bytes -> rig survives save/load (SYNC4)

Verified live at :8020: selfcheck ok=[A,FV,L,MBP,U]; save/load keeps cam+light.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 21:19:07 +10:00
type-two
ba29d0c6d6 [laneB] M4-B model: audio mutators, morphs track eval, importRhubarb
- timeline.js: morphs track kind evaluated via stage.setMorph (lerp per viseme
  name), audio mutators addAudio/moveAudio/setAudioGain/removeAudio (undo-able),
  importRhubarb(id, json, t0) mapping mouthCues → morph keys (per-cue ramp),
  load() ensures scene.audio[], addKey keyed by (t,name) for morphs too.
- stagestub.js: setMorph (records) + visemeTargets seam.
- timeline_test.mjs: morph lerp, rhubarb keys, audio mutators + undo round-trip.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 21:13:35 +10:00
type-two
355ad2fe33 [laneC] log: rhubarb endpoint
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 21:10:54 +10:00
type-two
fb59601cc3 [laneC] /rhubarb viseme endpoint (ffmpeg-style 503 gating)
GET /rhubarb?path= -> rhubarb -f json -> mouthCues for Lane A's viseme lane;
503 when the binary is absent (not installed on ultra yet). Path-guarded.
test_server.py: +503 gating assert (9 groups green).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 21:10:38 +10:00
type-two
a15612eed0 [sync4] applyState camera/light hotfix, M4 verified (audio mux + graft), viseme/audio-UI round unlocked
- stage.js: applyState no longer skips camera/light entities (source
  'upload' but no bytes needed) — loaded scenes were silently losing the
  directing rig; PLAN schema adds source.type 'none' for the clean fix
- lane files: A session-3 + C M4 accepted; A→absorb hotfix + viseme lane
  (M4-A), B→audio rows + Web Audio sync + morphs track (M4-B), C complete

Verified: 6/6 entities restored on scene load, PiP 178x99 capped+visible,
lady 26/26 textured, audio-muxed render h264+aac 8.000s both streams,
graft_limb --selftest green under Blender 5.0.1, node+server suites green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 21:04:09 +10:00
type-two
04e15d3e39 [laneA] M3-polish: PiP cap+swap, spec-gloss GLB fix (lady renders textured)
- PiP: sized in _resize (~24% w, 16:9, bottom-right) — CSS aspect-ratio is
  unreliable on <canvas> and had ballooned it. Click-to-swap main<->active cam;
  _render(cam,canvas) restores aspect after a pip blit so swap never distorts.
  renderActiveCamera/pause/resume contract unchanged (M3 finalRender safe).
- room3d: SpecGlossPlugin on GLTFLoader — CC/Reallusion GLBs use
  KHR_materials_pbrSpecularGlossiness (dropped in three r160) and rendered
  flat white; map spec-gloss -> MeshStandard. lady.glb now withMap=26/26.

Verified live at :8020, zero console errors.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 20:49:22 +10:00
type-two
c602ce8dea [laneC] log: M4 done
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 20:47:52 +10:00
type-two
be65355f6b [laneC] M4: audio mux, graft_limb.py, MODELBEAST proxy
- render/end muxes scene audio[] (adelay+volume+amix, aac, -shortest);
  audio paths path-guarded; render.js finalRender passes scene audio
- scripts/graft_limb.py: headless Blender limb graft (align root->target bone,
  join armatures, parent-keep-offset, join meshes; refusal exit 2 on mitten
  overlap; drops phantom importer icosphere; no finish/decimate). --selftest
  builds synthetic fixtures + verifies both paths under Blender
- /mb/submit MODELBEAST proxy (SCENEGOD_MB=1 gated, token from ~/Documents/
  backnforth/.env at request time never logged, /api/assets->/api/jobs)
- test_server.py: +audio-stream, +MB gating & two-step contract (mock MB);
  8 groups green

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 20:47:31 +10:00
type-two
65b7757fd2 [laneB] log: M2 session 3
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 20:41:59 +10:00
type-two
6200f945b0 [laneB] M2 session 3: absorb _mirror + tests, setDuration, snap-step, param capture
- stagestub.js: removeEntity now fires onChange WITH the removed entity (real
  Stage contract) so _mirror's getEntity(id)===null removal path is testable.
- timeline.js: setDuration(x) — clamps the playhead, keeps out-of-range keys
  (lossless), notifies UI via onLoad. (Reviewed orchestrator's _mirror seam.)
- tlui.js: dur field → setDuration; snap-increment select (frame/0.1/0.25/1s);
  keyable-param add via dblclick on a params lane (captures current param
  values, skips 'type'/bool descriptors); params lane only when keys exist or
  kind is camera/light (no empty lane for characters).
- timeline_test.mjs: _mirror lifecycle (add→keyframe→remove) + setDuration.

Verified in browser (throwaway harness, not committed): dock-adds mirror into
rows, param capture, setDuration clamp+bar refresh, snap-step math.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 20:41:39 +10:00
type-two
41f2c9fa8d [sync2+3] entity mirror seam, SYNC 2 demo + 1080p mp4 verified, M4/polish unlocked
- timeline.js: _mirror — Timeline subscribes stage.onChange, upserts/removes
  live stage entities into scene.entities (closes the dock-added-entity gap
  both lanes flagged; keyframing now works without a scene load)
- lane files: M2/M3 accepted for all lanes; A→PiP sizing + lady.glb material
  polish, B→absorb _mirror + test + setDuration mutator, C→M4 (audio mux,
  graft_limb, MB proxy)

SYNC 2: lady+man+street+2cams+sunset built via live dock path, all
frame-exact (probes in transcript). SYNC 3: finalRender → 240 frames
1920x1080 → ffmpeg h264 8.000s, cuts + sunset verified in extracted frames.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 20:34:32 +10:00
type-two
dfd4cebd2b [laneA] M2: PiP + cameras/lights toolbar, eventized clip drops, onChange-on-add
- dock: /assets/tree {name,path,formats,thumb}; +camera/+key/+ambient toolbar;
  clip apply dispatches scenegod:clipdrop when timeline present (else preview);
  drag-from-dock places onto the viewport (screenToFloor / entityAt)
- stage: PiP overlay from active camera; renderActiveCamera hardened as M3 target
  + pause()/resume(); backdrop rebuild-on-param; fire onChange on add+remove
  (fixes tlui empty-names + a gizmo mouseUp id-vs-entity bug); set-active-camera
- verified in the integrated app (:8020, Lane B timeline mounted), zero console
  errors. Found + logged the entity-mirror integration gap (timeline model isn't
  seeded from live stage adds) for the orchestrator.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 20:14:52 +10:00
type-two
bbb3f99016 [laneB] log: M2 session 2
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 20:11:03 +10:00
type-two
5d9950211c [laneB] M2: SYNC1 row/bar fixes, clipdrop, crossfade, params/cut lanes, box-select
- timeline.js: onLoad hook (rebuilds rows + refreshes scene bar after load()),
  scenegod:clipdrop handler (addClipDrop: block at playhead, out=clip duration,
  auto 0.25 fade where it abuts a neighbour), crossfade now fade-IN + fade-OUT
  (pre-roll pulls successor start earlier for continuous local time).
- tlui.js: reads tracks from the TIMELINE (authoritative) not the stage copy —
  fixes post-load addKey/clipdrop not appearing. Params lanes (per-key color),
  camera-cut lane add via dblclick, snap toggle, box-select + multi-drag +
  Delete, selection highlight, fade wedge on blocks, grow-to-content height.
- timeline_test.mjs: + onLoad, addClipDrop duration, crossfade weight sum=1.

Verified in browser (throwaway harness, not committed): SYNC #3a/#3b repro
fixed, both clip blocks render, clipdrop/add-key show immediately, cut-add and
box-select multi-drag work.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 20:10:07 +10:00
type-two
24b7b4d4b7 [laneC] log: M2/M3 render pipeline done
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 20:00:42 +10:00
type-two
b627a0f2ff [laneC] M2/M3: render pipeline (ffmpeg encode + render.js), thumbnails endpoint, reaper
- /render/begin|frame|end|status|out with background ffmpeg thread + on-disk
  status fallback; boot-time reap of dirs >48h
- /assets/thumb sidecar lookup (pass-through, no resize dep)
- web/render.js: draftRecord (MediaRecorder) + finalRender (deterministic
  timeline.step -> PNG -> POST, <=4 in flight) + download helper
- test_server.py: +render pipeline (ffmpeg synthetic frames -> mp4, ffprobe
  ~1s) +thumbnail; 6 groups green

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 20:00:14 +10:00
type-two
1f8ad13ed6 [sync1] wire TimelineUI to real Stage, null-mixer guard, M2 unlock + orchestrator updates
- index.html: mount TimelineUI, setMixerAuto(false) when timeline present
- timeline.js: _evalClips guards no-blocks / null-mixer (real Stage returns
  null for non-characters; the stub never did)
- tldev.html removed; stagestub.js kept as the node-test fixture
- lane files: M1 accepted for all three, answers to open questions, M2
  priorities (A: clipdrop event, PiP; B: row/scene-bar refresh; C: straight
  into M3 render pipeline)
- SCENEGOD launch.json (informational; preview uses MESHGOD's entry)

SYNC 1 verified in-browser: character+backdrop+retargeted clip scene loads,
scrub interpolates frame-exact, scene save/load round-trips via server.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 18:36:16 +10:00
type-two
de3416869f [laneA] M1: Stage viewport, retarget port, dock+inspector
- room3d.js: pure-module port of rigroom retarget, target parameterized;
  _selftest bakes across two mismatched-rest skeletons (maxErr 6.4e-12)
- stage.js: Stage class (IBL, gizmo, entities, clip bake cache) per PLAN §4.2
- dock.js: asset dock + inspector, offline file-drop + clip retarget
- index.html/style.css: grid layout, guarded timeline import (page survives B absent)

Verified in-browser end-to-end: Mixamo char loaded+normalized, backdrop plane,
gizmo move, running clip retargeted (66 tracks) and animating live.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 18:06:22 +10:00
type-two
b20354c53f [laneB] log: M1 session 1
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 17:57:52 +10:00
type-two
45debd3c10 [laneB] M1: timeline model+clock, panel UI, stub, self-check
- timeline.js: headless clock/tracks/evaluate (transform slerp, param lerp,
  clip blocks with fade-out, camera cuts, undo stack). No three/DOM import.
- tlui.js: canvas panel — ruler scrub, playhead, key add/drag/delete, clip
  blocks, camera cuts, save/load with localStorage fallback. Injects own CSS.
- stagestub.js: dev Stage per PLAN §4.2 (deleted at SYNC 1).
- timeline_test.mjs: node self-check, all assertions pass.
- tldev.html: dev-only browser harness (deleted at SYNC 1).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 17:57:18 +10:00
type-two
f6292f2c84 [laneC] log: M1 done
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 17:54:31 +10:00
type-two
bb85eef040 [laneC] M1 server: statics, assets tree/file, scenes CRUD+validation, render stubs, tests
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 17:53:52 +10:00
type-two
e29edc808b plan: SCENEGOD master plan + 3-lane instructions + log scaffolding
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 17:44:07 +10:00