Lane A: grammar.js (shot/angle/focal/light/mark data), presets.js (bbox shot solver, light/mark appliers, DIRECT bar UI, NL box), stage video plates (VideoTexture plane/corner/dome, screen kind, syncVideos), dock video cards. Lane B: scenegod:direct → grouped-undo keys/cuts, 🗣 lip-sync from audio via /rhubarb, Cmd+wheel zoom / wheel scroll, audio trim, video clock-follow via nullable entityVideo seam. Lane C: POST /director (LLM proxy, server-side op validation vs vocab+scene, mock-tested), render.js awaits stage.syncVideos. Sync6 (orchestrator): VIDEO_EXTS in server tree/media types (video primary, img poster as thumb), stage.entityVideo accessor, syncVideos forces videoTex.needsUpdate (rVFC never fires in hidden tabs → black plates). Verified: 9 server groups + timeline + presets suites green; browser SYNC: one-click MCU framing lands camera+fov keys+cut (single undo), golden_hour relight keys, plate scrub-follow frame-exact, finalRender mp4 shows the Flow golden-hour street plate behind an animated character. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
163 lines
9.5 KiB
Markdown
163 lines
9.5 KiB
Markdown
# Lane C log — append-only, newest at bottom
|
|
|
|
Format per session:
|
|
|
|
## <date> session N
|
|
DONE: …
|
|
DECISIONS: …
|
|
BLOCKED/REQUESTS: …
|
|
NEXT: …
|
|
|
|
## 2026-07-18 session 1
|
|
DONE: M1 code-complete. Commit bb85eef.
|
|
- `requirements.txt` (fastapi, uvicorn only); `.gitignore` += `.venv/`.
|
|
- `scenegod/server.py` single file: statics (`/` no-store placeholder,
|
|
`/web/{path}` short-cache, path-guarded), `/assets/tree` (5s cache,
|
|
per-category grouping), `/assets/file` (traversal-guarded), `/scenes`
|
|
CRUD with atomic write + slugify + `validate_scene`, render endpoints
|
|
(begin/frame accept + save; end/status/out 501 until M3).
|
|
- `scripts/test_server.py`: 5 groups green (tree grouping, file stream,
|
|
4 traversal attempts rejected, scene round-trip w/ slugify, 5 validation
|
|
failures + 1 fade-overlap-allowed). `OK: 5 test groups passed`.
|
|
DECISIONS:
|
|
- System python3 is 3.14 (fastapi present but pydantic missing); made a
|
|
`.venv` on python3.11 per CLAUDE.md stack rule. Tests + server use
|
|
`.venv/bin/python`. `.venv/` gitignored.
|
|
- `web/` is under `scenegod/web/` per PLAN §2 (not repo root) — `WEB`
|
|
points there; Lane A/B statics resolve via `/web/{path}`.
|
|
- Asset grouping: per-category primary ext set (models for chars/anims/
|
|
props, images for backdrops, audio for audio). Non-primary image sharing
|
|
a stem → `thumb`, not a `format`. `path` prefers glb else first model.
|
|
Entries with no primary format dropped (thumb-only stems aren't assets).
|
|
- Clip-overlap check: violation if `next.start < a.end - a.fade`, where
|
|
`a.end = start + (out-in)*loop`. Fade-region overlap allowed.
|
|
BLOCKED/REQUESTS: none.
|
|
NEXT: M2 (when unlocked): scene-validation hardening + asset thumbnails
|
|
endpoint. M3 render pipeline (ffmpeg encode + web/render.js) is designed in
|
|
C-server.md §C5, endpoints already stubbed — implement at SYNC-2/M3.
|
|
|
|
## 2026-07-18 session 2
|
|
DONE: M2 thumbnails + M3 render pipeline (orchestrator green-lit both). Commit b627a0f.
|
|
- `/render/begin` (uuid dir + meta.json, 503 if no ffmpeg), `/frame/{n}`
|
|
(PNG->frames/{n:06d}.png, 400 on empty), `/end` (spawns bg ffmpeg thread:
|
|
libx264 crf18 yuv420p, 400 if no frames), `/status` (queued|encoding|
|
|
done|error+log tail; on-disk out.mp4 fallback after restart), `/out.mp4`.
|
|
- Boot-time `reap_renders()` deletes render dirs >48h.
|
|
- `/assets/thumb?path=` serves same-stem image sidecar (404 if none).
|
|
- `web/render.js`: draftRecord (MediaRecorder webm), finalRender
|
|
(timeline.step -> renderActiveCamera -> toBlob -> POST, <=4 in flight,
|
|
resize+restore renderer), download() helper. node --check clean.
|
|
- `test_server.py`: added render group (ffmpeg lavfi -> 30 PNGs -> begin ->
|
|
post -> end -> poll -> ffprobe duration ~1s) + thumbnail asserts.
|
|
`OK: 6 test groups passed`.
|
|
DECISIONS:
|
|
- Render state in-memory (single-process local tool); status() infers from
|
|
disk after a restart. ponytail-commented.
|
|
- Thumbnails pass-through, no resize — Pillow would be a new dep (stack rule
|
|
says log a REQUEST first); not worth it until dock load is measurably slow.
|
|
- finalRender total frames = round(duration*fps); relies on timeline.step
|
|
being deterministic (orchestrator verified).
|
|
BLOCKED/REQUESTS:
|
|
- finalRender's per-frame path depends on Lane A `stage.renderActiveCamera`
|
|
as a real render target (their M2). Server + ffmpeg path fully tested with
|
|
synthetic frames; the browser client path is unexercised until A lands it.
|
|
NEXT: M4 is locked. When unlocked: ffmpeg audio mux (scene audio[]),
|
|
/mb/submit MODELBEAST proxy, scripts/graft_limb.py. Meanwhile available to
|
|
integration-test finalRender once Lane A's renderActiveCamera lands.
|
|
|
|
## 2026-07-18 session 3
|
|
DONE: M4 complete (unlocked after SYNC 3). Commit be65355.
|
|
- Audio mux: `/render/end` now accepts `{audio:[{path,start,gain}]}`, muxes via
|
|
ffmpeg adelay+volume+amix -> aac -shortest; audio paths path-guarded (400 on
|
|
missing). `render.js finalRender` passes `opts.audio` through.
|
|
- `scripts/graft_limb.py`: headless Blender (`blender -b -P ... -- --body B
|
|
--limb L --bone NAME --out O`). Align limb root head->target bone world head,
|
|
join armatures, parent limb root under target (keep offset), join meshes
|
|
(vgroup weights survive). Refusal: overlapping bones -> GraftRefused ->
|
|
exit 2 "re-skin, don't graft". No finish/decimate. `--selftest` builds
|
|
synthetic body/limb GLBs and verifies refusal + graft paths under Blender.
|
|
- `/mb/submit`: MODELBEAST proxy, SCENEGOD_MB=1 gated (503 otherwise), token
|
|
from ~/Documents/backnforth/.env at request time (never logged; errors
|
|
scrub it), /api/assets multipart -> /api/jobs, mirrors meshgod/ops.py.
|
|
- `test_server.py`: 8 groups green (added audio-stream assert, MB gating,
|
|
MB two-step contract via in-process mock — no farm jobs).
|
|
DECISIONS:
|
|
- Blender 5 glTF importer injects a stray unparented 42-vert icosphere on
|
|
every import; `_import` now keeps only rig-connected meshes (parented to
|
|
armature / has vgroups / armature modifier) and deletes strays. Right filter
|
|
for a graft tool regardless.
|
|
- graft refusal check: provides = limb bones - {limb root}; refuse if
|
|
provides ∩ body bones. Attach/root bone sharing the target name is expected.
|
|
- MB test uses a stdlib http.server mock + a 2nd scenegod instance; asserts
|
|
Bearer token forwarded, multipart upload, asset_id chained into job payload.
|
|
BLOCKED/REQUESTS: none.
|
|
BACKLOG (from orchestrator, not urgent): draft webm via MediaRecorder is
|
|
untested in the embedded preview (rAF suspension). Needs one verification in a
|
|
real Chrome window before calling draftRecord done — client-only, needs Lane A
|
|
stage running live; deferred.
|
|
NEXT: Lane C M1-M4 all shipped. Available for polish / integration support.
|
|
Blender binary here is /Applications/Blender.app/Contents/MacOS/Blender (not
|
|
on PATH as `blender` in this session, contrary to orchestrator note).
|
|
|
|
## 2026-07-18 session 3b
|
|
DONE: optional item from orchestrator — `/rhubarb?path=` endpoint. Commit fb59601.
|
|
- `GET /rhubarb?path=` (path-guarded) -> `rhubarb -f json <wav>` -> returns
|
|
`{metadata, mouthCues:[{start,end,value}]}` for Lane A's viseme lane.
|
|
503 with install hint when the binary is absent (ffmpeg-style gate).
|
|
- test_server.py: +503-when-absent assert (9 groups green).
|
|
DECISIONS: synchronous subprocess (VO clips are short); rhubarb NOT installed
|
|
on ultra, so only the 503 path is exercised here — happy path (JSON parse) is
|
|
untested until `rhubarb` is on PATH. Flagged same as MediaRecorder backlog.
|
|
BLOCKED/REQUESTS: none. To fully verify: install rhubarb-lip-sync on ultra,
|
|
then GET /rhubarb?path=audio/<some>.wav.
|
|
NEXT: nothing outstanding — Lane C M1-M4 + rhubarb shipped. Standing by.
|
|
|
|
## 2026-07-20 session 4
|
|
DONE: M5 `/director` + M6 render.js sliver.
|
|
- `scenegod/server.py`: `POST /director` {text, scene:{entities,time}} ->
|
|
OpenAI-compat chat (`SCENEGOD_LLM_URL` + `/v1/chat/completions`, model
|
|
`SCENEGOD_LLM_MODEL` default `qwen2.5:7b`, temp 0.1, response_format
|
|
json_object, stdlib urllib, 120s timeout). 503 + env hint when URL unset;
|
|
502 on LLM error/unreachable/unparseable output — raw model text is NEVER
|
|
forwarded. `_validate_ops`: ops shot/light/mark checked against vocab +
|
|
scene entity ids; violations dropped into `rejected` [{op, reason}];
|
|
response {ops, rejected}. Shot defaults angle="eye", focal=35 filled
|
|
server-side.
|
|
- Vocab (SHOTS/ANGLES/FOCALS/LIGHTS/MARKS) defined ONCE at top of the
|
|
director section, mirroring PLAN §5 M5.
|
|
- `web/render.js` finalRender: after `timeline.step(f)`, before capture:
|
|
`if (stage.syncVideos) await stage.syncVideos(timeline.time)` — guarded
|
|
no-op until Lane A lands it (M6 determinism hook).
|
|
- `scripts/test_server.py`: +director group (in-process http.server LLM mock,
|
|
MB-mock pattern): 503 when env unset, 3 valid ops pass through verbatim,
|
|
unknown op / unknown entity id / unknown preset land in `rejected` with
|
|
reasons, prompt carries 10 vocab words + roster ids + playhead, model name
|
|
+ low temp asserted. `OK: 9 test groups passed` (rhubarb-503 group now
|
|
self-skips — rhubarb is installed on ultra, so that group's absent-path
|
|
doesn't run; happy path was orchestrator-verified live).
|
|
DECISIONS:
|
|
- `web/grammar.js` does NOT exist yet (no presets.js either) — vocab lives in
|
|
server.py per spec fallback. **AT SYNC: re-point the SHOTS/ANGLES/FOCALS/
|
|
LIGHTS/MARKS lists + DIRECTOR_SYS at Lane A's grammar.js (read at boot)**
|
|
so server prompt and client presets can't drift. Op shapes I defined
|
|
(scenegod:direct): {op:"shot",subject,shot,angle,focal} /
|
|
{op:"light",preset} / {op:"mark",subject,mark,target?} — Lane A's
|
|
presets.js appliers must accept exactly these.
|
|
- Model may return `{"ops":[...]}` or a bare array — both accepted
|
|
(json_object mode forces an object wrapper; bare array tolerated for
|
|
servers that ignore response_format).
|
|
- Env read at request time (matches _mb_token pattern) so John can set
|
|
SCENEGOD_LLM_URL without a restart-order footgun; test env explicitly
|
|
pops it for the 503 assert.
|
|
- focal coerced via int() before vocab check (7b models love "35" strings).
|
|
BLOCKED/REQUESTS:
|
|
- Lane A: need `web/grammar.js` (data only) + presets.js appliers matching
|
|
the op shapes above; flag any shape disagreement at SYNC and I'll conform.
|
|
- Live end-to-end against real qwen2.5:7b on m4pro untested this session
|
|
(SCENEGOD_LLM_URL unset locally by design) — orchestrator: one smoke call
|
|
at SYNC with env set is the acceptance step.
|
|
NEXT: M6 FLOW VIDEO PLATES server side (unlocked on this acceptance):
|
|
flow_intake.py poster-jpg intake is already in scripts/; verify video exts
|
|
in /assets/tree (mp4 not in ASSET_EXTS yet — backdrops video category will
|
|
need it) + deterministic render check once Lane A's stage.syncVideos lands.
|