22c8a9a324
FEATURES (from a 5-lens audit: UX friction, feature gaps, bug hunt, visual quality, discoverability): - ACES filmic tone mapping + per-preset exposure. grammar.js runs physical sun intensities of 1.2-3.4; with NoToneMapping every lit face clipped to white and a warm key rendered neutral. Single biggest look win in the app. - Real shadows: PCFSoft, a shadow box fitted to the scene's bounds, and plates that no longer double as the shadow catcher (dedicated ShadowMaterial). - A Render button in the scene bar. finalRender had ZERO callers - the app's deliverable was console-only. - Save actually saves the stage you built (gizmo moves, inspector edits), and Save/Load stop lying about what happened. - 2x supersample + lanczos downscale; correct bt709 tagging and faststart. - Frame guide, camera-from-view, dock counts, delete confirmations. DEFECTS FOUND BY ADVERSARIAL REVIEW AND FIXED (all reproduced first): - BLOCKER: save wrote the playhead pose over the authored rest transform of keyed entities, so every save produced a different file. - renders were converted with the bt601 matrix while tagged bt709. - deleting an unkeyed camera stranded its cut -> scene 422s forever. - corner plates showed one photo at two exposures across the fold. - exposure was advertised as keyable but nothing keyed it, so a second lighting preset permanently poisoned the first. - the audio pre-flight could never fire (FastAPI does not route HEAD -> 405). - the two render buttons were not mutually exclusive. - frame guide letterboxed a narrow viewport that the render does not crop. ORCHESTRATOR (round 3): dangling camera cuts are pruned in Timeline.toJSON and skipped by cutCameraAt, rather than trying to keep every undo history clean - deleting a camera spans Stage (no undo) and Timeline (undo), so any older entry can resurrect a cut for a dead camera. Verified against the real validator. Also replaced a VACUOUS test that drained a stack whose only entry was a seeded no-op; timeline_test.mjs now replays both real histories and discriminates. Verified: 4 JS suites + 23 server groups + render client green; blocker repro now preserves the authored rest at every playhead position; save round-trips 200 with zero dangling cuts; zero console errors. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
743 lines
52 KiB
Markdown
743 lines
52 KiB
Markdown
# Lane C log — append-only, newest at bottom
|
||
|
||
Format per session:
|
||
|
||
## <date> session N
|
||
DONE: …
|
||
DECISIONS: …
|
||
BLOCKED/REQUESTS: …
|
||
NEXT: …
|
||
|
||
## 2026-07-18 session 1
|
||
DONE: M1 code-complete. Commit bb85eef.
|
||
- `requirements.txt` (fastapi, uvicorn only); `.gitignore` += `.venv/`.
|
||
- `scenegod/server.py` single file: statics (`/` no-store placeholder,
|
||
`/web/{path}` short-cache, path-guarded), `/assets/tree` (5s cache,
|
||
per-category grouping), `/assets/file` (traversal-guarded), `/scenes`
|
||
CRUD with atomic write + slugify + `validate_scene`, render endpoints
|
||
(begin/frame accept + save; end/status/out 501 until M3).
|
||
- `scripts/test_server.py`: 5 groups green (tree grouping, file stream,
|
||
4 traversal attempts rejected, scene round-trip w/ slugify, 5 validation
|
||
failures + 1 fade-overlap-allowed). `OK: 5 test groups passed`.
|
||
DECISIONS:
|
||
- System python3 is 3.14 (fastapi present but pydantic missing); made a
|
||
`.venv` on python3.11 per CLAUDE.md stack rule. Tests + server use
|
||
`.venv/bin/python`. `.venv/` gitignored.
|
||
- `web/` is under `scenegod/web/` per PLAN §2 (not repo root) — `WEB`
|
||
points there; Lane A/B statics resolve via `/web/{path}`.
|
||
- Asset grouping: per-category primary ext set (models for chars/anims/
|
||
props, images for backdrops, audio for audio). Non-primary image sharing
|
||
a stem → `thumb`, not a `format`. `path` prefers glb else first model.
|
||
Entries with no primary format dropped (thumb-only stems aren't assets).
|
||
- Clip-overlap check: violation if `next.start < a.end - a.fade`, where
|
||
`a.end = start + (out-in)*loop`. Fade-region overlap allowed.
|
||
BLOCKED/REQUESTS: none.
|
||
NEXT: M2 (when unlocked): scene-validation hardening + asset thumbnails
|
||
endpoint. M3 render pipeline (ffmpeg encode + web/render.js) is designed in
|
||
C-server.md §C5, endpoints already stubbed — implement at SYNC-2/M3.
|
||
|
||
## 2026-07-18 session 2
|
||
DONE: M2 thumbnails + M3 render pipeline (orchestrator green-lit both). Commit b627a0f.
|
||
- `/render/begin` (uuid dir + meta.json, 503 if no ffmpeg), `/frame/{n}`
|
||
(PNG->frames/{n:06d}.png, 400 on empty), `/end` (spawns bg ffmpeg thread:
|
||
libx264 crf18 yuv420p, 400 if no frames), `/status` (queued|encoding|
|
||
done|error+log tail; on-disk out.mp4 fallback after restart), `/out.mp4`.
|
||
- Boot-time `reap_renders()` deletes render dirs >48h.
|
||
- `/assets/thumb?path=` serves same-stem image sidecar (404 if none).
|
||
- `web/render.js`: draftRecord (MediaRecorder webm), finalRender
|
||
(timeline.step -> renderActiveCamera -> toBlob -> POST, <=4 in flight,
|
||
resize+restore renderer), download() helper. node --check clean.
|
||
- `test_server.py`: added render group (ffmpeg lavfi -> 30 PNGs -> begin ->
|
||
post -> end -> poll -> ffprobe duration ~1s) + thumbnail asserts.
|
||
`OK: 6 test groups passed`.
|
||
DECISIONS:
|
||
- Render state in-memory (single-process local tool); status() infers from
|
||
disk after a restart. ponytail-commented.
|
||
- Thumbnails pass-through, no resize — Pillow would be a new dep (stack rule
|
||
says log a REQUEST first); not worth it until dock load is measurably slow.
|
||
- finalRender total frames = round(duration*fps); relies on timeline.step
|
||
being deterministic (orchestrator verified).
|
||
BLOCKED/REQUESTS:
|
||
- finalRender's per-frame path depends on Lane A `stage.renderActiveCamera`
|
||
as a real render target (their M2). Server + ffmpeg path fully tested with
|
||
synthetic frames; the browser client path is unexercised until A lands it.
|
||
NEXT: M4 is locked. When unlocked: ffmpeg audio mux (scene audio[]),
|
||
/mb/submit MODELBEAST proxy, scripts/graft_limb.py. Meanwhile available to
|
||
integration-test finalRender once Lane A's renderActiveCamera lands.
|
||
|
||
## 2026-07-18 session 3
|
||
DONE: M4 complete (unlocked after SYNC 3). Commit be65355.
|
||
- Audio mux: `/render/end` now accepts `{audio:[{path,start,gain}]}`, muxes via
|
||
ffmpeg adelay+volume+amix -> aac -shortest; audio paths path-guarded (400 on
|
||
missing). `render.js finalRender` passes `opts.audio` through.
|
||
- `scripts/graft_limb.py`: headless Blender (`blender -b -P ... -- --body B
|
||
--limb L --bone NAME --out O`). Align limb root head->target bone world head,
|
||
join armatures, parent limb root under target (keep offset), join meshes
|
||
(vgroup weights survive). Refusal: overlapping bones -> GraftRefused ->
|
||
exit 2 "re-skin, don't graft". No finish/decimate. `--selftest` builds
|
||
synthetic body/limb GLBs and verifies refusal + graft paths under Blender.
|
||
- `/mb/submit`: MODELBEAST proxy, SCENEGOD_MB=1 gated (503 otherwise), token
|
||
from ~/Documents/backnforth/.env at request time (never logged; errors
|
||
scrub it), /api/assets multipart -> /api/jobs, mirrors meshgod/ops.py.
|
||
- `test_server.py`: 8 groups green (added audio-stream assert, MB gating,
|
||
MB two-step contract via in-process mock — no farm jobs).
|
||
DECISIONS:
|
||
- Blender 5 glTF importer injects a stray unparented 42-vert icosphere on
|
||
every import; `_import` now keeps only rig-connected meshes (parented to
|
||
armature / has vgroups / armature modifier) and deletes strays. Right filter
|
||
for a graft tool regardless.
|
||
- graft refusal check: provides = limb bones - {limb root}; refuse if
|
||
provides ∩ body bones. Attach/root bone sharing the target name is expected.
|
||
- MB test uses a stdlib http.server mock + a 2nd scenegod instance; asserts
|
||
Bearer token forwarded, multipart upload, asset_id chained into job payload.
|
||
BLOCKED/REQUESTS: none.
|
||
BACKLOG (from orchestrator, not urgent): draft webm via MediaRecorder is
|
||
untested in the embedded preview (rAF suspension). Needs one verification in a
|
||
real Chrome window before calling draftRecord done — client-only, needs Lane A
|
||
stage running live; deferred.
|
||
NEXT: Lane C M1-M4 all shipped. Available for polish / integration support.
|
||
Blender binary here is /Applications/Blender.app/Contents/MacOS/Blender (not
|
||
on PATH as `blender` in this session, contrary to orchestrator note).
|
||
|
||
## 2026-07-18 session 3b
|
||
DONE: optional item from orchestrator — `/rhubarb?path=` endpoint. Commit fb59601.
|
||
- `GET /rhubarb?path=` (path-guarded) -> `rhubarb -f json <wav>` -> returns
|
||
`{metadata, mouthCues:[{start,end,value}]}` for Lane A's viseme lane.
|
||
503 with install hint when the binary is absent (ffmpeg-style gate).
|
||
- test_server.py: +503-when-absent assert (9 groups green).
|
||
DECISIONS: synchronous subprocess (VO clips are short); rhubarb NOT installed
|
||
on ultra, so only the 503 path is exercised here — happy path (JSON parse) is
|
||
untested until `rhubarb` is on PATH. Flagged same as MediaRecorder backlog.
|
||
BLOCKED/REQUESTS: none. To fully verify: install rhubarb-lip-sync on ultra,
|
||
then GET /rhubarb?path=audio/<some>.wav.
|
||
NEXT: nothing outstanding — Lane C M1-M4 + rhubarb shipped. Standing by.
|
||
|
||
## 2026-07-20 session 4
|
||
DONE: M5 `/director` + M6 render.js sliver.
|
||
- `scenegod/server.py`: `POST /director` {text, scene:{entities,time}} ->
|
||
OpenAI-compat chat (`SCENEGOD_LLM_URL` + `/v1/chat/completions`, model
|
||
`SCENEGOD_LLM_MODEL` default `qwen2.5:7b`, temp 0.1, response_format
|
||
json_object, stdlib urllib, 120s timeout). 503 + env hint when URL unset;
|
||
502 on LLM error/unreachable/unparseable output — raw model text is NEVER
|
||
forwarded. `_validate_ops`: ops shot/light/mark checked against vocab +
|
||
scene entity ids; violations dropped into `rejected` [{op, reason}];
|
||
response {ops, rejected}. Shot defaults angle="eye", focal=35 filled
|
||
server-side.
|
||
- Vocab (SHOTS/ANGLES/FOCALS/LIGHTS/MARKS) defined ONCE at top of the
|
||
director section, mirroring PLAN §5 M5.
|
||
- `web/render.js` finalRender: after `timeline.step(f)`, before capture:
|
||
`if (stage.syncVideos) await stage.syncVideos(timeline.time)` — guarded
|
||
no-op until Lane A lands it (M6 determinism hook).
|
||
- `scripts/test_server.py`: +director group (in-process http.server LLM mock,
|
||
MB-mock pattern): 503 when env unset, 3 valid ops pass through verbatim,
|
||
unknown op / unknown entity id / unknown preset land in `rejected` with
|
||
reasons, prompt carries 10 vocab words + roster ids + playhead, model name
|
||
+ low temp asserted. `OK: 9 test groups passed` (rhubarb-503 group now
|
||
self-skips — rhubarb is installed on ultra, so that group's absent-path
|
||
doesn't run; happy path was orchestrator-verified live).
|
||
DECISIONS:
|
||
- `web/grammar.js` does NOT exist yet (no presets.js either) — vocab lives in
|
||
server.py per spec fallback. **AT SYNC: re-point the SHOTS/ANGLES/FOCALS/
|
||
LIGHTS/MARKS lists + DIRECTOR_SYS at Lane A's grammar.js (read at boot)**
|
||
so server prompt and client presets can't drift. Op shapes I defined
|
||
(scenegod:direct): {op:"shot",subject,shot,angle,focal} /
|
||
{op:"light",preset} / {op:"mark",subject,mark,target?} — Lane A's
|
||
presets.js appliers must accept exactly these.
|
||
- Model may return `{"ops":[...]}` or a bare array — both accepted
|
||
(json_object mode forces an object wrapper; bare array tolerated for
|
||
servers that ignore response_format).
|
||
- Env read at request time (matches _mb_token pattern) so John can set
|
||
SCENEGOD_LLM_URL without a restart-order footgun; test env explicitly
|
||
pops it for the 503 assert.
|
||
- focal coerced via int() before vocab check (7b models love "35" strings).
|
||
BLOCKED/REQUESTS:
|
||
- Lane A: need `web/grammar.js` (data only) + presets.js appliers matching
|
||
the op shapes above; flag any shape disagreement at SYNC and I'll conform.
|
||
- Live end-to-end against real qwen2.5:7b on m4pro untested this session
|
||
(SCENEGOD_LLM_URL unset locally by design) — orchestrator: one smoke call
|
||
at SYNC with env set is the acceptance step.
|
||
NEXT: M6 FLOW VIDEO PLATES server side (unlocked on this acceptance):
|
||
flow_intake.py poster-jpg intake is already in scripts/; verify video exts
|
||
in /assets/tree (mp4 not in ASSET_EXTS yet — backdrops video category will
|
||
need it) + deterministic render check once Lane A's stage.syncVideos lands.
|
||
|
||
## 2026-07-25 session 5
|
||
DONE: M7-C starter templates — server side complete, 10 test groups green.
|
||
- `templates/` at repo root (NOT gitignored — ships with the code, unlike
|
||
assets/scenes/renders). Three files, all valid against the live
|
||
`validate_scene`, every referenced path verified present in `assets/`:
|
||
- `templates/two-hander-golden-hour.json` — lady + man facing each other
|
||
(yaw +0.55 / -0.6; characters face +Z at yaw 0 per presets.js), Flow
|
||
golden-hour street plate (`mode:"video"`, `fit:"frustum"`, `groundTint`),
|
||
grammar.js `golden_hour` key+ambient verbatim (#ffb469 @2.6 / #ffd9b0 @0.55,
|
||
bg #d98a52, fog 0.02), cam1 "wide" fov40 @5m + cam2 "closer" fov32 @3.4m,
|
||
cut at t=4.5. 8s. Both carry `carrying.fbx`; lady's block is trimmed
|
||
`in:2,out:10` so the two performances don't move in lockstep.
|
||
- `templates/record-store.json` — record-store interior plate, man +
|
||
carrying.fbx, one camera fov38, warm key + ambient (fog 0.012), 6s.
|
||
- `templates/empty-stage.json` — one camera, `day` key+ambient, 10s, no
|
||
characters, no plate. The clean start.
|
||
- `scenegod/server.py:31` TEMPLATES root (`SCENEGOD_TEMPLATES`, default
|
||
`./templates`); `:261-293` the read-only endpoints: `GET /templates` ->
|
||
`[{name,title,description,thumb?}]`, `GET /templates/{name}` -> scene JSON,
|
||
both through `safe_under` + `slugify` (same guard discipline as /scenes).
|
||
No POST/DELETE by design — load a template, then `POST /scenes/{name}`.
|
||
- `scripts/test_server.py:137` templates group: lists exactly the three,
|
||
order-leads with the two-hander, round-trips each one, rejects
|
||
`../../etc/hosts` / encoded `..` / absolute / unknown (all >=400), write
|
||
verbs 404/405, and — the check that stops a rotten template shipping —
|
||
POSTs every template through the real `validate_scene` and asserts every
|
||
`source.path` / `params.video|image` / `clips[].path` / `audio[].path` /
|
||
thumb exists under `assets/`, plus camera+light `source.type == "none"`.
|
||
`OK: 10 test groups passed`.
|
||
- `.gitignore`: `*.log` (a 65KB `scenegod-8020.log` was sitting untracked at
|
||
repo root, one `git add` away from the repo).
|
||
DECISIONS:
|
||
- Presentation metadata rides INSIDE the scene JSON under a `template` key
|
||
({title, description, thumb, order}) rather than a sidecar or a filename
|
||
convention: `validate_scene` ignores unknown top-level keys and
|
||
`Timeline.load/toJSON` doesn't carry it, so a saved scene comes out as plain
|
||
scene JSON with no template residue. `order` is server-side sort only (not in
|
||
the response) so the picker leads with the wow template, not alphabetically.
|
||
- Verified before shipping, not assumed: lady.glb is a **CC_Base** rig (103
|
||
`CC_Base` strings, 0 mixamo) — safe only because Lane A's M7-A alias layer
|
||
landed; `_normalize` scales both characters to 1.7m with feet at y=0, so the
|
||
camera heights/fovs above frame them for real (wide vspan 3.65m, closer
|
||
1.97m). Both clips probed under headless Blender: carrying.fbx = 10.0s and
|
||
hiphop.fbx = 4.467s, and **both are in-place** (hips travel 10cm / 2cm), so
|
||
no template performer can wander out of frame.
|
||
- Video plates follow the orchestrator shape (`params.video`, no `image`):
|
||
`_buildBackdrop` reads `p.image || en.source.path`, and `source.path` is the
|
||
same mp4, so the fallback carries it. If Lane A ever drops that fallback,
|
||
add `image` alongside `video`.
|
||
- Cameras/lights get `source:{type:"none"}` (sync2-sunset's `"upload"` was the
|
||
bug). Light `rot` is not decorative — `_buildLight` aims a directional down
|
||
the wrapper's -Z, so every key light's euler is `lookAtEuler(pos,[0,1,0])`
|
||
computed with presets.js's own algebra.
|
||
BLOCKED/REQUESTS:
|
||
- @orchestrator: the uvicorn on :8020 is NOT `--reload` — it still 404s
|
||
`/templates`. Restart it before browser-checking.
|
||
- @Lane B (M7-B): the picker contract is `GET /templates` ->
|
||
`[{name,title,description,thumb?}]` (already ordered; `thumb` is
|
||
asset-relative -> `assets/file?path=` or `assets/thumb?path=`), then
|
||
`GET /templates/{name}` -> feed straight to `timeline.load()` /
|
||
`stage.applyState()`, and save with the existing `POST /scenes/{name}`.
|
||
NEXT: nothing outstanding in Lane C. Backlog if wanted: a fourth template
|
||
once a prop/screen asset bank exists, and the still-unverified MediaRecorder
|
||
draft path (carried from session 3).
|
||
|
||
## 2026-07-25 session 6
|
||
DONE: **M8-C MULTI-SHOT SEQUENCING** — the film-level object. 15 test groups green
|
||
(was 10). A scene is one take; a *sequence* is an ordered list of trimmed scene
|
||
windows joined by cuts/dissolves over a film-level audio bed.
|
||
- **Sequence doc + validator** — `scenegod/server.py:443 plan_sequence()` validates
|
||
AND resolves in ONE pass (save-time validation and render-time planning can never
|
||
disagree). Roots at `:32-33` (`SCENEGOD_SEQUENCES` -> ./sequences, `SCENEGOD_FILMS`
|
||
-> ./films, both mkdir'd on boot like scenes/renders). Violations, all 422 with a
|
||
message per offence: version!=1, empty/oversized shots (max 200), unknown scene,
|
||
bad scene name, in/out not numbers, in<0, in>=out, out past the scene's duration,
|
||
unknown transition, transitionDur out of 0..10s, dissolve with dur<=0, dissolve not
|
||
shorter than BOTH shots it joins (ffmpeg xfade requirement), scenes with mixed fps
|
||
("one film, one fps"), audio path traversal / missing file / bad start|gain.
|
||
- **Endpoints** `:563-800`: `GET /sequences` (name,title,mtime,shots,duration,ok),
|
||
`GET|POST|DELETE /sequences/{name}` (slugify + safe_under + capped 1MB + atomic
|
||
write), `POST /sequences/{name}/render` -> `{filmId,fps,width,height,duration,
|
||
warnings,shots[]}`, `POST /films/{id}/shot/{i} {renderId}`, `POST /films/{id}/end`,
|
||
`GET /films/{id}/status` -> `{state,shot,of,log}`, `GET /films/{id}/out.mp4`.
|
||
States: collecting -> queued -> encoding -> done|error.
|
||
- **The film render is a client-driven job** (`:648 _concat_film`): the browser is the
|
||
only thing that can render a scene, so the server hands out the resolved shot plan,
|
||
the client renders each shot through the UNCHANGED `/render/*` session endpoints and
|
||
reports renderId-per-index, and `/end` folds the shot mp4s with ONE ffmpeg
|
||
filter_complex: each input `scale=WxH,setsar=1,fps,format=yuv420p`, then a left fold
|
||
— cut = `concat=n=2:v=1:a=0`, dissolve = `xfade=transition=fade:duration=D:offset=`
|
||
(running accumulated length - D). Film-level `audio[]` -> adelay/volume/amix/apad,
|
||
`-shortest`. Film dirs reaped >48h (`:75 reap_dirs(root)`, now called for RENDERS
|
||
and FILMS).
|
||
- **`scenegod/web/film.js`** (new): `renderSequence(stage, timeline, seqName, opts)`
|
||
drives the whole loop — POST render -> per shot `GET scenes/{scene}` ->
|
||
`timeline.applyScene(json)` (Lane B's single apply path) -> `captureFrames` over
|
||
frames [in*fps, out*fps) -> end+poll the shot -> `POST films/{id}/shot/{i}` ->
|
||
`films/{id}/end` -> poll -> resolve with `films/{id}/out.mp4`. Plus
|
||
list/get/save/deleteSequence helpers so a UI lane never hand-rolls (or absolutises)
|
||
the URLs. Relative URLs only.
|
||
- **`web/render.js` refactored, not duplicated**: the frame loop is now
|
||
`captureFrames(stage,timeline,{renderId,from,to,onProgress})` + `withRenderSize` /
|
||
`beginRender` / `endRender` / `pollRender` / `postJSON` / `getJSON`, all exported;
|
||
`finalRender(stage,timeline,opts)` keeps its exact old signature and is now 6 lines
|
||
built from them. film.js imports them — one frame-post path in the repo.
|
||
- **`sequences/demo-two-shot.json`**: 3 shots, 11.5s — sync2-sunset 0..4.5 (cam1)
|
||
dissolve 0.5s into sync2-sunset 4.5..8.0 (cam2, same scene, other side of its cut),
|
||
cut to sync1-demo 0..4.0. Verified live: `GET /sequences` -> ok:true duration 11.5,
|
||
`POST /sequences/demo-two-shot/render` -> the 3-shot plan, and a full fake-shot
|
||
end-to-end (lavfi mp4s registered through the real endpoints) produced out.mp4 at
|
||
exactly 11.500000s.
|
||
- **`scripts/test_server.py`**: +5 groups (`:225` CRUD round-trip incl. slugify +
|
||
lossless read-back + DELETE 200/404; `:248` 13 validation failures each asserted for
|
||
its own message + traversal/unknown guards on GET and DELETE; `:281` film smoke —
|
||
two lavfi mp4s registered as shots, end, poll, ffprobe duration 5.0s = 2+3, output
|
||
64x64, plus guards: end-before-registration 400, bad index 400, traversal renderId
|
||
400, unknown renderId 400, 32x32 shot -> 400 "one resolution per film", unknown
|
||
filmId 404; `:340` dissolve variant -> 4.5s = 2+3-0.5 with an audio bed muxed;
|
||
`:370` the shipped example can't rot — every shot's scene must exist and every
|
||
`out` must be within that scene's duration). `OK: 15 test groups passed`.
|
||
DECISIONS:
|
||
- **One fps per film, validated** rather than resampling: `timeline.step(frame)` is
|
||
the deterministic seek and it divides by the SCENE's fps, so a film whose fps
|
||
differed from a scene's would silently render the wrong instants. Mixed-fps
|
||
sequences are a clear 422 and the film's fps comes from the scenes, not the client.
|
||
Client passes width/height only.
|
||
- **One resolution per film, enforced at registration** (compare `renders/{rid}/
|
||
meta.json` w/h/fps to the film's) so the failure is a clear 400 the moment it
|
||
happens, not a cryptic ffmpeg filter error after the last shot. The scale/setsar
|
||
filter stays as the belt to that braces (renders with no meta.json).
|
||
- **xfade offsets come from ffprobe'd durations** (`:636 _probe_dur`, nominal length
|
||
as fallback): nominal length can differ from the encoded mp4 by up to a frame, and
|
||
an absolute xfade offset that's off by a frame shows as a stutter at the join.
|
||
- **Shots render silent; the soundtrack is film-level.** Concat with per-shot audio
|
||
needs every shot to have an audio stream (mixed presence breaks the filter), and a
|
||
film's bed rarely respects shot boundaries anyway. Not silent-by-surprise: the
|
||
render-start response carries a `warnings[]` naming any referenced scene whose
|
||
`audio[]` won't be carried, and film.js logs them.
|
||
- Existing `/render/end` mux gained `apad` before `-shortest` (`:326`) — a VO shorter
|
||
than the shot was truncating the PICTURE. Same fix in the film mux; the M4 audio
|
||
assert still passes.
|
||
- `sequences/` ships (like `templates/`) but is user-writable, so `.gitignore` takes
|
||
`sequences/*` + `!sequences/demo-two-shot.json`: the example is tracked, John's own
|
||
sequences stay out of `git status`. `films/` fully ignored.
|
||
- Film plan + shot registrations persist in `films/{id}/meta.json` (atomic tmp+rename),
|
||
not just memory: a film spans minutes of client rendering and `/end` must survive a
|
||
server restart underneath it. Only the encode state/log is in-memory (renders model).
|
||
BLOCKED/REQUESTS:
|
||
- @orchestrator (deploy/, not my lane) — two one-line prod fixes so sequences persist:
|
||
1. `deploy/docker-compose.yml`: add `- SCENEGOD_SEQUENCES=/app/sequences` and
|
||
`- SCENEGOD_FILMS=/app/films` to environment, and
|
||
`- /opt/scenegod-data/sequences:/app/sequences` +
|
||
`- /opt/scenegod-data/films:/app/films` to volumes.
|
||
2. `deploy/deploy.sh`: `mkdir -p /opt/scenegod-data/{assets,scenes,renders,sequences,films}`
|
||
and add `--exclude films` to the rsync.
|
||
Without these the app still boots and works (it mkdirs both dirs inside the
|
||
container) but a rebuild eats every saved sequence. Nothing else new is needed in
|
||
prod: no LLM/MB env, ffmpeg is already in the image.
|
||
- @orchestrator — the :8020 uvicorn is not `--reload`; restart it before browser-driving
|
||
the film (it will 404 `/sequences`). Test instances only ever ran on 8097/8099.
|
||
- @Lane B (whenever it's unlocked) — the UI contract is ready and stable:
|
||
`import { renderSequence, listSequences, getSequence, saveSequence, deleteSequence }
|
||
from './web/film.js'`. A sequence editor is a list of `{scene,in,out,transition,
|
||
transitionDur}` rows over `GET /scenes`; `renderSequence(stage, timeline, name,
|
||
{width, height, onShot, onProgress})` does the rest and resolves with the mp4 URL.
|
||
NOTE it calls `timeline.applyScene()` per shot, so it CLOBBERS the loaded scene —
|
||
the UI should confirm via `timeline.hasContent()` first, same as new-from-template.
|
||
NEXT: nothing outstanding in Lane C. Backlog: per-scene audio carried into a film
|
||
(auto-derive the bed from each shot's scene `audio[]` offset into film time — needs a
|
||
silence-input path for shots with no audio); more transition types (xfade has ~50,
|
||
`wipeleft`/`slideup` are one vocab entry each); still-unverified MediaRecorder draft
|
||
path (carried from session 3).
|
||
|
||
## 2026-07-25 session 7
|
||
DONE: **M9-C1 audio trim at the mux** + **M9-C2 BEAT DETECTION**. 18 test groups green
|
||
(was 15). `OK: 18 test groups passed`, whole suite ~6.5s.
|
||
|
||
**M9-C1 — `in`/`out` honoured in BOTH muxes.**
|
||
- `scenegod/server.py:112 resolve_audio()` — ONE function now turns an
|
||
`audio[]` entry into a muxable dict for both lanes (`/render/{id}/end` at :441
|
||
and `plan_sequence` at :562), so a clip means the same thing in a shot and in a
|
||
film. Validates path (guarded), `start`/`gain`/`in`/`out` types, `in>=0`,
|
||
`in<out`. `_num` moved up to :103 next to `safe_under` for the same reason.
|
||
- `:364 audio_input()` puts the trim on the INPUT (`-ss in -t out-in`) and
|
||
`:381 audio_filters()` keeps the adelay/volume/amix/apad chain EXACTLY as
|
||
shipped. Both muxes call the pair (`:401` render, `:718` film).
|
||
- Absent `in`/`out` = byte-identical ffmpeg command to before (proved by the
|
||
untrimmed A/B in the test).
|
||
- Verified end-to-end, not by eyeballing the command line: `scripts/test_server.py:250`
|
||
renders 3s of picture over `trim.wav` (1s silence, then 1s of 880Hz).
|
||
Trimmed to the tone at `start:0.5` -> `volumedetect` reads -91dB at 0.0-0.4s,
|
||
**-6.1dB at 0.6-1.4s**, -91dB at 1.7-2.7s. The SAME clip untrimmed reads -91dB
|
||
at 0.6-1.4 and -6.1dB at 1.6-2.4 — i.e. the trim moved the sound, and it is not
|
||
a tautological test. `:429` does the same for the film-level bed (silent to
|
||
1.8s, tone at 2.1s) inside the existing dissolve film.
|
||
DECISIONS (M9-C1):
|
||
- **`-ss`/`-t` on the input, NOT `atrim` in the filter graph** — this is a bug
|
||
fix, not taste. `atrim,asetpts,adelay,...,amix,apad` on ffmpeg 8.x
|
||
(homebrew, 62.x libs) emits pad frames at ~INT64_MAX PTS; the muxer drops all
|
||
of them and you get an mp4 with an aac stream and **zero audio samples**, rc=0,
|
||
no warning. Reproduced 4 ways: old chain OK, +atrim broken, +atrim without apad
|
||
OK, +atrim with `apad=whole_dur=N` OK. Input seeking leaves the proven graph
|
||
untouched. (Sample-exact for wav; a few ms at worst for mp3.)
|
||
- `scripts/test_server.py:59 seg_db()` REFUSES to answer when a window decoded no
|
||
samples, instead of returning -99. Otherwise every "assert it's silent here"
|
||
passes on a stream that is silent because it is empty — which is exactly the
|
||
failure above, and it would have shipped green.
|
||
- A trim window past EOF made ffmpeg die mid-mux (`-22 Error muxing a packet` ->
|
||
a mystery failed render), so `resolve_audio` ffprobes the source **only when a
|
||
trim is asked for**: `in` past the end is a 400/422 that says so, `out` past the
|
||
end is just no tail trim (a UI dragging the tail wide shouldn't fail a render).
|
||
|
||
**M9-C2 — `GET /beats?path=` (`server.py:869-1090`).**
|
||
`{path,bpm,confidence,beats[],downbeats[],duration,meter,period,acPeak,onBeatContrast,analyzed}`.
|
||
Chain (all stdlib + one ffmpeg pipe, no new deps):
|
||
1. `:896` ffmpeg -> mono s16 PCM @22050 (`-map 0:a:0`, so mp4/webm work too),
|
||
read with `array`. `-t 900` cap: 15 min analysed, grid extrapolated past that
|
||
with `truncated:true` (a 40MB decode ceiling, not a 2-hour DJ set in RAM).
|
||
2. `:908` per-frame **log energy**, hop 256 (86.13 fps) -> half-wave-rectified
|
||
first difference -> 3-tap smooth -> **local-mean subtraction** (±0.37s).
|
||
3. `:933` normalised autocorrelation over 60..180 BPM, harmonics added
|
||
(`r[l] + .5r[2l] + .5r[l/2] + .3r[3l/2]`), times a log-gaussian prior centred
|
||
on **125 BPM**, peak parabola-interpolated.
|
||
4. `:1029` explicit octave check: score half / self / double with the comb, keep
|
||
everything within 15% of the best, prefer the candidate in 90..180 BPM.
|
||
5. `:968` comb-filter phase search over one period, then `:972` **weighted
|
||
least-squares refit** of period+phase onto the nearest onset peaks (sub-frame,
|
||
parabolic). Without the refit the raw AC lag is ±0.5 frame = seconds of drift
|
||
across a 4-minute track.
|
||
6. Emits an absolute-seconds grid across the whole file; `downbeats` = every 4th
|
||
beat, **4/4 assumed**, coset chosen as the strongest of the four (`meter:4` is
|
||
in the response so a caller knows it was an assumption, not a detection).
|
||
MEASURED ACCURACY (synthetic click tracks, `aevalsrc` with `mod(t,period)` so beat
|
||
k is at EXACTLY k*period): 12 tempos 90/100/110/118/120/124/126/128/132/140/150/174
|
||
-> **worst BPM error 0.03** (spec allowed ±2), **worst beat-time error 3.3 ms**
|
||
over the first 16 beats (spec allowed ~30 ms), confidence 0.85-0.94.
|
||
Non-music: pink noise conf 0.10, digital silence conf 0.0 with `beats:[]` and a
|
||
`note` (no crash, 200). Real 90s house from ~/Music/HOUSE: 127.41 / 126.89 /
|
||
121.20 / 126.95 BPM, and independently re-analysing two disjoint 90s windows of
|
||
each track reproduces the full-track BPM to within 0.15 on 3 of 4 (the 4th is an
|
||
intro/breakdown pair that confidence correctly calls mush, 0.16-0.20).
|
||
SPEED (ultra, measured): 4-min wav **0.41s**, 5-min mp3 0.66s, 7.6-min mp3 0.69s;
|
||
cached re-hit **2-3 ms**. Cache key is (path, mtime_ns, size), bounded at 32.
|
||
DECISIONS (M9-C2):
|
||
- **Energy flux, not spectral flux.** A pure-Python radix-2 FFT is ~20k transforms
|
||
for a 4-min track (tens of seconds of CPU) and the repo forbids numpy; for
|
||
four-to-the-floor the kick dominates broadband energy anyway. The log
|
||
compression + local-mean whitening is what buys back the discrimination — it is
|
||
the difference between conf 0.07 and 0.35 on a dense club mix.
|
||
- **`confidence` = sqrt(acPeak × onBeatContrast)**, both reported separately so a
|
||
caller can see why. acPeak = normalised autocorrelation at the chosen period
|
||
("is the envelope periodic at all"); onBeatContrast = (on-beat mean − all-frame
|
||
mean)/(sum) ("does the EMITTED grid explain the onsets"). Measured bands, in the
|
||
code comment: clicks 0.83-0.93, real house 0.32-0.65, noise 0.10, silence 0.0.
|
||
Read >0.5 as a solid grid, <0.2 as mush.
|
||
- Mean (not sum) comb score, deliberately: a double-tempo grid then pays for the
|
||
empty positions it adds, which is half the octave problem solved for free. The
|
||
half/double check + band preference is the other half — verified on `alt128`
|
||
(strong kick every 2nd beat, autocorrelates at 64) -> 128.01.
|
||
- The grid is CONSTANT tempo. No rubato/drift tracking, said plainly in the
|
||
response contract. For a DJ mix, tempo changes per track — analyse a segment.
|
||
- **The two fixtures in `assets/audio/test/` are not exact**: direct sample-level
|
||
onset timing puts `beat120.wav`'s kicks at 0.5 … 14.53 (28 intervals ->
|
||
**119.74 BPM**, it drifts ~0.3%), and `house128.wav` drifts similarly. My detector
|
||
says 119.62 / 127.62 on them, i.e. it is reading the FILE correctly. That is why
|
||
the test synthesises its own exactly-periodic clicks instead of asserting
|
||
against those two.
|
||
- New test group `scripts/test_server.py:525` boots an instance with
|
||
`PATH=/usr/bin:/bin` (no homebrew) and asserts `/beats` AND `/rhubarb` 503 —
|
||
the ffmpeg gate is now actually exercised, and so is rhubarb's 503 branch,
|
||
which had been unreachable on this box since rhubarb was installed.
|
||
- No client change was needed: `web/render.js:74 endRender` already forwards the
|
||
scene's `audio[]` objects verbatim, so `in`/`out` reach the server as-is.
|
||
BLOCKED/REQUESTS:
|
||
- @Lane B — `/beats` is ready for beat-snapping. Contract:
|
||
`GET beats?path=<asset-relative>` -> `{bpm, confidence, beats:[s,...],
|
||
downbeats:[s,...], duration, meter:4, period, acPeak, onBeatContrast}`; times
|
||
are absolute seconds in the FILE, so snapping a timeline key means
|
||
`beat + audioClip.start - (audioClip.in||0)`. Gate the UI on
|
||
`confidence` (>0.5 solid, <0.2 mush) rather than pretending every grid is good.
|
||
First call on a 5-min track is ~0.7s, after that it is instant — fetch once per
|
||
audio clip and cache client-side.
|
||
- @orchestrator — nothing needed for prod: ffmpeg is already in the image, so
|
||
`/beats` works on the VPS; no new env, no new dep (requirements.txt untouched).
|
||
The :8020 uvicorn still isn't `--reload`, so restart it before browser-driving
|
||
`/beats`. All my testing ran on ports 8097-8104 and 8098.
|
||
NEXT: nothing outstanding in Lane C. Backlog: per-scene audio auto-carried into a
|
||
film bed; more xfade transition types; the still-unverified MediaRecorder draft
|
||
path (carried from session 3); and — if beat-sync gets ambitious — a `?from=&to=`
|
||
window on `/beats` so a DJ mix can be analysed per track instead of assuming one
|
||
tempo for the whole hour.
|
||
|
||
## 2026-07-26 session 8 — M9-C2 tempo bias: root-caused and fixed
|
||
The orchestrator is right and my session-7 claim was wrong. **The fixtures are exact**
|
||
(`beat120.wav` kicks on 0.5000s to the microsecond, `house128.wav` on 0.468735s), and my
|
||
detector was **0.32% slow on both**. Worse, I had measured 120.002 correctly myself early
|
||
in that session and then believed a cruder second scan over it — the mistake was
|
||
disbelieving ground truth to protect the code. Corrected below.
|
||
|
||
**ROOT CAUSE — the onset front-end was measuring the wrong instant.**
|
||
A percussive hit is a high-frequency transient riding a slower low-frequency body.
|
||
`BEAT_SR=22050` resamples away everything above 11 kHz, and in `beat120.wav` the >11k
|
||
share of the attack grows ~20x across the file (peak sample 1402 at beat 1 -> 27711 at
|
||
beat 15). Measured, per beat, on the <11k signal that survived the decode:
|
||
beats 1/5/29 peak at **+0 ms** after the attack, beats 15/25 at **+30 / +50 ms**.
|
||
A lateness that VARIES per beat is a tempo error, not an offset — and the weighted
|
||
least-squares refit then fits those biased points perfectly. It was NOT the refit
|
||
(sub-millisecond over 200s click tracks) and NOT integer-lag AC quantisation
|
||
(the parabolic step already handled that; the AC's own estimate was 119.87, and the refit
|
||
moved it to 119.62 because its INPUT peaks were late).
|
||
**Fix: `BEAT_SR, BEAT_HOP = 44100, 512`** (`server.py:882`) — same 86.13 fps envelope, same
|
||
constants downstream, transient preserved. 172 fps was tried and REJECTED (`server.py:875`
|
||
comment): it breaks octave choice on real records (a 90s window of a house track went to
|
||
63.7 BPM) because the prior/harmonic weights are tuned for 86 fps.
|
||
`server.py:906 _frame_energy()` now decodes+frames in ONE STREAMING pass (Popen, ~4MB
|
||
reads) — 44.1k × 900s is 79MB of PCM and `subprocess.run` would hold it twice; peak memory
|
||
is now a few MB. It also returns the exact sample count, so `duration` is 15.0 not 14.988.
|
||
|
||
**SECOND BUG, found by sweeping tempos after the fix** (`server.py:963 _beat_tempo`):
|
||
a 118 BPM / 60s click track read **114.79**. Two defects compounding: (1) harmonics that
|
||
fall outside the lag table counted as ZERO, which silently penalises the lag that owns them
|
||
(lag 43 kept its 2x at 86, lag 44 lost its 88 — so a hair of score flipped the choice to
|
||
the wrong lag); (2) the parabolic interpolation was then applied to a point on a SLOPE,
|
||
which extrapolates: lag 43 became 44.95. Fixed by scoring only the harmonics that exist
|
||
(weight-normalised), then walking uphill to the local AC maximum before interpolating, and
|
||
clamping the correction to ±0.5 lag.
|
||
|
||
**RE-VERIFIED against the ground truth** (live over HTTP, `GET /beats`):
|
||
| file | truth | detected | err | max |beat − true grid| | conf |
|
||
|---|---|---|---|---|---|
|
||
| `beat120.wav` | 120.003 | **120.030** | +0.027 | **6.0 ms** (last beat +3.3) | 0.92 |
|
||
| `house128.wav` | 128.004 | **128.020** | +0.016 | **10.1 ms** (last beat +7.6) | 0.86 |
|
||
Was −0.387 / −0.388 and 45.1 / 52.5 ms of drift.
|
||
Synthetic sweep, 17 tempos 60…179 BPM on 60s files: **worst BPM error 0.010, worst drift
|
||
5.7 ms across the whole file**. Octave traps still correct (alt-strong-every-2nd 128.01,
|
||
64 BPM stays 64, kick+offbeat-hat 125.97). Real 90s house unchanged: 127.41 / 126.89 /
|
||
121.22 / 126.96, disjoint-window self-consistency within 0.13 BPM on 3 of 4.
|
||
Truncation path exercised (cap forced to 60s on a 200s file): grid extrapolates to 200.0s
|
||
with 1.0 ms of drift at the end. Speed: **4-min wav 0.48s, 5-min mp3 0.84s**, 7.6-min 0.78s,
|
||
cached 2-3 ms (was 0.66s at 22050 — the transient costs ~0.2s and is worth it).
|
||
|
||
**TESTS — this class of error cannot pass again** (`scripts/test_server.py:468`, 18 groups):
|
||
1. Every click case now asserts **whole-file drift** (`max|b − round(b/per)·per| < 20 ms`),
|
||
not just average BPM, and the sweep gained **118 BPM @ 60s** (pins the `_beat_tempo` fix).
|
||
2. **200s click track at 128** — end-of-file drift < 20 ms, and the grid must actually reach
|
||
past 195s (catches "correct tempo, grid stops early").
|
||
3. **`hftick.wav`** — a 16 kHz tick over a slow 60 Hz body, i.e. the failure mechanism
|
||
itself: the first 12 beats must land within **12 ms**. This is the one that pins
|
||
`BEAT_SR`, and it ships in the test (assets/ is gitignored).
|
||
4. **The fixtures as ground truth** when present: 120.003 / 128.004 ± 0.2 BPM and < 20 ms
|
||
drift, copied into the test's asset root, skipped with a printed note when absent.
|
||
Proven to discriminate, by monkeypatching the old code back in:
|
||
`22050/256` -> fixtures FAIL at 45.1 / 52.5 ms drift and hftick FAILS at 26.7 ms;
|
||
old `_beat_tempo` -> 118 BPM case FAILS at 114.79 (253 ms drift). Note the pure 200s click
|
||
track passes under BOTH old versions — a clean click track can never catch a spectral bias,
|
||
which is exactly why session 7's tests missed it.
|
||
DECISIONS:
|
||
- Ground truth wins. When a measurement disagrees with the code, the code is wrong until a
|
||
BETTER measurement says otherwise — and "better" means the one with the sharper
|
||
instrument (sample-level threshold crossing over the whole file), not the newer one.
|
||
- Front-end resolution is a tuning-coupled choice, not a free knob: 44100/512 keeps every
|
||
downstream constant valid; 44100/256 would need the prior and harmonic weights retuned.
|
||
The comment at `server.py:875` says so, so nobody "optimises" the decode rate back down.
|
||
BLOCKED/REQUESTS: none. Everything else from session 7 (trim work, speed, caching,
|
||
confidence, the 503 gate) is unchanged and still green.
|
||
NEXT: unchanged from session 7 (per-scene audio into a film bed, more transitions,
|
||
MediaRecorder draft path, `?from=&to=` windowing on /beats for DJ mixes).
|
||
|
||
## 2026-07-28 session 9 — render QUALITY pass (chrome, colour, supersample, symlinked banks)
|
||
Four items, all shipped. **21 test groups green** (was 18) + a new client test file.
|
||
`.venv/bin/python scripts/test_server.py` -> `OK: 21 test groups passed`;
|
||
`node scripts/test_render_client.mjs` -> `OK: render.js client checks passed`;
|
||
`node --check` clean on render.js + film.js.
|
||
|
||
**C1 — the gizmo/grid/camera-cones were in every mp4, and each render quadrupled the viewport.**
|
||
- `web/render.js:80,97` — `captureFrames` now brackets the loop with `stage.pause?.()` …
|
||
`finally { stage.resume?.() }`. Stage's SYNC7 `_chrome(false)` (stage.js:612) only fires when
|
||
`_paused`, and `pause()` (stage.js:621) is the ONLY thing that sets it — grep proved nothing in
|
||
web/ ever called it, so `_chrome()` had been dead code and the TransformControls arrows, the
|
||
60x60 GridHelper and the other cameras' cyan cones were baked into every frame. Doing it inside
|
||
captureFrames covers both callers (Lane B's ⏺ Render and film.js's per-shot loop) exactly once
|
||
each, and it stops the rAF director loop fighting for the GPU at render size.
|
||
- `web/render.js:40-52` — `withRenderSize` snapshotted `canvas.width/height` (DEVICE px) and
|
||
restored them through `setSize(w,h,false)` AFTER putting the pixel ratio back — and setSize
|
||
multiplies by the ratio again. On a retina Mac (dpr 2) the viewport buffer grew 2x per axis per
|
||
render (4x the pixels after one film, 16x after two) until a window resize. Now snapshots
|
||
`canvas.clientWidth/clientHeight` (the canvas is `width:100%;height:100%` of #view — style.css:24
|
||
— so clientWidth IS the CSS size, independent of the buffer), falling back to `canvas.width/dpr`
|
||
when the panel is hidden. No three import (header rule holds).
|
||
|
||
**C2 — colour-tagged, faststart, one lossy generation.**
|
||
- `server.py:408 video_args(crf=18)` — one helper, both call sites (`_encode` :464,
|
||
`_concat_film` :813): libx264 / preset slow / high@4.2 / yuv420p / crf / bt709 primaries+trc+
|
||
colorspace / range tv / `+faststart`.
|
||
- **MEASURED, and it changed the design**: on this box's ffmpeg 8.1.2 the OUTPUT options
|
||
`-color_primaries`/`-color_trc` are SILENTLY IGNORED (only `-colorspace` lands) — the png
|
||
decoder + auto-scaler hand the encoder frames tagged "unspecified" and frame properties win.
|
||
ffprobe on the result: `color_space=bt709, color_transfer=unknown, color_primaries=unknown`.
|
||
So `server.py:428 SETPARAMS` tags the FRAMES (`setparams=...`) and rides on every encode —
|
||
appended to the `-vf` chain in `_encode`, and as a `[vout]` node at the end of `_concat_film`'s
|
||
filter_complex (:805/:807, the map moved from `[{cur}]` to `[vout]`). ffprobe now reads bt709
|
||
on all three fields, asserted for both the shot mp4 and the film.
|
||
- Double-encode: `_encode` takes `crf=18` by default; film SHOTS pass 14 (`server.py:537`,
|
||
driven by `{intermediate:true}` in the `/render/{id}/end` body — `render.js:104 endRender(rid,
|
||
audio, extra)`, set by `film.js:48`). The concat stays at 18, so only one pass meaningfully
|
||
quantises. Asserted end-to-end by reading the crf out of x264's SEI (`crf=14.0` / `crf=18.0`
|
||
in the mp4 bytes), not by inspecting the command line.
|
||
|
||
**C3 — 2x supersample, resolved by lanczos in ffmpeg.**
|
||
- `web/render.js:59 ssFactor(w,h,ss=2)` (exported; film.js uses it too): 2, forced to 1 when
|
||
`w>1920 || h>1080`. `finalRender` (:123) and `renderSequence` (film.js:36) render at
|
||
`width*ss × height*ss` but still declare the OUTPUT size to `render/begin`, so meta.json keeps
|
||
describing the deliverable and the film-grid check is unaffected. Uniform scale = same aspect,
|
||
and `_render` only refits backdrops on an aspect CHANGE (stage.js:611) — framing untouched.
|
||
- `server.py:431 vf_args(meta)` reads the declared w/h from the render's meta.json and emits
|
||
`-vf scale={w}:{h}:flags=lanczos,<setparams>`, always (near-noop when sizes already match; also
|
||
catches a client/server size mismatch). No scale when begin declared no size — the pre-existing
|
||
`{name,fps}`-only begin body must not get upscaled to the 1280x720 default.
|
||
- `-vf` sits before the audio `-filter_complex`; they coexist because the video map is a plain
|
||
input (`0:v`), not a complex-graph label. **Verified with a real mux, not by reasoning** — the
|
||
test renders 640x360 frames with a tone and asserts both 320x180 output AND that the audio is
|
||
still there (-6dB at 0.05-0.25s).
|
||
- `server.py:506` frame cap 25MB -> **40MB** with the reason in a comment. Every other
|
||
`capped_body` limit untouched.
|
||
|
||
**C4 — symlinked asset banks (the documented way to get banks in) stop 400-ing.**
|
||
- `server.py:95 safe_under` is now LEXICAL: normpath (backslashes -> /), reject absolute /
|
||
`..` / `../…`, return `root / rel` UNRESOLVED. `.resolve()` follows symlinks, so
|
||
`ln -s ~/MESHGOD/assets3d/gallery assets/props/gallery` listed in `/assets/tree` and then 400'd
|
||
"bad path" on every fetch (dock: `failed: Error: asset fetch 400`) — characters, backdrops,
|
||
clips, audio, /beats, /rhubarb, all of it. Traversal is still caught before touching the disk
|
||
(normpath collapses `a/../../x` to `../x`). The docstring says all of this so a future
|
||
ship-check pass doesn't "fix" it back, and notes the accepted symlink-loop caveat.
|
||
- `server.py:178` `os.walk(base, followlinks=True)` so a bank symlinked one level deeper is
|
||
scanned at all.
|
||
- Side effect worth knowing: the lexical guard also rejects `..\..\x`, which the old
|
||
resolve()-based guard ACCEPTED on posix (backslash isn't a separator there, so it was just a
|
||
weird filename inside the root). Harmless either way; strictly safer now.
|
||
|
||
**TESTS — all four proved to DISCRIMINATE by putting the old code back:**
|
||
- `scripts/test_render_client.mjs` (NEW, plain node, no deps, no browser, no three): fake Stage /
|
||
Timeline / fetch. Old render.js -> **9 failures** (pause/resume never called, renders outside the
|
||
pause window, restore at 2560x1440 instead of 1280x720, hidden-panel fallback). Also asserts:
|
||
30 frames for 1s@30fps, one POST per frame, `render/begin` declares 1280x720 while setSize gets
|
||
2560x1440, ss forced to 1 above 1080p, ssFactor thresholds, and that `resume()` still runs when
|
||
a frame POST throws.
|
||
- `scripts/test_server.py:145` safe_under unit (6 traversal rejects, 3 accepts incl. empty=root)
|
||
+ integration (symlinked bank in `/assets/tree` and streamable, exact bytes). Old code:
|
||
followlinks off -> `symlinked bank not scanned`; old guard -> `(400, b'{"detail":"bad path"}')`.
|
||
- `:172` video_args/vf_args unit: bt709 on all four fields, `+faststart`, crf 14 vs 18, no
|
||
input-only flag (`-i/-framerate/-ss/-t/-f/-y`) in the output list, setparams on every encode.
|
||
- `:298`/`:308` render smoke: colour tags on out.mp4; 640x360 frames -> 320x180 output; the
|
||
`-vf` + `-filter_complex` coexistence with audio; crf SEI check. `:513` the FILM is bt709 too.
|
||
Old encoders -> `AssertionError: unknown` (primaries) and `('640','360')` (scale).
|
||
DECISIONS:
|
||
- **1080p still supersamples.** The item's prose said force ss=1 when `w>1920||h>1080`; its
|
||
sample test line said a 1920x1080 render should get ss=1. Those contradict, and the prose wins:
|
||
the default deliverable IS 1080p, so ss=1 there would make the whole feature a no-op for the
|
||
common case — and the same item's 40MB-cap rationale explicitly assumes "a 1080p deliverable
|
||
posts 3840x2160 PNGs". So >1080p output = no supersample; 1080p and below = 2x. Written into
|
||
the test as an explicit assert with that reasoning.
|
||
- `crf` is not a free client knob: `/render/end` takes a BOOLEAN `intermediate`, and the server
|
||
picks 14 or 18. A client can't ask for crf 51.
|
||
- `vf_args` skips the scale when begin declared no size rather than defaulting to 1280x720 —
|
||
the existing test posts 64x64 frames with a `{name,fps}`-only begin body and must not be
|
||
upscaled. Tags still ride on that path.
|
||
BLOCKED/REQUESTS:
|
||
- @Lane B (tlui.js, FYI not a blocker): `renderPlan()`'s object is spread into `finalRender`, so
|
||
adding `ss: 1` to it is all that's needed if you ever want a "fast draft" toggle — the option
|
||
already exists client-side. Also note a 1080p render now uploads 3840x2160 PNGs: expect it to
|
||
take roughly 2-4x longer than before and consider saying so on the button.
|
||
- @orchestrator — nothing needed for prod: no new env, no new dep, ffmpeg in the image already
|
||
has `setparams` (filter exists since 4.1). The :8020 uvicorn is still not `--reload`, so
|
||
restart it before browser-checking any of this. My tests ran on ports 8097-8104 only.
|
||
NEXT: unchanged backlog (per-scene audio auto-carried into a film bed; more xfade transition
|
||
types; MediaRecorder draft path still unverified in a real Chrome window; `?from=&to=` windowing
|
||
on /beats). New candidate: `draftRecord` should pause the stage too — it deliberately does NOT
|
||
(it records the LIVE canvas, so chrome is arguably wanted in a draft), but if John disagrees it
|
||
is a two-line change.
|
||
|
||
## 2026-07-28 session 10 — adversarial-review fixes (colour was a LIE, gizmo ghost, dropped frames)
|
||
**23 test groups green** (was 21) + 4 new client groups. `.venv/bin/python scripts/test_server.py`
|
||
-> `OK: 23 test groups passed`; `node scripts/test_render_client.mjs` -> `OK: render.js client
|
||
checks passed`; `node --check` clean on render.js, film.js, test_render_client.mjs.
|
||
|
||
**MAJOR 1 — session 9's colour work TAGGED the file bt709 and CONVERTED the pixels bt601.**
|
||
Reproduced with the shipped arg list before touching anything: pure green PNG in, first luma
|
||
sample out = **145** (bt601 green; bt709 is 173), file decodes through its own bt709 tag as
|
||
(0,216,0). Neon magenta (230,40,120) -> (244,61,119). Root cause is NOT "we forgot a flag": a
|
||
`setparams` filter PINS ITS OUTPUT LINK to bt709, which leaves the scale filter's output link
|
||
unspecified, so swscale's RGB->YUV falls back to bt601 — with **no** setparams at all the graph
|
||
negotiates bt709 off `-colorspace` and gets it right. i.e. session 9's tag *caused* the mislabel
|
||
it was meant to cure. Fix = state the conversion on the converting filter: `server.py:441
|
||
CONVERT = "out_color_matrix=bt709"`, used by `vf_args` (`:463`, both the sized and the no-size
|
||
chain) and by `_concat_film`'s per-shot scale (`:850`). Measured after: Y=173, round-trip
|
||
(0,255,1) / (230,38,119).
|
||
- The reviewer's sub-claim that the NO-SIZE path was equally broken did **not** reproduce:
|
||
setparams-only was already correct (the auto-inserted scaler sits AFTER setparams and honours
|
||
the tag). Made it explicit anyway — `scale=out_color_matrix=bt709` keeps iw:ih — so correctness
|
||
stops depending on where ffmpeg chooses to insert its scaler. Test pins that it does not resize.
|
||
- Regression: `test_server.py` `probe_rgb()` decodes frame 0 back to RGB and `near()` compares.
|
||
Three colours (green / neon magenta / sky) x both filter paths, plus the FILM (its untagged
|
||
lavfi fixtures are exactly the "old shot mp4 still in renders/" case). **Discriminates**: with
|
||
CONVERT removed -> `0x00FF00 scaled render decodes as (0, 216, 0)`; film -> `film decodes green
|
||
as (0, 216, 0)`. Tag asserts alone stay green through all of that — that is why they missed it.
|
||
|
||
**MAJOR 2 — pause() woke up dead code that left a GHOST GIZMO in the viewport.**
|
||
`stage.js _chrome(true)` (Lane A) re-shows chrome after every paused frame by FORCING
|
||
`_tc.visible = true`, overriding the `visible = false` that `select()`'s `detach()` just set.
|
||
So a render started with nothing selected (or with a frustum-fitted plate selected — same detach)
|
||
ended with an inert translate gizmo floating at the last object's position: the exact bug class
|
||
pause() was added to kill, re-introduced by pause() itself.
|
||
- Fix in my file: `render.js:81 restoreSelectionChrome(stage)`, called from `captureFrames`'s
|
||
finally (`:139`). Re-runs `select()` with the id Stage already holds, so attach/detach decides
|
||
the gizmo again. It re-selects only if `stage.getEntity(id)` still resolves — film.js swaps the
|
||
whole scene per shot, so the remembered id is often gone and re-attaching to a dead entity
|
||
would be worse than no gizmo.
|
||
- Regression: client group 7 models stage.js exactly (attach/detach drive `_tc.visible`, `_render`
|
||
brackets each paused draw with `_chrome(false)/_chrome(true)`); three cases — nothing selected,
|
||
something selected, selection since deleted. **Discriminates**: dropping the call -> 2 FAILs.
|
||
|
||
**BONUS (nobody caught this one) — a failed frame POST was SILENTLY DROPPED.**
|
||
Found while wiring the new tests: the `.finally(() => inflight.delete(p))` pulls each POST out of
|
||
the set as it settles, so a rejection nobody happened to be awaiting vanished. Measured with only
|
||
frame 3 failing: `captureFrames` **RESOLVED**, 29/30 frames stored, plus an unhandled rejection —
|
||
ffmpeg's `%06d` demuxer then stops at the gap and you get a short mp4 with `state=done`. Fixed at
|
||
`render.js:129` (`.catch` remembers the first error) + `:132`/`:136` re-throw, so it fails fast at
|
||
the frame that broke. Regression = client group 5b + a process-wide unhandledRejection sentinel
|
||
(group 10). **Discriminates**: old code -> 3 FAILs (resolves, no frame number, 29 unhandled).
|
||
|
||
**MINOR — vf_args' scale silently UPSCALED an undersized capture** (docstring claimed the
|
||
opposite: "catches any client/server size mismatch"). Now `_encode` ffprobes frame 0 (`_png_size`,
|
||
`server.py:466`) and errors before spawning when the frames are SMALLER than the declared
|
||
deliverable — the lost-WebGL-context / clamped-canvas case, previously a soft mushy mp4 with
|
||
state=done and no log line. Bigger is normal (that is the supersample); equal is fine. Docstrings
|
||
now describe what the code does. **Discriminates**: guard disabled -> `undersized frames encoded
|
||
anyway: {'state': 'done'...}` with 64x36 frames upscaled to 320x180.
|
||
|
||
**MINOR — pause/resume were optional-chained, so losing them would fail silently.** They are not
|
||
in PLAN §4.2's Stage contract and `stage.pause?.()` no-ops on any Stage without them, which would
|
||
put the gizmo back in every mp4 with all tests green. Now REQUIRED (`render.js:116` throws a
|
||
message that says why), and client group 9 reads Lane A's live `web/stage.js` and fails if
|
||
`pause()`/`resume()`/the `_paused` chrome gate/`_selected` stop existing — the first test in this
|
||
repo that pins a cross-lane seam the PLAN doesn't cover. **Discriminates**: reverting to `?.` ->
|
||
FAIL; and the stage.js regexes go false on a mutated copy with `pause()` renamed.
|
||
|
||
DECISIONS:
|
||
- A tag is not a conversion, and in ffmpeg they FIGHT: `setparams` breaks the colourspace
|
||
negotiation that would otherwise do the right thing. Both halves are now written next to each
|
||
other with the measurement in the comment (`server.py:428-441`) so neither gets "cleaned up".
|
||
- Colour assertions are per-PIXEL from here on. Session 9's tests asserted the metadata the code
|
||
writes, which can only ever prove the code did what it says, never that the result is right.
|
||
- `restoreSelectionChrome` reads `stage._selected` (a private field) rather than a getter, because
|
||
there isn't one. It is guarded by `'_selected' in stage`, so a Lane A rename degrades to "no
|
||
restore" rather than clearing John's selection — and group 9 fails loudly if that field goes.
|
||
BLOCKED/REQUESTS:
|
||
- @Lane A (stage.js — DO NOT let me edit it): `_chrome(visible)` forces `visible = true` on
|
||
everything it touches, so it cannot restore state it did not save. `_chrome(true)` un-hides the
|
||
DETACHED TransformControls (three r160: `attach()`/`detach()` drive `.visible` directly). Please
|
||
snapshot each object's `visible` in `_chrome(false)` and restore those values in `_chrome(true)`;
|
||
then my `restoreSelectionChrome` guard in render.js becomes belt-and-braces and can go. Same
|
||
latent hole covers `en.helper` and anything else that is legitimately hidden at render time.
|
||
- @orchestrator (deploy/ is not mine): prod nginx `client_max_body_size 30m`
|
||
(deploy/nginx-scenegod.conf) is now BELOW this server's 40MB frame cap (server.py:552) and its
|
||
comment ("render frames <=25MB + slack") is stale. Nothing hits either wall today — 4K PNG
|
||
frames measure ~3MB typical — but the binding limit in prod is the one Lane C cannot see, and a
|
||
413 there arrives with no message from us. Please bump to 48m and refresh the comment.
|
||
- @orchestrator, disclosing the FULL cost of session 9's ss=2-at-1080p deviation (I logged the
|
||
deviation but not its bill): the default ⏺ Render posts 3840x2160 PNGs, so /opt/scenegod-data/
|
||
renders holds ~2.6x the bytes for the 48h reap window (a film keeps every shot's frames), the
|
||
upload over the tunnel is ~2.6x, and the default render is 2-4x slower. There is no UI opt-out
|
||
because `renderPlan()` (tlui.js, Lane B) emits no `ss` — the option exists client-side, it just
|
||
is not wired to a control. If you want the acceptance line honoured instead (1920x1080 -> ss 1),
|
||
it is one number in `ssFactor` and one assert in test_render_client.mjs; say the word and I will
|
||
flip it rather than defend it.
|
||
NEXT: unchanged backlog (per-scene audio auto-carried into a film bed; more xfade transition
|
||
types; MediaRecorder draft path still unverified in a real Chrome window; `?from=&to=` windowing
|
||
on /beats). New candidate off this session: `_encode` could also refuse a frame COUNT that does
|
||
not match `duration*fps` — the same invisible-failure family as the undersized-frame guard.
|