SCENEGOD/lanes/C-server.md

# Lane C — Server: FastAPI, assets, scenes, render pipeline

You own: `scenegod/server.py`, `web/render.js`, `scripts/*`,
`requirements.txt`, `.gitignore`. Milestone: **M1**.

## C1. Scaffold (first commit)
- `requirements.txt`: `fastapi`, `uvicorn`. Nothing else.
- `.gitignore`: `.env`, `assets/`, `scenes/`, `renders/`, `__pycache__/`,
  `*.pyc`, `.DS_Store`.
- `scenegod/server.py`, single file (split only when it hurts — MESHGOD's
  server.py is one file at 800+ lines and fine). `SCENEGOD_PORT` 8020,
  bind `127.0.0.1` (env `SCENEGOD_BIND` to override later — not now).
- Statics: `GET /` → `web/index.html` with
  `Cache-Control: no-store` (MESHGOD stale-cache lesson); `/web/*` served
  with short cache. Use FastAPI `FileResponse`/`StaticFiles`, headers set
  explicitly for the HTML.

## C2. Assets endpoints
- Roots from env with defaults: `SCENEGOD_ASSETS=./assets`,
  `SCENEGOD_SCENES=./scenes`, `SCENEGOD_RENDERS=./renders` (create scenes/
  renders on boot; assets must exist — clear error if not).
- `GET /assets/tree`: walk `assets/{characters,animations,props,backdrops,audio}`.
  Group rigroom-library style: files sharing dir+stem = one entry with a
  `formats` list (`{"name":"lady","path":"characters/pack01/lady.glb",
  "formats":["glb","fbx"]}`). Extensions: glb/gltf/fbx/obj/bvh + jpg/png/webp
  (backdrops) + wav/mp3 (audio). Scan live, no index file, cache 5s.
- `GET /assets/file?path=`: **resolve and enforce
  `path.resolve().is_relative_to(ASSETS_ROOT)`** — reject `..`, absolute
  paths, symlink escapes (resolve first, then check). Correct content-types
  (`model/gltf-binary`, etc.). This guard pattern applies to every
  path-taking endpoint you ever add here (ship-check rule).
- Read `/Users/johnking/Documents/MESHGOD/meshgod/library.py` first — same
  idea, steal the grouping approach.

## C3. Scenes CRUD
- `GET /scenes` → `[{name, mtime, duration}]`; `GET /scenes/{name}`;
  `POST /scenes/{name}` with JSON body. `name` slugified `[a-z0-9-_]`, file
  `scenes/{name}.json`, atomic write (tmp + rename).
- Validate on save (stdlib, no jsonschema dep): `version==1`, entity ids
  unique, every track sorted by `t`, clip blocks non-overlapping per entity
  (fade overlap allowed), `cameraCuts` reference existing camera entities.
  Reject with a 422 listing every violation (agents and UI both rely on the
  messages).

## C4. scripts/test_server.py (M1 gate)
Stdlib `urllib` + `subprocess` (uvicorn on a test port, tmp asset/scene
dirs): asserts tree grouping, file streaming, traversal attempts → 4xx
(`?path=../../etc/hosts`, absolute, URL-encoded `..`), scene round-trip,
validation failures (unsorted keys, dup ids). `python3 scripts/test_server.py`
green = M1 code-done. Run with `| lm -l 2`.

## C5. Render pipeline — M3, but design lands now (endpoints stubbed 501)
- Session model: `POST /render/begin {name,fps,width,height}` → `{renderId}`
  (uuid, dir `renders/{id}/frames/`). `POST /render/{id}/frame/{n}`: raw PNG
  body → `frames/{n:06d}.png`. `POST /render/{id}/end {audio?}` → spawn
  ffmpeg in a background thread:
  `ffmpeg -framerate {fps} -i frames/%06d.png -c:v libx264 -pix_fmt yuv420p
  -crf 18 out.mp4` (audio inputs + `-map`/amix in M4). `GET /render/{id}/status`
  = queued|encoding|done|error(+log tail); `GET /render/{id}/out.mp4`.
  ffmpeg presence checked at boot (`shutil.which`), endpoint 503s w/ message
  if absent. Reap render dirs >48h old on boot.
- `web/render.js` (M3): exports `draftRecord(stage, timeline)` —
  `canvas.captureStream(fps)` + MediaRecorder while `timeline.play()`s, save
  webm; and `finalRender(stage, timeline, opts)` — resize canvas, loop
  `timeline.step(f)` → `stage.renderActiveCamera()` → `canvas.toBlob('image/png')`
  → POST frame (3-4 in flight, backpressure via await), then `/end`, poll
  status, download link. Deterministic: never uses wall-clock time.

## C6. M4 (locked): audio, MODELBEAST, graft
- ffmpeg audio mux per scene JSON `audio[]`.
- `POST /mb/submit` proxy re-implementing the contract in MESHGOD
  `meshgod/ops.py` (upload input to /api/assets first; token read from
  `~/Documents/backnforth/.env` at request time, never logged; endpoint only
  active when `SCENEGOD_MB=1`).
- `scripts/graft_limb.py` headless Blender (pattern:
  MESHGOD `scripts/glb2fbx.py`): `--body body.glb --limb hand.glb --bone
  mixamorig:RightHand` → align limb root bone to target bone head, join
  armatures, parent-keep-offset, join meshes (weights survive), export. If
  the body armature ALREADY has the bones the limb provides → exit 2 with
  "bones exist; mitten weights — re-skin, don't graft". Never route output
  through any finish/decimate path (rig contract).

## M1 acceptance (log it)
- `uvicorn scenegod.server:app --port 8020` serves Lane A's page (or a
  placeholder index if A hasn't landed — don't block on them).
- test_server.py green; paste the summary line in your log.

## ORCHESTRATOR UPDATES

### 2026-07-18 — M1 ACCEPTED. **M2 unlocked AND M3 render pipeline green-lit early.**
test_server.py green, tree/file/scenes verified live through the browser page
(scene save 200 + lossless reload). You're ahead of the other lanes, so skip
the wait: after the small M2 items (thumbnails endpoint; validation already
solid), go straight into §C5 — implement `/render/begin|frame|end|status|out`
+ ffmpeg encode, and `web/render.js`. Notes:
1. `timeline.step(frame)` is verified deterministic against the real Stage
   (orchestrator stepped 0/30/60/90/120 and got exact eased positions) — your
   final-render loop can rely on it today. `stage.renderActiveCamera` as a
   render target is Lane A M2 work; until it lands, test `finalRender` with
   the director view canvas, and test the ffmpeg path with synthetic PNGs
   from a script (extend test_server.py: begin → POST 30 generated frames →
   end → poll → assert mp4 exists + ffprobe duration ≈ 1s).
2. ffmpeg on ultra: check `shutil.which` at boot as designed; it's installed
   via homebrew here.
3. Test assets for the tree now exist under `assets/*/test/` (gitignored).
4. uvicorn for local dev: `.venv/bin/uvicorn scenegod.server:app --port 8020`
   — there's also a launch entry in MESHGOD/.claude/launch.json ("scenegod")
   used by the orchestrator's browser preview.

### 2026-07-18 — M2+M3 ACCEPTED, **SYNC 3 PASSED. M4 is UNLOCKED.**
Orchestrator ran your `finalRender` end-to-end in the integrated app against
Lane A's `renderActiveCamera`: 240 frames @1920×1080 → ffmpeg → out.mp4,
h264, exactly 8.000s, with camera cuts and sunset light lerp visible in
extracted frames. The browser client path you couldn't exercise is now
verified — no changes needed. Proceed with M4 in this order:
1. ffmpeg audio mux from scene `audio[]` (the sync2-sunset scene in
   `scenes/` is your fixture; add a wav to `assets/audio/test/`).
2. `scripts/graft_limb.py` (spec in §C6 — blender binary is on ultra's PATH
   as `blender`; verify with a mixamo body + any rigged hand GLB, and add
   the refusal-path test).
3. `/mb/submit` MODELBEAST proxy last (needs `SCENEGOD_MB=1` gating like
   riggermore; test only that auth/upload contract matches meshgod/ops.py —
   don't burn farm jobs on repeat runs).
One render-quality note for your backlog, not urgent: draft webm via
MediaRecorder is untested in the embedded preview (rAF suspension) — verify
it once in a real Chrome window before calling it done.

### 2026-07-19 — **M5 DIRECTOR MODE unlocked**: `POST /director` (spec PLAN §5 M5).
- rhubarb is now INSTALLED on ultra (built from source, arm64, v1.14.0,
  symlinked /opt/homebrew/bin/rhubarb) — your happy path is verified:
  `GET /rhubarb?path=audio/test/vo.wav` → real mouthCues. Nothing to do there.
- `POST /director {text, scene:{entities:[{id,kind,label}], time}}` → chat
  request to `SCENEGOD_LLM_URL` (OpenAI-compat; ultra's tailnet Ollama is
  `http://100.69.21.128:11434`, model via `SCENEGOD_LLM_MODEL`). Orchestrator
  checked /api/tags on that box: it serves `qwen2.5:7b` (+ an embed model) —
  default SCENEGOD_LLM_MODEL to `qwen2.5:7b`; John can pull a gemma later
  and it's just an env change. 503 with a hint when env unset. stdlib
  urllib only.
- System prompt: translate director language into a JSON array of ops
  matching Lane A's `scenegod:direct` contract EXACTLY (enumerate the shot/
  angle/focal/light/mark vocabularies from grammar.js in the prompt; read
  the file at boot so the vocab never drifts). `format: json` / low temp.
- VALIDATE server-side before returning: unknown op/shot/preset/entity id →
  drop the op and include it in a `rejected` list so the UI can toast it.
  Never forward raw model text to the client.
- test_server.py: mock the LLM (in-process http.server, same pattern as your
  MB mock): assert prompt carries the vocab, valid ops pass, garbage ops land
  in `rejected`, 503 when env unset.

### 2026-07-18 — M4 ACCEPTED. Lane C scope (M1–M4) is COMPLETE.
Orchestrator verified in the integrated app: audio-muxed finalRender →
h264+aac mp4, both streams exactly 8.000s; graft_limb `--selftest` green
under Blender 5.0.1 (both refusal + graft paths); 8 server test groups pass.
Your blender-path correction noted (it's `/Applications/Blender.app/...`,
not on PATH — my error). Standing by is fine; remaining work is A/B UI
(viseme lane, audio rows). One item if you want it: add a `rhubarb` check +
`/rhubarb?path=` endpoint (subprocess → JSON) IF rhubarb is installed —
503-with-message otherwise, same pattern as ffmpeg. Log a REQUEST first if
it needs more than that.

### 2026-07-19 — M6 FLOW VIDEO PLATES queued behind M5 (full spec PLAN §5 M6).
Do M5 first; M6 is unlocked the moment your M5 acceptance is logged. Test
media will be arriving in assets/backdrops/video/ via scripts/flow_intake.py.

### 2026-07-20 — M5+M6 ACCEPTED at SYNC 6 (all lanes ran as orchestrator subagents).
Verified live: DIRECT bar CU/low/frame → camera+keys+cut, ONE undo reverts;
golden_hour keys; video plate follows the clock frame-exact; finalRender shows
the Flow plate behind an animating character. Orchestrator seam fixes to absorb:
(1) server: VIDEO_EXTS in tree/media-types, video beats poster as primary;
(2) stage.entityVideo(id) added (Lane B seam); (3) syncVideos forces
videoTex.needsUpdate — r160 VideoTexture rides requestVideoFrameCallback which
never fires in hidden tabs → black plates in renders. Backlog carried: honor
audio in/out at mux (C), walk-to-mark prevPos pre-apply (A), live viseme asset
still owed by John.

### 2026-07-25 — M7-C ACCEPTED at SYNC 8.
Verified live after a server restart: /templates lists all three in order with thumbs;
two-hander-golden-hour loads 7 entities (plate, man, lady, cam1, cam2, sun, amb), 8s,
both performers carrying clips, and the cut fires cam1 to cam2 across t=4.5; record-store
(5 ents, 6s) and empty-stage (3 ents, 10s) both clean. 10 test groups green. The
anti-rot check (every referenced asset must exist) is exactly the right instinct — keep
that pattern for anything else that ships data alongside code.
NEXT for Lane C when you want it: the post-M7 backlog in PLAN §5 is yours to start —
multi-shot SEQUENCING (ordered shot list of scenes, rendered and ffmpeg-concatenated
into one film) is the biggest single unlock left in the project.

### 2026-07-25 — M8-C ACCEPTED at SYNC 10. **Sequencing works; SCENEGOD makes FILMS now.**
Verified live end-to-end, twice. (1) Your shipped `demo-two-shot` → one 11.500s mp4; the
frame at t=4.25 is visibly TWO shots superimposed, so the xfade really composites, and the
hard cut into shot 3 lands. (2) A fresh film built from the M7 templates — saved
two-hander-golden-hour and record-store as scenes, cut them into `milk-bar-film` (wide →
dissolve → closer → cut → interior) → 11.800s @960×540, correct on every boundary.
Your two flagged deploy lines are done by the orchestrator: compose now sets
SCENEGOD_SEQUENCES/SCENEGOD_FILMS with `/opt/scenegod-data/{sequences,films}` volumes, and
deploy.sh mkdirs both + excludes `films`/`*.log`.
Excellent instincts this round: ffprobe'ing real durations for xfade offsets rather than
trusting nominal length, validating dissolve-shorter-than-both-shots (xfade's actual
requirement), persisting the film plan so /end survives a restart, and catching the
pre-existing missing `apad` that let short audio truncate the PICTURE.
NEXT for Lane C: audio `in`/`out` at mux is STILL the open gap (Lane B's head-trim made it
easy to hit) — close that. Then beat-sync groundwork: a `POST /beats?path=` that runs a beat
detector over an audio asset and returns a grid the timeline can snap to (MODELBEAST demucs
is available on ultra; the endpoint should 503 cleanly on the VPS like the others).

### 2026-07-26 — M9-C ACCEPTED at SYNC 11 (after one bounce-back).
Trim-at-mux verified. Beat detection verified against the orchestrator's independent
ground truth: beat120 120.030 vs 120.003 true, house128 128.020 vs 128.004, worst offset
from the true grid 6.0 / 10.1 ms across the WHOLE file (was 45 / 52 ms). 18 groups green.
Three things worth carrying forward as lessons:
1. You initially explained a real bug away as "the fixtures aren't exact" — and had
   measured them correctly yourself earlier in the same session. When your output
   disagrees with a fixture, the fixture is the hypothesis to test LAST, not first.
2. The root cause is a keeper: decoding at 22050 discarded the HF transient, and because
   the attack's HF share GREW across the file, the surviving low-frequency onset landed
   progressively later — a per-beat-varying lateness reads as a tempo error, and the LS
   refit then faithfully fit the biased points. Diagnosing that from "0.3% low" is good work.
3. `hftick.wav` (16 kHz tick over a 60 Hz body) is now the regression that pins BEAT_SR,
   and the note about 172 fps breaking octave choice is exactly the kind of comment that
   stops someone "optimising" it back. Keep writing those.
Also good: proving the new tests DISCRIMINATE by monkeypatching the old code back, and
noticing that a pure click track passes under both versions — which is precisely why the
first round's tests missed it. Test with the pathological signal, not the clean one.
NEXT for Lane C: nothing queued. Lane B consumes /beats next (snap-to-beat + shot-list UI).