SCENEGOD/lanes/C-server.md

Lane C — Server: FastAPI, assets, scenes, render pipeline

You own: scenegod/server.py, web/render.js, scripts/*, requirements.txt, .gitignore. Milestone: M1.

C1. Scaffold (first commit)

• requirements.txt: fastapi, uvicorn. Nothing else.
• .gitignore: .env, assets/, scenes/, renders/, __pycache__/, *.pyc, .DS_Store.
• scenegod/server.py, single file (split only when it hurts — MESHGOD's server.py is one file at 800+ lines and fine). SCENEGOD_PORT 8020, bind 127.0.0.1 (env SCENEGOD_BIND to override later — not now).
• Statics: GET / → web/index.html with Cache-Control: no-store (MESHGOD stale-cache lesson); /web/* served with short cache. Use FastAPI FileResponse/StaticFiles, headers set explicitly for the HTML.

C2. Assets endpoints

• Roots from env with defaults: SCENEGOD_ASSETS=./assets, SCENEGOD_SCENES=./scenes, SCENEGOD_RENDERS=./renders (create scenes/ renders on boot; assets must exist — clear error if not).
• GET /assets/tree: walk assets/{characters,animations,props,backdrops,audio}. Group rigroom-library style: files sharing dir+stem = one entry with a formats list ({"name":"lady","path":"characters/pack01/lady.glb", "formats":["glb","fbx"]}). Extensions: glb/gltf/fbx/obj/bvh + jpg/png/webp (backdrops) + wav/mp3 (audio). Scan live, no index file, cache 5s.
• GET /assets/file?path=: resolve and enforce path.resolve().is_relative_to(ASSETS_ROOT) — reject .., absolute paths, symlink escapes (resolve first, then check). Correct content-types (model/gltf-binary, etc.). This guard pattern applies to every path-taking endpoint you ever add here (ship-check rule).
• Read /Users/johnking/Documents/MESHGOD/meshgod/library.py first — same idea, steal the grouping approach.

C3. Scenes CRUD

• GET /scenes → [{name, mtime, duration}]; GET /scenes/{name}; POST /scenes/{name} with JSON body. name slugified [a-z0-9-_], file scenes/{name}.json, atomic write (tmp + rename).
• Validate on save (stdlib, no jsonschema dep): version==1, entity ids unique, every track sorted by t, clip blocks non-overlapping per entity (fade overlap allowed), cameraCuts reference existing camera entities. Reject with a 422 listing every violation (agents and UI both rely on the messages).

C4. scripts/test_server.py (M1 gate)

Stdlib urllib + subprocess (uvicorn on a test port, tmp asset/scene dirs): asserts tree grouping, file streaming, traversal attempts → 4xx (?path=../../etc/hosts, absolute, URL-encoded ..), scene round-trip, validation failures (unsorted keys, dup ids). python3 scripts/test_server.py green = M1 code-done. Run with | lm -l 2.

C5. Render pipeline — M3, but design lands now (endpoints stubbed 501)

• Session model: POST /render/begin {name,fps,width,height} → {renderId} (uuid, dir renders/{id}/frames/). POST /render/{id}/frame/{n}: raw PNG body → frames/{n:06d}.png. POST /render/{id}/end {audio?} → spawn ffmpeg in a background thread: ffmpeg -framerate {fps} -i frames/%06d.png -c:v libx264 -pix_fmt yuv420p -crf 18 out.mp4 (audio inputs + -map/amix in M4). GET /render/{id}/status = queued|encoding|done|error(+log tail); GET /render/{id}/out.mp4. ffmpeg presence checked at boot (shutil.which), endpoint 503s w/ message if absent. Reap render dirs >48h old on boot.
• web/render.js (M3): exports draftRecord(stage, timeline) — canvas.captureStream(fps) + MediaRecorder while timeline.play()s, save webm; and finalRender(stage, timeline, opts) — resize canvas, loop timeline.step(f) → stage.renderActiveCamera() → canvas.toBlob('image/png') → POST frame (3-4 in flight, backpressure via await), then /end, poll status, download link. Deterministic: never uses wall-clock time.

C6. M4 (locked): audio, MODELBEAST, graft

• ffmpeg audio mux per scene JSON audio[].
• POST /mb/submit proxy re-implementing the contract in MESHGOD meshgod/ops.py (upload input to /api/assets first; token read from ~/Documents/backnforth/.env at request time, never logged; endpoint only active when SCENEGOD_MB=1).
• scripts/graft_limb.py headless Blender (pattern: MESHGOD scripts/glb2fbx.py): --body body.glb --limb hand.glb --bone mixamorig:RightHand → align limb root bone to target bone head, join armatures, parent-keep-offset, join meshes (weights survive), export. If the body armature ALREADY has the bones the limb provides → exit 2 with "bones exist; mitten weights — re-skin, don't graft". Never route output through any finish/decimate path (rig contract).

M1 acceptance (log it)

• uvicorn scenegod.server:app --port 8020 serves Lane A's page (or a placeholder index if A hasn't landed — don't block on them).
• test_server.py green; paste the summary line in your log.

ORCHESTRATOR UPDATES

2026-07-18 — M1 ACCEPTED. M2 unlocked AND M3 render pipeline green-lit early.

test_server.py green, tree/file/scenes verified live through the browser page (scene save 200 + lossless reload). You're ahead of the other lanes, so skip the wait: after the small M2 items (thumbnails endpoint; validation already solid), go straight into §C5 — implement /render/begin|frame|end|status|out

• ffmpeg encode, and web/render.js. Notes:

1. timeline.step(frame) is verified deterministic against the real Stage (orchestrator stepped 0/30/60/90/120 and got exact eased positions) — your final-render loop can rely on it today. stage.renderActiveCamera as a render target is Lane A M2 work; until it lands, test finalRender with the director view canvas, and test the ffmpeg path with synthetic PNGs from a script (extend test_server.py: begin → POST 30 generated frames → end → poll → assert mp4 exists + ffprobe duration ≈ 1s).
2. ffmpeg on ultra: check shutil.which at boot as designed; it's installed via homebrew here.
3. Test assets for the tree now exist under assets/*/test/ (gitignored).
4. uvicorn for local dev: .venv/bin/uvicorn scenegod.server:app --port 8020 — there's also a launch entry in MESHGOD/.claude/launch.json ("scenegod") used by the orchestrator's browser preview.

2026-07-18 — M2+M3 ACCEPTED, SYNC 3 PASSED. M4 is UNLOCKED.

Orchestrator ran your finalRender end-to-end in the integrated app against Lane A's renderActiveCamera: 240 frames @1920×1080 → ffmpeg → out.mp4, h264, exactly 8.000s, with camera cuts and sunset light lerp visible in extracted frames. The browser client path you couldn't exercise is now verified — no changes needed. Proceed with M4 in this order:

1. ffmpeg audio mux from scene audio[] (the sync2-sunset scene in scenes/ is your fixture; add a wav to assets/audio/test/).
2. scripts/graft_limb.py (spec in §C6 — blender binary is on ultra's PATH as blender; verify with a mixamo body + any rigged hand GLB, and add the refusal-path test).
3. /mb/submit MODELBEAST proxy last (needs SCENEGOD_MB=1 gating like riggermore; test only that auth/upload contract matches meshgod/ops.py — don't burn farm jobs on repeat runs). One render-quality note for your backlog, not urgent: draft webm via MediaRecorder is untested in the embedded preview (rAF suspension) — verify it once in a real Chrome window before calling it done.

2026-07-19 — M5 DIRECTOR MODE unlocked: POST /director (spec PLAN §5 M5).

• rhubarb is now INSTALLED on ultra (built from source, arm64, v1.14.0, symlinked /opt/homebrew/bin/rhubarb) — your happy path is verified: GET /rhubarb?path=audio/test/vo.wav → real mouthCues. Nothing to do there.
• POST /director {text, scene:{entities:[{id,kind,label}], time}} → chat request to SCENEGOD_LLM_URL (OpenAI-compat; ultra's tailnet Ollama is http://100.69.21.128:11434, model via SCENEGOD_LLM_MODEL). Orchestrator checked /api/tags on that box: it serves qwen2.5:7b (+ an embed model) — default SCENEGOD_LLM_MODEL to qwen2.5:7b; John can pull a gemma later and it's just an env change. 503 with a hint when env unset. stdlib urllib only.
• System prompt: translate director language into a JSON array of ops matching Lane A's scenegod:direct contract EXACTLY (enumerate the shot/ angle/focal/light/mark vocabularies from grammar.js in the prompt; read the file at boot so the vocab never drifts). format: json / low temp.
• VALIDATE server-side before returning: unknown op/shot/preset/entity id → drop the op and include it in a rejected list so the UI can toast it. Never forward raw model text to the client.
• test_server.py: mock the LLM (in-process http.server, same pattern as your MB mock): assert prompt carries the vocab, valid ops pass, garbage ops land in rejected, 503 when env unset.

2026-07-18 — M4 ACCEPTED. Lane C scope (M1–M4) is COMPLETE.

Orchestrator verified in the integrated app: audio-muxed finalRender → h264+aac mp4, both streams exactly 8.000s; graft_limb --selftest green under Blender 5.0.1 (both refusal + graft paths); 8 server test groups pass. Your blender-path correction noted (it's /Applications/Blender.app/..., not on PATH — my error). Standing by is fine; remaining work is A/B UI (viseme lane, audio rows). One item if you want it: add a rhubarb check + /rhubarb?path= endpoint (subprocess → JSON) IF rhubarb is installed — 503-with-message otherwise, same pattern as ffmpeg. Log a REQUEST first if it needs more than that.

2026-07-19 — M6 FLOW VIDEO PLATES queued behind M5 (full spec PLAN §5 M6).

Do M5 first; M6 is unlocked the moment your M5 acceptance is logged. Test media will be arriving in assets/backdrops/video/ via scripts/flow_intake.py.

2026-07-20 — M5+M6 ACCEPTED at SYNC 6 (all lanes ran as orchestrator subagents).

Verified live: DIRECT bar CU/low/frame → camera+keys+cut, ONE undo reverts; golden_hour keys; video plate follows the clock frame-exact; finalRender shows the Flow plate behind an animating character. Orchestrator seam fixes to absorb: (1) server: VIDEO_EXTS in tree/media-types, video beats poster as primary; (2) stage.entityVideo(id) added (Lane B seam); (3) syncVideos forces videoTex.needsUpdate — r160 VideoTexture rides requestVideoFrameCallback which never fires in hidden tabs → black plates in renders. Backlog carried: honor audio in/out at mux (C), walk-to-mark prevPos pre-apply (A), live viseme asset still owed by John.