SCENEGOD/lanes/C-server.md
type-two 41f2c9fa8d [sync2+3] entity mirror seam, SYNC 2 demo + 1080p mp4 verified, M4/polish unlocked
- timeline.js: _mirror — Timeline subscribes stage.onChange, upserts/removes
  live stage entities into scene.entities (closes the dock-added-entity gap
  both lanes flagged; keyframing now works without a scene load)
- lane files: M2/M3 accepted for all lanes; A→PiP sizing + lady.glb material
  polish, B→absorb _mirror + test + setDuration mutator, C→M4 (audio mux,
  graft_limb, MB proxy)

SYNC 2: lady+man+street+2cams+sunset built via live dock path, all
frame-exact (probes in transcript). SYNC 3: finalRender → 240 frames
1920x1080 → ffmpeg h264 8.000s, cuts + sunset verified in extracted frames.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 20:34:32 +10:00

7.3 KiB
Raw Blame History

Lane C — Server: FastAPI, assets, scenes, render pipeline

You own: scenegod/server.py, web/render.js, scripts/*, requirements.txt, .gitignore. Milestone: M1.

C1. Scaffold (first commit)

  • requirements.txt: fastapi, uvicorn. Nothing else.
  • .gitignore: .env, assets/, scenes/, renders/, __pycache__/, *.pyc, .DS_Store.
  • scenegod/server.py, single file (split only when it hurts — MESHGOD's server.py is one file at 800+ lines and fine). SCENEGOD_PORT 8020, bind 127.0.0.1 (env SCENEGOD_BIND to override later — not now).
  • Statics: GET /web/index.html with Cache-Control: no-store (MESHGOD stale-cache lesson); /web/* served with short cache. Use FastAPI FileResponse/StaticFiles, headers set explicitly for the HTML.

C2. Assets endpoints

  • Roots from env with defaults: SCENEGOD_ASSETS=./assets, SCENEGOD_SCENES=./scenes, SCENEGOD_RENDERS=./renders (create scenes/ renders on boot; assets must exist — clear error if not).
  • GET /assets/tree: walk assets/{characters,animations,props,backdrops,audio}. Group rigroom-library style: files sharing dir+stem = one entry with a formats list ({"name":"lady","path":"characters/pack01/lady.glb", "formats":["glb","fbx"]}). Extensions: glb/gltf/fbx/obj/bvh + jpg/png/webp (backdrops) + wav/mp3 (audio). Scan live, no index file, cache 5s.
  • GET /assets/file?path=: resolve and enforce path.resolve().is_relative_to(ASSETS_ROOT) — reject .., absolute paths, symlink escapes (resolve first, then check). Correct content-types (model/gltf-binary, etc.). This guard pattern applies to every path-taking endpoint you ever add here (ship-check rule).
  • Read /Users/johnking/Documents/MESHGOD/meshgod/library.py first — same idea, steal the grouping approach.

C3. Scenes CRUD

  • GET /scenes[{name, mtime, duration}]; GET /scenes/{name}; POST /scenes/{name} with JSON body. name slugified [a-z0-9-_], file scenes/{name}.json, atomic write (tmp + rename).
  • Validate on save (stdlib, no jsonschema dep): version==1, entity ids unique, every track sorted by t, clip blocks non-overlapping per entity (fade overlap allowed), cameraCuts reference existing camera entities. Reject with a 422 listing every violation (agents and UI both rely on the messages).

C4. scripts/test_server.py (M1 gate)

Stdlib urllib + subprocess (uvicorn on a test port, tmp asset/scene dirs): asserts tree grouping, file streaming, traversal attempts → 4xx (?path=../../etc/hosts, absolute, URL-encoded ..), scene round-trip, validation failures (unsorted keys, dup ids). python3 scripts/test_server.py green = M1 code-done. Run with | lm -l 2.

C5. Render pipeline — M3, but design lands now (endpoints stubbed 501)

  • Session model: POST /render/begin {name,fps,width,height}{renderId} (uuid, dir renders/{id}/frames/). POST /render/{id}/frame/{n}: raw PNG body → frames/{n:06d}.png. POST /render/{id}/end {audio?} → spawn ffmpeg in a background thread: ffmpeg -framerate {fps} -i frames/%06d.png -c:v libx264 -pix_fmt yuv420p -crf 18 out.mp4 (audio inputs + -map/amix in M4). GET /render/{id}/status = queued|encoding|done|error(+log tail); GET /render/{id}/out.mp4. ffmpeg presence checked at boot (shutil.which), endpoint 503s w/ message if absent. Reap render dirs >48h old on boot.
  • web/render.js (M3): exports draftRecord(stage, timeline)canvas.captureStream(fps) + MediaRecorder while timeline.play()s, save webm; and finalRender(stage, timeline, opts) — resize canvas, loop timeline.step(f)stage.renderActiveCamera()canvas.toBlob('image/png') → POST frame (3-4 in flight, backpressure via await), then /end, poll status, download link. Deterministic: never uses wall-clock time.

C6. M4 (locked): audio, MODELBEAST, graft

  • ffmpeg audio mux per scene JSON audio[].
  • POST /mb/submit proxy re-implementing the contract in MESHGOD meshgod/ops.py (upload input to /api/assets first; token read from ~/Documents/backnforth/.env at request time, never logged; endpoint only active when SCENEGOD_MB=1).
  • scripts/graft_limb.py headless Blender (pattern: MESHGOD scripts/glb2fbx.py): --body body.glb --limb hand.glb --bone mixamorig:RightHand → align limb root bone to target bone head, join armatures, parent-keep-offset, join meshes (weights survive), export. If the body armature ALREADY has the bones the limb provides → exit 2 with "bones exist; mitten weights — re-skin, don't graft". Never route output through any finish/decimate path (rig contract).

M1 acceptance (log it)

  • uvicorn scenegod.server:app --port 8020 serves Lane A's page (or a placeholder index if A hasn't landed — don't block on them).
  • test_server.py green; paste the summary line in your log.

ORCHESTRATOR UPDATES

2026-07-18 — M1 ACCEPTED. M2 unlocked AND M3 render pipeline green-lit early.

test_server.py green, tree/file/scenes verified live through the browser page (scene save 200 + lossless reload). You're ahead of the other lanes, so skip the wait: after the small M2 items (thumbnails endpoint; validation already solid), go straight into §C5 — implement /render/begin|frame|end|status|out

  • ffmpeg encode, and web/render.js. Notes:
  1. timeline.step(frame) is verified deterministic against the real Stage (orchestrator stepped 0/30/60/90/120 and got exact eased positions) — your final-render loop can rely on it today. stage.renderActiveCamera as a render target is Lane A M2 work; until it lands, test finalRender with the director view canvas, and test the ffmpeg path with synthetic PNGs from a script (extend test_server.py: begin → POST 30 generated frames → end → poll → assert mp4 exists + ffprobe duration ≈ 1s).
  2. ffmpeg on ultra: check shutil.which at boot as designed; it's installed via homebrew here.
  3. Test assets for the tree now exist under assets/*/test/ (gitignored).
  4. uvicorn for local dev: .venv/bin/uvicorn scenegod.server:app --port 8020 — there's also a launch entry in MESHGOD/.claude/launch.json ("scenegod") used by the orchestrator's browser preview.

2026-07-18 — M2+M3 ACCEPTED, SYNC 3 PASSED. M4 is UNLOCKED.

Orchestrator ran your finalRender end-to-end in the integrated app against Lane A's renderActiveCamera: 240 frames @1920×1080 → ffmpeg → out.mp4, h264, exactly 8.000s, with camera cuts and sunset light lerp visible in extracted frames. The browser client path you couldn't exercise is now verified — no changes needed. Proceed with M4 in this order:

  1. ffmpeg audio mux from scene audio[] (the sync2-sunset scene in scenes/ is your fixture; add a wav to assets/audio/test/).
  2. scripts/graft_limb.py (spec in §C6 — blender binary is on ultra's PATH as blender; verify with a mixamo body + any rigged hand GLB, and add the refusal-path test).
  3. /mb/submit MODELBEAST proxy last (needs SCENEGOD_MB=1 gating like riggermore; test only that auth/upload contract matches meshgod/ops.py — don't burn farm jobs on repeat runs). One render-quality note for your backlog, not urgent: draft webm via MediaRecorder is untested in the embedded preview (rAF suspension) — verify it once in a real Chrome window before calling it done.