plan: SCENEGOD master plan + 3-lane instructions + log scaffolding
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
commit
e29edc808b
7
.gitignore
vendored
Normal file
7
.gitignore
vendored
Normal file
@ -0,0 +1,7 @@
|
||||
.env
|
||||
assets/
|
||||
scenes/
|
||||
renders/
|
||||
__pycache__/
|
||||
*.pyc
|
||||
.DS_Store
|
||||
61
CLAUDE.md
Normal file
61
CLAUDE.md
Normal file
@ -0,0 +1,61 @@
|
||||
# SCENEGOD — machinima director's sandbox (iClone/SFM UX, GODVERSE stack)
|
||||
|
||||
Drop characters + animation clips + image backdrops on a 3D stage, keyframe
|
||||
transforms/cameras/lights on a timeline, render mp4. Web app: FastAPI +
|
||||
three.js, same conventions as MESHGOD (`~/Documents/MESHGOD`).
|
||||
|
||||
## You are one of THREE parallel lane agents
|
||||
1. **Read, in order:** this file → `PLAN.md` → your lane file in `lanes/` →
|
||||
your log in `logs/`. Your lane file is your source of truth — the
|
||||
orchestrator (Fable) updates it between your sessions. Re-read it at the
|
||||
start of EVERY session; instructions may have changed since your last one.
|
||||
2. **Only touch files your lane owns** (ownership map in PLAN.md §3). If you
|
||||
need a change in another lane's file, write the request in your log under
|
||||
BLOCKED/REQUESTS — do not edit it yourself.
|
||||
3. **Log protocol** — append (never rewrite) to `logs/lane<X>.md` at the end
|
||||
of every session, and mid-session when you make a design decision:
|
||||
|
||||
```
|
||||
## 2026-07-18 session N
|
||||
DONE: what shipped, with file:line pointers
|
||||
DECISIONS: choices made + why (schema tweaks, deps, algorithms)
|
||||
BLOCKED/REQUESTS: what you need from another lane or the orchestrator
|
||||
NEXT: what you'd do next session
|
||||
```
|
||||
4. **Commit early, commit often**, on `main`, prefixed `[laneA]`/`[laneB]`/
|
||||
`[laneC]`. `git pull --rebase` before every commit — other lanes commit to
|
||||
the same branch. File ownership makes conflicts near-zero; if you hit one
|
||||
anyway, resolve only your own files and log it.
|
||||
5. **Don't exceed your lane's current milestone.** Milestones and sync points
|
||||
are in PLAN.md §5. Finishing early = polish + tests + log, not starting
|
||||
the next milestone uninvited.
|
||||
|
||||
## Stack rules
|
||||
- Python 3.11 stdlib + `fastapi` + `uvicorn` only (requirements.txt). No
|
||||
other server deps without logging a REQUEST first.
|
||||
- Frontend: vanilla ES modules + three.js via importmap (copy the pattern
|
||||
from `/Users/johnking/Documents/MESHGOD/meshgod/web/rigroom.html`). No
|
||||
bundler, no npm, no framework.
|
||||
- Server: `uvicorn scenegod.server:app --port 8020`, binds 127.0.0.1 by
|
||||
default (local tool on ultra; VPS deploy is a later problem).
|
||||
- Secrets/env in `.env` (gitignored). Never commit tokens.
|
||||
- Reference code you may READ (never edit): the MESHGOD repo, especially
|
||||
`meshgod/web/rigroom.html` (retarget bake), `meshgod/library.py` (folder
|
||||
scanning), `meshgod/ops.py` (MODELBEAST submit contract),
|
||||
`scripts/glb2fbx.py` (headless Blender pattern).
|
||||
|
||||
## Gotchas inherited from MESHGOD (learned the hard way — do not relearn)
|
||||
- trimesh/some GLBs omit `metallicFactor` → glTF default 1.0 → renders black.
|
||||
Viewer must use an IBL environment (`RoomEnvironment` + PMREM), not just
|
||||
lights.
|
||||
- Serve HTML **no-cache** — a stale cache once hid UI buttons for a day.
|
||||
- Mixamo rig quirks (all already solved in rigroom's `bakeRetarget` — port,
|
||||
don't reinvent): namespaces differ per download (`mixamorig4Hips`), FBX
|
||||
files carry duplicate hierarchies, quaternion sign continuity must be
|
||||
enforced, Blender-exported GLB rest poses differ from FBX rests → retarget
|
||||
= world-space rotation-delta bake at 30fps, never raw track copy.
|
||||
- Rigged GLBs must NEVER be sent through any Blender join/decimate "finish"
|
||||
path — it spawns phantom joint meshes and mauls the armature.
|
||||
- Path traversal: every endpoint that takes a path must resolve + verify it
|
||||
stays inside its root (ship-check rule).
|
||||
- Verbose commands (installs, test runs, ffmpeg): pipe through `| lm -l 2`.
|
||||
208
PLAN.md
Normal file
208
PLAN.md
Normal file
@ -0,0 +1,208 @@
|
||||
# SCENEGOD master plan
|
||||
|
||||
Owner: John. Orchestrator: Fable (reviews `logs/`, updates `lanes/`).
|
||||
Executors: three Opus 4.8 lane agents working in parallel on `main`.
|
||||
|
||||
## 1. What we're building
|
||||
|
||||
A browser page where John can:
|
||||
- drag characters (rigged GLB/FBX), props (MeshGod gallery GLBs), and image
|
||||
backdrops onto a lit 3D stage;
|
||||
- drop Mixamo-style animation clips onto characters (client-side retarget,
|
||||
ported from MESHGOD rigroom);
|
||||
- keyframe entity transforms, camera moves + cuts, and lighting on a
|
||||
timeline with a scrubber;
|
||||
- save/load scenes as JSON;
|
||||
- render the timeline to mp4 (draft: MediaRecorder webm; final: per-frame
|
||||
PNG → server ffmpeg);
|
||||
- later: audio tracks, MODELBEAST tts/voice-clone/lipsync, viseme lip sync
|
||||
for blendshape characters, and a headless-Blender limb-graft script.
|
||||
|
||||
Non-goals (v1): multiplayer, physics, in-browser modeling, VPS deploy,
|
||||
game-engine export. It renders video, it is not a game engine.
|
||||
|
||||
## 2. Repo layout (create as you go — no empty scaffolding)
|
||||
|
||||
```
|
||||
scenegod/
|
||||
server.py # FastAPI app (Lane C)
|
||||
web/
|
||||
index.html # page skeleton + importmap + mount divs (Lane A)
|
||||
style.css # (Lane A; B may append a clearly-marked timeline block)
|
||||
room3d.js # loaders + retarget, ported from rigroom (Lane A)
|
||||
stage.js # Stage class: scene/entities/gizmos (Lane A)
|
||||
dock.js # asset dock UI (Lane A)
|
||||
timeline.js # Timeline class: clock, tracks, evaluation (Lane B)
|
||||
tlui.js # timeline DOM/canvas UI (Lane B)
|
||||
render.js # draft+final render client (Lane C)
|
||||
scripts/
|
||||
graft_limb.py # headless Blender limb graft (Lane C, M4)
|
||||
test_server.py # endpoint smoke tests (Lane C)
|
||||
requirements.txt # fastapi, uvicorn (Lane C)
|
||||
lanes/ logs/ # orchestrator + agents
|
||||
```
|
||||
|
||||
## 3. File ownership (hard rule)
|
||||
|
||||
| Lane | Owns | May read everything, edits ONLY its own files |
|
||||
|---|---|---|
|
||||
| A — Stage | `web/index.html`, `web/style.css`, `web/room3d.js`, `web/stage.js`, `web/dock.js` | |
|
||||
| B — Timeline | `web/timeline.js`, `web/tlui.js` | appends to `style.css` only inside `/* === LANE B === */` block |
|
||||
| C — Server | `scenegod/server.py`, `web/render.js`, `scripts/*`, `requirements.txt`, `.gitignore` | |
|
||||
|
||||
`PLAN.md` + `lanes/*` = orchestrator only. `logs/lane<X>.md` = its agent only.
|
||||
Schema changes (§4): propose in your log, orchestrator amends PLAN.md, then
|
||||
everyone follows the new version.
|
||||
|
||||
## 4. Contracts (the reason three lanes can run in parallel)
|
||||
|
||||
### 4.1 Scene JSON — schema v1
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"version": 1,
|
||||
"name": "street-scene",
|
||||
"fps": 30,
|
||||
"duration": 20.0, // seconds
|
||||
"entities": [
|
||||
{
|
||||
"id": "e1", // unique, stable
|
||||
"kind": "character", // character|prop|backdrop|camera|light
|
||||
"label": "lady",
|
||||
"source": { "type": "assets", // assets|upload (upload = client-side only, warn on save)
|
||||
"path": "characters/pack01/lady.glb" },
|
||||
"params": {}, // kind-specific, see below
|
||||
"transform": { "pos": [0,0,0], "rot": [0,0,0], "scale": 1 }, // rest pose (used when no keys)
|
||||
"tracks": {
|
||||
"transform": [ // sorted by t; euler rad; ease: linear|in|out|inout|step
|
||||
{ "t": 0, "pos": [0,0,0], "rot": [0,0,0], "scale": 1, "ease": "linear" }
|
||||
],
|
||||
"params": [ // keyable scalars/colors, e.g. camera fov, light intensity
|
||||
{ "t": 0, "key": "fov", "value": 45, "ease": "inout" }
|
||||
],
|
||||
"clips": [ // characters only; non-overlapping except fade regions
|
||||
{ "path": "animations/pack01/walk.fbx", "clipIndex": 0,
|
||||
"start": 1.0, // timeline seconds where block begins
|
||||
"in": 0.0, "out": 2.4, // trim within source clip
|
||||
"loop": 2, // repeat count (1 = once)
|
||||
"fade": 0.25 } // crossfade seconds into NEXT block
|
||||
]
|
||||
}
|
||||
}
|
||||
],
|
||||
"cameraCuts": [ { "t": 0, "camera": "e7" } ], // active camera entity id from t onward
|
||||
"audio": [ { "path": "audio/vo1.wav", "start": 0.0, "gain": 1.0 } ] // M4
|
||||
}
|
||||
```
|
||||
|
||||
`params` by kind — `camera`: `{fov}`; `light`: `{type:"key"|"ambient", color:"#fff",
|
||||
intensity, castShadow}`; `backdrop`: `{mode:"plane"|"corner"|"dome", image:"backdrops/street.jpg",
|
||||
width}`. Exactly one `light` entity of type `ambient` per scene (hemisphere);
|
||||
`key` lights are directional, multiple allowed.
|
||||
|
||||
### 4.2 JS module contracts
|
||||
|
||||
`room3d.js` (Lane A) exports:
|
||||
```js
|
||||
parseAny(arrayBuffer, filename) -> Promise<{root, anims}> // glb|fbx|obj|bvh
|
||||
canon(name) -> string
|
||||
boneMap(root) -> {canonName: Bone}
|
||||
captureRest(root) -> {rest: Map<Object3D,{wq,parent}>, order: Object3D[]}
|
||||
bakeRetarget(clip, srcRoot, srcRest, tgtRoot, tgtRest) -> AnimationClip // parameterized target (rigroom closes over a global — fix that in the port)
|
||||
disposeRoot(root); stats(root) -> {tris, meshes, bones}
|
||||
```
|
||||
|
||||
`stage.js` (Lane A) exports `class Stage`:
|
||||
```js
|
||||
new Stage(viewportEl) // renderer, scene, IBL env, grid, orbit "director cam"
|
||||
async addEntity(desc) -> entity // desc = scene-JSON entity; loads asset, builds object
|
||||
removeEntity(id); getEntity(id); entities() -> entity[]
|
||||
select(id|null); onSelect(cb); onChange(cb) // gizmo edits fire onChange(entity)
|
||||
entityTransform(id) -> {pos,rot,scale} // current gizmo state (for "capture keyframe")
|
||||
setTransform(id, {pos,rot,scale}) // timeline drives this every tick
|
||||
setParam(id, key, value) // fov, light color/intensity, ...
|
||||
prepareClip(id, path, clipIndex) -> Promise<AnimationClip> // fetch+parse+retarget, CACHED
|
||||
entityMixer(id) -> AnimationMixer
|
||||
setActiveCamera(id|null) // null = director orbit cam
|
||||
renderActiveCamera(canvasOrNull) // PiP preview / final-render target
|
||||
captureState() -> sceneJsonEntities; async applyState(sceneJson)
|
||||
```
|
||||
|
||||
`timeline.js` (Lane B) exports `class Timeline`:
|
||||
```js
|
||||
new Timeline(stage) // talks to Stage ONLY via the API above
|
||||
load(sceneJson); toJSON() -> sceneJson // owns duration/fps/tracks/cameraCuts
|
||||
time; playing; play(); pause(); seek(t); onTick(cb(t))
|
||||
evaluate(t) // interpolate transform+param keys → stage.setTransform/…
|
||||
// activate clip blocks → mixer weights/time, crossfades
|
||||
// camera cuts → stage.setActiveCamera
|
||||
step(frame) // deterministic: seek(frame/fps)+evaluate — used by final render
|
||||
```
|
||||
Until Lane A ships Stage (M1), Lane B develops against `stagestub.js`
|
||||
(Lane B writes it, same API, logs to console) — delete at integration.
|
||||
|
||||
### 4.3 HTTP endpoints (Lane C)
|
||||
|
||||
```
|
||||
GET / stageroom page (no-cache) + all /web statics
|
||||
GET /assets/tree {characters:[...], animations:[...], props:[...], backdrops:[...], audio:[...]}
|
||||
scans SCENEGOD_ASSETS folder tree live (rigroom-library style:
|
||||
dir+stem grouping, no index file); props may also merge the
|
||||
MESHGOD gallery if MESHGOD_GALLERY_URL is set (later, M3)
|
||||
GET /assets/file?path= streams one asset (path-traversal-guarded)
|
||||
GET /scenes list; GET /scenes/{name} one
|
||||
POST /scenes/{name} save scene JSON (validated: version, ids unique, sorted tracks)
|
||||
POST /render/begin {fps,width,height,name} -> {renderId}
|
||||
POST /render/{id}/frame/{n} raw PNG body
|
||||
POST /render/{id}/end {audio?: [...]}: ffmpeg PNG seq (+audio M4) -> mp4
|
||||
GET /render/{id}/status ; GET /render/{id}/out.mp4
|
||||
```
|
||||
Env: `SCENEGOD_ASSETS` (default `./assets`), `SCENEGOD_SCENES` (default
|
||||
`./scenes`), `SCENEGOD_PORT` (8020). John symlinks/rsyncs his existing
|
||||
banks into `assets/` — the server never writes into it except `/render`.
|
||||
|
||||
## 5. Milestones & sync points
|
||||
|
||||
**M1 — walking skeleton** (all lanes, independent):
|
||||
- A: index.html + Stage with IBL viewer, load a GLB character + an image
|
||||
backdrop from the dock, gizmo move/rotate/scale, retarget port proven
|
||||
(load fbx clip onto character, plays once via a temp button).
|
||||
- B: Timeline UI against stagestub — tracks list, scrubber, add/drag/delete
|
||||
transform keyframes, play/pause with interpolation visible in console.
|
||||
- C: server serving the page + /assets/tree + /assets/file + /scenes CRUD,
|
||||
with scripts/test_server.py passing.
|
||||
- **SYNC 1** (orchestrator): wire B to real Stage, delete stub. Definition of
|
||||
done: load scene JSON → character walks between two keyframed positions
|
||||
in front of a street backdrop, scrubbed by the timeline.
|
||||
|
||||
**M2 — direction**: A: multiple cameras + PiP + light entities + param
|
||||
plumbing. B: clip blocks with trim/loop/crossfade, params track, camera-cut
|
||||
track, snapping, undo (simple command stack). C: scene validation hardening,
|
||||
asset thumbnails endpoint. **SYNC 2**: the "lady + man + street + 2 cameras +
|
||||
sunset" demo scene plays end to end.
|
||||
|
||||
**M3 — render**: C: render session endpoints + ffmpeg + render.js client
|
||||
(draft MediaRecorder + final frame-step using `timeline.step`). A/B: whatever
|
||||
SYNC 2 revealed + polish pass from orchestrator notes. **SYNC 3**: 10-second
|
||||
1080p mp4 rendered deterministically.
|
||||
|
||||
**M4 — sound & flesh**: C: audio in scene JSON + ffmpeg mux + MODELBEAST
|
||||
tts/voice-clone submit (mb contract from meshgod/ops.py, token from
|
||||
`~/Documents/backnforth/.env`, NEVER logged) + `scripts/graft_limb.py`.
|
||||
B: audio tracks on the timeline UI, Web Audio sync. A: viseme lane — detect
|
||||
morph targets, Rhubarb-JSON → morph keyframes, greyed out when absent.
|
||||
|
||||
## 6. How John launches the lanes
|
||||
|
||||
Three Claude Code sessions (Opus 4.8), cwd `~/Documents/SCENEGOD`, one prompt
|
||||
each — see README-LANES section at the bottom of each lane file, or just:
|
||||
|
||||
> You are Lane A (or B / C). Read CLAUDE.md, PLAN.md, lanes/A-stage.md and
|
||||
> logs/laneA.md, then continue the work from where the log leaves off. Obey
|
||||
> the file-ownership map and the log protocol. Stop at your current
|
||||
> milestone boundary.
|
||||
|
||||
Orchestrator loop (Fable, this session or any MESHGOD/SCENEGOD session):
|
||||
read `logs/*.md` → answer BLOCKED/REQUESTS by editing `lanes/*.md` (append a
|
||||
dated **ORCHESTRATOR UPDATE** section) → run the SYNC integration when all
|
||||
three logs report their milestone DONE.
|
||||
100
lanes/A-stage.md
Normal file
100
lanes/A-stage.md
Normal file
@ -0,0 +1,100 @@
|
||||
# Lane A — Stage: viewport, assets on stage, retarget port
|
||||
|
||||
You own: `scenegod/web/index.html`, `style.css`, `room3d.js`, `stage.js`, `dock.js`.
|
||||
Milestone: **M1** (do not start M2 items until the orchestrator flips this line).
|
||||
|
||||
## A1. index.html skeleton (first commit)
|
||||
- Layout: full-viewport CSS grid — left dock `#dock` (280px), center `#view`,
|
||||
bottom `#timeline` (260px, empty div — Lane B mounts into it), right
|
||||
`#inspector` (260px). Dark UI, system font; steal the visual language of
|
||||
MESHGOD's rigroom (read `/Users/johnking/Documents/MESHGOD/meshgod/web/rigroom.html`).
|
||||
- Importmap: copy rigroom's exactly (same three.js version + addons paths) so
|
||||
behavior matches the code you're porting.
|
||||
- Script entry: `<script type="module" src="/web/main-stage.js">`? No — keep
|
||||
entry inline in index.html: `import {Stage} from './stage.js'` etc., create
|
||||
the Stage, Dock, and (when it exists) `new Timeline(stage)` guarded by a
|
||||
dynamic import that tolerates timeline.js not existing yet:
|
||||
`try { const {Timeline} = await import('./timeline.js'); ... } catch {}`.
|
||||
That keeps lanes decoupled — your page must never be broken by B's absence.
|
||||
- Style: `style.css`; leave a `/* === LANE B === */` marker at the bottom.
|
||||
|
||||
## A2. room3d.js — port from rigroom.html (the crown jewels)
|
||||
Source: `/Users/johnking/Documents/MESHGOD/meshgod/web/rigroom.html` lines
|
||||
~125–290 (loaders, canon/boneMap, captureRest, bakeRetarget, stats,
|
||||
disposeRoot). Port rules:
|
||||
- Make it a real ES module with the exports listed in PLAN §4.2. No DOM, no
|
||||
globals, no toast() — throw Errors, callers handle UI.
|
||||
- **Parameterize the target**: rigroom's `bakeRetarget` closes over the
|
||||
global `char`. New signature
|
||||
`bakeRetarget(clip, srcRoot, srcRest, tgtRoot, tgtRest) -> AnimationClip`.
|
||||
Keep the algorithm byte-for-byte otherwise: 30fps sampling, world-space
|
||||
rotation delta from source rest applied to target rest, parent-first world
|
||||
quat propagation, hips world-position track scaled by target/source hips
|
||||
height ratio, quaternion sign continuity (dot < 0 → negate), and the
|
||||
duplicate-FBX-hierarchy dedupe (bind only first bone per canon key).
|
||||
- Keep `canon()` exactly as-is (`mixamorig\d*[:_]?` strip) — it encodes real
|
||||
Mixamo quirks.
|
||||
- Self-check: `room3d.js` gets a tiny exported `_selftest()` that builds two
|
||||
3-bone synthetic skeletons with different rests, bakes a 1s clip across,
|
||||
and asserts end-pose world quats match within 1e-3. Wire it to
|
||||
`?selftest=1` on the page; log the result.
|
||||
|
||||
## A3. stage.js — the Stage class (PLAN §4.2 API)
|
||||
- Renderer setup: copy rigroom's (antialias, PMREM + **RoomEnvironment IBL**
|
||||
— metals gotcha), hemisphere + key directional light as *default* lights
|
||||
(replaced by scene light entities when present), circle floor + grid,
|
||||
OrbitControls director camera.
|
||||
- `addEntity(desc)`:
|
||||
- `character`/`prop`: fetch `/assets/file?path=`, `parseAny`, normalize
|
||||
scale like rigroom (fit ~1.7m for characters; props keep native size but
|
||||
expose scale), ground to y=0, attach to a `THREE.Group` wrapper — ALL
|
||||
transforms (gizmo + timeline) act on the wrapper, never the loaded root.
|
||||
For characters: `captureRest` once at load, store on the entity; create
|
||||
`AnimationMixer` on the wrapper.
|
||||
- `backdrop`: `params.mode` — `plane`: textured PlaneGeometry, height from
|
||||
image aspect × `params.width` (default 10m); `corner`: plane + floor
|
||||
plane sharing the image bottom third (cheap "L" street corner); `dome`:
|
||||
40m sphere, `side: BackSide`, texture on inside. `SRGBColorSpace` on all
|
||||
image textures.
|
||||
- `camera`: PerspectiveCamera in a wrapper + a small frustum helper mesh so
|
||||
it's visible/selectable in the director view. `params.fov`.
|
||||
- `light`: `key` → DirectionalLight + helper; `ambient` → replaces the
|
||||
default hemisphere intensity. Adding the first `key` light dims defaults.
|
||||
- Selection: raycast on click → `select(id)`, outline or bbox highlight,
|
||||
`TransformControls` gizmo (W/E/R = move/rotate/scale) on the wrapper;
|
||||
gizmo drag end → `onChange(entity)`.
|
||||
- `prepareClip(id, path, clipIndex)`: fetch+parse the clip source, cache
|
||||
`{srcRoot, srcRest, anims}` per path (Map), bake per (entity, path, index)
|
||||
and cache the baked clip. Baking is the expensive step — never re-bake on
|
||||
seek.
|
||||
- `renderActiveCamera(canvas)`: render the active camera to a small PiP
|
||||
canvas overlay (M2) and, for Lane C's final render, to the main canvas at
|
||||
exact size. Keep director-orbit rendering the default main view.
|
||||
- `captureState()/applyState()`: round-trip with scene JSON entities. Assets
|
||||
with `source.type:"upload"` (drag-and-dropped local files) live only in
|
||||
memory — mark them in the dock and warn on save.
|
||||
|
||||
## A4. dock.js
|
||||
- Tabs: Characters / Animations / Props / Backdrops from `GET /assets/tree`
|
||||
(Lane C; until it exists use a hardcoded `mock=1` tree + local file-drop —
|
||||
drag any glb/fbx/image onto the page adds it as `source.type:"upload"`).
|
||||
- Click or drag-onto-viewport → `stage.addEntity`. Animations are NOT added
|
||||
to the stage; dragging one onto a character in the viewport (raycast under
|
||||
drop point) or onto its inspector calls `prepareClip` + M1 temp "play once"
|
||||
button. In M2, B takes over clip placement (dock drag → timeline track).
|
||||
- Inspector panel: selected entity's label, kind, transform numbers (editable),
|
||||
params (fov slider for cameras, color+intensity for lights, backdrop mode),
|
||||
delete button, "⏺ key" button that emits
|
||||
`window.dispatchEvent(new CustomEvent('scenegod:capturekey',{detail:{id}}))`
|
||||
— Lane B listens; you don't call timeline code directly.
|
||||
|
||||
## M1 acceptance (log it with evidence)
|
||||
- `?selftest=1` retarget check passes.
|
||||
- Load lady.glb (put test files in `assets/` yourself — grab from MESHGOD
|
||||
library banks), drop street.jpg as plane backdrop, gizmo-move the lady,
|
||||
drop walk.fbx on her → she walks in place. Screenshot in log dir welcome
|
||||
(`logs/shots/`, yours to create).
|
||||
- Page loads clean with timeline.js absent AND with server absent (mock=1).
|
||||
|
||||
## ORCHESTRATOR UPDATES
|
||||
(none yet — check back each session)
|
||||
81
lanes/B-timeline.md
Normal file
81
lanes/B-timeline.md
Normal file
@ -0,0 +1,81 @@
|
||||
# Lane B — Timeline: master clock, tracks, keyframes, evaluation
|
||||
|
||||
You own: `scenegod/web/timeline.js`, `tlui.js` (+ your marked block at the
|
||||
bottom of `style.css`, + your dev-only `stagestub.js` until SYNC 1).
|
||||
Milestone: **M1**. You develop against the Stage API in PLAN §4.2 — code to
|
||||
the contract, not to Lane A's implementation.
|
||||
|
||||
## B1. stagestub.js (first commit, deleted at SYNC 1)
|
||||
Implements the full Stage API from PLAN §4.2 with plain objects: entities
|
||||
are `{id, kind, transform, params}`, `setTransform`/`setParam` record into a
|
||||
`stub.applied` array and `console.debug`. `prepareClip` returns a fake
|
||||
`{duration: 2.4}` clip. This lets every timeline feature be built and tested
|
||||
headless before Lane A lands.
|
||||
|
||||
## B2. timeline.js — the model + clock (no DOM in this file)
|
||||
- Owns the authoritative scene fields: `fps`, `duration`, `entities[].tracks`,
|
||||
`cameraCuts`. `load(sceneJson)` / `toJSON()` round-trip **losslessly** —
|
||||
unknown fields on entities are preserved (Lane A owns them).
|
||||
- Master clock: `play()` uses `requestAnimationFrame` deltas; `seek(t)`
|
||||
clamps to `[0, duration]`; every advance calls `evaluate(t)` then
|
||||
`onTick` listeners. `step(frame)` = `seek(frame/fps)` with NO raf — pure
|
||||
and deterministic (Lane C's final render calls it in a loop).
|
||||
- `evaluate(t)` per entity:
|
||||
1. **Transform track**: binary-search the bracketing keys, interpolate —
|
||||
pos/scale lerp, rot via Quaternion.slerp (convert stored eulers once at
|
||||
load, cache quats on the key objects), easing (`linear|in|out|inout` =
|
||||
cubic, `step` = hold) applied to the segment's normalized u. 0 keys →
|
||||
rest transform; 1 key → hold. Result → `stage.setTransform(id, …)`.
|
||||
2. **Params track**: same interpolation per `key` name (numbers lerp,
|
||||
colors lerp in sRGB — fine for v1) → `stage.setParam`.
|
||||
3. **Clip blocks** (characters): a block covers
|
||||
`[start, start + (out-in)*loop]`. If t inside: local clip time =
|
||||
`in + ((t-start) % (out-in))`; drive via the entity's mixer — one
|
||||
`AnimationAction` per block, `action.time = localTime;
|
||||
action.weight = w; mixer.update(0)` (set-time style, NOT free-running —
|
||||
scrubbing must be exact). Crossfade: in the last `fade` seconds of a
|
||||
block that has a successor, ramp weight 1→0 while successor ramps 0→1.
|
||||
Outside any block: weight 0 (character holds rest / last transform).
|
||||
4. **Camera cuts**: last cut with `cut.t <= t` → `stage.setActiveCamera`.
|
||||
- Keyframe ops API (tlui + Lane A's event use these): `addKey(id, track, key)`,
|
||||
`moveKey`, `deleteKey`, `addClipBlock(id, block)`, `moveClipBlock`,
|
||||
`trimClipBlock`, `addCut(t, cameraId)` — every mutator keeps arrays sorted
|
||||
by `t` and pushes an inverse op onto a simple undo stack (M2: wire to
|
||||
Ctrl+Z; the stack itself costs nothing now).
|
||||
- Listen for `scenegod:capturekey` (see Lane A dock): on it,
|
||||
`addKey(id, 'transform', {t: this.time, ...stage.entityTransform(id), ease:'inout'})`.
|
||||
|
||||
## B3. tlui.js — the panel (mounts into `#timeline`)
|
||||
- Left column: track names (one row per entity, auto-synced to
|
||||
stage.entities via `onChange`/poll; + Cameras row + a per-character clip
|
||||
sub-row). Right: time ruler + canvas-drawn track lanes (a `<canvas>` per
|
||||
lane or one big one — your call, log the decision).
|
||||
- Interactions (M1): click ruler = seek; drag playhead; space = play/pause;
|
||||
double-click a transform lane = add key at that t from current stage state;
|
||||
drag key = move (snap to frame, hold Alt = free); right-click key = delete;
|
||||
drag a clip block horizontally; drag block edges = trim. Home/End = 0/duration.
|
||||
- Scene bar: name field, duration field, Save/Load buttons →
|
||||
`POST/GET /scenes/{name}` (Lane C; until then, localStorage fallback —
|
||||
keep the fetch code, catch and fall back).
|
||||
- Keep ALL DOM here; timeline.js must run headless under node for tests.
|
||||
|
||||
## B4. self-check — `web/timeline_test.mjs`
|
||||
Runs under plain `node` (import timeline.js + stagestub): build a scene with
|
||||
2 transform keys + 1 clip block + 2 camera cuts, `step()` through 90 frames,
|
||||
assert: interpolated midpoint position, ease-step hold, clip local time at
|
||||
3 sample frames, active camera flips at the cut, `toJSON()` deep-equals a
|
||||
re-`load`ed copy. `node web/timeline_test.mjs` green = M1 code-done.
|
||||
|
||||
## M1 acceptance (log it)
|
||||
- timeline_test.mjs passes.
|
||||
- In the browser against the stub: scrub bar moves entities in the stub log,
|
||||
keys draggable, play/pause works, save/load round-trips via localStorage.
|
||||
- Do NOT integrate with the real Stage yourself — that's SYNC 1, the
|
||||
orchestrator runs it (your code should make it a one-line swap).
|
||||
|
||||
## M2 preview (locked until orchestrator flips it)
|
||||
Clip crossfades polish, params lanes UI, camera-cut lane UI, snapping
|
||||
config, undo wiring, box-select + multi-drag of keys.
|
||||
|
||||
## ORCHESTRATOR UPDATES
|
||||
(none yet — check back each session)
|
||||
89
lanes/C-server.md
Normal file
89
lanes/C-server.md
Normal file
@ -0,0 +1,89 @@
|
||||
# Lane C — Server: FastAPI, assets, scenes, render pipeline
|
||||
|
||||
You own: `scenegod/server.py`, `web/render.js`, `scripts/*`,
|
||||
`requirements.txt`, `.gitignore`. Milestone: **M1**.
|
||||
|
||||
## C1. Scaffold (first commit)
|
||||
- `requirements.txt`: `fastapi`, `uvicorn`. Nothing else.
|
||||
- `.gitignore`: `.env`, `assets/`, `scenes/`, `renders/`, `__pycache__/`,
|
||||
`*.pyc`, `.DS_Store`.
|
||||
- `scenegod/server.py`, single file (split only when it hurts — MESHGOD's
|
||||
server.py is one file at 800+ lines and fine). `SCENEGOD_PORT` 8020,
|
||||
bind `127.0.0.1` (env `SCENEGOD_BIND` to override later — not now).
|
||||
- Statics: `GET /` → `web/index.html` with
|
||||
`Cache-Control: no-store` (MESHGOD stale-cache lesson); `/web/*` served
|
||||
with short cache. Use FastAPI `FileResponse`/`StaticFiles`, headers set
|
||||
explicitly for the HTML.
|
||||
|
||||
## C2. Assets endpoints
|
||||
- Roots from env with defaults: `SCENEGOD_ASSETS=./assets`,
|
||||
`SCENEGOD_SCENES=./scenes`, `SCENEGOD_RENDERS=./renders` (create scenes/
|
||||
renders on boot; assets must exist — clear error if not).
|
||||
- `GET /assets/tree`: walk `assets/{characters,animations,props,backdrops,audio}`.
|
||||
Group rigroom-library style: files sharing dir+stem = one entry with a
|
||||
`formats` list (`{"name":"lady","path":"characters/pack01/lady.glb",
|
||||
"formats":["glb","fbx"]}`). Extensions: glb/gltf/fbx/obj/bvh + jpg/png/webp
|
||||
(backdrops) + wav/mp3 (audio). Scan live, no index file, cache 5s.
|
||||
- `GET /assets/file?path=`: **resolve and enforce
|
||||
`path.resolve().is_relative_to(ASSETS_ROOT)`** — reject `..`, absolute
|
||||
paths, symlink escapes (resolve first, then check). Correct content-types
|
||||
(`model/gltf-binary`, etc.). This guard pattern applies to every
|
||||
path-taking endpoint you ever add here (ship-check rule).
|
||||
- Read `/Users/johnking/Documents/MESHGOD/meshgod/library.py` first — same
|
||||
idea, steal the grouping approach.
|
||||
|
||||
## C3. Scenes CRUD
|
||||
- `GET /scenes` → `[{name, mtime, duration}]`; `GET /scenes/{name}`;
|
||||
`POST /scenes/{name}` with JSON body. `name` slugified `[a-z0-9-_]`, file
|
||||
`scenes/{name}.json`, atomic write (tmp + rename).
|
||||
- Validate on save (stdlib, no jsonschema dep): `version==1`, entity ids
|
||||
unique, every track sorted by `t`, clip blocks non-overlapping per entity
|
||||
(fade overlap allowed), `cameraCuts` reference existing camera entities.
|
||||
Reject with a 422 listing every violation (agents and UI both rely on the
|
||||
messages).
|
||||
|
||||
## C4. scripts/test_server.py (M1 gate)
|
||||
Stdlib `urllib` + `subprocess` (uvicorn on a test port, tmp asset/scene
|
||||
dirs): asserts tree grouping, file streaming, traversal attempts → 4xx
|
||||
(`?path=../../etc/hosts`, absolute, URL-encoded `..`), scene round-trip,
|
||||
validation failures (unsorted keys, dup ids). `python3 scripts/test_server.py`
|
||||
green = M1 code-done. Run with `| lm -l 2`.
|
||||
|
||||
## C5. Render pipeline — M3, but design lands now (endpoints stubbed 501)
|
||||
- Session model: `POST /render/begin {name,fps,width,height}` → `{renderId}`
|
||||
(uuid, dir `renders/{id}/frames/`). `POST /render/{id}/frame/{n}`: raw PNG
|
||||
body → `frames/{n:06d}.png`. `POST /render/{id}/end {audio?}` → spawn
|
||||
ffmpeg in a background thread:
|
||||
`ffmpeg -framerate {fps} -i frames/%06d.png -c:v libx264 -pix_fmt yuv420p
|
||||
-crf 18 out.mp4` (audio inputs + `-map`/amix in M4). `GET /render/{id}/status`
|
||||
= queued|encoding|done|error(+log tail); `GET /render/{id}/out.mp4`.
|
||||
ffmpeg presence checked at boot (`shutil.which`), endpoint 503s w/ message
|
||||
if absent. Reap render dirs >48h old on boot.
|
||||
- `web/render.js` (M3): exports `draftRecord(stage, timeline)` —
|
||||
`canvas.captureStream(fps)` + MediaRecorder while `timeline.play()`s, save
|
||||
webm; and `finalRender(stage, timeline, opts)` — resize canvas, loop
|
||||
`timeline.step(f)` → `stage.renderActiveCamera()` → `canvas.toBlob('image/png')`
|
||||
→ POST frame (3-4 in flight, backpressure via await), then `/end`, poll
|
||||
status, download link. Deterministic: never uses wall-clock time.
|
||||
|
||||
## C6. M4 (locked): audio, MODELBEAST, graft
|
||||
- ffmpeg audio mux per scene JSON `audio[]`.
|
||||
- `POST /mb/submit` proxy re-implementing the contract in MESHGOD
|
||||
`meshgod/ops.py` (upload input to /api/assets first; token read from
|
||||
`~/Documents/backnforth/.env` at request time, never logged; endpoint only
|
||||
active when `SCENEGOD_MB=1`).
|
||||
- `scripts/graft_limb.py` headless Blender (pattern:
|
||||
MESHGOD `scripts/glb2fbx.py`): `--body body.glb --limb hand.glb --bone
|
||||
mixamorig:RightHand` → align limb root bone to target bone head, join
|
||||
armatures, parent-keep-offset, join meshes (weights survive), export. If
|
||||
the body armature ALREADY has the bones the limb provides → exit 2 with
|
||||
"bones exist; mitten weights — re-skin, don't graft". Never route output
|
||||
through any finish/decimate path (rig contract).
|
||||
|
||||
## M1 acceptance (log it)
|
||||
- `uvicorn scenegod.server:app --port 8020` serves Lane A's page (or a
|
||||
placeholder index if A hasn't landed — don't block on them).
|
||||
- test_server.py green; paste the summary line in your log.
|
||||
|
||||
## ORCHESTRATOR UPDATES
|
||||
(none yet — check back each session)
|
||||
11
logs/laneA.md
Normal file
11
logs/laneA.md
Normal file
@ -0,0 +1,11 @@
|
||||
# Lane A log — append-only, newest at bottom
|
||||
|
||||
Format per session:
|
||||
|
||||
## <date> session N
|
||||
DONE: …
|
||||
DECISIONS: …
|
||||
BLOCKED/REQUESTS: …
|
||||
NEXT: …
|
||||
|
||||
(no sessions yet)
|
||||
11
logs/laneB.md
Normal file
11
logs/laneB.md
Normal file
@ -0,0 +1,11 @@
|
||||
# Lane B log — append-only, newest at bottom
|
||||
|
||||
Format per session:
|
||||
|
||||
## <date> session N
|
||||
DONE: …
|
||||
DECISIONS: …
|
||||
BLOCKED/REQUESTS: …
|
||||
NEXT: …
|
||||
|
||||
(no sessions yet)
|
||||
11
logs/laneC.md
Normal file
11
logs/laneC.md
Normal file
@ -0,0 +1,11 @@
|
||||
# Lane C log — append-only, newest at bottom
|
||||
|
||||
Format per session:
|
||||
|
||||
## <date> session N
|
||||
DONE: …
|
||||
DECISIONS: …
|
||||
BLOCKED/REQUESTS: …
|
||||
NEXT: …
|
||||
|
||||
(no sessions yet)
|
||||
Loading…
Reference in New Issue
Block a user