mrp3gi/SPEC.md

# MRP3GI — Specification & Execution Plan v1.0

**For the implementing agent:** this document is the contract. Work the
milestones in order; each has acceptance criteria you can verify yourself
(headless where possible). When this spec and convenience disagree, the
spec wins. When the spec is silent, follow the Architecture Invariants.

---

## 1. What this is

MRP3GI renders classic Sierra-style adventure games in **3D** (three.js,
rigged GLB characters, real camera) while the entire game — rooms, verbs,
hotspots, dialogue, flags, score, timers, death, saves — runs inside
**`mrpci-core`**, the existing Rust SCI-generation engine, compiled to
WASM. This is the Grim Fandango model: 3D presentation over a 2D brain.

Why this shape:

- `mrpci-core` is already headless, deterministic (fixed 30Hz ticks, seeded
  RNG, no wall clock), and driven entirely by a JSON Command/Event bus. Its
  native-only deps (`tiny_http`) are already fenced behind
  `cfg(not(target_arch = "wasm32"))`. It was built for this.
- Every MRPCI game (starting with the Neon Precinct demo) becomes a 3D game
  for free, and every future MRP3GI game is *also* playable in the 2D
  MRPCI GUI, testable over MCP, and replayable in CI.
- The sim's coordinate space is 2D (x 0..320, y 0..190, y = depth). Actors
  walk on a ground plane. That is exactly what a 3D adventure needs: the
  renderer maps sim (x, y) → world (X, Z) and stands a model there.

## 2. Architecture invariants (do not violate)

1. **No game rules in TypeScript.** If it changes game state, it is a
   `Command` into the core. If the player learns something, it arrived as
   an `Event`. The TS layer may keep *presentation* state only (camera,
   animation phase, UI open/closed).
2. **The bridge surface is JSON strings + byte buffers.** No fine-grained
   wasm-bindgen types for game objects — the wire format is the same one
   the JSONL/HTTP/MCP surfaces already speak (see `../mrpci/docs/CONTROL.md`).
3. **Determinism is a feature.** The golden-replay test (M1) must pass on
   every commit: the WASM core, fed the canonical playthrough script,
   produces byte-identical event JSON to the native binary.
4. **`mrpci-core` is consumed, not forked.** Path dependency `../mrpci`.
   If the core needs a change (it will — see §4.1), make it in the mrpci
   repo, keep it native-compatible, run mrpci's own tests, commit there.
5. **60fps presentation, 30Hz sim.** The renderer interpolates actor
   positions between sim ticks; it never runs the sim faster to look
   smoother.

## 3. Repo layout

```
mrp3gi/
  SPEC.md                     # this file
  README.md
  package.json                # vite + typescript + three + vitest
  vite.config.ts
  tsconfig.json               # strict: true
  bridge/                     # Rust: the WASM story core
    Cargo.toml                # crate mrp3gi-bridge; dep mrpci-core = { path = "../../mrpci/mrpci-core" }
    src/lib.rs                # wasm-bindgen exports (§4)
  src/
    main.ts                   # boot: load bundle, start loop
    bus.ts                    # typed Command/Event wrappers over the bridge
    world3d.ts                # scene graph: set, ground, lights, camera rig
    mapping.ts                # sim-space ↔ world-space (§5)
    actors.ts                 # GLB registry, AnimationMixer state machines (§7)
    sets.ts                   # backdrop + control-screen extrusion (§6)
    picking.ts                # raycast → sim coords → VerbAt/MoveTo
    ui.ts                     # HTML overlay: verb bar, dialogue, inventory, log
    audio.ts                  # WebAudio sink for cue/music events (§8)
    interp.ts                 # per-actor position/facing interpolation
  public/
    models/                   # GLBs (robot.glb first; from the MeshGod/Mixamo pipeline)
    games/neon-precinct.bundle.json   # generated (§4.2)
  tests/
    golden.test.ts            # determinism replay vs recorded native events
    mapping.test.ts           # coordinate round-trips
  scripts/
    make-bundle.mjs           # game folder → bundle JSON (calls native headless)
    record-golden.mjs         # regenerates tests/golden.events.jsonl from native
```

## 4. The WASM bridge

### 4.1 Prerequisite changes in `../mrpci` (small, do these first)

1. **`Event` needs `Deserialize`** (tests compare parsed events) — add
   `Deserialize` to the derive on `Event` and its payload structs, or keep
   comparison as raw strings (acceptable; then skip this).
2. **Bundle must carry everything a filesystem-less build needs.**
   `WorldBundle` already carries manifest + rooms + scripts. Add
   `sprites: HashMap<String, (u32 w, u32 h, Vec<u8> rgba)>` (serde with
   base64 or plain array) populated by `World::to_bundle()`, and make
   `World::from_memory`/`apply_bundle` accept them. Background `pics/`
   PNGs may be skipped in v1 — Neon Precinct is fully procedural.
3. **Expose a screens accessor** if not public already: the bridge needs
   `visual/priority/control/hotspot` slices plus `render_indices()` and
   `effective_palette()` (both exist).
4. Confirm `rhai` builds for `wasm32-unknown-unknown` with the existing
   `["sync", "no_time"]` features (it does; `no_time` exists precisely for
   wasm). `image` crate is PNG-only and wasm-safe.

### 4.2 Bundle format

`scripts/make-bundle.mjs` runs the native `mrpci-headless` (build it from
`../mrpci`) with a new tiny flag to add there: `--export-bundle out.json`
(serializes `world.to_bundle()`, including sprites per 4.1.2). The bundle
is committed under `public/games/` so the web app needs no Rust at runtime.

### 4.3 `bridge/src/lib.rs` exports (complete list)

```rust
#[wasm_bindgen]
pub struct Engine { gs: GameState }

#[wasm_bindgen]
impl Engine {
    /// Boot from a bundle JSON string. Never panics: bad JSON → Err(String).
    #[wasm_bindgen(constructor)]
    pub fn new(bundle_json: &str) -> Result<Engine, JsError>;

    /// One Command JSON in → JSON array of Events out.
    pub fn apply(&mut self, command_json: &str) -> String;

    /// Wall-clock driver: accumulates dt (seconds) into fixed 30Hz cycles.
    pub fn tick(&mut self, dt: f32) -> String;          // events JSON

    pub fn snapshot(&self) -> String;                    // StateSnapshot JSON

    // --- presentation feeds (no JSON; raw buffers, zero-copy where easy) ---
    pub fn render_indices(&self) -> Vec<u8>;             // 320*190 palette indices (scene+weather)
    pub fn effective_palette(&self) -> Vec<u8>;          // 256*4 RGBA, cycle LUT applied
    pub fn control_screen(&self) -> Vec<u8>;             // walls/water (set extrusion + debug)
    pub fn priority_screen(&self) -> Vec<u8>;
    pub fn hotspot_screen(&self) -> Vec<u8>;

    /// Actor transforms for the renderer, cheaper than full snapshots:
    /// JSON [{name, x, y, dir, scale, walking, visible, talking}] — ego first.
    pub fn actors(&self) -> String;

    // --- audio content (bytes are WAV; browser decodes once, caches) ---
    pub fn cue_wav(&self, name: &str) -> Vec<u8>;
    pub fn music_wav(&self, mood: u8) -> Vec<u8>;
}
```

Notes:
- `actors()` needs a small core addition or can be assembled from
  `snapshot()` + ego fields; prefer a dedicated method (cheap, per-frame).
  Include NPCs and the ego with their current `dir` and `walking` flags —
  the animation state machine keys off these.
- Build with `wasm-pack build bridge --target web --out-dir ../src/wasm`.
  Commit nothing from `src/wasm` (gitignore); `npm run bridge` regenerates.

## 5. Coordinate mapping (`mapping.ts`)

- `PIC_W = 320`, `PIC_H = 190`, `UNITS_PER_PIXEL = 1/32`.
- Sim `(x, y)` → world `(X, 0, Z)`: `X = (x - 160) * UPP`,
  `Z = (y - 190) * UPP` (so the room front edge is Z = 0, deeper rooms go
  negative Z; +Y is up). Inverse for picking; round-trip test required.
- Character height: sim actors are ~36px tall ⇒ target GLB display height
  `36 * UPP * heightScale(actor)`; the room `ScaleTable` is **ignored for
  size** in 3D (real perspective does that job) but `scale()` still feeds
  walk-speed in the core — leave the core alone.
- Camera rig per room (presentation data, §6.3): default is a SCI-style
  fixed camera at `(0, 2.2, 1.8)` looking at `(0, 0.5, -2.2)`, vertical
  FOV 50°. All values overridable per room.

## 6. Sets (three phases, shipped in this order)

### 6.1 Backdrop mode (M2)
The room's own 256-color frame becomes the set:
- Ground plane: `PIC_W*UPP × PIC_H*UPP`, textured with the
  `render_indices` + palette composite (build an offscreen canvas texture;
  `NearestFilter`, `SRGBColorSpace`). Rebuild the texture only when a
  `room_changed` event arrives or every N frames while palette cycles run
  (cheap: 60KB).
- Backdrop plane: same frame, standing at the room's far edge, so the
  camera sees painted walls behind the 3D actors. Crude and charming; it
  proves the whole pipe.

### 6.2 Extruded blockout (M4)
Generate set geometry from the invisible screens:
- March the **control screen**: cells with `CTL_BLOCK` become merged boxes
  (greedy meshing, 4px grid resolution), height 1.4 world units, textured
  by sampling the visual screen at the wall's base row. Water cells get a
  translucent blue plane at y=0.02.
- The visual frame still textures the ground; walls now occlude actors in
  real 3D — the priority screen's job, inherited by the depth buffer.
- Hotspot debug view (`?debug=hotspots`): tinted transparent prisms.

### 6.3 Authored sets + camera (M5)
`public/games/<game>.3d.json` (presentation-only, never in the core):
```json
{
  "rooms": {
    "0": {
      "camera": { "pos": [0,2.2,1.8], "lookAt": [0,0.5,-2.2], "fov": 50 },
      "set":    { "glb": "models/sets/neon-row.glb" },
      "lights": [ {"type":"point","pos":[1.2,1.5,-2],"color":"#ff8844","intensity":2} ],
      "actors": { "vend-bot": "models/vendbot.glb" }
    }
  },
  "actors": { "ego": "models/robot.glb", "sergeant": "models/sergeant.glb" }
}
```
Missing entries fall back to 6.2, then 6.1. A GLB set replaces extruded
geometry but the control screen **still owns collision** — sets are looks.

## 7. Actors (`actors.ts`)

- Loader: GLTFLoader + DRACO optional. Registry maps actor name → model
  URL via `.3d.json`, falling back to `models/robot.glb`, falling back to
  a capsule + name sprite (never crash on a missing model — placeholder
  and console warn, MRPCI's no-Error-47 rule extends to assets).
- Clips: models follow the house Mixamo convention — clips named `idle`,
  `walk`, `talk` (extras ignored in v1). State machine per actor:
  `walking → walk`, `in_dialogue with this actor → talk`, else `idle`;
  crossfade 0.15s.
- Facing: sim `dir` (1..8) → yaw; when walking, face the interpolated
  velocity vector instead (smoother on A* paths); damp yaw at 10 rad/s.
- Interpolation (`interp.ts`): keep the last two sim positions per actor
  with their tick stamps; render at `renderTime - one tick` with linear
  interpolation. Teleports (place_ego, room change) snap: distance > 24px
  in one tick ⇒ no lerp.
- Weather: `room_changed` carries the room; read `weather` from snapshot.
  Rain/snow/embers as a THREE.Points system (~300 particles) matching the
  core's look. Purely visual; do not consume core RNG (mirror of the
  particles.rs rule).

## 8. UI & audio

- **Verb bar** (HTML/CSS overlay, not WebGL): WALK/LOOK/DO/TALK + BAG +
  score + room name. Right-click cycles verbs. Number keys answer
  dialogue. Enter opens the parser line → `{"cmd":"parse"}` — the text
  parser ships in 3D too, because it's free.
- **Dialogue**: `dialogue_open` events render the classic window as HTML
  (white, red border); choices clickable.
- **Picking**: raycast the ground plane + extruded walls + actor meshes.
  Actor hit → `verb_at` at that actor's sim coords; ground hit → inverse
  mapping → `move_to`/`verb_at`. Walls hit → the sim coords of the wall
  base pixel (hotspots live on the control/hotspot screens there).
- **Audio**: on `audio`/`music` events, decode `cue_wav`/`music_wav` bytes
  through WebAudio (decode once, cache buffers; music loops, ~0.13 gain).
  Honor a mute toggle. Files under `<game>/sfx|music/` may override later
  — v1 uses the synth only.
- **Transcript log**: last 3 lines, bottom overlay, fading.

## 9. Milestones & acceptance criteria

**M0 — scaffold.** vite+TS strict+three+vitest; `bridge/` compiles with
wasm-pack; `npm run bridge && npm run dev` shows a lit empty scene and an
FPS meter. CI script (plain `npm test`) green.

**M1 — the brain in the browser.** mrpci changes from §4.1 landed (in
`../mrpci`, its tests still green). Bundle exporter works; Engine boots
Neon Precinct in a vitest node environment; **golden test**: running
`tests/playthrough.jsonl` (copy the canonical one from the mrpci session
logs or re-record with `scripts/record-golden.mjs`) through the WASM
Engine produces event JSON byte-identical to `tests/golden.events.jsonl`
recorded from the native binary. This test is sacred; it runs forever.

**M2 — backdrop 3D.** Ground+backdrop textured from the live frame;
capsule actors move when you click; edge exits and the precinct door
change rooms (with a fade); verb bar works; you can play start-to-finish
(win 25/25) with capsules. Weather points visible in rooms 0 and 2.

**M3 — rigged robots.** `robot.glb` (produce with the existing
MeshGod/Mixamo pipeline; idle/walk/talk clips) replaces the ego capsule;
NPC models or recolored fallbacks for sergeant/vend-bot; animation state
machine + interpolation + facing damping feel right at 60fps; dialogue UI
styled; full playthrough again, now looking like a game.

**M4 — extruded sets + lighting.** Control-screen greedy meshing; walls
occlude actors; per-room ambient+key light defaults derived from the
room's palette average; `?debug=` views for control/hotspot/priority.

**M5 — authored polish + ship.** `.3d.json` camera/lights/models per
room for Neon Precinct; save/restore UI over core SaveGame/RestoreGame
(slots in the core's save dir don't exist on web — add bridge methods
`save_json()`/`restore_json(s)` wrapping SaveData to/from localStorage);
`npm run build` produces a static bundle; deploy per the fleet deploy-map
(games VPS / partly.party, dockerized like the other web games). Run
`/ship-check` before exposing.

## 10. Testing bar

- `golden.test.ts` — determinism (M1, forever).
- `mapping.test.ts` — sim↔world round-trips, edge pixels.
- A `smoke.test.ts` that boots Engine, ticks 300, asserts no `error`
  events from scripts and a stable snapshot shape.
- Visual checks: puppeteer screenshot script (`npm run shot -- --room 2`)
  for eyeballing; not CI-gating in v1.

## 11. Out of scope for v1 (do not build yet)

Free camera / player-relative controls; physics; navmesh (the control
screen *is* the navmesh); shadows beyond a cheap blob; mobile controls;
multiplayer; asset streaming; a TS reimplementation of anything the core
does.

## 12. Open questions (decide during M3, note decisions in this file)

- Lip-flap/talk loop timing from transcript length?
- Palette-cycling neon in 3D: emissive planes sampling the cycled palette
  vs. baked flipbook — try emissive planes first.
- Whether extruded walls should sample the *backdrop* texture region
  instead of base-row color (probably yes, later).