mrp3gi/SPEC.md

MRP3GI — Specification & Execution Plan v1.0

For the implementing agent: this document is the contract. Work the milestones in order; each has acceptance criteria you can verify yourself (headless where possible). When this spec and convenience disagree, the spec wins. When the spec is silent, follow the Architecture Invariants.

---

1. What this is

MRP3GI renders classic Sierra-style adventure games in 3D (three.js, rigged GLB characters, real camera) while the entire game — rooms, verbs, hotspots, dialogue, flags, score, timers, death, saves — runs inside mrpci-core, the existing Rust SCI-generation engine, compiled to WASM. This is the Grim Fandango model: 3D presentation over a 2D brain.

Why this shape:

• mrpci-core is already headless, deterministic (fixed 30Hz ticks, seeded RNG, no wall clock), and driven entirely by a JSON Command/Event bus. Its native-only deps (tiny_http) are already fenced behind cfg(not(target_arch = "wasm32")). It was built for this.
• Every MRPCI game (starting with the Neon Precinct demo) becomes a 3D game for free, and every future MRP3GI game is also playable in the 2D MRPCI GUI, testable over MCP, and replayable in CI.
• The sim's coordinate space is 2D (x 0..320, y 0..190, y = depth). Actors walk on a ground plane. That is exactly what a 3D adventure needs: the renderer maps sim (x, y) → world (X, Z) and stands a model there.

2. Architecture invariants (do not violate)

1. No game rules in TypeScript. If it changes game state, it is a Command into the core. If the player learns something, it arrived as an Event. The TS layer may keep presentation state only (camera, animation phase, UI open/closed).
2. The bridge surface is JSON strings + byte buffers. No fine-grained wasm-bindgen types for game objects — the wire format is the same one the JSONL/HTTP/MCP surfaces already speak (see ../mrpci/docs/CONTROL.md).
3. Determinism is a feature. The golden-replay test (M1) must pass on every commit: the WASM core, fed the canonical playthrough script, produces byte-identical event JSON to the native binary.
4. mrpci-core is consumed, not forked. Path dependency ../mrpci. If the core needs a change (it will — see §4.1), make it in the mrpci repo, keep it native-compatible, run mrpci's own tests, commit there.
5. 60fps presentation, 30Hz sim. The renderer interpolates actor positions between sim ticks; it never runs the sim faster to look smoother.

3. Repo layout

mrp3gi/
  SPEC.md                     # this file
  README.md
  package.json                # vite + typescript + three + vitest
  vite.config.ts
  tsconfig.json               # strict: true
  bridge/                     # Rust: the WASM story core
    Cargo.toml                # crate mrp3gi-bridge; dep mrpci-core = { path = "../../mrpci/mrpci-core" }
    src/lib.rs                # wasm-bindgen exports (§4)
  src/
    main.ts                   # boot: load bundle, start loop
    bus.ts                    # typed Command/Event wrappers over the bridge
    world3d.ts                # scene graph: set, ground, lights, camera rig
    mapping.ts                # sim-space ↔ world-space (§5)
    actors.ts                 # GLB registry, AnimationMixer state machines (§7)
    sets.ts                   # backdrop + control-screen extrusion (§6)
    picking.ts                # raycast → sim coords → VerbAt/MoveTo
    ui.ts                     # HTML overlay: verb bar, dialogue, inventory, log
    audio.ts                  # WebAudio sink for cue/music events (§8)
    interp.ts                 # per-actor position/facing interpolation
  public/
    models/                   # GLBs (robot.glb first; from the MeshGod/Mixamo pipeline)
    games/neon-precinct.bundle.json   # generated (§4.2)
  tests/
    golden.test.ts            # determinism replay vs recorded native events
    mapping.test.ts           # coordinate round-trips
  scripts/
    make-bundle.mjs           # game folder → bundle JSON (calls native headless)
    record-golden.mjs         # regenerates tests/golden.events.jsonl from native

4. The WASM bridge

4.1 Prerequisite changes in ../mrpci (small, do these first)

1. Event needs Deserialize (tests compare parsed events) — add Deserialize to the derive on Event and its payload structs, or keep comparison as raw strings (acceptable; then skip this).
2. Bundle must carry everything a filesystem-less build needs. WorldBundle already carries manifest + rooms + scripts. Add sprites: HashMap<String, (u32 w, u32 h, Vec<u8> rgba)> (serde with base64 or plain array) populated by World::to_bundle(), and make World::from_memory/apply_bundle accept them. Background pics/ PNGs may be skipped in v1 — Neon Precinct is fully procedural.
3. Expose a screens accessor if not public already: the bridge needs visual/priority/control/hotspot slices plus render_indices() and effective_palette() (both exist).
4. Confirm rhai builds for wasm32-unknown-unknown with the existing ["sync", "no_time"] features (it does; no_time exists precisely for wasm). image crate is PNG-only and wasm-safe.

4.2 Bundle format

scripts/make-bundle.mjs runs the native mrpci-headless (build it from ../mrpci) with a new tiny flag to add there: --export-bundle out.json (serializes world.to_bundle(), including sprites per 4.1.2). The bundle is committed under public/games/ so the web app needs no Rust at runtime.

4.3 bridge/src/lib.rs exports (complete list)

#[wasm_bindgen]
pub struct Engine { gs: GameState }

#[wasm_bindgen]
impl Engine {
    /// Boot from a bundle JSON string. Never panics: bad JSON → Err(String).
    #[wasm_bindgen(constructor)]
    pub fn new(bundle_json: &str) -> Result<Engine, JsError>;

    /// One Command JSON in → JSON array of Events out.
    pub fn apply(&mut self, command_json: &str) -> String;

    /// Wall-clock driver: accumulates dt (seconds) into fixed 30Hz cycles.
    pub fn tick(&mut self, dt: f32) -> String;          // events JSON

    pub fn snapshot(&self) -> String;                    // StateSnapshot JSON

    // --- presentation feeds (no JSON; raw buffers, zero-copy where easy) ---
    pub fn render_indices(&self) -> Vec<u8>;             // 320*190 palette indices (scene+weather)
    pub fn effective_palette(&self) -> Vec<u8>;          // 256*4 RGBA, cycle LUT applied
    pub fn control_screen(&self) -> Vec<u8>;             // walls/water (set extrusion + debug)
    pub fn priority_screen(&self) -> Vec<u8>;
    pub fn hotspot_screen(&self) -> Vec<u8>;

    /// Actor transforms for the renderer, cheaper than full snapshots:
    /// JSON [{name, x, y, dir, scale, walking, visible, talking}] — ego first.
    pub fn actors(&self) -> String;

    // --- audio content (bytes are WAV; browser decodes once, caches) ---
    pub fn cue_wav(&self, name: &str) -> Vec<u8>;
    pub fn music_wav(&self, mood: u8) -> Vec<u8>;
}

Notes:

• actors() needs a small core addition or can be assembled from snapshot() + ego fields; prefer a dedicated method (cheap, per-frame). Include NPCs and the ego with their current dir and walking flags — the animation state machine keys off these.
• Build with wasm-pack build bridge --target web --out-dir ../src/wasm. Commit nothing from src/wasm (gitignore); npm run bridge regenerates.

5. Coordinate mapping (mapping.ts)

• PIC_W = 320, PIC_H = 190, UNITS_PER_PIXEL = 1/32.
• Sim (x, y) → world (X, 0, Z): X = (x - 160) * UPP, Z = (y - 190) * UPP (so the room front edge is Z = 0, deeper rooms go negative Z; +Y is up). Inverse for picking; round-trip test required.
• Character height: sim actors are ~36px tall ⇒ target GLB display height 36 * UPP * heightScale(actor); the room ScaleTable is ignored for size in 3D (real perspective does that job) but scale() still feeds walk-speed in the core — leave the core alone.
• Camera rig per room (presentation data, §6.3): default is a SCI-style fixed camera at (0, 2.2, 1.8) looking at (0, 0.5, -2.2), vertical FOV 50°. All values overridable per room.

6. Sets (three phases, shipped in this order)

6.1 Backdrop mode (M2)

The room's own 256-color frame becomes the set:

• Ground plane: PIC_W*UPP × PIC_H*UPP, textured with the render_indices + palette composite (build an offscreen canvas texture; NearestFilter, SRGBColorSpace). Rebuild the texture only when a room_changed event arrives or every N frames while palette cycles run (cheap: 60KB).
• Backdrop plane: same frame, standing at the room's far edge, so the camera sees painted walls behind the 3D actors. Crude and charming; it proves the whole pipe.

6.2 Extruded blockout (M4)

Generate set geometry from the invisible screens:

• March the control screen: cells with CTL_BLOCK become merged boxes (greedy meshing, 4px grid resolution), height 1.4 world units, textured by sampling the visual screen at the wall's base row. Water cells get a translucent blue plane at y=0.02.
• The visual frame still textures the ground; walls now occlude actors in real 3D — the priority screen's job, inherited by the depth buffer.
• Hotspot debug view (?debug=hotspots): tinted transparent prisms.

6.3 Authored sets + camera (M5)

public/games/<game>.3d.json (presentation-only, never in the core):

{
  "rooms": {
    "0": {
      "camera": { "pos": [0,2.2,1.8], "lookAt": [0,0.5,-2.2], "fov": 50 },
      "set":    { "glb": "models/sets/neon-row.glb" },
      "lights": [ {"type":"point","pos":[1.2,1.5,-2],"color":"#ff8844","intensity":2} ],
      "actors": { "vend-bot": "models/vendbot.glb" }
    }
  },
  "actors": { "ego": "models/robot.glb", "sergeant": "models/sergeant.glb" }
}

Missing entries fall back to 6.2, then 6.1. A GLB set replaces extruded geometry but the control screen still owns collision — sets are looks.

7. Actors (actors.ts)

• Loader: GLTFLoader + DRACO optional. Registry maps actor name → model URL via .3d.json, falling back to models/robot.glb, falling back to a capsule + name sprite (never crash on a missing model — placeholder and console warn, MRPCI's no-Error-47 rule extends to assets).
• Clips: models follow the house Mixamo convention — clips named idle, walk, talk (extras ignored in v1). State machine per actor: walking → walk, in_dialogue with this actor → talk, else idle; crossfade 0.15s.
• Facing: sim dir (1..8) → yaw; when walking, face the interpolated velocity vector instead (smoother on A* paths); damp yaw at 10 rad/s.
• Interpolation (interp.ts): keep the last two sim positions per actor with their tick stamps; render at renderTime - one tick with linear interpolation. Teleports (place_ego, room change) snap: distance > 24px in one tick ⇒ no lerp.
• Weather: room_changed carries the room; read weather from snapshot. Rain/snow/embers as a THREE.Points system (~300 particles) matching the core's look. Purely visual; do not consume core RNG (mirror of the particles.rs rule).

8. UI & audio

• Verb bar (HTML/CSS overlay, not WebGL): WALK/LOOK/DO/TALK + BAG + score + room name. Right-click cycles verbs. Number keys answer dialogue. Enter opens the parser line → {"cmd":"parse"} — the text parser ships in 3D too, because it's free.
• Dialogue: dialogue_open events render the classic window as HTML (white, red border); choices clickable.
• Picking: raycast the ground plane + extruded walls + actor meshes. Actor hit → verb_at at that actor's sim coords; ground hit → inverse mapping → move_to/verb_at. Walls hit → the sim coords of the wall base pixel (hotspots live on the control/hotspot screens there).
• Audio: on audio/music events, decode cue_wav/music_wav bytes through WebAudio (decode once, cache buffers; music loops, ~0.13 gain). Honor a mute toggle. Files under <game>/sfx|music/ may override later — v1 uses the synth only.
• Transcript log: last 3 lines, bottom overlay, fading.

9. Milestones & acceptance criteria

M0 — scaffold. vite+TS strict+three+vitest; bridge/ compiles with wasm-pack; npm run bridge && npm run dev shows a lit empty scene and an FPS meter. CI script (plain npm test) green.

M1 — the brain in the browser. mrpci changes from §4.1 landed (in ../mrpci, its tests still green). Bundle exporter works; Engine boots Neon Precinct in a vitest node environment; golden test: running tests/playthrough.jsonl (copy the canonical one from the mrpci session logs or re-record with scripts/record-golden.mjs) through the WASM Engine produces event JSON byte-identical to tests/golden.events.jsonl recorded from the native binary. This test is sacred; it runs forever.

M2 — backdrop 3D. Ground+backdrop textured from the live frame; capsule actors move when you click; edge exits and the precinct door change rooms (with a fade); verb bar works; you can play start-to-finish (win 25/25) with capsules. Weather points visible in rooms 0 and 2.

M3 — rigged robots. robot.glb (produce with the existing MeshGod/Mixamo pipeline; idle/walk/talk clips) replaces the ego capsule; NPC models or recolored fallbacks for sergeant/vend-bot; animation state machine + interpolation + facing damping feel right at 60fps; dialogue UI styled; full playthrough again, now looking like a game.

M4 — extruded sets + lighting. Control-screen greedy meshing; walls occlude actors; per-room ambient+key light defaults derived from the room's palette average; ?debug= views for control/hotspot/priority.

M5 — authored polish + ship. .3d.json camera/lights/models per room for Neon Precinct; save/restore UI over core SaveGame/RestoreGame (slots in the core's save dir don't exist on web — add bridge methods save_json()/restore_json(s) wrapping SaveData to/from localStorage); npm run build produces a static bundle; deploy per the fleet deploy-map (games VPS / partly.party, dockerized like the other web games). Run /ship-check before exposing.

10. Testing bar

• golden.test.ts — determinism (M1, forever).
• mapping.test.ts — sim↔world round-trips, edge pixels.
• A smoke.test.ts that boots Engine, ticks 300, asserts no error events from scripts and a stable snapshot shape.
• Visual checks: puppeteer screenshot script (npm run shot -- --room 2) for eyeballing; not CI-gating in v1.

11. Out of scope for v1 (do not build yet)

Free camera / player-relative controls; physics; navmesh (the control screen is the navmesh); shadows beyond a cheap blob; mobile controls; multiplayer; asset streaming; a TS reimplementation of anything the core does.

12. Open questions (decide during M3, note decisions in this file)

• Lip-flap/talk loop timing from transcript length?
• Palette-cycling neon in 3D: emissive planes sampling the cycled palette vs. baked flipbook — try emissive planes first.
• Whether extruded walls should sample the backdrop texture region instead of base-row color (probably yes, later).