diff --git a/docs/HANDOVER.md b/docs/HANDOVER.md new file mode 100644 index 0000000..80e0d0b --- /dev/null +++ b/docs/HANDOVER.md @@ -0,0 +1,262 @@ +# MRPGI — Handover: making the engine fully programmatically controllable + +> **Status:** design / discussion complete. **No engine code has been changed yet.** +> **Date:** 2026-07-05 +> **Purpose:** capture the vision, the current-state analysis, the decisions locked in, and the phased plan so any future session (human or AI) can pick up exactly where we left off. + +--- + +## 1. The dream (in the owner's words) + +> "It's an engine for a text adventure but it's got its own paint app inside it and can import art or images and you can assign functions to bands or objects, so you can build a Sierra-style game with it. I want it **100% fully programmatically controllable via CLI / terminal / endpoints / hooks / websockets / MCP / Python**, so an AI can create a whole game from prompts and I can just drop JSON text lore in for expansions. As many features as the OG engine had, plus newer tech. Use **Strudel** to make sounds, which can also be controlled. Always want **options**." + +The engine is **MRPGI — Monster Robot Party Game Interpreter**: a new-school, Apple-Silicon-native reimagining of Sierra's 1980s AGI engine (King's Quest / Space Quest / Police Quest), written in Rust + [macroquad](https://macroquad.rs). + +--- + +## 2. Decisions locked this session + +| Question | Decision | +|---|---| +| Primary control surfaces | **MCP server + CLI/stdio (JSONL) + HTTP/WebSocket** (all three). Python rides on HTTP. | +| Runtime model | **Headless-first, GUI optional.** The engine core runs with no window; macroquad becomes one optional viewer. | +| Audio | **Offer all backends, toggle any combination on/off** — native chiptune synth, audio files (mp3/ogg/wav), and Strudel. | +| Guiding principle | **"Always want options."** Every subsystem is a swappable, runtime-toggleable backend; nothing is locked in. | + +--- + +## 3. Architecture at a glance + +![MRPGI target architecture](architecture.svg) + +```mermaid +flowchart TD + subgraph FE["Front-ends — how you drive it (any/all at once)"] + GUI["macroquad GUI
play + paint · optional"] + CLI["CLI · stdio
JSONL in, events out"] + HTTP["HTTP + WebSocket
REST · SSE · Python"] + MCP["MCP server
Claude authors + plays"] + end + + BUS(["Command / event bus
commands down · events up"]) + + subgraph CORE["mrpgi-core — headless engine (zero macroquad)"] + GS["GameState
apply(cmd) → events · tick(dt) → events"] + PARSE["Parser + AI lane
verb·noun → authoritative logic"] + FB["FrameBuffer
visual + priority buffers"] + WORLD["World
rooms · exits · flags/vars"] + RENDER["render_visual()
→ PNG, no window"] + end + + DATA["A game = a folder
game.json · rooms/*.json · sprites/*.png · audio"] + AUDIO["Audio out — toggle both/either/off
SynthSink · FileSink (mp3/ogg/wav) · StrudelSink"] + + GUI --> BUS + CLI --> BUS + HTTP --> BUS + MCP --> BUS + BUS --> CORE + DATA -->|load_game · lore upsert| CORE + CORE -->|AudioSink events| AUDIO +``` + +**The one insight:** "CLI or terminal or endpoints or hooks or websockets or MCP or Python" is **one feature, not seven** — a headless core behind a typed command/event bus. Every surface is a thin adapter that pushes `Command`s in and reads `Event`s out. + +--- + +## 4. Current-state assessment (the honest read) + +MRPGI is a solid, coherent ~3,500-line AGI recreation whose foundations are **much closer to headless-ready than they look.** + +**What's genuinely good and reusable as-is:** +- The real AGI priority/depth trick: dual `visual` + `priority` buffers, a `band(y)` depth table, and per-pixel `pri >= room.priority[i]` compositing (`compose()` at `src/main.rs:387`). This is the actual "walk behind the tree" mechanism, not a fake z-sort. +- A deterministic verb-noun parser (`run_command()` at `src/main.rs:586`) with an **AI fallback that cannot cheat** — the LLM only translates free text into an *already-allowed* command, which is then re-run through the real logic (`src/ai.rs`). This is the correct architecture; most homages let the LLM invent state. +- A 1,362-line in-engine paint editor with dual LOOK + MEANING layers, so art and walkability/depth are painted together (`src/editor.rs`). +- **The whole data model already derives `serde`** with disciplined `#[serde(default)]`: `RoomDoc`, `ObjDef`, `DlgNode`, `Stroke`, `Meaning` (`src/editor.rs:104-146`). Rooms already round-trip to `rooms/roomN.json`; there is already a `kit/` of themed objects and locations. + +**The single weakness — it is structural, not rot:** +`src/main.rs` fuses three things into one macroquad loop: +1. **game state** — locals at `src/main.rs:68-76` (`ego`, `inventory`, `taken`, `won`, `dlg`, `command`, `transcript`); +2. **update logic** — `ego.update()` (283), `run_command()` (188), the exit-transition scan (289–324), the AI poll (234); +3. **presentation** — `get_frame_time()`, `get_char_pressed()`, `mouse_position()`, `compose()`, `draw_texture_ex()`, `next_frame().await`. + +There is currently **no way to advance the game without rendering a frame.** Everything the owner wants is downstream of fixing that one fusion. + +**Why it's ~80% ready and doesn't know it:** `ScreenObj::update(&FrameBuffer)`, the entire `picture::*` / `FrameBuffer` / `Cel` / `PicOp` pipeline, and `run_command()` are **already pure computation with no macroquad dependency.** They already take `&mut FrameBuffer` and return values. This is a *give-it-a-spine* job, not a rewrite. + +--- + +## 5. The core architectural move + +Split the crate into a Cargo workspace: +- **`mrpgi-core`** — a library with **zero** macroquad dependency (a compile error is the guardrail). +- **`mrpgi`** — the macroquad GUI, now just *one consumer* of the core. + +Hoist the scattered `main()` locals into one owned struct with two pure methods and a typed bus: + +```rust +struct GameState { + ego: ScreenObj, + world: World, // rooms + current + FrameBuffer (today: the Editor struct) + inventory: Vec, + taken: Vec<(u32, String)>, + won: bool, + dlg: Option, + transcript: Vec, + flags: HashMap, // P4 + vars: HashMap, // P4 +} + +impl GameState { + fn apply(&mut self, cmd: Command) -> Vec; // wraps run_command, goto_room, move_target, paint + fn tick(&mut self, dt: f32) -> Vec; // wraps ego.update + exit-transition scan +} +``` + +- `compose()` splits into a pure `render_visual(&self) -> Vec` (lives in core) + a 3-line macroquad texture blit (lives in the GUI). +- The clock becomes a `dt: f32` **parameter** instead of `get_frame_time()`, so scripted replays are bit-identical. +- Input becomes `Command`s pushed onto the bus instead of direct `is_key_down` reads. +- **The AI lane needs no change** — it already runs on a worker thread and reports over an `mpsc` channel (`src/ai.rs:81-88`), which is the event-bus pattern in miniature. It just moves from a `main.rs` local into the engine. + +### Command / Event sketch (starting point) + +```rust +enum Command { + Walk { dir: u8 }, MoveTo { x: i32, y: i32 }, + Parse { text: String }, Verb { verb: String, noun: String }, + GotoRoom { n: u32 }, + LoadRoom { n: u32, doc: RoomDoc }, UpsertObject { room: u32, obj: ObjDef }, + PaintStroke { stroke: Stroke }, + SetMusic { mood: u8 }, PlaySfx { cue: String }, + NewGame, Query, +} + +enum Event { + Transcript(String), RoomChanged(u32), InventoryChanged, + DialoguePrompt { node: usize }, Won, + Audio(AudioEvent), FrameReady, Error(String), +} +``` + +--- + +## 6. Control surfaces (all thin adapters over the bus) + +| Surface | How it works | Effort | +|---|---|---| +| **In-process Rust API** | `Engine::apply(Command) -> Vec` — the substrate everything wraps. | M | +| **CLI (stdio, JSONL)** | `mrpgi headless` reads newline JSON commands on stdin, writes newline JSON events on stdout. `--script foo.jsonl` replays a command log for deterministic tests. | S | +| **HTTP + WebSocket** | axum listener on a worker thread (exactly like `ai.rs` spawns its translate thread). `POST /command`, `GET /state`, `GET /events` (SSE), `GET /frame.png` (headless render), `POST /rooms/{n}`, `POST /games`. | M | +| **MCP server** | Wrap the bus as MCP tools: `mrpgi_apply_command`, `mrpgi_get_state`, `mrpgi_load_room`, `mrpgi_paint_stroke`, `mrpgi_render_frame` (returns the image so the model can *see* its own moves), `mrpgi_new_game`. | M | +| **Python** | Thin client over the HTTP server (or a pyo3 binding). Falls out for free. | S | + +**Concurrency model:** prefer a single owner thread with an `mpsc` command channel (mirroring `ai.rs`) over a `Mutex` — keeps tick ordering deterministic, avoids lock contention. + +--- + +## 7. Audio — "always want options" + +Model audio as a `trait AudioSink` behind the bus. Sound calls (currently imperative: `sfx.pickup()` at `src/main.rs:162,199,217-224`) become `Event::Audio(..)` on the same event bus every control surface already consumes. Then pick sink(s) at runtime and/or declare them per-game in `game.json`: + +- **`SynthSink`** — the existing `src/sound.rs` runtime chiptune synth. Offline default, procedural, tiny, LLM-authorable as text. +- **`FileSink`** — plays `.mp3` / `.ogg` / `.wav` from the game folder. macroquad's `load_sound` auto-detects all three (`load_sound_from_bytes` is already what `sound.rs` calls, so this is a data-source swap, not a new system). **Prefer OGG for looping music** — MP3 encoders pad a few ms of silence that clicks at loop seams; fine for one-shot SFX. Likely the **first** audio backend to build (easier than the Strudel bridge). +- **`StrudelSink`** — Strudel is browser/Web-Audio JS; the engine is native Rust, so they meet at a **message boundary, not shared memory.** The engine emits audio *descriptors* (`{sfx:"door"}`, `{pattern:"note(\"c e g\").s(\"sawtooth\")"}`) over the WebSocket to a tiny browser page running `@strudel/core` + `@strudel/webaudio`, which is the actual synth. Rooms gain an optional `music_pattern: Option` so an LLM-authored room can carry a Strudel one-liner in its JSON. + +Any combination runs at once, or none (silent). This mirrors the existing `ai.rs` provider switch (`ollama / openrouter / off`) — the template for every option in the engine. + +**The rule that keeps options from becoming chaos:** options live at the *edges*, never in the core. `GameState` neither knows nor cares which sink or control surface is attached — it only emits `Event`s and applies `Command`s. That's what makes "add another option later" a 30-line trait impl instead of a rewrite. + +--- + +## 8. Games as data (the "drop JSON lore in" milestone) + +The bones already exist (`RoomDoc`, `ObjDef`, `DlgNode`, `Stroke` all serde; rooms round-trip; `kit/` holds JSON archetypes). The gap: a "game" is not yet a first-class loadable unit — the engine hardcodes `rooms/roomN.json` paths, room 0 as boot, and the seven verbs in code. + +1. **Game manifest** — a `GameManifest` (`game.json`): `{ name, start_room, start_spawn, rooms_dir, sprites_dir, verbs, intro_text, win_text, flags, music_patterns }`. A game becomes a folder: `games//{game.json, rooms/*.json, sprites/*.png, sfx/*, music/*}`. `World::load_game(dir)` replaces the hardcoded paths. +2. **Data-driven verbs** — lift the verb table (today split across `canon_verb` `src/main.rs:551`, `build_prompt` `src/ai.rs:73`, and the `run_command` match) into `Vec` in the manifest. `build_prompt` then generates its whitelist *from* this table, so the AI and the parser can never drift. +3. **Flags & vars** — add `flags`/`vars` to `GameState`; give `ObjDef`/`DlgChoice` optional `requires`/`sets` (all `#[serde(default)]` so existing rooms keep loading). Turns one-shot `wins` and tree dialogue into real gated puzzles. +4. **Lore ingestion** — because the types are serde, an LLM expansion is just JSON that deserializes. Add `UpsertRoom` / `UpsertObject` / `PatchDialogue` commands so a running game extends live over HTTP/MCP. Ship a schemars-derived **JSON Schema** as the authoring contract. Validate ingested rooms with the existing `find_spawn` (`src/editor.rs`) + exit-target resolution before committing. + +--- + +## 9. AGI-soul gaps surfaced (real, but downstream of the core move) + +Not on the critical path to programmability — sequence *after* the bus/server land, or the refactor never ships: +- Only one active room at a time (`editor.current: u32`) — no cross-room NPCs/state. +- Click-to-walk uses naive greedy steering (`sprite.rs`), no pathfinding — gets stuck on diagonal obstacles. +- Dialogue is tree-only, no conditions on choices. +- Victory is a single `won: bool` (no flags/vars) — addressed in P4. +- No item combining; take/drop are simple vec push/remove. +- Room dimensions hardcoded 160×168 (`framebuffer.rs`). + +--- + +## 10. Phased roadmap (each phase ships something usable) + +| Phase | Goal | Unlocks | +|---|---|---| +| **P0 · Carve the core** | Workspace: `mrpgi-core` (lib, no macroquad) + `mrpgi` (GUI). Move the pure pieces over. No behaviour change. | A compilable proof the sim has no rendering dependency — precondition for everything. | +| **P1 · GameState + bus** | `GameState` with `apply(Command)`/`tick(dt)`; GUI rewritten to drive them; `compose()` split; AI worker delivers a `Command::Parse` result as an `Event`. | Engine fully drivable & observable in-process without rendering. Golden-transcript tests possible. | +| **P2 · Headless CLI + game folders** | `mrpgi headless` (JSONL, `--script` replay); `GameManifest` + `World::load_game`; data-driven verb table. | LLM/pipe control today; CI golden tests; authoring a game without touching Rust. **("Drop JSON lore in.")** | +| **P3 · HTTP/WS + MCP** | axum server (incl. headless `/frame.png`); MCP tools incl. `render_frame`; Python client; JSON Schema authoring contract. | Claude/Python/curl drive **and** author games; the model can *see* frames it produces. **The headline scenario.** | +| **P4 · AGI soul: flags/vars/conditions** | `flags`/`vars` in `GameState`; `requires`/`sets` on objects & dialogue; upsert commands with validation. | Real puzzles, gated dialogue, multi-step quests — all authorable as JSON. | +| **P5 · Audio sinks** | `trait AudioSink`; `sound.rs` → `SynthSink`; `FileSink` (mp3/ogg/wav) + `music/`+`sfx/` folder convention; `StrudelSink` browser sidecar over WS; `music_pattern` on rooms. | Programmatic, LLM-authored music/sfx; every backend toggleable. | +| **P6 · Headless editor** | `PaintStroke`/`UpsertObject` commands so the editor's stroke pipeline runs without `frame()`; GUI editor becomes a thin front-end over the same commands. | An LLM paints rooms via the same bus a human uses. | + +--- + +## 11. Quick wins (afternoon-sized, near-zero risk — the recommended first move) + +1. **`mrpgi --render-room rooms/room0.json out.png`** — bake a `RoomDoc` to PNG headless via the existing compose path. Proves headless rendering with almost no refactor; gives the LLM a way to preview authored rooms *today*. +2. **Lift `run_command()` into `mrpgi-core` and unit-test it** with golden `(input → lines, understood)` cases. Zero risk; instant safety net for the P1 refactor. +3. **Generate the AI verb list in `build_prompt` (`src/ai.rs:68`) *from* `canon_verb`'s table** instead of a hardcoded string, so the two can never drift. ~15 min; also preps data-driven verbs. +4. **Wrap sound calls behind `trait AudioSink`** with the existing `Sfx`/`Music` as the impl (no behaviour change). Mechanical; unlocks the whole audio-options path later. + +--- + +## 12. Risks & guardrails + +- **macroquad owns the window/audio/GL context** via `#[macroquad::main]`; the core must never call into it. Enforce with the workspace boundary — a compile error is the guardrail. Watch `assets.rs`: the `quantize()`/`Cel` path is pure (`image` crate) but any macroquad `Image`/`Texture` use stays in the front-end. +- **Determinism:** the headless core must take `dt` explicitly and fix the cycle rate (`CYCLE_DT = 1/20`) so replays are bit-identical. Keep the (non-deterministic) AI lane *out* of the deterministic core path — it produces a `Command` that then runs through deterministic logic (the current design already respects this). +- **Backward-compat:** every new `RoomDoc`/`ObjDef` field MUST be `#[serde(default)]` (the codebase already does this) so the existing `rooms/*.json` and `kit/` packs keep loading. +- **Strudel is a genuine native/web split** — accept a browser sidecar as the real path; don't over-invest in a native mini-notation interpreter until a native-only user needs it. +- **Editor/Play share state** — the core split is the chance to make `World` the single source of truth. Keep the GUI editor driving the *same* commands the server uses, rather than a parallel path, to avoid regressing the working editor. + +--- + +## 13. Open questions / next step + +**Recommended opening play:** P0 (carve `mrpgi-core`) + the four quick wins, on a branch — gets a headless engine rendering a room and passing its first tests without touching the fun stuff. + +**Or keep designing first:** nail down the exact `Command`/`Event` enums and the `game.json` manifest schema before any code moves. + +Still to decide when building starts: +- HTTP framework: axum vs. a minimal hyper/tiny_http (weigh against build times). +- MCP transport: stdio (simplest) to start. +- Whether `FileSink` lands in P2 (alongside game folders) or waits for P5. + +--- + +## 14. Source map (current engine) + +| File | Lines | Role | +|---|---|---| +| `src/main.rs` | 728 | Window, game loop, mode switch, parser (`run_command`), compositing, HUD. **The fusion to break up.** | +| `src/editor.rs` | 1362 | In-engine paint tool; also the de-facto world database (`RoomDoc`, `ObjDef`, `DlgNode`, sprite catalog). | +| `src/assets.rs` | 440 | PNG → EGA quantize + dither; tile / pour-into-shape / object insertion; hot-reload. | +| `src/picture.rs` | 283 | Vector room painting (`PicOp` rects / lines / pixels), `stamp_cel_fit`. | +| `src/sprite.rs` | 166 | The ego: movement, click-to-walk, collision, walk animation; `Prop`. | +| `src/ai.rs` | 134 | AI parser lane (Ollama / OpenRouter / off) — **the template for the whole option/adapter pattern.** | +| `src/sound.rs` | 151 | Runtime square-wave chiptune synth + 5 music moods → WAV bytes. | +| `src/framebuffer.rs` | 87 | The `visual` + `priority` buffers and the `band(y)` depth table. | +| `src/room.rs` | 72 | Hand-authored demo `Scene`. | +| `src/view.rs` | 51 | Sprites as loops + cels, authored from ASCII grids. | +| `src/palette.rs` | 40 | The 16-color EGA palette (indices → RGBA). | + +Resources: `rooms/*.json` (room docs), `kit/` (themed objects + locations — proto game-folders), `sprites/*.png`, `assets/*.png`, `palettes/`. Docs: `README.md`, `MANUAL.md`, `EDITOR_GUIDE.md`. + +--- + +## 15. Provenance + +This handover was produced from a full read of the engine source (`src/*.rs`) plus a parallel architecture-mapping pass over every subsystem (game loop, editor, render pipeline, actors, AI + sound, resources) and a synthesis design pass. It reflects the state of `main @ d0caf98` (Initial commit) and the design conversation of 2026-07-05. Diagram source: [`architecture.svg`](architecture.svg). diff --git a/docs/architecture.svg b/docs/architecture.svg new file mode 100644 index 0000000..c737e2e --- /dev/null +++ b/docs/architecture.svg @@ -0,0 +1,91 @@ + + + + + + + + + One engine, many front-ends + Blue = how you drive it · teal = the headless engine core + + + + + macroquad GUI + play + paint, optional + + + + CLI · stdio + JSONL in, events out + + + + HTTP + WebSocket + REST · SSE · Python + + + + MCP server + Claude authors + plays + + + + + + + + + + Command / event bus + commands down · events up + + + + + + + mrpgi-core · headless (no macroquad) + + + GameState + apply(cmd) → events · tick(dt) → events + + + Parser + AI lane + verb·noun → authoritative logic + + + FrameBuffer + visual + priority buffers + + + World + rooms · exits · flags/vars + + + render_visual() + → PNG, no window + + + load_game · lore upsert + + AudioSink events + + + + A game = a folder + games/<name>/ + game.json — manifest, verbs + rooms/*.json — art, objects, dialogue + sprites/*.png — cels + + + + Audio out — toggle both / either / off + + Native chiptune · SynthSink (offline) + + FileSink (mp3/ogg/wav) · StrudelSink (browser) +