# NPCvector

**Turn any scraped text corpus + pgvector into a cast of game NPCs with authentic
voices — without ever shipping a word of the original corpus.**

First deployment: **GABGOD** on ultra — Soulstrut (818k embedded posts) + VeryGoodPlus
(59k) forum corpus → sarcastic shopkeepers for PROCITY's 90s Australian shopping town.
But the pattern is generic: any corpus you can embed, any cast of characters, any game.

```
                        THE PATTERN
┌─────────────────────────────────────────────────────────────────┐
│  corpus (pgvector)          personas.json                       │
│  forum posts, reviews,      who talks, how they talk,           │
│  transcripts, chat logs     what situations they're in          │
│        │                          │                             │
│        └───────► distill.py ◄─────┘                             │
│   1. embed the situation (nomic-embed-text, 768d)               │
│   2. pgvector top-k similar corpus posts = STYLE REFERENCE      │
│   3. writer LLM (toggleable, see below) writes ORIGINAL lines   │
│        │                                                        │
│        ▼                                                        │
│   line_bank table  (approved = NULL until a human reviews)      │
│        │                                                        │
│        ├──► in-game retrieval dialogue (line bank + small       │
│        │    embedding model in the browser via LiteRT.js —      │
│        │    player types anything, semantic match picks the     │
│        │    NPC's comeback; tiny download; fully offline)       │
│        │                                                        │
│        └──► export_sft.py → SFT JSONL → LoRA a local model      │
│             (bake the VOICE into a model; facts stay in RAG)    │
└─────────────────────────────────────────────────────────────────┘
```

## Why distill instead of RAG-ing the corpus straight into the game?

1. **Real people wrote the corpus.** Verbatim forum posts coming out of an NPC's
   mouth in a public game is a legal grey zone and recognizably *someone*. The
   corpus is used as **voice and attitude reference only**; the prompt forbids
   copying, and provenance (`source_post_ids`) is kept so any line can be audited
   against its references.
2. **Review gate.** Every generated line lands with `approved = NULL`. A human
   eyeballs before anything ships (house rule — same as mocapgod QC clips).
   Essential when the writer model is uncensored (below).
3. **Tiny runtime.** A reviewed line bank + a small embedding model runs in the
   browser (LiteRT.js / WASM / WebGPU) — no server, no 2GB model download for
   players, works offline on monsterrobot.games.

## The abliterated toggle

```bash
./gabgod abliterated status   # which writer distill.py will use
./gabgod abliterated on       # uncensored writer (local, private generation)
./gabgod abliterated off      # stock writer (safe default)
```

`config.json → writer.abliterated` flips between two OpenAI-compatible endpoints:

| | model | where | when |
|---|---|---|---|
| **off** | qwen2.5:7b | m4pro Ollama | safe default, boring but obedient |
| **on** | supergemma4-26b-uncensored (16GB GGUF) | ultra's own Ollama | commits to the bit, no refusals |

(First choice was gemma-4-E4B-it-OBLITERATED — abandoned after it drifted into
Chinese mid-run and failed 25/40 topics. Abliteration damages small models;
the 26B holds together. See "Earned gotchas" below.)

**House rule: abliterated output never ships publicly unreviewed.** The toggle
gates *generation*; the `approved` column gates *shipping*. Both must be green
for a line to reach a public game. For private/LAN free-chat NPCs, point the
game at the abliterated endpoint directly and skip the bank — your house, your
rules.

## Using it on something else later (the reusable bit)

1. **Get a corpus into pgvector.** Any table with `(id, body_text, embedding vector(768))`
   embedded with `nomic-embed-text` works. Add the table name to
   `config.json → retrieval.sources`. (In discogs_full you already have embedded:
   soulstrut_post, vgplus_post, ishkur_style, discogs_reviews 414k, yt_transcript,
   release_wikipedia 1.3M, artist_wikipedia, semantic_gold…)
2. **Write personas.json.** Per character: `voice` (who they are, how they talk)
   and `topics` (situations they'll need lines for). The topics double as the
   retrieval queries — write them in the corpus's vocabulary for better matches.
3. **Run `./gabgod distill`.** Heartbeat in `~/.jobs/gabgod-distill.status`,
   resume-safe (persona-topics with enough lines are skipped on rerun).
4. **Review** (`./gabgod sample <persona>`, then set `approved` in psql/pgweb),
   **export** (`./gabgod export` → `sft/train.jsonl` + `valid.jsonl`), and either
   ship the bank or LoRA the voice (`mlx_lm.lora --data sft/ ...` on m3ultra).

Ideas already on the list: RECORDGOD store-assistant voice from discogs_reviews;
MRPGI narrator personas; toastsim's judge from the same forums; Ishkur-flavoured
genre-pedant NPC from ishkur_style (153 embedded style blurbs).

## Files

| file | what |
|---|---|
| `distill.py` | the forge: retrieve → write → insert. `--persona X --topics N --dry-run` to smoke-test |
| `personas.json` | the cast (currently: PROCITY's six shopkeepers) |
| `config.json` | endpoints + toggle + retrieval knobs (gitignored; start from `config.example.json`) |
| `schema.sql` | `gabgod.line_bank` (+ schema) — run once per database |
| `export_sft.py` | approved lines → chat-format JSONL for LoRA |
| `gabgod` | CLI: `abliterated on/off/status · distill · status · sample · counts · export` |

## Ops notes

- Deployment lives on **ultra** `~/Documents/GABGOD` (the data's there; scripts use
  `/opt/homebrew/bin/python3` per TCC rule). This repo is the canonical source.
- Long runs follow the heartbeat convention (`~/.jobs/`, overwrite-not-append,
  die loudly, checkpoint+resume).
- Embeddings: nomic-embed-text (768d) — **must match whatever embedded the corpus**,
  or retrieval silently degrades. Check dims with `atttypmod` on the vector column.
- Postgres gotcha that bit here: `n_live_tup` stats on the corpus tables read 0
  (stale autovacuum); trust `count(*)`, not `pg_stat_user_tables`.
- Ollama structured-output note: small models ignore "exactly N lines" — the
  parser takes whatever valid JSON array comes back (raw_decode from first `[`).

## Earned gotchas (one bad evening's tuition)

1. **Abliterated small models are brain-damaged.** The E4B weight-edit produced
   Chinese, word salad, and malformed JSON under a real workload despite passing
   a simple smoke test. Test with the PRODUCTION prompt, not "say 3 phrases".
2. **Reasoning models + long prompts = empty answers.** supergemma via the
   OpenAI-compat `/v1` endpoint burned its entire 4096-token budget on internal
   drafts (`finish_reason: length`, 12k chars of reasoning, zero content). Fix:
   use ollama's native `/api/chat` with `"think": false` — 12/12 lines in 9s vs
   nothing in 5 minutes. distill.py falls back automatically for models with no
   think switch.
3. **Keep the reference block small.** >6 refs × 400 chars measurably worsens
   deliberation spirals; retrieval `top_k` can stay high, but cap what you show
   the writer.
4. **`sane_line()` is load-bearing.** Language-drift and fragment junk WILL
   reach the DB without the ascii-ratio/length gate; purging after the fact is
   `delete ... where length(regexp_replace(line,'[[:ascii:]]','','g')) > 3`.
5. **Resume counts must be per-writer-model** (`writer_model=%s` in the skip
   query), or a failed model's junk "completes" topics and blocks the good
   model from refilling them.