NPCvector/README.md

NPCvector

Turn any scraped text corpus + pgvector into a cast of game NPCs with authentic voices — without ever shipping a word of the original corpus.

First deployment: GABGOD on ultra — Soulstrut (818k embedded posts) + VeryGoodPlus (59k) forum corpus → sarcastic shopkeepers for PROCITY's 90s Australian shopping town. But the pattern is generic: any corpus you can embed, any cast of characters, any game.

                        THE PATTERN
┌─────────────────────────────────────────────────────────────────┐
│  corpus (pgvector)          personas.json                       │
│  forum posts, reviews,      who talks, how they talk,           │
│  transcripts, chat logs     what situations they're in          │
│        │                          │                             │
│        └───────► distill.py ◄─────┘                             │
│   1. embed the situation (nomic-embed-text, 768d)               │
│   2. pgvector top-k similar corpus posts = STYLE REFERENCE      │
│   3. writer LLM (toggleable, see below) writes ORIGINAL lines   │
│        │                                                        │
│        ▼                                                        │
│   line_bank table  (approved = NULL until a human reviews)      │
│        │                                                        │
│        ├──► in-game retrieval dialogue (line bank + small       │
│        │    embedding model in the browser via LiteRT.js —      │
│        │    player types anything, semantic match picks the     │
│        │    NPC's comeback; tiny download; fully offline)       │
│        │                                                        │
│        └──► export_sft.py → SFT JSONL → LoRA a local model      │
│             (bake the VOICE into a model; facts stay in RAG)    │
└─────────────────────────────────────────────────────────────────┘

Why distill instead of RAG-ing the corpus straight into the game?

1. Real people wrote the corpus. Verbatim forum posts coming out of an NPC's mouth in a public game is a legal grey zone and recognizably someone. The corpus is used as voice and attitude reference only; the prompt forbids copying, and provenance (source_post_ids) is kept so any line can be audited against its references.
2. Review gate. Every generated line lands with approved = NULL. A human eyeballs before anything ships (house rule — same as mocapgod QC clips). Essential when the writer model is uncensored (below).
3. Tiny runtime. A reviewed line bank + a small embedding model runs in the browser (LiteRT.js / WASM / WebGPU) — no server, no 2GB model download for players, works offline on monsterrobot.games.

The abliterated toggle

./gabgod abliterated status   # which writer distill.py will use
./gabgod abliterated on       # uncensored writer (local, private generation)
./gabgod abliterated off      # stock writer (safe default)

config.json → writer.abliterated flips between two OpenAI-compatible endpoints:

	model	where	when
off	qwen2.5:7b	m4pro Ollama	safe default, boring but obedient
on	supergemma4-26b-uncensored (16GB GGUF)	ultra's own Ollama	commits to the bit, no refusals

(First choice was gemma-4-E4B-it-OBLITERATED — abandoned after it drifted into Chinese mid-run and failed 25/40 topics. Abliteration damages small models; the 26B holds together. See "Earned gotchas" below.)

House rule: abliterated output never ships publicly unreviewed. The toggle gates generation; the approved column gates shipping. Both must be green for a line to reach a public game. For private/LAN free-chat NPCs, point the game at the abliterated endpoint directly and skip the bank — your house, your rules.

Using it on something else later (the reusable bit)

1. Get a corpus into pgvector. Any table with (id, body_text, embedding vector(768)) embedded with nomic-embed-text works. Add the table name to config.json → retrieval.sources. (In discogs_full you already have embedded: soulstrut_post, vgplus_post, ishkur_style, discogs_reviews 414k, yt_transcript, release_wikipedia 1.3M, artist_wikipedia, semantic_gold…)
2. Write personas.json. Per character: voice (who they are, how they talk) and topics (situations they'll need lines for). The topics double as the retrieval queries — write them in the corpus's vocabulary for better matches.
3. Run ./gabgod distill. Heartbeat in ~/.jobs/gabgod-distill.status, resume-safe (persona-topics with enough lines are skipped on rerun).
4. Review (./gabgod sample <persona>, then set approved in psql/pgweb), export (./gabgod export → sft/train.jsonl + valid.jsonl), and either ship the bank or LoRA the voice (mlx_lm.lora --data sft/ ... on m3ultra).

Ideas already on the list: RECORDGOD store-assistant voice from discogs_reviews; MRPGI narrator personas; toastsim's judge from the same forums; Ishkur-flavoured genre-pedant NPC from ishkur_style (153 embedded style blurbs).

Files

file	what
distill.py	the forge: retrieve → write → insert. --persona X --topics N --dry-run to smoke-test
personas.json	the cast (currently: PROCITY's six shopkeepers)
config.json	endpoints + toggle + retrieval knobs (gitignored; start from config.example.json)
schema.sql	gabgod.line_bank (+ schema) — run once per database
export_sft.py	approved lines → chat-format JSONL for LoRA
gabgod	CLI: abliterated on/off/status · distill · status · sample · counts · export

Ops notes

• Deployment lives on ultra ~/Documents/GABGOD (the data's there; scripts use /opt/homebrew/bin/python3 per TCC rule). This repo is the canonical source.
• Long runs follow the heartbeat convention (~/.jobs/, overwrite-not-append, die loudly, checkpoint+resume).
• Embeddings: nomic-embed-text (768d) — must match whatever embedded the corpus, or retrieval silently degrades. Check dims with atttypmod on the vector column.
• Postgres gotcha that bit here: n_live_tup stats on the corpus tables read 0 (stale autovacuum); trust count(*), not pg_stat_user_tables.
• Ollama structured-output note: small models ignore "exactly N lines" — the parser takes whatever valid JSON array comes back (raw_decode from first [).

Earned gotchas (one bad evening's tuition)

1. Abliterated small models are brain-damaged. The E4B weight-edit produced Chinese, word salad, and malformed JSON under a real workload despite passing a simple smoke test. Test with the PRODUCTION prompt, not "say 3 phrases".
2. Reasoning models + long prompts = empty answers. supergemma via the OpenAI-compat /v1 endpoint burned its entire 4096-token budget on internal drafts (finish_reason: length, 12k chars of reasoning, zero content). Fix: use ollama's native /api/chat with "think": false — 12/12 lines in 9s vs nothing in 5 minutes. distill.py falls back automatically for models with no think switch.
3. Keep the reference block small. >6 refs × 400 chars measurably worsens deliberation spirals; retrieval top_k can stay high, but cap what you show the writer.
4. sane_line() is load-bearing. Language-drift and fragment junk WILL reach the DB without the ascii-ratio/length gate; purging after the fact is delete ... where length(regexp_replace(line,'[[:ascii:]]','','g')) > 3.
5. Resume counts must be per-writer-model (writer_model=%s in the skip query), or a failed model's junk "completes" topics and blocks the good model from refilling them.