RECORDGOD/RECORDGOD_PLAN.md
type-two d75f27a691 feat: RecordGod founding — service scaffold + MariaDB→Postgres migration
- FastAPI twin of DealGod (async SQLAlchemy/asyncpg). wowplatter/v1: ping,
  inventory/lookup, inventory/intake (enriches from discogs_full by release_id).
- schema v2: unified inventory (vinyl+general), slim crate, sales/sale_items.
  Discogs catalog NOT copied — joined from discogs_full at read time.
- migrate.py: 34,543 shop rows moved (vs ~3.2M in the WowPlatter clone).
- Smoke test green; plan/readme carry the architecture + UI boundary.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-19 17:18:50 +10:00

228 lines
11 KiB
Markdown

# RecordGod — The Plan
_Founding document. The map all four codebases build to. 2026-06-19._
> **For the DealGod / PRICEGOD / WowPlatter Claudes reading this:** this is RecordGod's
> charter. It does not change your briefs — it names who fulfils the store-API side of them
> and how the back-office moves out of WordPress without breaking the live shop.
---
## 0. TL;DR
- **RecordGod** = the records back-office engine. Postgres-native, runs **outside** WordPress,
on the same VPS pattern as `api.dealgod.pro`.
- It **ingests WowPlatter's job, not its code.** ~5% of WowPlatter's surface is the prize;
the rest stays WP-native or dies with the founder.
- It serves the **`wowplatter/v1`** contract (the five routes from the WowPlatter brief). That
contract is the **seam** — PRICEGOD codes against it; whether PHP-in-WP or RecordGod answers
is an implementation detail that moves over time.
- It **calls DealGod** for market value. DealGod stays a separate product. RecordGod is its
customer, same as PRICEGOD.
- **Source of truth by concern:** RecordGod/local owns intake & catalog; WordPress/Woo owns
sale-state & checkout. Non-destructive UPSERT on `release_id` + SKU. Never overwrite a sold
or hand-edited row.
---
## 1. The mandate
**One line:** RecordGod is the records back-office engine — inventory truth, import,
enrichment, labels, copy, queries — Postgres-native, outside WordPress, calling DealGod for
market value and feeding tidy data out to the storefront.
**Honour the founder.** WowPlatter keeps running the live floor and is the *reference spec*.
RecordGod matches what WowPlatter already does right — its schema, the exact label layout, the
sale-state rules — before it retires anything. Copying the founder's hard-won correctness is
both the respectful move and the lazy one: don't redesign what works.
**`release_id` (Discogs release id) is the universal join key.** Lead with it everywhere.
---
## 2. The cast & the map
```mermaid
flowchart TB
PG["PRICEGOD<br/>seller extension (Discogs cockpit)"]
DG[("DealGod<br/>market brain · api.dealgod.pro")]
RG[["RecordGod<br/>records engine · Postgres, outside WP"]]
MIR[("Local mirrors @ ultra<br/>discogs_full · MusicBrainz · enwiki")]
WP["WowPlatter<br/>WP plugin · live storefront"]
COM["WooCommerce · Square · Aus Post · Mail<br/>checkout · sale-state"]
PG -->|"X-API-Key · price-suggest, /api/me"| DG
PG -->|"wowplatter/v1 contract"| RG
RG -->|"asks market value"| DG
RG -->|"enrich: instant, unlimited"| MIR
RG -->|"publish: UPSERT on release_id+SKU"| WP
WP --> COM
COM -.->|"sale-state is authoritative"| RG
```
- **PRICEGOD** — the brains on top. Talks to exactly **two** backends (DealGod + the store
API). Treats any `404` as "pending" and degrades gracefully, so routes can ship one at a time.
- **DealGod** — the market brain. Untouched except one add (`GET /api/me`). Its own product.
- **RecordGod** — the muscle. Serves `wowplatter/v1`, enriches from the local mirrors, calls
DealGod for value, publishes tidy data to WordPress.
- **WowPlatter** — the founder. Live storefront, 3D store, and the commerce stack (Woo, Square,
Aus Post rates, transactional mail). Keeps these forever.
- **Local mirrors @ ultra** — full Discogs + MusicBrainz + enwiki dumps in Postgres. Already
proven (the apple-id WP-CLI command `pg_connect`s to `discogs_full` on `127.0.0.1:5432`).
---
## 3. The contract seam — `wowplatter/v1`
The five routes from `../PRICEGOD/WOWPLATTER_BRIEF.md` are RecordGod's front door:
| Route | Purpose | First home | Final home |
|---|---|---|---|
| `GET /ping` | connection test + store identity chip | PHP-in-WP | RecordGod |
| `GET /inventory/lookup?release_id=` | "do WE have this?" (the daily driver) | PHP-in-WP | RecordGod |
| `POST /inventory/price` | accept-target write → push Woo + Square | **stays WP** (commerce) | **stays WP** |
| `GET /labels/templates` | so PRICEGOD prints OUR label design | PHP-in-WP | RecordGod |
| `POST /inventory/intake` | new-stock intake → local staging | PHP-in-WP | RecordGod |
**The reframe that matters:** the WowPlatter brief describes this as a PHP facade *inside*
WordPress. That is correct — as **step one of a strangler, not the permanent home.** The
*contract* is the seam; the *implementation* starts in PHP over today's MariaDB (fast, unblocks
PRICEGOD now) and migrates into RecordGod behind the frozen contract, route by route. PRICEGOD
never notices.
The one route that **never leaves WordPress** is `inventory/price`'s write side — pushing the
accepted price to WooCommerce + Square is commerce, and commerce is WP-native. RecordGod stages
the price; the thin WP plugin does the push.
---
## 4. The triage — ingest the value, let the bloat die with the founder
WowPlatter is ~7,700 first-party PHP files. "Best of WowPlatter" is a fraction of it.
### Ingest into RecordGod (the prize)
- `inventory-db` / change-logger / items-handler — inventory truth
- `import-*` (artist / label / master / release / style) + batch-processor — **rebuilt on the
local mirrors**, not live `api.discogs.com`. The worst MariaDB pain and the biggest win.
- `discogs-adapter` / `musicbrainz` / `wikidata` enrichment — repointed at `discogs_full` / MB
/ wiki on ultra
- `label-designer` — templates as JSON first, designer UI later
- reports / queries — the stuff MariaDB-in-WP chokes on
### Stays WordPress / Woo-native (never moves — the money & shopfront)
- checkout, orders, Square sync, Aus Post rates, the storefront + 3D store, transactional mail
*sending*
### Let it die with the founder (do NOT carry across)
- ai-chatbot, matrix-appservice, beeper-bridge, gig-guide, ticketmaster, facebook-catalog /
ebay channels (maybe later), css/js optimizers, image-scanner, alt-text, the PHP
backup/cron/retry plumbing — RecordGod gets its own Python/Postgres-native infra, not a port.
> Every line you *don't* ingest is the win. This triage is the difference between a 3-week
> build and a 9-month one.
---
## 5. The "+more" — only what Postgres-outside-WP actually unlocks
Not a feature parade. The handful that are *hard in MariaDB/WP and easy in RecordGod*:
1. **Import enrichment from local mirrors** — instant / unlimited vs rate-limited Discogs API.
2. **pgvector copy** — newsletter blurbs, "records like this", semantic dedupe of the dirty
catalog. The LLM writes; pgvector retrieves. Default to Claude when wired. *(Add pgvector
only when keyword/trigram search measurably falls short — not day one.)*
3. **Queries / reporting** the WP admin can't do at speed.
4. **DealGod arbitrage native** — buy-signals on your *own* shelves, not just suggested prices.
5. **Staging → publish pipeline** with the non-destructive UPSERT (§6).
---
## 6. Source of truth — the 3am gotcha, already solved
Split by **concern**, not by table (the WowPlatter brief got this right — lock it in):
- **Live (WordPress/Woo) is authoritative for sale-state.** Never overwrite sold / ordered /
hand-edited rows.
- **Local (RecordGod) is authoritative for intake & catalog.**
- **Publish = non-destructive UPSERT keyed on `release_id` + SKU.** Pull live's current
SKUs/sold-status into local *first*, then publish.
- Keep the Google-Sheet import as a fallback until the local pipeline earns trust.
This is the one decision that's expensive to get wrong. Everything else is reversible.
---
## 7. Build order (the lazy sequence)
1. **Unblock PRICEGOD this week.** Ship `wowplatter/v1` routes `ping` + `lookup` + `price` as a
thin PHP facade over today's MariaDB. A few hundred lines over data that already exists.
Not throwaway — it's the contract.
2. **Add DealGod's `GET /api/me`** (two lines off `require_key`) — the only thing blocking
PRICEGOD's green light.
3. **Stand up RecordGod** (Python + Postgres + nginx, same pattern as `api.dealgod.pro`).
First migrate **import-on-local-mirrors** — that's the pain you actually named.
4. **Migrate routes' guts** into RecordGod behind the frozen contract, then flip the base URL
(or per-route proxy) once enough have moved. PRICEGOD codes against the contract, not the
location.
5. Then labels (JSON templates → port the pliceclogs layout), then intake/staging, then the
pgvector copy + reporting wins.
**Migrate the schema, don't redesign it.** Day one: dump the relevant MariaDB tables → load
into Postgres as-is, *then* improve indexes / add pgvector. Reshaping before moving is how you
strand the live store.
---
## 8. Open decisions
- **Auth scheme.** DealGod uses `X-API-Key` (UUID); the WowPlatter brief says `Authorization:
Bearer` (WP app-password / api-tokens). Two schemes is fine **if deliberate** (one for the
market brain, one for the store) — but PRICEGOD has to hold both. Confirm that's intended, or
unify, before the facade ships. An auth scheme baked into a shipped extension is annoying to
change later.
---
## 9. Stack
Reuse what's already operated daily — a new framework is a tax paid forever.
- **Python + Postgres**, fronted by **nginx**, on the **same VPS**, same shape as
`api.dealgod.pro`. No WordPress process involved.
- Enrichment reads the **local mirrors on ultra** over Tailscale (`discogs_full` / MB / wiki).
- pgvector when retrieval/similarity is the real need, not before.
---
## 10. UI integration boundary — shared shell, separate engine
The web UI is **shared** with DealGod; the engine and data are **not**. RecordGod renders as an
**enterprise-gated section** of the DealGod web app, but its pages are served by RecordGod and
call RecordGod's API. Share the glass, not the muscle.
- RecordGod serves its own UI from its own service/repo, styled with DealGod's shared theme.
- DealGod's shell mounts it via **three things only**:
1. **nginx path route**`app.dealgod.pro/store/*` → RecordGod; everything else → DealGod.
2. **shared auth** — one login session; RecordGod trusts DealGod's auth and reads the plan;
`plan=enterprise` lights the `/store` nav entry.
3. **shared theme** — same `_theme.html` / `themeify`, so it looks like one site.
- **No code overlap.** The two repos meet at nginx + a cookie + a CSS file + the API contract.
- **Dependency direction:** DealGod knows RecordGod's mount URL and gates on plan; RecordGod
trusts DealGod's auth. RecordGod never imports DealGod internals.
**Why this matters for staffing:** the DealGod-side IA reorg (role-based nav: `admin` /
`shopper: basic, pro` / `store: enterprise → RecordGod`) and the RecordGod build run **in
parallel** against this seam without colliding — different repos, one small contract. The
DealGod-repo change is just the mount point (one nav link + the nginx route + trust-the-cookie),
owned by whoever holds DealGod.
---
## 11. Pointers
- `../PRICEGOD/DEALGOD_BRIEF.md` — DealGod's one add (`/api/me`) + two optional niceties.
- `../PRICEGOD/WOWPLATTER_BRIEF.md` — the five-route contract + the local-clone pipeline.
- `../PRICEGOD/src/lib/wowplatter.js` — the exact shapes the extension already calls.
- `../dealgod` `media_routes.py:1043` — the live `POST /api/price-suggest` RecordGod will call.
- `../wowplatter` — the founder; reference spec for schema, labels, sale-state rules.