feat: RecordGod founding — service scaffold + MariaDB→Postgres migration
- FastAPI twin of DealGod (async SQLAlchemy/asyncpg). wowplatter/v1: ping, inventory/lookup, inventory/intake (enriches from discogs_full by release_id). - schema v2: unified inventory (vinyl+general), slim crate, sales/sale_items. Discogs catalog NOT copied — joined from discogs_full at read time. - migrate.py: 34,543 shop rows moved (vs ~3.2M in the WowPlatter clone). - Smoke test green; plan/readme carry the architecture + UI boundary. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
commit
d75f27a691
5
.gitignore
vendored
Normal file
5
.gitignore
vendored
Normal file
@ -0,0 +1,5 @@
|
||||
__pycache__/
|
||||
*.pyc
|
||||
.env
|
||||
.venv/
|
||||
venv/
|
||||
51
README.md
Normal file
51
README.md
Normal file
@ -0,0 +1,51 @@
|
||||
# RecordGod
|
||||
|
||||
**The records back-office engine — the best of WowPlatter, reborn on Postgres, outside WordPress.**
|
||||
|
||||
RecordGod ingests WowPlatter's *job* (inventory, import, enrichment, labels, copy, queries)
|
||||
but not WowPlatter's *code*. It runs as its own service on the VPS, calls DealGod for market
|
||||
value, and feeds WordPress the tidy data it likes for the storefront.
|
||||
|
||||
WowPlatter is the **ancestral founder** — still running the live shop floor, and the reference
|
||||
spec RecordGod matches before it retires anything. Honour the founder: don't redesign what
|
||||
already works, copy it.
|
||||
|
||||
## Start here
|
||||
- **[RECORDGOD_PLAN.md](RECORDGOD_PLAN.md)** — the plan, the map, the keep/drop triage, build
|
||||
order, and the open decisions. Read this first.
|
||||
|
||||
## The family (sibling repos on this machine)
|
||||
| Repo | What it is | Talks to RecordGod via |
|
||||
|---|---|---|
|
||||
| `../dealgod` | The **market** brain — cross-store value, arbitrage. Its own product. | RecordGod *calls* it (`X-API-Key`) for value |
|
||||
| `../PRICEGOD` | The seller **extension** (Discogs cockpit). Successor to pliceclogs. | Calls RecordGod's `wowplatter/v1` contract |
|
||||
| `../wowplatter` | The founder — WP/WooCommerce plugin running the live store. | RecordGod publishes tidy data to it |
|
||||
|
||||
Briefs that define the contracts already live in `../PRICEGOD/DEALGOD_BRIEF.md` and
|
||||
`../PRICEGOD/WOWPLATTER_BRIEF.md`. All three still hold; RecordGod is who fulfils the
|
||||
WowPlatter side of them.
|
||||
|
||||
## Run it (dev)
|
||||
```bash
|
||||
python3 -m venv .venv && ./.venv/bin/pip install -r requirements.txt
|
||||
RECORDGOD_TOKEN=dev-tok ./.venv/bin/uvicorn app.main:app --reload --port 8010
|
||||
./.venv/bin/python test_smoke.py # self-check — no DB/infra needed
|
||||
```
|
||||
Postgres schema: `psql recordgod < schema.sql`.
|
||||
|
||||
Live now: `GET /wowplatter/v1/ping`, `GET /wowplatter/v1/inventory/lookup?release_id=`.
|
||||
Routes 3–5 (price / labels / intake) are intentionally **404** — PRICEGOD reads that as
|
||||
"pending" and degrades gracefully, so an undefined route is the correct "not built yet" signal.
|
||||
|
||||
## Layout
|
||||
```
|
||||
app/
|
||||
main.py FastAPI app + /healthz
|
||||
db.py async SQLAlchemy over asyncpg (twin of DealGod's db.py)
|
||||
auth.py require_token — Bearer; the one place the auth scheme lives
|
||||
wowplatter_v1.py the contract router (ping + lookup live)
|
||||
schema.sql minimal inventory table, release_id-keyed, store_id seam
|
||||
test_smoke.py infra-free self-check
|
||||
```
|
||||
|
||||
_Status: founding — scaffold runs, smoke test green. 2026-06-19._
|
||||
227
RECORDGOD_PLAN.md
Normal file
227
RECORDGOD_PLAN.md
Normal file
@ -0,0 +1,227 @@
|
||||
# RecordGod — The Plan
|
||||
|
||||
_Founding document. The map all four codebases build to. 2026-06-19._
|
||||
|
||||
> **For the DealGod / PRICEGOD / WowPlatter Claudes reading this:** this is RecordGod's
|
||||
> charter. It does not change your briefs — it names who fulfils the store-API side of them
|
||||
> and how the back-office moves out of WordPress without breaking the live shop.
|
||||
|
||||
---
|
||||
|
||||
## 0. TL;DR
|
||||
|
||||
- **RecordGod** = the records back-office engine. Postgres-native, runs **outside** WordPress,
|
||||
on the same VPS pattern as `api.dealgod.pro`.
|
||||
- It **ingests WowPlatter's job, not its code.** ~5% of WowPlatter's surface is the prize;
|
||||
the rest stays WP-native or dies with the founder.
|
||||
- It serves the **`wowplatter/v1`** contract (the five routes from the WowPlatter brief). That
|
||||
contract is the **seam** — PRICEGOD codes against it; whether PHP-in-WP or RecordGod answers
|
||||
is an implementation detail that moves over time.
|
||||
- It **calls DealGod** for market value. DealGod stays a separate product. RecordGod is its
|
||||
customer, same as PRICEGOD.
|
||||
- **Source of truth by concern:** RecordGod/local owns intake & catalog; WordPress/Woo owns
|
||||
sale-state & checkout. Non-destructive UPSERT on `release_id` + SKU. Never overwrite a sold
|
||||
or hand-edited row.
|
||||
|
||||
---
|
||||
|
||||
## 1. The mandate
|
||||
|
||||
**One line:** RecordGod is the records back-office engine — inventory truth, import,
|
||||
enrichment, labels, copy, queries — Postgres-native, outside WordPress, calling DealGod for
|
||||
market value and feeding tidy data out to the storefront.
|
||||
|
||||
**Honour the founder.** WowPlatter keeps running the live floor and is the *reference spec*.
|
||||
RecordGod matches what WowPlatter already does right — its schema, the exact label layout, the
|
||||
sale-state rules — before it retires anything. Copying the founder's hard-won correctness is
|
||||
both the respectful move and the lazy one: don't redesign what works.
|
||||
|
||||
**`release_id` (Discogs release id) is the universal join key.** Lead with it everywhere.
|
||||
|
||||
---
|
||||
|
||||
## 2. The cast & the map
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
PG["PRICEGOD<br/>seller extension (Discogs cockpit)"]
|
||||
DG[("DealGod<br/>market brain · api.dealgod.pro")]
|
||||
RG[["RecordGod<br/>records engine · Postgres, outside WP"]]
|
||||
MIR[("Local mirrors @ ultra<br/>discogs_full · MusicBrainz · enwiki")]
|
||||
WP["WowPlatter<br/>WP plugin · live storefront"]
|
||||
COM["WooCommerce · Square · Aus Post · Mail<br/>checkout · sale-state"]
|
||||
|
||||
PG -->|"X-API-Key · price-suggest, /api/me"| DG
|
||||
PG -->|"wowplatter/v1 contract"| RG
|
||||
RG -->|"asks market value"| DG
|
||||
RG -->|"enrich: instant, unlimited"| MIR
|
||||
RG -->|"publish: UPSERT on release_id+SKU"| WP
|
||||
WP --> COM
|
||||
COM -.->|"sale-state is authoritative"| RG
|
||||
```
|
||||
|
||||
- **PRICEGOD** — the brains on top. Talks to exactly **two** backends (DealGod + the store
|
||||
API). Treats any `404` as "pending" and degrades gracefully, so routes can ship one at a time.
|
||||
- **DealGod** — the market brain. Untouched except one add (`GET /api/me`). Its own product.
|
||||
- **RecordGod** — the muscle. Serves `wowplatter/v1`, enriches from the local mirrors, calls
|
||||
DealGod for value, publishes tidy data to WordPress.
|
||||
- **WowPlatter** — the founder. Live storefront, 3D store, and the commerce stack (Woo, Square,
|
||||
Aus Post rates, transactional mail). Keeps these forever.
|
||||
- **Local mirrors @ ultra** — full Discogs + MusicBrainz + enwiki dumps in Postgres. Already
|
||||
proven (the apple-id WP-CLI command `pg_connect`s to `discogs_full` on `127.0.0.1:5432`).
|
||||
|
||||
---
|
||||
|
||||
## 3. The contract seam — `wowplatter/v1`
|
||||
|
||||
The five routes from `../PRICEGOD/WOWPLATTER_BRIEF.md` are RecordGod's front door:
|
||||
|
||||
| Route | Purpose | First home | Final home |
|
||||
|---|---|---|---|
|
||||
| `GET /ping` | connection test + store identity chip | PHP-in-WP | RecordGod |
|
||||
| `GET /inventory/lookup?release_id=` | "do WE have this?" (the daily driver) | PHP-in-WP | RecordGod |
|
||||
| `POST /inventory/price` | accept-target write → push Woo + Square | **stays WP** (commerce) | **stays WP** |
|
||||
| `GET /labels/templates` | so PRICEGOD prints OUR label design | PHP-in-WP | RecordGod |
|
||||
| `POST /inventory/intake` | new-stock intake → local staging | PHP-in-WP | RecordGod |
|
||||
|
||||
**The reframe that matters:** the WowPlatter brief describes this as a PHP facade *inside*
|
||||
WordPress. That is correct — as **step one of a strangler, not the permanent home.** The
|
||||
*contract* is the seam; the *implementation* starts in PHP over today's MariaDB (fast, unblocks
|
||||
PRICEGOD now) and migrates into RecordGod behind the frozen contract, route by route. PRICEGOD
|
||||
never notices.
|
||||
|
||||
The one route that **never leaves WordPress** is `inventory/price`'s write side — pushing the
|
||||
accepted price to WooCommerce + Square is commerce, and commerce is WP-native. RecordGod stages
|
||||
the price; the thin WP plugin does the push.
|
||||
|
||||
---
|
||||
|
||||
## 4. The triage — ingest the value, let the bloat die with the founder
|
||||
|
||||
WowPlatter is ~7,700 first-party PHP files. "Best of WowPlatter" is a fraction of it.
|
||||
|
||||
### Ingest into RecordGod (the prize)
|
||||
- `inventory-db` / change-logger / items-handler — inventory truth
|
||||
- `import-*` (artist / label / master / release / style) + batch-processor — **rebuilt on the
|
||||
local mirrors**, not live `api.discogs.com`. The worst MariaDB pain and the biggest win.
|
||||
- `discogs-adapter` / `musicbrainz` / `wikidata` enrichment — repointed at `discogs_full` / MB
|
||||
/ wiki on ultra
|
||||
- `label-designer` — templates as JSON first, designer UI later
|
||||
- reports / queries — the stuff MariaDB-in-WP chokes on
|
||||
|
||||
### Stays WordPress / Woo-native (never moves — the money & shopfront)
|
||||
- checkout, orders, Square sync, Aus Post rates, the storefront + 3D store, transactional mail
|
||||
*sending*
|
||||
|
||||
### Let it die with the founder (do NOT carry across)
|
||||
- ai-chatbot, matrix-appservice, beeper-bridge, gig-guide, ticketmaster, facebook-catalog /
|
||||
ebay channels (maybe later), css/js optimizers, image-scanner, alt-text, the PHP
|
||||
backup/cron/retry plumbing — RecordGod gets its own Python/Postgres-native infra, not a port.
|
||||
|
||||
> Every line you *don't* ingest is the win. This triage is the difference between a 3-week
|
||||
> build and a 9-month one.
|
||||
|
||||
---
|
||||
|
||||
## 5. The "+more" — only what Postgres-outside-WP actually unlocks
|
||||
|
||||
Not a feature parade. The handful that are *hard in MariaDB/WP and easy in RecordGod*:
|
||||
|
||||
1. **Import enrichment from local mirrors** — instant / unlimited vs rate-limited Discogs API.
|
||||
2. **pgvector copy** — newsletter blurbs, "records like this", semantic dedupe of the dirty
|
||||
catalog. The LLM writes; pgvector retrieves. Default to Claude when wired. *(Add pgvector
|
||||
only when keyword/trigram search measurably falls short — not day one.)*
|
||||
3. **Queries / reporting** the WP admin can't do at speed.
|
||||
4. **DealGod arbitrage native** — buy-signals on your *own* shelves, not just suggested prices.
|
||||
5. **Staging → publish pipeline** with the non-destructive UPSERT (§6).
|
||||
|
||||
---
|
||||
|
||||
## 6. Source of truth — the 3am gotcha, already solved
|
||||
|
||||
Split by **concern**, not by table (the WowPlatter brief got this right — lock it in):
|
||||
|
||||
- **Live (WordPress/Woo) is authoritative for sale-state.** Never overwrite sold / ordered /
|
||||
hand-edited rows.
|
||||
- **Local (RecordGod) is authoritative for intake & catalog.**
|
||||
- **Publish = non-destructive UPSERT keyed on `release_id` + SKU.** Pull live's current
|
||||
SKUs/sold-status into local *first*, then publish.
|
||||
- Keep the Google-Sheet import as a fallback until the local pipeline earns trust.
|
||||
|
||||
This is the one decision that's expensive to get wrong. Everything else is reversible.
|
||||
|
||||
---
|
||||
|
||||
## 7. Build order (the lazy sequence)
|
||||
|
||||
1. **Unblock PRICEGOD this week.** Ship `wowplatter/v1` routes `ping` + `lookup` + `price` as a
|
||||
thin PHP facade over today's MariaDB. A few hundred lines over data that already exists.
|
||||
Not throwaway — it's the contract.
|
||||
2. **Add DealGod's `GET /api/me`** (two lines off `require_key`) — the only thing blocking
|
||||
PRICEGOD's green light.
|
||||
3. **Stand up RecordGod** (Python + Postgres + nginx, same pattern as `api.dealgod.pro`).
|
||||
First migrate **import-on-local-mirrors** — that's the pain you actually named.
|
||||
4. **Migrate routes' guts** into RecordGod behind the frozen contract, then flip the base URL
|
||||
(or per-route proxy) once enough have moved. PRICEGOD codes against the contract, not the
|
||||
location.
|
||||
5. Then labels (JSON templates → port the pliceclogs layout), then intake/staging, then the
|
||||
pgvector copy + reporting wins.
|
||||
|
||||
**Migrate the schema, don't redesign it.** Day one: dump the relevant MariaDB tables → load
|
||||
into Postgres as-is, *then* improve indexes / add pgvector. Reshaping before moving is how you
|
||||
strand the live store.
|
||||
|
||||
---
|
||||
|
||||
## 8. Open decisions
|
||||
|
||||
- **Auth scheme.** DealGod uses `X-API-Key` (UUID); the WowPlatter brief says `Authorization:
|
||||
Bearer` (WP app-password / api-tokens). Two schemes is fine **if deliberate** (one for the
|
||||
market brain, one for the store) — but PRICEGOD has to hold both. Confirm that's intended, or
|
||||
unify, before the facade ships. An auth scheme baked into a shipped extension is annoying to
|
||||
change later.
|
||||
|
||||
---
|
||||
|
||||
## 9. Stack
|
||||
|
||||
Reuse what's already operated daily — a new framework is a tax paid forever.
|
||||
|
||||
- **Python + Postgres**, fronted by **nginx**, on the **same VPS**, same shape as
|
||||
`api.dealgod.pro`. No WordPress process involved.
|
||||
- Enrichment reads the **local mirrors on ultra** over Tailscale (`discogs_full` / MB / wiki).
|
||||
- pgvector when retrieval/similarity is the real need, not before.
|
||||
|
||||
---
|
||||
|
||||
## 10. UI integration boundary — shared shell, separate engine
|
||||
|
||||
The web UI is **shared** with DealGod; the engine and data are **not**. RecordGod renders as an
|
||||
**enterprise-gated section** of the DealGod web app, but its pages are served by RecordGod and
|
||||
call RecordGod's API. Share the glass, not the muscle.
|
||||
|
||||
- RecordGod serves its own UI from its own service/repo, styled with DealGod's shared theme.
|
||||
- DealGod's shell mounts it via **three things only**:
|
||||
1. **nginx path route** — `app.dealgod.pro/store/*` → RecordGod; everything else → DealGod.
|
||||
2. **shared auth** — one login session; RecordGod trusts DealGod's auth and reads the plan;
|
||||
`plan=enterprise` lights the `/store` nav entry.
|
||||
3. **shared theme** — same `_theme.html` / `themeify`, so it looks like one site.
|
||||
- **No code overlap.** The two repos meet at nginx + a cookie + a CSS file + the API contract.
|
||||
- **Dependency direction:** DealGod knows RecordGod's mount URL and gates on plan; RecordGod
|
||||
trusts DealGod's auth. RecordGod never imports DealGod internals.
|
||||
|
||||
**Why this matters for staffing:** the DealGod-side IA reorg (role-based nav: `admin` /
|
||||
`shopper: basic, pro` / `store: enterprise → RecordGod`) and the RecordGod build run **in
|
||||
parallel** against this seam without colliding — different repos, one small contract. The
|
||||
DealGod-repo change is just the mount point (one nav link + the nginx route + trust-the-cookie),
|
||||
owned by whoever holds DealGod.
|
||||
|
||||
---
|
||||
|
||||
## 11. Pointers
|
||||
|
||||
- `../PRICEGOD/DEALGOD_BRIEF.md` — DealGod's one add (`/api/me`) + two optional niceties.
|
||||
- `../PRICEGOD/WOWPLATTER_BRIEF.md` — the five-route contract + the local-clone pipeline.
|
||||
- `../PRICEGOD/src/lib/wowplatter.js` — the exact shapes the extension already calls.
|
||||
- `../dealgod` `media_routes.py:1043` — the live `POST /api/price-suggest` RecordGod will call.
|
||||
- `../wowplatter` — the founder; reference spec for schema, labels, sale-state rules.
|
||||
1
app/__init__.py
Normal file
1
app/__init__.py
Normal file
@ -0,0 +1 @@
|
||||
__version__ = "0.1.0"
|
||||
22
app/auth.py
Normal file
22
app/auth.py
Normal file
@ -0,0 +1,22 @@
|
||||
import os
|
||||
from fastapi import Header, HTTPException
|
||||
|
||||
# Single-tenant v1: one store, one token from the environment.
|
||||
STORE = os.getenv("RECORDGOD_STORE", "Monster Robot Party")
|
||||
PLAN = os.getenv("RECORDGOD_PLAN", "enterprise")
|
||||
TOKEN = os.getenv("RECORDGOD_TOKEN") # no default — unset means "deny everything", not "open"
|
||||
|
||||
|
||||
async def require_token(authorization: str | None = Header(None)):
|
||||
# ponytail: Bearer per WOWPLATTER_BRIEF. The auth scheme is still an OPEN decision
|
||||
# (DealGod's X-API-Key vs this Bearer) — keep the resolver in this one function so
|
||||
# flipping it, or swapping the env token for a tokens table / shared DealGod session,
|
||||
# is a single-file change.
|
||||
if not TOKEN:
|
||||
raise HTTPException(503, "RECORDGOD_TOKEN not configured")
|
||||
if not authorization or not authorization.startswith("Bearer "):
|
||||
raise HTTPException(401, "missing bearer token")
|
||||
if authorization[7:].strip() != TOKEN:
|
||||
raise HTTPException(401, "invalid token")
|
||||
# ponytail: store_id is always 1 for now — the seam for multi-shop, not the room.
|
||||
return {"store": STORE, "plan": PLAN, "store_id": 1}
|
||||
22
app/db.py
Normal file
22
app/db.py
Normal file
@ -0,0 +1,22 @@
|
||||
from sqlalchemy.ext.asyncio import create_async_engine, AsyncSession, async_sessionmaker
|
||||
import os
|
||||
|
||||
# Same shape as DealGod's db.py — async SQLAlchemy over asyncpg. RecordGod owns its OWN db.
|
||||
DATABASE_URL = os.getenv("DATABASE_URL", "postgresql+asyncpg://localhost/recordgod")
|
||||
|
||||
engine = create_async_engine(
|
||||
DATABASE_URL, pool_size=5, max_overflow=5,
|
||||
pool_timeout=30, pool_recycle=900, pool_pre_ping=True)
|
||||
SessionLocal = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
|
||||
|
||||
|
||||
async def get_db():
|
||||
async with SessionLocal() as session:
|
||||
yield session
|
||||
|
||||
|
||||
# Read-only enrichment mirror — the full Discogs dump (same local Postgres, different db).
|
||||
# We DON'T copy the catalog into RecordGod; intake joins it on release_id at write time.
|
||||
MIRROR_URL = os.getenv("DISCOGS_MIRROR_URL", "postgresql+asyncpg://localhost/discogs_full")
|
||||
mirror_engine = create_async_engine(MIRROR_URL, pool_size=2, max_overflow=2, pool_pre_ping=True)
|
||||
MirrorSession = async_sessionmaker(mirror_engine, class_=AsyncSession, expire_on_commit=False)
|
||||
12
app/main.py
Normal file
12
app/main.py
Normal file
@ -0,0 +1,12 @@
|
||||
from fastapi import FastAPI
|
||||
|
||||
from . import __version__
|
||||
from .wowplatter_v1 import router as wowplatter_v1_router
|
||||
|
||||
app = FastAPI(title="RECORDGOD", version=__version__)
|
||||
app.include_router(wowplatter_v1_router)
|
||||
|
||||
|
||||
@app.get("/healthz")
|
||||
async def healthz():
|
||||
return {"ok": True, "service": "recordgod", "version": __version__}
|
||||
106
app/wowplatter_v1.py
Normal file
106
app/wowplatter_v1.py
Normal file
@ -0,0 +1,106 @@
|
||||
import base64
|
||||
import secrets
|
||||
|
||||
from fastapi import APIRouter, Depends, Query
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import text
|
||||
|
||||
from . import __version__
|
||||
from .auth import require_token
|
||||
from .db import get_db, MirrorSession
|
||||
|
||||
# The wowplatter/v1 contract — RecordGod's front door (see RECORDGOD_PLAN.md §3).
|
||||
# Routes 1, 2, 5 live here. Routes 3-4 (price/labels) are deliberately NOT defined yet:
|
||||
# PRICEGOD treats 404 as "pending" and degrades gracefully. ponytail: 404 already says it.
|
||||
router = APIRouter(prefix="/wowplatter/v1", tags=["wowplatter/v1"])
|
||||
|
||||
|
||||
@router.get("/ping")
|
||||
async def ping(ident=Depends(require_token)):
|
||||
"""Connection test + identity — lights PRICEGOD's green dot and store chip."""
|
||||
return {"ok": True, "store": ident["store"], "plan": ident["plan"], "version": __version__}
|
||||
|
||||
|
||||
@router.get("/inventory/lookup")
|
||||
async def inventory_lookup(
|
||||
release_id: int | None = Query(None),
|
||||
barcode: str | None = Query(None),
|
||||
ident=Depends(require_token),
|
||||
db=Depends(get_db),
|
||||
):
|
||||
""""Do WE have this?" — the most-used call. release_id first, identifier (barcode) fallback."""
|
||||
if release_id is None and barcode is None:
|
||||
return {"ok": False, "reason": "need release_id or barcode"}
|
||||
|
||||
where, params = (
|
||||
("release_id = :rid", {"rid": release_id})
|
||||
if release_id is not None
|
||||
else ("identifier = :bc", {"bc": barcode})
|
||||
)
|
||||
params["sid"] = ident["store_id"]
|
||||
|
||||
row = (await db.execute(text(
|
||||
f"SELECT sku, price, qty, condition, status, product_url, release_id "
|
||||
f"FROM inventory WHERE {where} AND store_id = :sid AND in_stock LIMIT 1"
|
||||
), params)).mappings().first()
|
||||
|
||||
if not row:
|
||||
return {"ok": True, "in_stock": False, "release_id": release_id}
|
||||
return {
|
||||
"ok": True, "in_stock": True, **dict(row),
|
||||
"price": float(row["price"]) if row["price"] is not None else None,
|
||||
}
|
||||
|
||||
|
||||
class IntakeIn(BaseModel):
|
||||
release_id: int | None = None
|
||||
identifier: str | None = None
|
||||
condition: str | None = None
|
||||
sleeve: str | None = None
|
||||
notes: str | None = None
|
||||
price: float | None = None
|
||||
sku: str | None = None
|
||||
kind: str = "vinyl"
|
||||
|
||||
|
||||
def _new_sku() -> str:
|
||||
# ponytail: MRP- + 8 base32 chars ≈ 1e12 space — collisions negligible at shop scale; the
|
||||
# sku PK + ON CONFLICT upsert means a clash just updates that row. Tighten if it ever bites.
|
||||
return "MRP-" + base64.b32encode(secrets.token_bytes(5)).decode().rstrip("=")[:8]
|
||||
|
||||
|
||||
@router.post("/inventory/intake")
|
||||
async def inventory_intake(body: IntakeIn, ident=Depends(require_token), db=Depends(get_db)):
|
||||
"""New-stock intake. Enriches from the discogs_full mirror by release_id, stages locally
|
||||
(staged=true). Publishing to live is a separate, non-destructive step (PLAN §6)."""
|
||||
title = artist = None
|
||||
weight = None
|
||||
if body.release_id:
|
||||
try: # ponytail: mirror down → stage un-enriched rather than fail the intake
|
||||
async with MirrorSession() as mirror:
|
||||
m = (await mirror.execute(text(
|
||||
"SELECT title, artists_sort, estimated_weight FROM release WHERE id = :r"
|
||||
), {"r": body.release_id})).mappings().first()
|
||||
if m:
|
||||
title, artist, weight = m["title"], m["artists_sort"], m["estimated_weight"]
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
sku = body.sku or _new_sku()
|
||||
await db.execute(text("""
|
||||
INSERT INTO inventory (sku, store_id, kind, release_id, identifier, title, price,
|
||||
condition, sleeve_cond, weight_g, notes, staged, status)
|
||||
VALUES (:sku, :sid, :kind, :rid, :ident, :title, :price,
|
||||
:cond, :sleeve, :wt, :notes, true, 'staged')
|
||||
ON CONFLICT (sku) DO UPDATE SET
|
||||
release_id = EXCLUDED.release_id, title = EXCLUDED.title, price = EXCLUDED.price,
|
||||
condition = EXCLUDED.condition, sleeve_cond = EXCLUDED.sleeve_cond,
|
||||
notes = EXCLUDED.notes, updated_at = now()
|
||||
"""), {
|
||||
"sku": sku, "sid": ident["store_id"], "kind": body.kind, "rid": body.release_id,
|
||||
"ident": body.identifier, "title": title, "price": body.price,
|
||||
"cond": body.condition, "sleeve": body.sleeve, "wt": weight, "notes": body.notes,
|
||||
})
|
||||
await db.commit()
|
||||
return {"ok": True, "sku": sku, "staged": True,
|
||||
"title": title, "artist": artist, "enriched": title is not None}
|
||||
159
migrate.py
Normal file
159
migrate.py
Normal file
@ -0,0 +1,159 @@
|
||||
#!/usr/bin/env python3
|
||||
"""One-shot MariaDB (WowPlatter) -> Postgres (RecordGod) migration.
|
||||
|
||||
Migrates the shop's OWN data only — inventory (two tables unified into one), sales history,
|
||||
and a slim crate table. The ~3.2M-row Discogs CATALOG is deliberately NOT copied: it lives in
|
||||
the discogs_full mirror and is joined on release_id at read time. See RECORDGOD_PLAN.md.
|
||||
|
||||
python migrate.py # idempotent: create schema, upsert (ON CONFLICT DO NOTHING)
|
||||
python migrate.py --truncate # wipe the 4 tables first, then reload
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import pathlib
|
||||
import re
|
||||
import sys
|
||||
from datetime import datetime, timezone
|
||||
|
||||
import pymysql
|
||||
import psycopg
|
||||
from psycopg.types.json import Jsonb
|
||||
|
||||
WP_CONFIG = os.getenv("WP_CONFIG", "/opt/homebrew/var/www/monsterrobot.localsite/wp-config.php")
|
||||
PG_DSN = os.getenv("RECORDGOD_DSN", "postgresql://localhost/recordgod")
|
||||
SOCK = os.getenv("MARIA_SOCK", "/opt/homebrew/var/mysql/mysql.sock")
|
||||
HERE = pathlib.Path(__file__).parent
|
||||
|
||||
|
||||
def wp_creds():
|
||||
t = pathlib.Path(WP_CONFIG).read_text()
|
||||
g = lambda k: re.search(rf"'{k}',\s*'([^']*)'", t).group(1)
|
||||
return g("DB_NAME"), g("DB_USER"), g("DB_PASSWORD")
|
||||
|
||||
|
||||
def yn(v):
|
||||
return v == "y"
|
||||
|
||||
|
||||
def js(v):
|
||||
if not v:
|
||||
return None
|
||||
try:
|
||||
return json.loads(v)
|
||||
except Exception:
|
||||
return None # ponytail: skip a malformed json blob rather than abort the whole migration
|
||||
|
||||
|
||||
def jb(v):
|
||||
return Jsonb(v) if v is not None else None
|
||||
|
||||
|
||||
def insert_many(pg, table, cols, dicts):
|
||||
rows = [tuple(d.get(c) for c in cols) for d in dicts]
|
||||
if not rows:
|
||||
return 0
|
||||
ph = ",".join(["%s"] * len(cols))
|
||||
sql = f"INSERT INTO {table} ({','.join(cols)}) VALUES ({ph}) ON CONFLICT DO NOTHING"
|
||||
with pg.cursor() as cur:
|
||||
cur.executemany(sql, rows)
|
||||
return len(rows)
|
||||
|
||||
|
||||
def main():
|
||||
truncate = "--truncate" in sys.argv
|
||||
now = datetime.now(timezone.utc)
|
||||
name, user, pw = wp_creds()
|
||||
my = pymysql.connect(unix_socket=SOCK, user=user, password=pw, database=name,
|
||||
cursorclass=pymysql.cursors.DictCursor)
|
||||
pg = psycopg.connect(PG_DSN, autocommit=False)
|
||||
pg.execute(open(HERE / "schema.sql").read())
|
||||
pg.commit()
|
||||
if truncate:
|
||||
pg.execute("TRUNCATE inventory, sale_items, sales, crate")
|
||||
|
||||
c = my.cursor()
|
||||
|
||||
# --- inventory: disc_inventory (vinyl) -----------------------------------------------------
|
||||
c.execute("SELECT * FROM wp_rmp_disc_inventory")
|
||||
vinyl = [dict(
|
||||
sku=r["sku"], store_id=1, kind="vinyl", release_id=r["release_id"],
|
||||
price=r["price"], qty=1, in_stock=yn(r["instock"]),
|
||||
condition=r["media_condition"], sleeve_cond=r["sleeve_condition"],
|
||||
location=r["location"], crate_id=r["crate_id"], slot_number=r["slot_number"],
|
||||
status="publish" if yn(r["instock"]) else "sold", notes=r["comment"],
|
||||
square_item_id=r["square_item_id"], square_catalog_object_id=r["square_catalog_object_id"],
|
||||
square_sku=r["square_sku"], square_synced_at=r["square_synced_at"],
|
||||
staged=False, sold_date=r["sold_date"], created_at=now, updated_at=now,
|
||||
) for r in c.fetchall()]
|
||||
|
||||
# --- inventory: disc_general_inventory (books/mags/etc.) -----------------------------------
|
||||
c.execute("SELECT * FROM wp_rmp_disc_general_inventory")
|
||||
general = []
|
||||
for g in c.fetchall():
|
||||
attrs = js(g["attributes"]) or {}
|
||||
dims = js(g["dimensions"])
|
||||
if dims:
|
||||
attrs["dimensions"] = dims
|
||||
general.append(dict(
|
||||
sku=g["sku"], store_id=1, kind=(g["type"] or g["category"] or "other").lower(),
|
||||
identifier=g["identifier"] or g["upc"], title=g["title"], price=g["price"],
|
||||
qty=g["quantity"] or 1, in_stock=yn(g["instock"]), condition=g["condition_desc"],
|
||||
brand=g["brand"], model=g["model"], weight_g=g["weight"], location=g["location"],
|
||||
crate_id=g["crate_id"], slot_number=g["slot_number"],
|
||||
status="publish" if yn(g["instock"]) else "sold",
|
||||
square_item_id=g["square_item_id"], square_catalog_object_id=g["square_catalog_object_id"],
|
||||
square_sku=g["square_sku"], square_synced_at=g["square_synced_at"],
|
||||
est_market_value=g["est_market_value"], lowest_competitor=g["lowest_competitor"],
|
||||
target_price=g["target_price"], attributes=jb(attrs or None),
|
||||
images=jb(js(g["images"])), raw_json=jb(js(g["raw_json"])), staged=False,
|
||||
created_at=g["created_at"] or now, updated_at=g["updated_at"] or g["created_at"] or now,
|
||||
))
|
||||
|
||||
inv_cols = ["sku", "store_id", "kind", "release_id", "identifier", "title", "price", "qty",
|
||||
"in_stock", "condition", "sleeve_cond", "brand", "model", "weight_g", "location",
|
||||
"crate_id", "slot_number", "status", "notes", "est_market_value",
|
||||
"lowest_competitor", "target_price", "square_item_id", "square_catalog_object_id",
|
||||
"square_sku", "square_synced_at", "staged", "attributes", "images", "raw_json",
|
||||
"sold_date", "created_at", "updated_at"]
|
||||
|
||||
# --- crate (slim) --------------------------------------------------------------------------
|
||||
c.execute("SELECT id, name, label_text, crate_purpose, notes FROM wp_rmp_disc_virtual_crate")
|
||||
crates = [dict(id=v["id"], name=v["name"], label_text=v["label_text"],
|
||||
purpose=v["crate_purpose"], notes=v["notes"]) for v in c.fetchall()]
|
||||
|
||||
# --- sales + sale_items (reporting copy; WP owns live sale-state) ---------------------------
|
||||
c.execute("SELECT * FROM wp_rmp_disc_sales")
|
||||
sales = [dict(id=s["id"], sale_number=s["sale_number"], customer_id=s["customer_id"],
|
||||
total=s["total_amount"] or s["total"], tax_amount=s["tax_amount"],
|
||||
status=s["payment_status"] or s["status"], payment_method=s["payment_method"],
|
||||
sale_date=s["sale_date"], created_at=s["created_at"]) for s in c.fetchall()]
|
||||
c.execute("SELECT * FROM wp_rmp_disc_sales_items")
|
||||
items = [dict(id=i["id"], sale_id=i["sale_id"], sku=(i["sku"] or None),
|
||||
release_id=i["release_id"], item_name=i["item_name"], qty=i["quantity"],
|
||||
unit_price=i["unit_price"], line_total=i["line_total"]) for i in c.fetchall()]
|
||||
# drop orphan line-items that point at a sale that no longer exists (real junk in the source)
|
||||
sale_ids = {s["id"] for s in sales}
|
||||
n_orphan = sum(1 for i in items if i["sale_id"] not in sale_ids)
|
||||
items = [i for i in items if i["sale_id"] in sale_ids]
|
||||
|
||||
# order matters: crate + sales before things that could reference them
|
||||
n = {}
|
||||
n["crate"] = insert_many(pg, "crate", ["id", "name", "label_text", "purpose", "notes"], crates)
|
||||
n["inventory(vinyl)"] = insert_many(pg, "inventory", inv_cols, vinyl)
|
||||
n["inventory(general)"] = insert_many(pg, "inventory", inv_cols, general)
|
||||
n["sales"] = insert_many(pg, "sales",
|
||||
["id", "sale_number", "customer_id", "total", "tax_amount", "status",
|
||||
"payment_method", "sale_date", "created_at"], sales)
|
||||
n["sale_items"] = insert_many(pg, "sale_items",
|
||||
["id", "sale_id", "sku", "release_id", "item_name", "qty",
|
||||
"unit_price", "line_total"], items)
|
||||
pg.commit()
|
||||
for k, v in n.items():
|
||||
print(f" {k:22} {v:>6}")
|
||||
if n_orphan:
|
||||
print(f" (skipped {n_orphan} orphan sale_items pointing at deleted sales)")
|
||||
print("done.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
3
requirements-migrate.txt
Normal file
3
requirements-migrate.txt
Normal file
@ -0,0 +1,3 @@
|
||||
# Migration-only deps (not needed by the running service). Run: pip install -r requirements-migrate.txt
|
||||
pymysql
|
||||
psycopg[binary]
|
||||
5
requirements.txt
Normal file
5
requirements.txt
Normal file
@ -0,0 +1,5 @@
|
||||
fastapi
|
||||
uvicorn[standard]
|
||||
sqlalchemy[asyncio]
|
||||
asyncpg
|
||||
httpx
|
||||
90
schema.sql
Normal file
90
schema.sql
Normal file
@ -0,0 +1,90 @@
|
||||
-- RecordGod schema v2.
|
||||
-- ONE inventory table (vinyl + general goods unified), sales history, slim crates.
|
||||
-- The Discogs CATALOG is deliberately NOT stored here — it lives in the discogs_full Postgres
|
||||
-- mirror and is joined on release_id at read time. We keep the shop's OWN data only:
|
||||
-- WowPlatter crammed a ~3.2M-row Discogs clone into the shop DB; RecordGod keeps ~40k real rows.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS inventory (
|
||||
sku text PRIMARY KEY,
|
||||
store_id int NOT NULL DEFAULT 1, -- always 1 for now; the multi-shop seam
|
||||
kind text NOT NULL DEFAULT 'vinyl', -- vinyl|book|magazine|comic|card|game|other
|
||||
release_id bigint, -- Discogs id (vinyl) → join discogs_full.release
|
||||
identifier text, -- ISBN/UPC/barcode (general goods lookup key)
|
||||
title text, -- denormalised cache; vinyl filled from mirror at intake
|
||||
price numeric(10,2),
|
||||
qty int NOT NULL DEFAULT 1,
|
||||
in_stock boolean NOT NULL DEFAULT true, -- was enum('y','n')
|
||||
condition text, -- media grade (vinyl) / condition desc (general)
|
||||
sleeve_cond text, -- vinyl sleeve grade
|
||||
brand text,
|
||||
model text,
|
||||
weight_g numeric(10,3),
|
||||
location text,
|
||||
crate_id int,
|
||||
slot_number int,
|
||||
status text DEFAULT 'publish',
|
||||
notes text,
|
||||
product_url text,
|
||||
-- DealGod price intel (the engine writes these)
|
||||
est_market_value numeric(10,2),
|
||||
lowest_competitor numeric(10,2),
|
||||
target_price numeric(10,2),
|
||||
priced_at timestamptz,
|
||||
-- Square sync (Square/WP authoritative)
|
||||
square_item_id text,
|
||||
square_catalog_object_id text,
|
||||
square_sku text,
|
||||
square_synced_at timestamptz,
|
||||
-- intake / staging (live owns sale-state; local owns intake — see RECORDGOD_PLAN.md §6)
|
||||
staged boolean NOT NULL DEFAULT false, -- true = intaken locally, not yet published live
|
||||
attributes jsonb, -- per-kind extras (dimensions, etc.) — was longtext+CHECK
|
||||
images jsonb,
|
||||
raw_json jsonb, -- enrichment cache
|
||||
sold_date timestamptz,
|
||||
created_at timestamptz NOT NULL DEFAULT now(),
|
||||
updated_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS inventory_release_idx ON inventory (store_id, release_id) WHERE release_id IS NOT NULL;
|
||||
CREATE INDEX IF NOT EXISTS inventory_identifier_idx ON inventory (store_id, identifier) WHERE identifier IS NOT NULL;
|
||||
CREATE INDEX IF NOT EXISTS inventory_instock_idx ON inventory (store_id, in_stock) WHERE in_stock;
|
||||
CREATE INDEX IF NOT EXISTS inventory_crate_idx ON inventory (crate_id, slot_number);
|
||||
CREATE INDEX IF NOT EXISTS inventory_staged_idx ON inventory (staged) WHERE staged;
|
||||
|
||||
CREATE TABLE IF NOT EXISTS crate (
|
||||
id int PRIMARY KEY,
|
||||
name text NOT NULL,
|
||||
label_text text,
|
||||
purpose text DEFAULT 'stock', -- stock|spacer|display|returns|hold|sale
|
||||
notes text
|
||||
-- 3D geometry (pos/rotation/mesh/genre-lists) intentionally stays in WowPlatter — front-end concern.
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS sales (
|
||||
id bigint PRIMARY KEY,
|
||||
sale_number text UNIQUE,
|
||||
customer_id bigint,
|
||||
total numeric(10,2) NOT NULL DEFAULT 0, -- consolidated (WP had total + total_amount)
|
||||
tax_amount numeric(10,2) NOT NULL DEFAULT 0,
|
||||
status text DEFAULT 'pending', -- consolidated (WP had status + payment_status)
|
||||
payment_method text,
|
||||
sale_date timestamptz,
|
||||
created_at timestamptz
|
||||
-- WooCommerce/WP stays authoritative for live sale-state; this is the reporting/analytics copy.
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS sales_date_idx ON sales (sale_date);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS sale_items (
|
||||
id bigint PRIMARY KEY,
|
||||
sale_id bigint REFERENCES sales(id) ON DELETE CASCADE,
|
||||
sku text,
|
||||
release_id bigint,
|
||||
item_name text,
|
||||
qty int NOT NULL DEFAULT 1,
|
||||
unit_price numeric(10,2) NOT NULL DEFAULT 0,
|
||||
line_total numeric(10,2) NOT NULL DEFAULT 0
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS sale_items_sale_idx ON sale_items (sale_id);
|
||||
CREATE INDEX IF NOT EXISTS sale_items_release_idx ON sale_items (release_id);
|
||||
|
||||
-- pgvector (semantic search / "records like this" / newsletter copy): add when keyword search
|
||||
-- measurably falls short, not before.
|
||||
73
test_smoke.py
Normal file
73
test_smoke.py
Normal file
@ -0,0 +1,73 @@
|
||||
import os
|
||||
os.environ["RECORDGOD_TOKEN"] = "test-tok"
|
||||
|
||||
from fastapi.testclient import TestClient
|
||||
from app.main import app
|
||||
from app.db import get_db
|
||||
|
||||
AUTH = {"Authorization": "Bearer test-tok"}
|
||||
|
||||
|
||||
class _FakeResult:
|
||||
def __init__(self, row): self._row = row
|
||||
def mappings(self): return self
|
||||
def first(self): return self._row
|
||||
|
||||
|
||||
def _override(row):
|
||||
async def _dep():
|
||||
class _DB:
|
||||
async def execute(self, *a, **k): return _FakeResult(row)
|
||||
async def commit(self): pass
|
||||
yield _DB()
|
||||
return _dep
|
||||
|
||||
|
||||
def test_ping_ok():
|
||||
r = TestClient(app).get("/wowplatter/v1/ping", headers=AUTH)
|
||||
assert r.status_code == 200
|
||||
b = r.json()
|
||||
assert b["ok"] and b["store"] and b["plan"] and b["version"]
|
||||
|
||||
|
||||
def test_auth_required():
|
||||
c = TestClient(app)
|
||||
assert c.get("/wowplatter/v1/ping").status_code == 401
|
||||
assert c.get("/wowplatter/v1/ping", headers={"Authorization": "Bearer wrong"}).status_code == 401
|
||||
|
||||
|
||||
def test_lookup_in_stock():
|
||||
app.dependency_overrides[get_db] = _override({
|
||||
"sku": "MRP-AB12CD", "price": 39.95, "qty": 1, "condition": "VG+",
|
||||
"status": "publish", "product_url": "https://store/x", "release_id": 12345})
|
||||
r = TestClient(app).get("/wowplatter/v1/inventory/lookup?release_id=12345", headers=AUTH)
|
||||
app.dependency_overrides.clear()
|
||||
assert r.status_code == 200
|
||||
b = r.json()
|
||||
assert b["in_stock"] is True and b["sku"] == "MRP-AB12CD" and b["price"] == 39.95
|
||||
|
||||
|
||||
def test_lookup_not_stocked():
|
||||
app.dependency_overrides[get_db] = _override(None)
|
||||
r = TestClient(app).get("/wowplatter/v1/inventory/lookup?release_id=999", headers=AUTH)
|
||||
app.dependency_overrides.clear()
|
||||
assert r.json() == {"ok": True, "in_stock": False, "release_id": 999}
|
||||
|
||||
|
||||
def test_intake_stages_without_mirror():
|
||||
# no release_id → no mirror call; just generate a sku and stage.
|
||||
app.dependency_overrides[get_db] = _override(None)
|
||||
r = TestClient(app).post("/wowplatter/v1/inventory/intake",
|
||||
headers=AUTH, json={"kind": "vinyl", "price": 25.0, "condition": "VG+"})
|
||||
app.dependency_overrides.clear()
|
||||
b = r.json()
|
||||
assert b["ok"] and b["staged"] is True and b["sku"].startswith("MRP-")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
test_ping_ok()
|
||||
test_auth_required()
|
||||
test_lookup_in_stock()
|
||||
test_lookup_not_stocked()
|
||||
test_intake_stages_without_mirror()
|
||||
print("ok — ping, auth, lookup(in/out), intake all pass")
|
||||
Loading…
Reference in New Issue
Block a user