feat(inventory): cost_price + cost_source + new/used split (distro-ingest foundations)

Buying stock from distros (RareWaves discount sales) → need per-COPY cost + a new/used flag.
- inventory.cost_price (per copy — costs vary per copy of the same release) + cost_source ('rarewaves #592619')
- inventory.condition_type 'new'|'used' (default used); backfilled 451 sealed → new
RareWaves analysed = Shopify; order pages give barcode (in /products/<ean>- handle) + title + qty + unit cost
($X.XX/ea). Wishlist = Klevu app (harder). Full plan in RAREWAVES_INGEST_PLAN.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
type-two 2026-06-27 10:49:51 +10:00
parent 1c8f15514c
commit 4c462861f5
2 changed files with 65 additions and 0 deletions

61
RAREWAVES_INGEST_PLAN.md Normal file
View File

@ -0,0 +1,61 @@
# Distro ingest (RareWaves first) + cost tracking + new/used — plan (2026-06-27)
Buy stock from distros (RareWaves discount sales, ~$815 capped shipping → margin in volume), scrape the
**order** + **wishlist** account pages via the PRICEGOD extension, land **cost price per copy** in inventory,
and use the **wishlist** to gauge scarcity (how many stores / Discogs sellers stock it). Prep Rocket / Inertia /
Bertus next (they send invoices to ingest).
## ✅ Done (foundations, live on recordgod)
- `inventory.cost_price numeric(10,2)` — per-COPY buy price (costs vary per copy, so inventory-level not release-level).
- `inventory.cost_source text` — provenance, e.g. `rarewaves #592619`.
- `inventory.condition_type text NOT NULL DEFAULT 'used'` — the **new/used binary split**. Backfilled: 451 sealed
items (`notes ILIKE '%seal%'`) → `new`; everything else `used`. (Distro-bought stock defaults `new`.)
## What the RareWaves dumps revealed (analysis)
RareWaves = **Shopify**. Source files on ultra `~/Documents/recordgod/rarw*.txt`.
**Orders (rarworder*.txt)** — new Shopify customer-account React pages. Each line item:
```
<a href="/products/0603497816163-rumours-2025?variant=55398669779318">
<img alt="Rumours"> … quantity 4 … $42.99/ea
```
**barcode (EAN-13 = leading digits of the handle)** + **title (img alt)** + **year (in slug)** + **qty** +
**unit cost (`$X.XX/ea`)** + variant id + order # (`Order #592619`). CSS classes are obfuscated hashes — key off
the **`/products/<barcode>-` URL**, the **`/ea` price text**, ARIA `role="row"`, and `quantity N`. **High confidence.**
**Wishlist (rarwwish*.txt)** — rendered by the **Klevu** app into collapsible `data-wishlist-row-id` rows. No clean
barcode in the dump (the 13-digit hits were CDN cache-bust hashes `?v=…`, false positives). **Harder** — needs
row→product mapping, likely via Klevu's data or by following each row's product link. Lower confidence; phase 2.
## Build plan (sequenced)
**1. Order scraper → cost into inventory (HIGH value, HIGH confidence — do first).**
- PRICEGOD content script on `rarewaves.com/account/orders/*` (+ legacy `/orders/*`): walk each `role="row"`,
pull `{barcode, title, year(slug), qty, unit_cost, variant_id}` + the order number.
- New RecordGod endpoint `POST /admin/intake/distro` `{source:'rarewaves', order_ref, items:[…]}` → per item:
resolve **barcode → release_id** (local `disc_release_identifier` `type='Barcode'`, **normalised**: strip
non-digits, try with/without leading 0; Discogs `/database/search?barcode=` fallback), enrich, **stage** with
`cost_price=unit_cost`, `cost_source='rarewaves #<order>'`, `condition_type='new'`, one SKU per copy (qty → N rows).
- Reuses the existing `_stage`/`_enrich` + the staged review queue. Token = the RecordGod store token already minted.
**2. New-stock approval landing (staff confirm release_id before commit).**
- A `/admin` "New stock" review view over `staged` rows that came from distro ingest: shows the scraped title +
barcode + auto-resolved release_id (with cover) + the cost; staff **confirm or re-pick** the release, set
retail price, then **publish** (staged→live). Resolve heuristics John named: almost always **reissues → newest
year**; assume **black vinyl** unless the slug/title says coloured; barcode pins it when present.
**3. New/used in the inventory UI.** Add `condition_type` (new/used toggle) + `cost_price` to the inventory
edit form + show on the list; surface **margin** (price cost) where both exist.
**4. Wishlist scarcity (phase 2, speculative).** Map wishlist rows → release_id, then count **how many stores /
Discogs sellers** stock each (DealGod has the cross-store + Discogs-seller data; reach via its API). Shows relative
scarcity to prioritise buys. Needs the Klevu row→product decode first.
**5. Other distros (Rocket / Inertia / Bertus).** They send **invoices** (PDF/CSV) not web pages → an invoice
ingest (upload → parse line items → same `/admin/intake/distro` core). Generic `source` + `cost_source` already
support it. Likely an LLM/parse step per invoice format.
## Open questions for John
- Order URL pattern: is it `rarewaves.com/account/orders/<id>` (new accounts) — confirm so the content-script match is right.
- Retail price on distro stock: auto-suggest from DealGod median, or staff set per item at approval?
- Wishlist scarcity: worth the Klevu-decode effort now, or park until orders+cost are proven in daily use?

View File

@ -106,6 +106,10 @@ _STARTUP_DDL = [
"ALTER TABLE staff ADD COLUMN IF NOT EXISTS pay_rate numeric(10,2)",
"ALTER TABLE staff ADD COLUMN IF NOT EXISTS password_hash text",
"CREATE UNIQUE INDEX IF NOT EXISTS staff_email_uniq ON staff (lower(email)) WHERE email IS NOT NULL",
# cost tracking (per-copy buy price from distro orders) + new/used split
"ALTER TABLE inventory ADD COLUMN IF NOT EXISTS cost_price numeric(10,2)",
"ALTER TABLE inventory ADD COLUMN IF NOT EXISTS cost_source text", # e.g. 'rarewaves #592619'
"ALTER TABLE inventory ADD COLUMN IF NOT EXISTS condition_type text NOT NULL DEFAULT 'used'", # 'new' | 'used'
]