4c462861f5
Buying stock from distros (RareWaves discount sales) → need per-COPY cost + a new/used flag.
- inventory.cost_price (per copy — costs vary per copy of the same release) + cost_source ('rarewaves #592619')
- inventory.condition_type 'new'|'used' (default used); backfilled 451 sealed → new
RareWaves analysed = Shopify; order pages give barcode (in /products/<ean>- handle) + title + qty + unit cost
($X.XX/ea). Wishlist = Klevu app (harder). Full plan in RAREWAVES_INGEST_PLAN.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
62 lines
4.4 KiB
Markdown
62 lines
4.4 KiB
Markdown
# Distro ingest (RareWaves first) + cost tracking + new/used — plan (2026-06-27)
|
||
|
||
Buy stock from distros (RareWaves discount sales, ~$8–15 capped shipping → margin in volume), scrape the
|
||
**order** + **wishlist** account pages via the PRICEGOD extension, land **cost price per copy** in inventory,
|
||
and use the **wishlist** to gauge scarcity (how many stores / Discogs sellers stock it). Prep Rocket / Inertia /
|
||
Bertus next (they send invoices to ingest).
|
||
|
||
## ✅ Done (foundations, live on recordgod)
|
||
- `inventory.cost_price numeric(10,2)` — per-COPY buy price (costs vary per copy, so inventory-level not release-level).
|
||
- `inventory.cost_source text` — provenance, e.g. `rarewaves #592619`.
|
||
- `inventory.condition_type text NOT NULL DEFAULT 'used'` — the **new/used binary split**. Backfilled: 451 sealed
|
||
items (`notes ILIKE '%seal%'`) → `new`; everything else `used`. (Distro-bought stock defaults `new`.)
|
||
|
||
## What the RareWaves dumps revealed (analysis)
|
||
RareWaves = **Shopify**. Source files on ultra `~/Documents/recordgod/rarw*.txt`.
|
||
|
||
**Orders (rarworder*.txt)** — new Shopify customer-account React pages. Each line item:
|
||
```
|
||
<a href="/products/0603497816163-rumours-2025?variant=55398669779318">
|
||
<img alt="Rumours"> … quantity 4 … $42.99/ea
|
||
```
|
||
→ **barcode (EAN-13 = leading digits of the handle)** + **title (img alt)** + **year (in slug)** + **qty** +
|
||
**unit cost (`$X.XX/ea`)** + variant id + order # (`Order #592619`). CSS classes are obfuscated hashes — key off
|
||
the **`/products/<barcode>-` URL**, the **`/ea` price text**, ARIA `role="row"`, and `quantity N`. **High confidence.**
|
||
|
||
**Wishlist (rarwwish*.txt)** — rendered by the **Klevu** app into collapsible `data-wishlist-row-id` rows. No clean
|
||
barcode in the dump (the 13-digit hits were CDN cache-bust hashes `?v=…`, false positives). **Harder** — needs
|
||
row→product mapping, likely via Klevu's data or by following each row's product link. Lower confidence; phase 2.
|
||
|
||
## Build plan (sequenced)
|
||
|
||
**1. Order scraper → cost into inventory (HIGH value, HIGH confidence — do first).**
|
||
- PRICEGOD content script on `rarewaves.com/account/orders/*` (+ legacy `/orders/*`): walk each `role="row"`,
|
||
pull `{barcode, title, year(slug), qty, unit_cost, variant_id}` + the order number.
|
||
- New RecordGod endpoint `POST /admin/intake/distro` `{source:'rarewaves', order_ref, items:[…]}` → per item:
|
||
resolve **barcode → release_id** (local `disc_release_identifier` `type='Barcode'`, **normalised**: strip
|
||
non-digits, try with/without leading 0; Discogs `/database/search?barcode=` fallback), enrich, **stage** with
|
||
`cost_price=unit_cost`, `cost_source='rarewaves #<order>'`, `condition_type='new'`, one SKU per copy (qty → N rows).
|
||
- Reuses the existing `_stage`/`_enrich` + the staged review queue. Token = the RecordGod store token already minted.
|
||
|
||
**2. New-stock approval landing (staff confirm release_id before commit).**
|
||
- A `/admin` "New stock" review view over `staged` rows that came from distro ingest: shows the scraped title +
|
||
barcode + auto-resolved release_id (with cover) + the cost; staff **confirm or re-pick** the release, set
|
||
retail price, then **publish** (staged→live). Resolve heuristics John named: almost always **reissues → newest
|
||
year**; assume **black vinyl** unless the slug/title says coloured; barcode pins it when present.
|
||
|
||
**3. New/used in the inventory UI.** Add `condition_type` (new/used toggle) + `cost_price` to the inventory
|
||
edit form + show on the list; surface **margin** (price − cost) where both exist.
|
||
|
||
**4. Wishlist scarcity (phase 2, speculative).** Map wishlist rows → release_id, then count **how many stores /
|
||
Discogs sellers** stock each (DealGod has the cross-store + Discogs-seller data; reach via its API). Shows relative
|
||
scarcity to prioritise buys. Needs the Klevu row→product decode first.
|
||
|
||
**5. Other distros (Rocket / Inertia / Bertus).** They send **invoices** (PDF/CSV) not web pages → an invoice
|
||
ingest (upload → parse line items → same `/admin/intake/distro` core). Generic `source` + `cost_source` already
|
||
support it. Likely an LLM/parse step per invoice format.
|
||
|
||
## Open questions for John
|
||
- Order URL pattern: is it `rarewaves.com/account/orders/<id>` (new accounts) — confirm so the content-script match is right.
|
||
- Retail price on distro stock: auto-suggest from DealGod median, or staff set per item at approval?
|
||
- Wishlist scarcity: worth the Klevu-decode effort now, or park until orders+cost are proven in daily use?
|