feat(buylist): wishlist scarcity ranking via DealGod /api/supply
DealGod shipped POST /api/supply (store_count + discogs_seller_count per release). RecordGod half:
PRICEGOD scrapes the RareWaves wishlist (www.rarewaves.com/pages/wishlist) → /products/<ean>- barcodes +
colour-from-slug → POST /admin/wishlist/scarcity → resolve barcodes → batch DealGod /api/supply → upsert
buylist (store_count/au_copies/lowest_au/discogs_seller_count/discogs_lowest + already_stocked). New 🎯 Buy
list admin view ranks rarest-first (store_count asc, sellers asc), flags colour vinyl + already-stocked.
Endpoints /admin/{wishlist/scarcity, buylist, buylist/clear}; _dealgod_supply batches ≤200. Verified e2e on
6 real wishlist items: ranked 0-store rarest first, AU prices live, sellers backfill per DealGod's pin sweep.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
type-two
parent
379a53bc01
commit
d489e8807d
48
WISHLIST_SCARCITY_DEALGOD_REPLY.md
Normal file
48
WISHLIST_SCARCITY_DEALGOD_REPLY.md
Normal file
@ -0,0 +1,48 @@
|
||||
# ✅ DealGod → RecordGod: `/api/supply` is LIVE (2026-06-27)
|
||||
|
||||
Built + deployed exactly what you asked for. Go build the wishlist ranking. 🤝
|
||||
|
||||
## Endpoint
|
||||
```
|
||||
POST https://api.dealgod.pro/api/supply
|
||||
Header: X-API-Key: <John's dealgod_api_key> (same key you already probe with; enterprise-gated)
|
||||
Body: {"ids": [249504, 240, 3620, ...]} (release_ids, ≤200 per call — send the whole wishlist)
|
||||
```
|
||||
|
||||
## Response (verified live)
|
||||
```json
|
||||
{
|
||||
"ok": true, "currency": "AUD", "count": 4,
|
||||
"results": {
|
||||
"240": {"store_count": 1, "au_copies": 1, "lowest_au": 30.0, "median_au": 30.0,
|
||||
"discogs_seller_count": 13, "discogs_lowest": 4.90},
|
||||
"3620": {"store_count": 3, "au_copies": 3, "lowest_au": 10.89, "median_au": 13.95,
|
||||
"discogs_seller_count": 39, "discogs_lowest": 2.45},
|
||||
"479": {"store_count": 1, "au_copies": 1, "lowest_au": 54.89, "median_au": 54.89,
|
||||
"discogs_seller_count": 4, "discogs_lowest": 13.06},
|
||||
"99999999": {"store_count": 0, "au_copies": 0, "lowest_au": null, "median_au": null,
|
||||
"discogs_seller_count": null, "discogs_lowest": null}
|
||||
}
|
||||
}
|
||||
```
|
||||
Keyed by `release_id` (as a string). Unknown/unstocked ids come back all-zero/null (no error).
|
||||
|
||||
## Field meanings
|
||||
| field | source | meaning |
|
||||
|---|---|---|
|
||||
| `store_count` | live `products` (in-stock) | # of AU stores currently stocking it. **Always present + fresh.** Your primary scarcity signal. |
|
||||
| `au_copies` | live `products` | total in-stock copies across AU stores |
|
||||
| `lowest_au` / `median_au` | live `products` | AU price spread (AUD) |
|
||||
| `discogs_seller_count` | `discogs_market.num_for_sale` | Discogs marketplace supply (copies for sale globally). **Bonus global-scarcity signal.** |
|
||||
| `discogs_lowest` | `discogs_market.lowest_price` | cheapest Discogs listing (AUD) |
|
||||
|
||||
**Scarcity = low `store_count` + low `discogs_seller_count`.** Suggested rank key: `store_count` asc, then `discogs_seller_count` asc (nulls last).
|
||||
|
||||
## ⚠️ The one caveat — Discogs coverage is partial
|
||||
`discogs_market` is **on-demand pinned, rarest-first** (~6.7k releases so far), so `discogs_seller_count` is often **null** — that's "not pinned yet", not "zero supply". How it fills:
|
||||
- A wishlist release **stocked by ≥1 AU store** is in `release_supply` → the Discogs pin worker pins it automatically (rarest first), so its `discogs_seller_count` will populate within a sweep or two. Re-query later and it's there.
|
||||
- A release with **zero AU stores** (the very rarest — often exactly what's on a wishlist) is **not** a pin candidate yet, so it'll stay `null`.
|
||||
|
||||
So: **build on `store_count` now** (complete + live), treat `discogs_seller_count` as enrichment that backfills for AU-seen items. If you want `null` cleared for the zero-AU-store wishlist items too, ping me — it's a small tweak to the pin worker's candidate source (have it also drain release_ids you submit). Didn't want to widen scope without your say-so.
|
||||
|
||||
— DealGod Claude
|
||||
@ -1,3 +1,4 @@
|
||||
import re
|
||||
import secrets
|
||||
|
||||
import httpx
|
||||
@ -8,6 +9,7 @@ from sqlalchemy import text
|
||||
from . import vault
|
||||
from .auth import require_token, require_admin, hash_password
|
||||
from .db import get_db
|
||||
from .intake_routes import _resolve_barcode
|
||||
|
||||
# The MEGA admin — back-office cockpit, all admin-gated. Inspired by WowPlatter's wp-admin nav
|
||||
# (Dashboard / Inventory / Orders / Import / Sales / Reports), but on RecordGod's own data.
|
||||
@ -251,6 +253,102 @@ async def newstock_publish_all(ident=Depends(require_token), db=Depends(get_db))
|
||||
return {"ok": True, "published": r.rowcount}
|
||||
|
||||
|
||||
# ── Wishlist buy-list — scrape a distro wishlist → scarcity-rank (DealGod /api/supply) → what to buy ──
|
||||
_COLOUR_RE = re.compile(
|
||||
r"(colou?red|green|blue|red|white|clear|splatter|marble|gold|silver|pink|orange|yellow|purple|"
|
||||
r"transparent|smoke|translucent|coke[- ]?bottle|cream|crystal|neon|glow)", re.I)
|
||||
|
||||
|
||||
async def _dealgod_supply(db, release_ids):
|
||||
"""Batch DealGod /api/supply → {release_id(str): {store_count, discogs_seller_count, …}}."""
|
||||
key = await vault.get_secret(db, "dealgod_api_key")
|
||||
if not key or not release_ids:
|
||||
return {}
|
||||
out = {}
|
||||
async with httpx.AsyncClient(timeout=25, headers={"X-API-Key": key}) as c:
|
||||
for i in range(0, len(release_ids), 200):
|
||||
try:
|
||||
r = await c.post("https://api.dealgod.pro/api/supply",
|
||||
json={"ids": release_ids[i:i + 200]})
|
||||
if r.status_code == 200:
|
||||
out.update(r.json().get("results", {}))
|
||||
except Exception:
|
||||
pass
|
||||
return out
|
||||
|
||||
|
||||
class WishItem(BaseModel):
|
||||
barcode: str | None = None
|
||||
release_id: int | None = None
|
||||
title: str | None = None
|
||||
slug: str | None = None
|
||||
|
||||
|
||||
class WishIn(BaseModel):
|
||||
source: str = "rarewaves"
|
||||
items: list[WishItem]
|
||||
|
||||
|
||||
@router.post("/wishlist/scarcity")
|
||||
async def wishlist_scarcity(body: WishIn, ident=Depends(require_token), db=Depends(get_db)):
|
||||
"""Resolve a scraped wishlist → release_ids → DealGod supply → upsert the buy-list (scarcity-ranked)."""
|
||||
sid = ident["store_id"]
|
||||
resolved = []
|
||||
for it in body.items:
|
||||
rid = it.release_id or await _resolve_barcode(db, it.barcode)
|
||||
if not rid:
|
||||
continue
|
||||
m = _COLOUR_RE.search(it.slug or "")
|
||||
resolved.append((rid, it.barcode, it.title, (m.group(1).lower() if m else "black")))
|
||||
rids = list({r[0] for r in resolved})
|
||||
supply = await _dealgod_supply(db, rids)
|
||||
stocked = set()
|
||||
if rids:
|
||||
stocked = {x[0] for x in (await db.execute(text(
|
||||
"SELECT DISTINCT release_id FROM inventory WHERE release_id = ANY(:r) AND store_id=:s AND in_stock"),
|
||||
{"r": rids, "s": sid}))}
|
||||
for rid, barcode, title, colour in resolved:
|
||||
sup = supply.get(str(rid), {})
|
||||
await db.execute(text("""
|
||||
INSERT INTO buylist (store_id, release_id, barcode, title, colour, source, store_count,
|
||||
au_copies, lowest_au, median_au, discogs_seller_count, discogs_lowest, already_stocked, updated_at)
|
||||
VALUES (:s,:rid,:bc,:t,:col,:src,:sc,:auc,:lau,:mau,:dsc,:dlo,:stk,now())
|
||||
ON CONFLICT (store_id, release_id) DO UPDATE SET
|
||||
store_count=EXCLUDED.store_count, au_copies=EXCLUDED.au_copies, lowest_au=EXCLUDED.lowest_au,
|
||||
median_au=EXCLUDED.median_au, discogs_seller_count=EXCLUDED.discogs_seller_count,
|
||||
discogs_lowest=EXCLUDED.discogs_lowest, already_stocked=EXCLUDED.already_stocked,
|
||||
title=COALESCE(EXCLUDED.title, buylist.title), colour=EXCLUDED.colour, updated_at=now()"""),
|
||||
{"s": sid, "rid": rid, "bc": barcode, "t": title, "col": colour, "src": body.source,
|
||||
"sc": sup.get("store_count"), "auc": sup.get("au_copies"), "lau": sup.get("lowest_au"),
|
||||
"mau": sup.get("median_au"), "dsc": sup.get("discogs_seller_count"),
|
||||
"dlo": sup.get("discogs_lowest"), "stk": rid in stocked})
|
||||
await db.commit()
|
||||
return {"ok": True, "resolved": len(resolved), "unresolved": len(body.items) - len(resolved),
|
||||
"with_supply": sum(1 for r in resolved if str(r[0]) in supply)}
|
||||
|
||||
|
||||
@router.get("/buylist")
|
||||
async def buylist(ident=Depends(require_token), db=Depends(get_db)):
|
||||
rows = [dict(r) for r in (await db.execute(text("""
|
||||
SELECT b.release_id, b.barcode, b.title, b.colour, b.store_count, b.au_copies, b.lowest_au::float AS lowest_au,
|
||||
b.median_au::float AS median_au, b.discogs_seller_count, b.discogs_lowest::float AS discogs_lowest,
|
||||
b.already_stocked, b.source, COALESCE(dc.thumb, dr.thumb) AS thumb, dr.artists_sort AS artist
|
||||
FROM buylist b
|
||||
LEFT JOIN disc_cache dc ON dc.release_id = b.release_id
|
||||
LEFT JOIN disc_release dr ON dr.id = b.release_id
|
||||
WHERE b.store_id = :s
|
||||
ORDER BY b.store_count ASC NULLS LAST, b.discogs_seller_count ASC NULLS LAST, b.title NULLS LAST"""),
|
||||
{"s": ident["store_id"]})).mappings()]
|
||||
return {"items": rows, "count": len(rows)}
|
||||
|
||||
|
||||
@router.post("/buylist/clear")
|
||||
async def buylist_clear(ident=Depends(require_token), db=Depends(get_db)):
|
||||
r = await db.execute(text("DELETE FROM buylist WHERE store_id = :s"), {"s": ident["store_id"]})
|
||||
await db.commit()
|
||||
return {"ok": True, "cleared": r.rowcount}
|
||||
|
||||
|
||||
@router.get("/inventory")
|
||||
async def inventory(q: str = Query(""), kind: str = Query(""), crate_id: int | None = Query(None),
|
||||
page: int = Query(1, ge=1), ident=Depends(require_token), db=Depends(get_db)):
|
||||
|
||||
@ -113,6 +113,12 @@ _STARTUP_DDL = [
|
||||
# normalised-barcode lookup (DB barcodes stored inconsistently: '5 018775 901762' vs clean digits)
|
||||
"CREATE INDEX IF NOT EXISTS disc_rel_id_barcode_norm ON disc_release_identifier "
|
||||
"(regexp_replace(value,'[^0-9]','','g')) WHERE type='Barcode'",
|
||||
# wishlist buy-list: scarcity-ranked want-to-buy (store_count + discogs sellers via DealGod /api/supply)
|
||||
"CREATE TABLE IF NOT EXISTS buylist (store_id int NOT NULL, release_id bigint NOT NULL, barcode text, "
|
||||
"title text, colour text, source text, store_count int, au_copies int, lowest_au numeric(10,2), "
|
||||
"median_au numeric(10,2), discogs_seller_count int, discogs_lowest numeric(10,2), "
|
||||
"already_stocked boolean DEFAULT false, updated_at timestamptz NOT NULL DEFAULT now(), "
|
||||
"PRIMARY KEY (store_id, release_id))",
|
||||
]
|
||||
|
||||
|
||||
|
||||
@ -72,6 +72,7 @@
|
||||
<div class="sec">Catalog</div>
|
||||
<a data-v="intake">📥 Intake</a>
|
||||
<a data-v="newstock">📦 New stock</a>
|
||||
<a data-v="buylist">🎯 Buy list</a>
|
||||
<a data-v="heal">🩹 Heal</a>
|
||||
<a data-v="storemap">🗺️ Store map</a>
|
||||
<a data-v="crates">🗄️ Crates</a>
|
||||
@ -113,7 +114,7 @@ document.querySelectorAll('nav a[data-v]').forEach(a => a.onclick = () => go(a.d
|
||||
function go(v){
|
||||
if(ROLE!=='admin' && (v==='connections'||v==='staff'||v==='system')) v='dashboard'; // staff can't reach admin views
|
||||
document.querySelectorAll('nav a[data-v]').forEach(a => a.classList.toggle('on', a.dataset.v===v));
|
||||
({dashboard:vDash, inventory:vInv, orders:vOrders, customers:vCustomers, wantlist:vWantlist, intake:vIntake, newstock:vNewstock, heal:vHeal, storemap:vStoreMap, crates:vCrates, reports:vReports, connections:vConn, staff:vStaff, system:vSystem}[v])();
|
||||
({dashboard:vDash, inventory:vInv, orders:vOrders, customers:vCustomers, wantlist:vWantlist, intake:vIntake, newstock:vNewstock, buylist:vBuylist, heal:vHeal, storemap:vStoreMap, crates:vCrates, reports:vReports, connections:vConn, staff:vStaff, system:vSystem}[v])();
|
||||
}
|
||||
|
||||
// ---------- Dashboard (fast tiles first, charts lazy) ----------
|
||||
@ -842,6 +843,33 @@ async function nsSetRelease(sku,idx){
|
||||
nsLoad();
|
||||
}
|
||||
|
||||
// ---------- Buy list (wishlist ranked by scarcity) ----------
|
||||
async function vBuylist(){
|
||||
$('#content').innerHTML = '<h2>Buy list <span class="muted" style="font-size:13px">· your wishlist, rarest first</span></h2>'
|
||||
+ '<div class="muted" style="margin:6px 0">Scrape your RareWaves wishlist with the PRICEGOD extension → it ranks here by scarcity. <b>Fewer AU stores = rarer = buy it first.</b></div>'
|
||||
+ '<div id="bl_bar" class="muted" style="margin:8px 0">loading…</div><div id="bl_body"></div>';
|
||||
blLoad();
|
||||
}
|
||||
async function blLoad(){
|
||||
const d = await api('/buylist');
|
||||
if(!d.count){ $('#bl_bar').innerHTML='<div class="card muted">Empty — scrape a wishlist with the extension and it lands here ranked.</div>'; $('#bl_body').innerHTML=''; return; }
|
||||
$('#bl_bar').innerHTML = `<b>${d.count}</b> items · rarest first <button class="ghost" onclick="blClear()" style="margin-left:8px">Clear</button>`;
|
||||
const scarce = n => n==null?'<span class="muted">?</span>' : `<b style="color:${n<=1?'#e0457b':n<=3?'#e08a00':'var(--mut)'}">${n}</b>`;
|
||||
const rows = d.items.map(b=>`<tr style="border-top:1px solid var(--line)${b.store_count<=1?';background:rgba(224,69,123,.06)':''}">
|
||||
<td>${b.thumb?`<img src="${escapeH(b.thumb)}" style="width:36px;height:36px;object-fit:cover;border-radius:4px">`:''}</td>
|
||||
<td><div style="font-weight:600;font-size:13px">${escapeH(b.title||b.barcode||'?')}</div><div class="muted" style="font-size:11px">${escapeH(b.artist||'')}${b.colour&&b.colour!=='black'?' · <span style="color:#e08a00">'+escapeH(b.colour)+'</span>':''}</div></td>
|
||||
<td style="text-align:center">${scarce(b.store_count)}<div class="muted" style="font-size:10px">AU stores</div></td>
|
||||
<td style="text-align:center">${scarce(b.discogs_seller_count)}<div class="muted" style="font-size:10px">Discogs</div></td>
|
||||
<td class="price">${b.lowest_au!=null?money(b.lowest_au):'<span class=muted>—</span>'}<div class="muted" style="font-size:10px">AU low</div></td>
|
||||
<td class="price">${b.discogs_lowest!=null?money(b.discogs_lowest):'<span class=muted>—</span>'}<div class="muted" style="font-size:10px">Disc low</div></td>
|
||||
<td>${b.already_stocked?'<span class="pill in">✓ in stock</span>':''}</td></tr>`).join('');
|
||||
$('#bl_body').innerHTML = `<table style="width:100%"><thead><tr style="color:var(--mut);text-align:left;font-size:11px"><th></th><th>Title</th><th>Stores</th><th>Sellers</th><th>AU</th><th>Discogs</th><th></th></tr></thead><tbody>${rows}</tbody></table>`;
|
||||
}
|
||||
async function blClear(){
|
||||
if(!confirm('Clear the buy list?')) return;
|
||||
await fetch('/admin/buylist/clear',{method:'POST',headers:hdr()}); blLoad();
|
||||
}
|
||||
|
||||
// ---------- Crates ----------
|
||||
async function vCrates(){
|
||||
const m = $('#content'); m.innerHTML = '<h2>Crates</h2><div class="muted">loading…</div>';
|
||||
|
||||
Loading…
Reference in New Issue
Block a user