feat(collections): Collections CRUD + fallback locator + Reorganize scan/apply/compress API + Collections UI

- collections_routes.py: /nav/collections CRUD over virtual_collection + /nav/locate (exact-then-rules)
- navigator_routes.py: /nav/reorg/{scan,apply,compress} (atomic re-file)
- disc_cache gets year/genre_ids/style_ids cols for rule matching (backfill TODO)
- /search: collections list + locator fallback shows the should-be-here zone for unfiled hits

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
type-two 2026-06-22 20:56:34 +10:00
parent 0767778931
commit 879a58992f
5 changed files with 364 additions and 8 deletions

View File

@ -151,6 +151,47 @@ Start P0→P1: that already gives a clickable store → rack → level → crate
---
## 6b. Collections + the fallback location map (captured 2026-06-22)
**Data already migrated:** `virtual_collection` (14 rows). Model = a named **zone**:
`name, color, crate_ids[] (JSON, in click order), priority, sort_method` +
**filing rules** `genre_ids[] / style_ids[] / label_ids[] / format_descriptions[] / price_min / price_max /
year_min / year_max`, `is_active`. Comment in the source: *"grouping of crates with filing rules for smart
placement."* Real examples: `2000s HOUSE` = House style + price≥$15 + year 20002009 → 12 crates;
`OLD SCHOOL RAP` = year 19801986 → 1 crate; `BREAKS 15+` = 4 breaks styles + $15+ → 10 crates.
**Collections UI** (Collections tab): list (colour swatch + crate count) → editor where you **click crates on the
map and they're kept in click order** as removable chips, set name/colour/priority, `sort_method`
(alpha_artist/title, year, price, date_added), filing rules, and a live **match panel**
(`N matching · N located · N not located · N overlap`). Save → `virtual_collection`.
**THE fallback locator (John's headline ask).** `GET /nav/locate?release_id=` (or sku):
1. exact location first — if the record has `crate_id`+`slot_number`, return that.
2. else match the record's `genre/style/label/format/price/year` against **active collections, `priority` DESC**;
first hit → return its `crate_ids` as the **"should-be-here" zone** ("House $1215 → these 4 crates").
The match panel's four numbers fall out of this: *matching* = records a collection's rules select, *located* = of
those, how many actually sit in its crates, *not located* = match-but-misfiled, *overlap* = matched by >1 collection.
**One data dependency to decide:** collections match on discogs `genre_ids/style_ids` + `year`, which RecordGod's
slim `disc_cache` doesn't carry (only title/artist/thumb/weight). Options: **(a)** enrich `disc_cache` with
`genre/style/year` (backfill from `discogs_full` on ultra) so the matcher is self-contained — recommended; or
**(b)** denormalise genre/style/year onto `inventory` at intake. Price is already on `inventory`.
## 6c. Reorganize — the A-Z re-file planner (P4)
Server contract (thin; planning is client-side, copy WowPlatter's `class-wowplatter-ajax-store-reorg.php`):
- `POST /nav/reorg/scan {crate_ids[]}` → all in-stock items (sku, release_id, crate_id, slot, price, title, artist,
year, thumb), ordered crate→slot. (Sources can be crate ids/ranges OR a collection's crates.)
- **Client plans:** apply filters (price≤ / label / year-range) → pick a **distribution target** (start crate +
count + slots/crate) → **sort** by `sort_method` → fill target crates slot-by-slot → produce a move list
`[{sku, from_crate/slot, to_crate/slot}]` + leftovers. Card-by-card **walkthrough modal** (prev/next, MERGE, Undo).
- `POST /nav/reorg/apply {matched[], leftovers[]}` → ONE transaction: per sku `UPDATE inventory SET crate_id,
slot_number, location_updated_*`; leftovers → `crate_id/slot=NULL, location=archive`. Guard SKU-miss → roll back.
- Slot housekeeping: `compress_crate_slots` (close gaps), `insert_with_shift` (insert at slot, push others down).
Build order: **Collections + fallback locator first** (location-map foundation, data already there, every locate
benefits), **then Reorganize** (heavier planner UI). Both reuse the §4 `/nav/*` router.
## 7. Open checks before building
1. Confirm `virtual_rack_type_level` + the crate-tag table actually came across in the migration (column names).
2. Confirm the slot-ordering rule WowPlatter uses (pos_x ascending vs an explicit `slot_number` on the crate).

187
app/collections_routes.py Normal file
View File

@ -0,0 +1,187 @@
import json
from fastapi import APIRouter, Depends, HTTPException, Query
from pydantic import BaseModel
from sqlalchemy import text
from .auth import require_token
from .db import get_db
# Collections = named crate zones + filing rules (genre/style/label/format + price/year) + priority.
# Migrated verbatim into virtual_collection. Powers the Collections tab AND the fallback locator:
# a record with no exact crate still resolves to "where it should be" via the highest-priority matching rule.
# See RECORDGOD_NAVIGATOR_PLAN.md §6b. (JSON list columns came across as text — parse in Python.)
router = APIRouter(prefix="/nav", tags=["collections"])
def _arr(v):
if isinstance(v, list):
return v
try:
return json.loads(v) if v else []
except Exception:
return []
class CollectionIn(BaseModel):
id: int | None = None
name: str
color: str = "#3498db"
crate_ids: list[int] = []
genre_ids: list[int] = []
style_ids: list[int] = []
label_ids: list[int] = []
format_descriptions: list[str] = []
price_min: float | None = None
price_max: float | None = None
year_min: int | None = None
year_max: int | None = None
sort_method: str = "alpha_artist"
priority: int = 0
is_active: bool = True
@router.get("/collections")
async def list_collections(ident=Depends(require_token), db=Depends(get_db)):
rows = (await db.execute(text("""
SELECT id, name, color, priority, (is_active = 1 OR is_active = true) AS is_active,
crate_ids, style_ids, genre_ids, price_min::float, price_max::float, year_min, year_max
FROM virtual_collection ORDER BY priority DESC, name
"""))).mappings()
out = []
for r in rows:
d = dict(r)
out.append({"id": d["id"], "name": d["name"], "color": d["color"], "priority": d["priority"],
"is_active": d["is_active"], "crate_count": len(_arr(d["crate_ids"])),
"n_styles": len(_arr(d["style_ids"])), "n_genres": len(_arr(d["genre_ids"])),
"price_min": d["price_min"], "price_max": d["price_max"],
"year_min": d["year_min"], "year_max": d["year_max"]})
return {"collections": out}
@router.get("/collections/{cid}")
async def get_collection(cid: int, ident=Depends(require_token), db=Depends(get_db)):
c = (await db.execute(text("SELECT * FROM virtual_collection WHERE id=:i"), {"i": cid})).mappings().first()
if not c:
raise HTTPException(404, "not found")
crate_ids = _arr(c["crate_ids"])
crates = []
if crate_ids:
crates = [dict(r) for r in (await db.execute(text("""
SELECT id, name, label_text,
(SELECT count(*) FROM inventory i WHERE i.crate_id = c.id AND i.in_stock) AS items
FROM virtual_crate c WHERE c.id = ANY(:ids)
"""), {"ids": crate_ids})).mappings()]
order = {cid_: n for n, cid_ in enumerate(crate_ids)} # keep click order
crates.sort(key=lambda x: order.get(x["id"], 999))
return {"collection": {
"id": c["id"], "name": c["name"], "color": c["color"], "priority": c["priority"],
"is_active": bool(c["is_active"]), "sort_method": c["sort_method"],
"crate_ids": crate_ids, "genre_ids": _arr(c["genre_ids"]), "style_ids": _arr(c["style_ids"]),
"label_ids": _arr(c["label_ids"]), "format_descriptions": _arr(c["format_descriptions"]),
"price_min": float(c["price_min"]) if c["price_min"] is not None else None,
"price_max": float(c["price_max"]) if c["price_max"] is not None else None,
"year_min": c["year_min"], "year_max": c["year_max"]},
"crates": crates}
@router.post("/collections")
async def save_collection(body: CollectionIn, ident=Depends(require_token), db=Depends(get_db)):
p = {"name": body.name, "color": body.color, "crates": json.dumps(body.crate_ids),
"genres": json.dumps(body.genre_ids), "styles": json.dumps(body.style_ids),
"labels": json.dumps(body.label_ids), "formats": json.dumps(body.format_descriptions),
"pmin": body.price_min, "pmax": body.price_max, "ymin": body.year_min, "ymax": body.year_max,
"sort": body.sort_method, "prio": body.priority, "active": 1 if body.is_active else 0}
if body.id:
await db.execute(text("""
UPDATE virtual_collection SET name=:name, color=:color, crate_ids=:crates, genre_ids=:genres,
style_ids=:styles, label_ids=:labels, format_descriptions=:formats, price_min=:pmin,
price_max=:pmax, year_min=:ymin, year_max=:ymax, sort_method=:sort, priority=:prio,
is_active=:active, updated_at=now() WHERE id=:id
"""), {**p, "id": body.id})
cid = body.id
else:
cid = (await db.execute(text("""
INSERT INTO virtual_collection (name, color, crate_ids, genre_ids, style_ids, label_ids,
format_descriptions, price_min, price_max, year_min, year_max, sort_method, priority, is_active)
VALUES (:name,:color,:crates,:genres,:styles,:labels,:formats,:pmin,:pmax,:ymin,:ymax,:sort,:prio,:active)
RETURNING id
"""), p)).first()[0]
await db.commit()
return {"ok": True, "id": cid}
@router.delete("/collections/{cid}")
async def delete_collection(cid: int, ident=Depends(require_token), db=Depends(get_db)):
await db.execute(text("DELETE FROM virtual_collection WHERE id=:i"), {"i": cid})
await db.commit()
return {"ok": True}
@router.get("/locate")
async def locate(release_id: int | None = Query(None), sku: str | None = Query(None),
ident=Depends(require_token), db=Depends(get_db)):
"""Where IS this record (exact crate+slot) and, failing that, where SHOULD it be (matching collection)."""
if sku:
inv = (await db.execute(text("SELECT * FROM inventory WHERE sku=:s"), {"s": sku})).mappings().first()
elif release_id:
inv = (await db.execute(text(
"SELECT * FROM inventory WHERE release_id=:r AND in_stock ORDER BY updated_at DESC LIMIT 1"
), {"r": release_id})).mappings().first()
else:
raise HTTPException(400, "release_id or sku required")
if not inv:
raise HTTPException(404, "not in inventory")
exact = None
if inv.get("crate_id"):
cr = (await db.execute(text("""
SELECT c.id, c.label_text, c.name, c.rack_id, c.rack_level_index::int AS level, r.name AS rack
FROM virtual_crate c LEFT JOIN virtual_rack r ON r.id=c.rack_id WHERE c.id=:i
"""), {"i": inv["crate_id"]})).mappings().first()
if cr:
exact = {"crate_id": cr["id"], "crate": cr["label_text"] or cr["name"], "rack": cr["rack"],
"level": cr["level"], "slot": inv.get("slot_number")}
# fallback: match the record against active collections by priority. genre/style needs the disc_cache
# enrichment (added but backfilled separately) — until then a style-ruled collection won't match.
meta = (await db.execute(text(
"SELECT year, genre_ids, style_ids FROM disc_cache WHERE release_id=:r"
), {"r": inv.get("release_id")})).mappings().first() if inv.get("release_id") else None
price = float(inv["price"]) if inv.get("price") is not None else None
year = meta["year"] if meta else None
rg = set(meta["genre_ids"] or []) if meta and "genre_ids" in meta.keys() else set()
rs = set(meta["style_ids"] or []) if meta and "style_ids" in meta.keys() else set()
suggestion = None
cols = (await db.execute(text("""
SELECT id, name, color, crate_ids, genre_ids, style_ids, price_min::float AS pmin,
price_max::float AS pmax, year_min, year_max
FROM virtual_collection WHERE is_active = 1 OR is_active = true ORDER BY priority DESC, name
"""))).mappings()
for c in cols:
gmin, gmax = c["pmin"], c["pmax"]
if gmin is not None and (price is None or price < gmin):
continue
if gmax is not None and (price is None or price > gmax):
continue
if c["year_min"] is not None and (year is None or year < c["year_min"]):
continue
if c["year_max"] is not None and (year is None or year > c["year_max"]):
continue
cgen, csty = set(_arr(c["genre_ids"])), set(_arr(c["style_ids"]))
if cgen and not (rg & cgen):
continue
if csty and not (rs & csty):
continue
if not (gmin or gmax or c["year_min"] or c["year_max"] or cgen or csty):
continue # a ruleless collection matches nothing
ids = _arr(c["crate_ids"])
names = [dict(r) for r in (await db.execute(text(
"SELECT id, label_text, name FROM virtual_crate WHERE id = ANY(:ids)"), {"ids": ids})).mappings()]
suggestion = {"id": c["id"], "name": c["name"], "color": c["color"],
"crates": [{"id": n["id"], "label": n["label_text"] or n["name"]} for n in names]}
break
return {"ok": True, "sku": inv["sku"], "release_id": inv.get("release_id"),
"exact": exact, "suggested": suggestion}

View File

@ -23,6 +23,7 @@ from .layout_routes import router as layout_router # noqa: E402
from .shop_routes import router as shop_router # noqa: E402
from .sales_routes import router as sales_router # noqa: E402
from .navigator_routes import router as navigator_router # noqa: E402
from .collections_routes import router as collections_router # noqa: E402
from .db import engine # noqa: E402
from sqlalchemy import text as _sqltext # noqa: E402
@ -35,6 +36,7 @@ app.include_router(layout_router)
app.include_router(shop_router)
app.include_router(sales_router)
app.include_router(navigator_router)
app.include_router(collections_router)
_STARTUP_DDL = [
@ -48,6 +50,10 @@ _STARTUP_DDL = [
"CREATE TABLE IF NOT EXISTS sales_setting (setting_key text PRIMARY KEY, setting_value text, is_active boolean NOT NULL DEFAULT true, updated_at timestamptz DEFAULT now())",
# a Guest customer always available at the counter
"INSERT INTO customer (id, first_name, last_name, email, is_guest) VALUES (0, 'Guest', 'Sale', NULL, true) ON CONFLICT (id) DO NOTHING",
# disc_cache enrichment for the collection fallback locator (backfilled from discogs metadata separately)
"ALTER TABLE disc_cache ADD COLUMN IF NOT EXISTS year int",
"ALTER TABLE disc_cache ADD COLUMN IF NOT EXISTS genre_ids int[]",
"ALTER TABLE disc_cache ADD COLUMN IF NOT EXISTS style_ids int[]",
# POS fields on the migrated sales tables
"ALTER TABLE sales ADD COLUMN IF NOT EXISTS subtotal numeric(10,2)",
"ALTER TABLE sales ADD COLUMN IF NOT EXISTS discount_amount numeric(10,2) DEFAULT 0",

View File

@ -1,4 +1,5 @@
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
from sqlalchemy import text
from .auth import require_token
@ -100,3 +101,90 @@ async def crate_contents(crate_id: int, ident=Depends(require_token), db=Depends
ORDER BY i.slot_number NULLS LAST, i.sku
"""), {"id": crate_id})).mappings()]
return {"ok": True, "crate": dict(crate), "items": items}
# ── Reorganize — A-Z re-file planner (server scans + applies; client plans the moves) ──────────
class ScanIn(BaseModel):
crate_ids: list[int]
class Move(BaseModel):
sku: str
new_crate_id: int
new_slot: int
class Leftover(BaseModel):
sku: str
archive_location: str | None = None
class ApplyIn(BaseModel):
matched: list[Move] = []
leftovers: list[Leftover] = []
@router.post("/reorg/scan")
async def reorg_scan(body: ScanIn, ident=Depends(require_token), db=Depends(get_db)):
"""Every in-stock item across the source crates — the pool the client sorts + redistributes."""
ids = [c for c in body.crate_ids if c and c > 0]
if not ids:
raise HTTPException(400, "no crate ids")
rows = [dict(r) for r in (await db.execute(text("""
SELECT i.sku, i.release_id, i.crate_id, i.slot_number::int AS slot, i.price::float AS price,
i.condition, coalesce(i.title, dc.title) AS title, dc.artist, dc.year, dc.thumb,
c.label_text AS crate_label
FROM inventory i
LEFT JOIN disc_cache dc ON dc.release_id = i.release_id
LEFT JOIN virtual_crate c ON c.id = i.crate_id
WHERE i.crate_id = ANY(:ids) AND i.in_stock
ORDER BY i.crate_id, i.slot_number NULLS LAST
"""), {"ids": ids})).mappings()]
return {"ok": True, "items": rows, "count": len(rows), "crate_ids": ids}
@router.post("/reorg/apply")
async def reorg_apply(body: ApplyIn, ident=Depends(require_token), db=Depends(get_db)):
"""Commit the plan in ONE transaction: matched → new crate+slot, leftovers → unfiled/archived."""
if not body.matched and not body.leftovers:
raise HTTPException(400, "nothing to apply")
moved = 0
for m in body.matched:
if not m.sku or m.new_crate_id < 1 or m.new_slot < 1:
continue
r = await db.execute(text("""
UPDATE inventory SET crate_id=:c, slot_number=:s, updated_at=now()
WHERE sku=:sku
"""), {"c": m.new_crate_id, "s": m.new_slot, "sku": m.sku})
moved += r.rowcount or 0
archived = 0
for lo in body.leftovers:
if not lo.sku:
continue
r = await db.execute(text("""
UPDATE inventory SET crate_id=NULL, slot_number=NULL, location=:loc, updated_at=now()
WHERE sku=:sku
"""), {"loc": lo.archive_location, "sku": lo.sku})
archived += r.rowcount or 0
if moved == 0 and body.matched:
# nothing matched a real SKU — roll back rather than half-apply
await db.rollback()
raise HTTPException(409, f"0 of {len(body.matched)} items matched an inventory SKU — nothing written")
await db.commit()
return {"ok": True, "matched_updated": moved, "leftovers_archived": archived}
@router.post("/reorg/compress/{crate_id}")
async def reorg_compress(crate_id: int, ident=Depends(require_token), db=Depends(get_db)):
"""Close gaps in a crate's slot numbering — renumber 1..N by current slot order."""
await db.execute(text("""
WITH ranked AS (
SELECT sku, row_number() OVER (ORDER BY slot_number NULLS LAST, sku) AS rn
FROM inventory WHERE crate_id = :c AND in_stock)
UPDATE inventory i SET slot_number = ranked.rn, updated_at = now()
FROM ranked WHERE i.sku = ranked.sku
"""), {"c": crate_id})
n = (await db.execute(text(
"SELECT count(*) FROM inventory WHERE crate_id=:c AND in_stock"), {"c": crate_id})).scalar()
await db.commit()
return {"ok": True, "slots": n}

View File

@ -57,7 +57,11 @@
<div class="bar" style="justify-content:space-between">
<div><span id="ccName" class="muted">Crate contents</span> <span id="ccMeta" class="muted"></span></div>
</div>
<div id="ccList" style="max-height:34vh;overflow:auto;margin-top:8px"><div class="muted">pick a crate on the map →</div></div>
<div id="ccList" style="max-height:28vh;overflow:auto;margin-top:8px"><div class="muted">pick a crate on the map →</div></div>
</div>
<div class="panel">
<div class="bar" style="justify-content:space-between"><b style="font-size:13px">📚 Collections</b><span id="colCount" class="muted"></span></div>
<div id="colList" style="max-height:22vh;overflow:auto;margin-top:8px"><div class="muted">loading…</div></div>
</div>
</div>
@ -102,7 +106,7 @@ async function signin(){
if(!(await fetch('/admin/stats',{headers:hdr()})).ok){ $('#gerr').textContent='invalid token'; return; }
localStorage.setItem('rg_token',TOKEN);
$('#gate').style.display='none'; $('#app').style.display='grid';
$('#q').focus(); await loadSpaces(); toStore();
$('#q').focus(); await loadSpaces(); toStore(); loadCollections();
}
async function loadSpaces(){
@ -249,18 +253,48 @@ async function run(){
if(!q){ $('#res').innerHTML=''; $('#count').textContent='type to search — click a hit to locate it'; return; }
const d=await get('/admin/inventory?q='+encodeURIComponent(q));
$('#count').textContent=`${d.total} match${d.total===1?'':'es'} — click to locate`;
$('#res').innerHTML=(d.items||[]).map(it=>`<div class="ri" onclick='locate(${it.crate_id||'null'})'>
$('#res').innerHTML=(d.items||[]).map(it=>`<div class="ri" onclick='locate(${it.crate_id||'null'}, ${JSON.stringify(it.sku||'')})'>
${it.thumb?`<img src="${it.thumb}">`:'<span class=ph></span>'}
<div class="t"><div class="nm">${esc(it.title||it.sku)}</div>
<div class="muted">${esc(it.artist||'')} ${it.condition?'· '+esc(it.condition):''} · <b class="${it.in_stock?'pink':'oos'}">${money(it.price)}</b></div>
${it.crate?`<span class="loc">📍 ${esc(it.crate)}</span>`:'<span class="muted">· no bin</span>'}</div></div>`).join('')
|| '<div class="muted" style="padding:8px">no match anywhere in the store</div>';
}
async function locate(crateId){
if(!crateId){ alert('That item has no bin assigned yet.'); return; }
const d=await get('/nav/crate/'+crateId); const c=d.crate;
if(c.rack_id){ await openRack(c.rack_id, crateId); }
loadCrate(crateId);
async function locate(crateId, sku){
if(crateId){ goCrate(crateId); return; }
// no exact bin → fallback: where SHOULD it be (collection zone)
const d=await get('/nav/locate?sku='+encodeURIComponent(sku||''));
if(d.suggested) showSuggestion(d.suggested);
else { $('#ccName').className='muted'; $('#ccName').textContent='No bin & no matching collection zone'; $('#ccMeta').textContent=''; $('#ccList').innerHTML='<div class="muted">this record isn\'t filed and no collection rule matches it yet</div>'; }
}
async function goCrate(id){
const d=await get('/nav/crate/'+id);
if(d.crate && d.crate.rack_id) await openRack(d.crate.rack_id, id);
loadCrate(id);
}
function showSuggestion(sug){
$('#ccName').className='pink'; $('#ccName').innerHTML='🎯 Should be in: '+esc(sug.name);
$('#ccMeta').textContent='· fallback zone ('+sug.crates.length+' crates)';
$('#ccList').innerHTML=sug.crates.length? sug.crates.map(c=>`<div class="ri" onclick="goCrate(${c.id})"><div class="t"><div class="nm">${esc(c.label||('Crate '+c.id))}</div></div><span class="loc">go →</span></div>`).join('')
: '<div class="muted">collection has no crates assigned</div>';
}
// ── collections ──
async function loadCollections(){
const d=await get('/nav/collections');
$('#colCount').textContent=(d.collections||[]).length+' saved';
$('#colList').innerHTML=(d.collections||[]).map(c=>{
const rule=[c.price_min!=null?('$'+c.price_min+'+'):'', c.year_min?(c.year_min+(c.year_max?'-'+c.year_max:'+')):'', c.n_styles?(c.n_styles+'st'):'', c.n_genres?(c.n_genres+'g'):''].filter(Boolean).join(' · ');
return `<div class="ri" onclick="openCollection(${c.id})"><span class="sw" style="background:${esc(c.color||'#3498db')}"></span>
<div class="t"><div class="nm">${esc(c.name)}</div><div class="muted">${c.crate_count} crates${rule?' · '+esc(rule):''}</div></div></div>`;
}).join('')||'<div class="muted">no collections</div>';
}
async function openCollection(id){
const d=await get('/nav/collections/'+id); const c=d.collection;
$('#ccName').className='pink'; $('#ccName').innerHTML='📚 '+esc(c.name);
$('#ccMeta').textContent='· '+(d.crates||[]).length+' crates';
$('#ccList').innerHTML=(d.crates||[]).length? (d.crates||[]).map(x=>`<div class="ri" onclick="goCrate(${x.id})"><div class="t"><div class="nm">${esc(x.label_text||x.name||('Crate '+x.id))}</div><div class="muted">${x.items||0} items</div></div><span class="loc">go →</span></div>`).join('')
: '<div class="muted">no crates in this collection</div>';
}
if(TOKEN){ $('#tok').value=TOKEN; signin(); }