Lane F R23: execute the mid-round merge of G's branch (charter law #5)
The merge amendment says G's branch merges mid-round once Fable-m3 reviews. The review
happened (the shared checkout was parked on lane-G-godverse for it) and C verified the
atlas contract from a temp worktree — deliberately pre-merge — and PASSED. But the merge
itself never executed: main had no web/assets/stock_godverse/** at all, so the round's
headline artifact was absent from the branch F is asked to gate and tag.
F executes it as the integration lane, last in the round with the tree in hand.
Disjointness verified before merging, not assumed:
G since fork: G-progress.md, docs/G3_ECONOMY_DESIGN.md, pipeline/_godverse/*,
pipeline/godverse_stock.py, web/assets/stock_godverse/3962749/*
main since fork: C-progress.md, docs/LANES/LANE_C_PUB.md
overlap: none. Zero conflicts, as charter law #5 predicted.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
m3ultra
commit
43b035aa2e
@ -3,6 +3,78 @@
|
|||||||
*Lane brief: `docs/LANES/LANE_G_GODVERSE.md`. This file follows your progress-file convention —
|
*Lane brief: `docs/LANES/LANE_G_GODVERSE.md`. This file follows your progress-file convention —
|
||||||
newest round first. Fable-on-ultra writing.*
|
newest round first. Fable-on-ultra writing.*
|
||||||
|
|
||||||
|
## Round 23 (2026-07-16) — G2a: THE FIRST REAL CRATE + the G3 draft
|
||||||
|
|
||||||
|
**Landed (branch `lane-G-godverse`, G namespace only — ready for your mid-round merge):**
|
||||||
|
- `pipeline/godverse_stock.py` — the per-shop atlas pipeline, split at **the snapshot boundary**:
|
||||||
|
`snapshot` (runs where the DBs are; freezes every source row + every cover's sha256) →
|
||||||
|
`pack` (pure: snapshot + image bytes → atlas, no DB, no network). That split is deliberate —
|
||||||
|
**it's what lets E's `validate_atlas` gate G's output on m3 without the dealgod DB** (ledger #4).
|
||||||
|
- `pipeline/_godverse/monsterrobot_550.snapshot.json` — the artifact of record (committed).
|
||||||
|
- `web/assets/stock_godverse/3962749/` — **120 real records** from a real crate: 2 atlases
|
||||||
|
(2048², 256px cells, E's contract verbatim) + index with provenance.
|
||||||
|
- `docs/G3_ECONOMY_DESIGN.md` — the draft (ledger #2), incl. **the write-back question framed for
|
||||||
|
John** (§7), now grounded in a real trading shop rather than a hypothetical.
|
||||||
|
- **Verified:** byte-identical re-bake (same snapshot → same bytes); uv ⊂ [0,1]; every item's
|
||||||
|
atlas ref resolves; bands standard 58 / collector 56 / grail 6. Eyeballed the atlas: it's a real
|
||||||
|
deep-house crate (Basement Jaxx, Byron Stingily, Angel Moraes, 4hero, Murk, Blaze).
|
||||||
|
|
||||||
|
**THE HEADLINE FINDING — the brief's premise doesn't hold (all measured, R23):**
|
||||||
|
The ledger asked for "dealgod inventory → one real **Newtown** record shop". That can't be built:
|
||||||
|
- **No Newtown record shop has stock in any database.** Egg Records / Papa Disquo / Haze Guitars /
|
||||||
|
The Vintage Record / Downtown Music: **0 items each**.
|
||||||
|
- **`dealgod.stores` is not a shop census** — 8,754 of 9,285 are **Discogs seller accounts**.
|
||||||
|
Zero Newtown stores, zero census name matches. It's an item *feed*, not a shop directory.
|
||||||
|
- **`thriftgod.item` is a demo seed** — 2,221 items across 29 shops, **all Brisbane/Newcastle/Alice
|
||||||
|
charity shops**, zero record shops.
|
||||||
|
- **`mint()` cannot produce tier 1**: `TABLESAMPLE SYSTEM (0.5)` + `random.random/uniform/shuffle`,
|
||||||
|
and **`random.seed` appears nowhere in `server.py`**. Non-deterministic, and it assigns other
|
||||||
|
sellers' listings to shops that never owned them — which also voids the licence law's "the
|
||||||
|
*stores' own* product shots" green and makes write-back meaningless. (G3 §2.)
|
||||||
|
- This is **not** risk #3's "fuzzy matching" — it's a population mismatch. Nothing to match *to*.
|
||||||
|
|
||||||
|
**What we found instead — and it's better.** `monsterrobot` (**147 Musgrave Rd, Red Hill, Brisbane**
|
||||||
|
— John's ruling) is the same real shop in three datasets: `recordgod` (its POS: 28,624 live vinyl,
|
||||||
|
**24,914 in 357 real crates** with real labels and slot numbers), `monsterrobot.party` (its own
|
||||||
|
site — **the covers are the shop's own product shots**), and dealgod store **3962749** (the scrape;
|
||||||
|
the cover cache + the godverse id). **99.3%** of its crate stock joins to a cached cover by Discogs
|
||||||
|
`release_id`. Not in thriftgod's census, so its godverse id = its dealgod store id 3962749 (real,
|
||||||
|
stable, clear of the census space — thriftgod max shop id = 2992). Drops counted: 120 of 122 in
|
||||||
|
crate 550 baked; 2 dropped (no own photo).
|
||||||
|
|
||||||
|
**Rulings received from John this round (recorded):**
|
||||||
|
1. **The crate ships as the shop it really is** — Red Hill, Brisbane, not Newtown.
|
||||||
|
2. **Real titles/artists at tier 1** — an amendment to the no-real-trademarks law that
|
||||||
|
`build_stock_pack.py` enforces (it parody-transforms metadata). Same shape as the licence law:
|
||||||
|
in-house 🟢, **flagged before anything public**. Recorded in every atlas's provenance.
|
||||||
|
|
||||||
|
**Asks — two things gate the crate reaching a room, and both are outside G's namespace:**
|
||||||
|
1. **[C + F] Per-shop packs aren't wired at runtime.** `stockpack.js` caches by `type` alone
|
||||||
|
(`_packs`/`_resolved`), and `interior_mode.js:192` does `getStockPack(shop.type)` with `base`
|
||||||
|
fixed to `assets/models/`. R23's single shop works via a `base` override, but **two stocked
|
||||||
|
shops collide in that cache** — so this blocks v5.0-beta, not just ledger #3. Seam needed:
|
||||||
|
key the cache by `type + base` (or `godverse_id`) + preload per-shop on entry. **C owns the
|
||||||
|
wording (LANE_C_PUB); G re-emits to whatever lands.**
|
||||||
|
2. **[E, via Fable] The crate has no street.** `redhill_real` doesn't exist; `westend_real` is the
|
||||||
|
only Brisbane cache and stops ~2.3 km south. Needs a `build_towns.py` TOWNS entry
|
||||||
|
(~`center: (-27.4553, 153.0064), span_km: 2.4`) + one Overpass fetch, then
|
||||||
|
`godverse_town.py redhill_real redhill_godverse`. **That's E's file and an outward-facing
|
||||||
|
fetch — G touched neither.** Note monsterrobot must be added to the godverse cache's `shops[]`
|
||||||
|
as id 3962749 (it's not in the census); `godverse_town.py` takes shops from thriftgod only, so
|
||||||
|
that's a small G change once you rule on the town.
|
||||||
|
|
||||||
|
**Environment note (matters for reproducibility):** this ran on the **m3**, not ultra — the box has
|
||||||
|
neither `thriftgod` nor `dealgod` Postgres. Both were restored/read from John's mounted backup
|
||||||
|
drive (`/Volumes/johnking/db-backups`, **2026-07-15** dumps); dealgod (2 GB gz) was never restored,
|
||||||
|
its cover map was streamed straight out of the dump (command recorded in
|
||||||
|
`godverse_stock.py:load_image_map`). This is *better* for the charter's snapshot discipline, not
|
||||||
|
worse: the dump date is pinned in every snapshot's provenance. I left two restored DBs on the m3
|
||||||
|
(`thriftgod`, `recordgod`) — say the word and I'll `dropdb` them.
|
||||||
|
|
||||||
|
**Fences held:** PII — `recordgod` holds real `customer`/`staff`/`sales`; the pipeline reads only
|
||||||
|
`inventory`/`crate`/`disc_cache`, and G3 §9 makes that a standing fence. Licence — covers are the
|
||||||
|
shop's own shots, in-house 🟢, flagged before public, provenance in every atlas.
|
||||||
|
|
||||||
## Round 1 (2026-07-16, ultra) — the marriage's first artifact
|
## Round 1 (2026-07-16, ultra) — the marriage's first artifact
|
||||||
|
|
||||||
**Landed:**
|
**Landed:**
|
||||||
|
|||||||
247
docs/G3_ECONOMY_DESIGN.md
Normal file
247
docs/G3_ECONOMY_DESIGN.md
Normal file
@ -0,0 +1,247 @@
|
|||||||
|
# G3 — THE ECONOMY BACKEND (design doc, DRAFT for ratification)
|
||||||
|
|
||||||
|
*Lane G (ultra), R23. Drafted per the v5 charter ([V5_REAL_SHOP.md](V5_REAL_SHOP.md)) ledger #2.
|
||||||
|
Ratification: both Fables + John, at R24 kickoff. **Nothing server-side ships before that.** This
|
||||||
|
round is paper — plus the tier-1 artifact that made the paper honest (G2a, merged this round).*
|
||||||
|
|
||||||
|
Status of each section: **[MEASURED]** = verified against the real databases this round.
|
||||||
|
**[DESIGN]** = proposed, not built. **[JOHN]** = a decision reserved to John by charter law.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. What the data actually is [MEASURED]
|
||||||
|
|
||||||
|
The charter assumed one join: thriftgod's census of real shops → dealgod's real items. **That join
|
||||||
|
does not exist.** The three datasets are three different populations:
|
||||||
|
|
||||||
|
| dataset | what it really holds | inventory? |
|
||||||
|
|---|---|---|
|
||||||
|
| `thriftgod.shop` | 2,928 real AU secondhand shops, OSM-anchored (the census) | **no** |
|
||||||
|
| `dealgod.stores` | 9,285 online sellers — **8,754 are Discogs accounts**, not physical shops | 1.77M listings |
|
||||||
|
| `recordgod` | **one real trading record shop's POS** — crates, slots, prices, grading | 28,624 live |
|
||||||
|
|
||||||
|
Measured consequences:
|
||||||
|
|
||||||
|
- **No Newtown record shop has any stock, in any database.** Egg Records, Papa Disquo, Haze
|
||||||
|
Guitars, The Vintage Record, Downtown Music: 0 items each. dealgod has zero Newtown stores and
|
||||||
|
zero census name matches.
|
||||||
|
- `thriftgod.item` (the minted table) holds 2,221 items across **29 shops — all Brisbane/
|
||||||
|
Newcastle/Alice charity shops**. Zero record shops. It is a demo seed, not a stock system.
|
||||||
|
- This is **not** charter risk #3 ("fuzzy matching"). It is a population mismatch. Fuzzy matching
|
||||||
|
cannot fix it, because there is nothing on the other side of the join to match *to*.
|
||||||
|
|
||||||
|
### 1a. The one shop where all three meet [MEASURED]
|
||||||
|
|
||||||
|
`monsterrobot` — **147 Musgrave Road, Red Hill, Brisbane QLD** (John, R23) — is the same real shop
|
||||||
|
in three forms, and that is the whole reason tier 1 can be honest:
|
||||||
|
|
||||||
|
```
|
||||||
|
recordgod (its POS) ──┐
|
||||||
|
monsterrobot.party ├── one real shop: real crates · real slots · real prices · its OWN photos
|
||||||
|
dealgod store 3962749 ─┘
|
||||||
|
```
|
||||||
|
|
||||||
|
- 28,624 live vinyl; **24,914 physically placed in 357 real crates** with real labels
|
||||||
|
("DEEP HOUSE", "$12–$15 TECHNO") and real slot numbers — with gaps where records actually sold.
|
||||||
|
- Covers are the shop's own product shots on the shop's own domain, cached by dealgod
|
||||||
|
(23,392 releases). **99.3%** of its crate stock joins to a cached cover by Discogs `release_id`.
|
||||||
|
- It is **not in thriftgod's OSM census**, so its godverse id is its dealgod store id `3962749`
|
||||||
|
(real, stable, clear of the census space — thriftgod max shop id = 2992).
|
||||||
|
|
||||||
|
**Shipped this round (G2a):** crate 550 "DEEP HOUSE" → 120 real records → `web/assets/
|
||||||
|
stock_godverse/3962749/`. Byte-identical re-bake verified.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Why `mint()` cannot be the source of any tier [MEASURED]
|
||||||
|
|
||||||
|
thriftgod's `mint()` is the obvious candidate for stocking shops. It disqualifies itself three ways:
|
||||||
|
|
||||||
|
1. **It is not deterministic.** `FEED_RECORDS` uses `TABLESAMPLE SYSTEM (0.5)`; `mint()` uses
|
||||||
|
`random.random()`, `random.uniform()`, `random.shuffle()`. **`random.seed` appears nowhere in
|
||||||
|
`server.py`.** Re-running mint gives different stock at different prices. That is fatal to tier
|
||||||
|
1 (charter law #2), and it is *why* tier 1 reads the POS instead: a crate is a table of facts.
|
||||||
|
2. **The shop assignment is fictional.** mint samples arbitrary Discogs sellers' listings and
|
||||||
|
assigns them to a census shop that never owned them.
|
||||||
|
3. **Therefore it breaks two laws downstream.** The licence law's green rests on "the *stores'
|
||||||
|
own* product shots" — a Discogs seller's photo shown as Egg Records' stock isn't that. And the
|
||||||
|
write-back question (§7) is **meaningless** for minted stock: you cannot mark real inventory
|
||||||
|
sold for an item the shop never had.
|
||||||
|
|
||||||
|
mint stays what it is: a fun demo seeder for parody tier 0. It is not an inventory system.
|
||||||
|
*(If a future round wants seeded mint for tier 0 variety, seeding it is a small, safe change —
|
||||||
|
`random.Random(shop_id)` + an ordered `ORDER BY id` sample instead of `TABLESAMPLE`.)*
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. The tier ladder, as built [MEASURED for 0–1, DESIGN for 2]
|
||||||
|
|
||||||
|
| tier | source | deterministic | offline |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 0 · parody | seeded canvas / E's parody packs | yes | yes |
|
||||||
|
| 1 · static real | **per-shop atlas baked from the POS snapshot** (G2a, shipped) | **yes — verified** | **yes — files** |
|
||||||
|
| 2 · live | thriftgod server over the POS (this doc) | **no, by design** | **enriches only** |
|
||||||
|
|
||||||
|
**The offline law is structural, not a promise.** `stockpack.js` already returns `null` when an
|
||||||
|
index or atlas is missing, and every caller falls back to parody canvas. Tier 2 rides the same
|
||||||
|
seam: the server is an *enrichment* that upgrades an already-playable tier-1 room. There is no
|
||||||
|
boot path, gate, or tag that may touch the network.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. API surface [DESIGN]
|
||||||
|
|
||||||
|
Read-only in v5.0; the write verbs are drafted here but **gated on §7**.
|
||||||
|
|
||||||
|
```
|
||||||
|
GET /godverse/v1/shop/<godverse_id>/stock?crate=<id>&since=<iso>
|
||||||
|
-> { shop_id, crate, tier, items:[{sku, slot, release_id, artist, title,
|
||||||
|
price, condition, sleeve_cond, cover, state}],
|
||||||
|
gone:[sku], etag, served_at }
|
||||||
|
GET /godverse/v1/shop/<godverse_id>/crates -> crate list + counts (the rotation menu)
|
||||||
|
POST /godverse/v1/reserve { sku, session } -> { ok, holds_until } [GATED — §7]
|
||||||
|
POST /godverse/v1/buy { sku, session, hold } -> { ok, receipt } [GATED — §7]
|
||||||
|
```
|
||||||
|
|
||||||
|
Design rules:
|
||||||
|
|
||||||
|
- **The atlas is never served by the API.** Covers stay static files (tier 1). The server sends
|
||||||
|
*facts* (what's still there, what it costs), never pixels. A tier-2 shop with a dead server is
|
||||||
|
visually identical to tier 1 — that's the fail-soft.
|
||||||
|
- **`gone[]` is the whole point of tier 2.** The client already holds the tier-1 crate; the server
|
||||||
|
says only what changed. A sold record is a `sku` in `gone[]` → `collapseBuyItem()` (already
|
||||||
|
built, R9) zero-areas the sleeve. Sold-means-gone is a *diff*, not a re-fetch.
|
||||||
|
- **`etag`/`since`** so a re-enter is a 304, not a payload.
|
||||||
|
- Shop id is the **godverse id** (§1a), never a POS internal id.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Async enrichment + caching (risk #1) [DESIGN]
|
||||||
|
|
||||||
|
**Law: the dig never blocks on the network.** Not "usually" — never. The mechanism:
|
||||||
|
|
||||||
|
1. Room builds at tier 1 from static files. Playable. Riffling works. **No await.**
|
||||||
|
2. The stock fetch is fired in parallel, never awaited by any build path, with a **hard ~400 ms
|
||||||
|
budget** and `AbortController`.
|
||||||
|
3. If it lands in time: apply `gone[]` + price deltas before the player reaches the crate.
|
||||||
|
4. If it lands late: apply on arrival (a record vanishing mid-riffle is *correct* — that's a real
|
||||||
|
shop). If it never lands: the room stays tier 1, silently. **Zero console noise** — a dead
|
||||||
|
server is an expected state, not an error.
|
||||||
|
5. Cache the response per `(shop, crate)` in memory + `sessionStorage`, keyed by `etag`. One
|
||||||
|
fetch per shop per session, not per crate entry.
|
||||||
|
6. **One flight at a time per shop**; re-entering a shop reuses the cache. No polling loop.
|
||||||
|
|
||||||
|
The v1 machinery makes this cheap: the pack is a shared material + per-item UVs, so applying
|
||||||
|
`gone[]` is a vertex collapse on an existing merged mesh — no rebuild, no re-upload, no stutter.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Identity: census ↔ lot ↔ POS (risk #3) [DESIGN, informed by §1]
|
||||||
|
|
||||||
|
The R22 contract said "atlas per shop keyed by godverse shop id". §1 shows the id must come from
|
||||||
|
whichever dataset actually *has* the shop, and mismatches must fail soft, never mis-stock:
|
||||||
|
|
||||||
|
- **A shop is stocked only by explicit, recorded identity** — a `godverse_id` that resolves to a
|
||||||
|
real POS. There is no fuzzy name matching, ever. A shop we cannot identify with certainty gets
|
||||||
|
tier 0. **Mis-stocking is worse than no stock**: it puts one shop's records in another's crate,
|
||||||
|
which is a lie the whole epoch exists to avoid.
|
||||||
|
- **Every unmatched census shop is counted, not guessed** (the drops law). R23's count: 2,928
|
||||||
|
census shops, 1 identified real POS.
|
||||||
|
- **The census is not a stock oracle.** Being in the census means the shop exists, not that we
|
||||||
|
know what's in it. v5.0-beta's "every hero shop stocked" is therefore bounded by *real POS
|
||||||
|
access*, not by census coverage — see §9.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. THE WRITE-BACK QUESTION [JOHN — charter law #4]
|
||||||
|
|
||||||
|
**Does an in-game purchase ever touch real thriftgod/recordgod inventory?**
|
||||||
|
|
||||||
|
This is now a live question, not a hypothetical: `monsterrobot` is a **real trading shop** whose
|
||||||
|
POS syncs to **Square**, with real customers buying these exact records off the real shelf. Stated
|
||||||
|
per charter law #4 with trade-offs and **no default**.
|
||||||
|
|
||||||
|
Relevant measured facts: the POS already has `in_stock`, `sold_date`, **`hold_expires_at`**, `qty`
|
||||||
|
and `square_item_id`. A reservation primitive therefore already exists — the question is whether
|
||||||
|
the game is allowed to pull it.
|
||||||
|
|
||||||
|
### Option A — Sandbox (read-live, sell-local)
|
||||||
|
Game reads real stock; a purchase never leaves the game. Sold-means-gone is per-session.
|
||||||
|
- **For:** zero real-world risk; no payment, tax, or fulfilment surface; a bug can't cost real
|
||||||
|
stock; ships fastest.
|
||||||
|
- **Against:** the one-of-a-kind thrill is fiction — the real crate never changes, and two players
|
||||||
|
(or the same player, next boot) buy the same record. Tier 2 becomes "live prices" more than
|
||||||
|
"live stock".
|
||||||
|
|
||||||
|
### Option B — Reservation (a real, expiring hold)
|
||||||
|
A game purchase places a real hold on the real SKU via the existing `hold_expires_at`. The record
|
||||||
|
genuinely leaves the real crate for N hours; unclaimed, it returns.
|
||||||
|
- **For:** genuinely one-of-a-kind, and **reversible** — the strongest truth-per-unit-risk; uses a
|
||||||
|
primitive the POS already has.
|
||||||
|
- **Against:** a game action removes real sellable stock from real in-store customers. Needs a
|
||||||
|
hold cap/budget, a staff-visible reason code ("held: GODVERSE"), and a race policy against
|
||||||
|
Square in-store sales. Someone must decide what happens when a held record sells in the shop
|
||||||
|
anyway (the game must lose, gracefully).
|
||||||
|
|
||||||
|
### Option C — Full write-back (game sale = real sale)
|
||||||
|
A purchase marks the real SKU sold and syncs to Square.
|
||||||
|
- **For:** total truth — buy it in the game and it is gone from the real shop, forever.
|
||||||
|
- **Against:** irreversible real-world inventory and revenue consequences; the game becomes a
|
||||||
|
sales channel (real payment, tax, postage, fulfilment, refunds, chargebacks); a bug destroys
|
||||||
|
real stock; a game wallet is not money. This is a business decision, not an engineering one.
|
||||||
|
|
||||||
|
**Lane G asks nothing and recommends nothing here.** Until ratified: **tier 2 is read-live,
|
||||||
|
sell-sandboxed** (the charter's standing state, Option A by default *only* because law #4 says so).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Failure modes [DESIGN]
|
||||||
|
|
||||||
|
| failure | behaviour | gate class |
|
||||||
|
|---|---|---|
|
||||||
|
| server unreachable / DNS dead | tier 1, silent, zero console noise | tier-2 smoke |
|
||||||
|
| server dies **mid-session** | room keeps last-good state; no re-fetch storm; no stutter | tier-2 smoke |
|
||||||
|
| server slow (> budget) | request aborted; tier 1 stands; late arrival applies or is dropped | tier-2 smoke |
|
||||||
|
| 5xx / malformed JSON | treated as unreachable — never a partial apply | tier-2 smoke |
|
||||||
|
| atlas missing but server up | **tier 0** — no stock without covers; never text-only sleeves | tier-1 gate |
|
||||||
|
| `gone[]` names an unknown sku | ignored + counted, never an error | tier-2 smoke |
|
||||||
|
| clock skew on holds | server time is authoritative; client never computes expiry | tier-2 smoke |
|
||||||
|
|
||||||
|
**No tier-2 failure may fail a gate or block a tag** (charter law #1). Tier 2 gets its own smoke
|
||||||
|
class: reachability, fail-soft, latency. Kill the server mid-session and the game must not stutter
|
||||||
|
— that is v5.0's release gate, and it is a *client* test, not a server test.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Fences (non-negotiable, both already load-bearing)
|
||||||
|
|
||||||
|
- **PII.** `recordgod` holds real `customer`, `customer_session`, `mailing_list`, `staff`,
|
||||||
|
`sales` — real private-individual data. The pipeline and the server read **only**
|
||||||
|
`inventory` / `crate` / `disc_cache`. No exceptions, no joins, no "just for debugging".
|
||||||
|
Charter law #3: no private-individual data, ever.
|
||||||
|
- **Licence.** Covers are the shop's own product shots → in-house 🟢, **flagged before any
|
||||||
|
public/commercial release**. Real titles/artists ship at tier 1 per John's R23 amendment to the
|
||||||
|
no-real-trademarks law, under the same flag. Every atlas carries its provenance.
|
||||||
|
- **Scope reality check for v5.0-beta.** "Every hero shop stocked" needs a real POS per shop. We
|
||||||
|
have exactly one (John's own). Stocking a *second* real shop means a second real shopkeeper's
|
||||||
|
data and consent — a licence and relationship question, not a pipeline question. Beta should be
|
||||||
|
scoped to **monsterrobot's 357 crates** (24,646 real records — the whole shop, crate rotation
|
||||||
|
per risk #2) rather than to more shops, unless John says otherwise.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. Open items for the reviewers
|
||||||
|
|
||||||
|
1. **[Fable-m3 / C]** Per-shop pack resolution is not wired: `stockpack.js` caches by `type`
|
||||||
|
alone (`_packs`/`_resolved`), and `interior_mode.js:192` resolves `getStockPack(shop.type)`
|
||||||
|
with `base` fixed at `assets/models/`. R23's single shop works via a `base` override; **two
|
||||||
|
stocked shops will collide in that cache.** The seam C/F need: key the cache by
|
||||||
|
`type + base` (or `godverse_id`), and preload per-shop on shop entry. G re-emits to whatever
|
||||||
|
wording C lands (LANE_C_PUB).
|
||||||
|
2. **[Fable-m3 / E]** `redhill_real` doesn't exist — the only Brisbane cache (`westend_real`)
|
||||||
|
stops ~2.3 km south of Red Hill. The crate has no street until E's `build_towns.py` gets a
|
||||||
|
Red Hill entry (~`center: (-27.4553, 153.0064), span_km: 2.4`) and one Overpass fetch. That's
|
||||||
|
E's file and an outward call — **G has not touched either.**
|
||||||
|
3. **[John, R24]** §7, the write-back question.
|
||||||
|
4. **[Both Fables]** §9's beta scoping — depth (one real shop, 357 crates) vs breadth (more
|
||||||
|
shops, which needs other people's data).
|
||||||
1487
pipeline/_godverse/monsterrobot_550.snapshot.json
Normal file
1487
pipeline/_godverse/monsterrobot_550.snapshot.json
Normal file
File diff suppressed because it is too large
Load Diff
245
pipeline/godverse_stock.py
Normal file
245
pipeline/godverse_stock.py
Normal file
@ -0,0 +1,245 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Lane G — G2a: per-shop REAL stock atlases (tier 1) for `?stock=real`.
|
||||||
|
|
||||||
|
THE FIRST REAL CRATE. One real shop's real crate — real sleeves, real titles, real prices —
|
||||||
|
baked to a static atlas keyed by godverse shop id. No server, no runtime query: tier 1 is files.
|
||||||
|
|
||||||
|
Two stages, split at THE SNAPSHOT BOUNDARY (the charter's determinism rule, and the same
|
||||||
|
discipline build_towns.py applies to Overpass raw):
|
||||||
|
|
||||||
|
snapshot POS DB + image map -> pipeline/_godverse/<shop>_<crate>.snapshot.json
|
||||||
|
pack snapshot + images -> web/assets/stock_godverse/<shop>/stock_record_*.{json,webp}
|
||||||
|
|
||||||
|
The snapshot is the artifact of record: it runs where the databases are, pins every source row
|
||||||
|
AND every cover's sha256, and is committed. The pack stage is pure — same snapshot + same image
|
||||||
|
bytes => byte-identical atlas, on any machine, with no DB. That split is what lets Lane E's
|
||||||
|
validate_atlas gate G's output on m3 without the dealgod DB.
|
||||||
|
|
||||||
|
python3 pipeline/godverse_stock.py snapshot --crate 550 --image-map <tsv>
|
||||||
|
~/Documents/MODELBEAST/venvs/mflux/bin/python pipeline/godverse_stock.py pack \
|
||||||
|
pipeline/_godverse/monsterrobot_550.snapshot.json # needs PIL (house convention)
|
||||||
|
|
||||||
|
THE SHOP (John's ruling, R23): `monsterrobot` — a real, trading record shop at 147 Musgrave Road,
|
||||||
|
Red Hill, Brisbane QLD. It is the same shop in three datasets, which is what makes tier 1 honest:
|
||||||
|
recordgod its POS: real crates, real slot numbers, real prices, real grading
|
||||||
|
monsterrobot.party its own website — the covers are the SHOP'S OWN product shots
|
||||||
|
dealgod store 3962749 the scrape of that site; the cover cache + the godverse shop id
|
||||||
|
It is NOT in thriftgod's OSM census (see G-progress R23), so its godverse id is its dealgod store
|
||||||
|
id 3962749 — real, stable, and clear of the census id space (thriftgod max shop id = 2992).
|
||||||
|
|
||||||
|
DETERMINISM: ordered queries, no RNG, no sampling, no clock in the output. Prices/titles/artists
|
||||||
|
are FACTS copied from the POS, not minted (thriftgod's mint() is TABLESAMPLE + unseeded random —
|
||||||
|
it cannot produce tier 1; see G3_ECONOMY_DESIGN.md §2).
|
||||||
|
|
||||||
|
LICENCE (charter law #3): covers are the shop's own product shots from the shop's own domain,
|
||||||
|
cached by dealgod. In-house 🟢. FLAG BEFORE ANY PUBLIC/COMMERCIAL RELEASE. Real titles/artists
|
||||||
|
ship at tier 1 per John's R23 amendment to the no-real-trademarks law (in-house green, same
|
||||||
|
flag). Provenance rides in every index. PII FENCE: recordgod holds real customer/staff/sales
|
||||||
|
tables — this pipeline reads ONLY inventory/crate/disc_cache and must never read the others.
|
||||||
|
"""
|
||||||
|
import argparse, hashlib, json, os, subprocess, sys
|
||||||
|
|
||||||
|
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||||
|
SNAP_DIR = os.path.join(ROOT, "pipeline", "_godverse")
|
||||||
|
STOCK_DIR = os.path.join(ROOT, "web", "assets", "stock_godverse")
|
||||||
|
IMG_CACHE = os.environ.get("DEALGOD_IMG_CACHE", "/Volumes/johnking/dealgod_backup/images")
|
||||||
|
|
||||||
|
# The shop, per John's R23 ruling. `godverse_id` = dealgod store id (see module docstring).
|
||||||
|
SHOP = {
|
||||||
|
"godverse_id": 3962749, "slug": "monsterrobot", "name": "Monster Robot Party",
|
||||||
|
"address": "147 Musgrave Road", "suburb": "Red Hill", "city": "Brisbane",
|
||||||
|
"state": "QLD", "country": "AU", "type": "record",
|
||||||
|
}
|
||||||
|
OWN_PHOTO_HOST = "monsterrobot.party" # the licence test: the shop's own domain, nothing else
|
||||||
|
|
||||||
|
CELL_W, CELL_H, GRID, ATLAS_PX = 256, 256, 8, 2048 # Lane E's record contract (build_stock_pack.py)
|
||||||
|
PER_ATLAS = GRID * (ATLAS_PX // CELL_H) # 64
|
||||||
|
|
||||||
|
|
||||||
|
def price_band(price):
|
||||||
|
"""Lane E's record bands, verbatim — C's dig UI reads this (grail => `rare`)."""
|
||||||
|
p = price if price is not None else 4
|
||||||
|
return "bargain" if p < 8 else "standard" if p < 25 else "collector" if p < 60 else "grail"
|
||||||
|
|
||||||
|
|
||||||
|
def psql(db, sql):
|
||||||
|
"""Read-only query -> list of field-lists. psql CLI, not psycopg2: the m3 has no psycopg2, and
|
||||||
|
a snapshot stage that shells out has one less dep between the DB and the frozen file."""
|
||||||
|
out = subprocess.run(["psql", "-d", db, "-tAF\t", "-c", sql],
|
||||||
|
capture_output=True, text=True, check=True).stdout
|
||||||
|
return [ln.split("\t") for ln in out.splitlines() if ln.strip()]
|
||||||
|
|
||||||
|
|
||||||
|
def _n(v):
|
||||||
|
return None if v in ("", "\\N") else v
|
||||||
|
|
||||||
|
|
||||||
|
# ── stage 1: snapshot ────────────────────────────────────────────────────────────────────
|
||||||
|
def load_image_map(path):
|
||||||
|
"""release_id -> local_image, restricted to the SHOP'S OWN photos (the licence test).
|
||||||
|
|
||||||
|
On ultra (dealgod present) this is a query. On the m3 dealgod is a 2 GB dump and never
|
||||||
|
restored, so the map is streamed out of it once — the command is recorded in the snapshot's
|
||||||
|
provenance so the extraction is reproducible:
|
||||||
|
|
||||||
|
gzcat dealgod-<date>.sql.gz | awk -F'\\t' '/^COPY public.products \\(/{c=1;next}
|
||||||
|
c && /^\\\\.$/{c=0;next} c && $50!="\\\\N"{print $50"\\t"$5"\\t"$34"\\t"$52"\\t"$56}'
|
||||||
|
(fields: 50=release_id 5=image_url 34=local_image 52=current_price 56=discogs_artist_name)
|
||||||
|
"""
|
||||||
|
m = {}
|
||||||
|
with open(path) as fh:
|
||||||
|
for ln in fh:
|
||||||
|
f = ln.rstrip("\n").split("\t")
|
||||||
|
if len(f) < 3:
|
||||||
|
continue
|
||||||
|
rel, url, local = f[0], f[1], f[2]
|
||||||
|
if OWN_PHOTO_HOST not in url or not local.startswith("/img/"):
|
||||||
|
continue
|
||||||
|
m.setdefault(rel, local) # first wins; ids are sorted upstream => stable
|
||||||
|
return m
|
||||||
|
|
||||||
|
|
||||||
|
def snapshot(args):
|
||||||
|
crate = psql("recordgod", f"SELECT id,name,COALESCE(label_text,'') FROM crate WHERE id={args.crate}")
|
||||||
|
if not crate:
|
||||||
|
sys.exit(f"crate {args.crate} not found in recordgod")
|
||||||
|
cid, cname, clabel = crate[0][0], crate[0][1], crate[0][2]
|
||||||
|
|
||||||
|
# ONLY inventory/crate/disc_cache — the PII fence (see module docstring).
|
||||||
|
rows = psql("recordgod", f"""
|
||||||
|
SELECT i.sku, COALESCE(i.slot_number::text,''), i.release_id,
|
||||||
|
COALESCE(d.artist,''), COALESCE(d.title, COALESCE(i.title,'')),
|
||||||
|
i.price, COALESCE(i.condition,''), COALESCE(i.sleeve_cond,'')
|
||||||
|
FROM inventory i
|
||||||
|
JOIN crate c ON c.id = i.crate_id
|
||||||
|
LEFT JOIN disc_cache d USING (release_id)
|
||||||
|
WHERE i.in_stock AND c.id = {args.crate} AND i.release_id IS NOT NULL
|
||||||
|
ORDER BY i.slot_number NULLS LAST, i.sku""")
|
||||||
|
|
||||||
|
imgs = load_image_map(args.image_map)
|
||||||
|
items, dropped = [], {"no_own_photo": 0, "file_missing": 0, "no_price": 0}
|
||||||
|
for sku, slot, rel, artist, title, price, cond, sleeve in rows:
|
||||||
|
local = imgs.get(rel)
|
||||||
|
if not local:
|
||||||
|
dropped["no_own_photo"] += 1
|
||||||
|
continue
|
||||||
|
path = os.path.join(IMG_CACHE, local[len("/img/"):])
|
||||||
|
if not os.path.isfile(path):
|
||||||
|
dropped["file_missing"] += 1
|
||||||
|
continue
|
||||||
|
if not _n(price):
|
||||||
|
dropped["no_price"] += 1
|
||||||
|
continue
|
||||||
|
with open(path, "rb") as fh:
|
||||||
|
sha = hashlib.sha256(fh.read()).hexdigest()
|
||||||
|
items.append({"sku": sku, "slot": int(slot) if slot else None, "release_id": int(rel),
|
||||||
|
"artist": artist, "title": title, "price": float(price),
|
||||||
|
"condition": cond, "sleeve_cond": sleeve,
|
||||||
|
"cover": local, "cover_sha256": sha})
|
||||||
|
|
||||||
|
if args.max_items:
|
||||||
|
items = items[:args.max_items]
|
||||||
|
|
||||||
|
snap = {
|
||||||
|
"version": 1, "kind": "record", "tier": 1,
|
||||||
|
"shop": SHOP,
|
||||||
|
"crate": {"id": int(cid), "name": cname, "label": clabel},
|
||||||
|
"counts": {"in_crate": len(rows), "baked": len(items), "dropped": dropped},
|
||||||
|
"source": {
|
||||||
|
"pos_db": "recordgod", "pos_dump": args.pos_dump,
|
||||||
|
"image_map_dump": args.image_map_dump, "image_cache": IMG_CACHE,
|
||||||
|
"captured": args.captured,
|
||||||
|
"extraction": "see load_image_map() docstring — the products projection command",
|
||||||
|
},
|
||||||
|
"licence": {
|
||||||
|
"covers": "the shop's own product shots (monsterrobot.party), cached by dealgod",
|
||||||
|
"zone": "in-house-green", "flag": "FLAG BEFORE ANY PUBLIC/COMMERCIAL RELEASE",
|
||||||
|
"metadata": "real titles/artists — John's R23 amendment to the no-real-trademarks law",
|
||||||
|
"pii": "none — inventory/crate/disc_cache only; customer/staff/sales never read",
|
||||||
|
},
|
||||||
|
"items": items,
|
||||||
|
}
|
||||||
|
os.makedirs(SNAP_DIR, exist_ok=True)
|
||||||
|
dst = os.path.join(SNAP_DIR, f"{SHOP['slug']}_{cid}.snapshot.json")
|
||||||
|
with open(dst, "w") as fh:
|
||||||
|
json.dump(snap, fh, indent=1, sort_keys=False)
|
||||||
|
fh.write("\n")
|
||||||
|
print(f"snapshot: crate {cid} '{clabel or cname}' — {len(items)} real records "
|
||||||
|
f"(of {len(rows)} in crate; dropped {dropped}) -> {dst}")
|
||||||
|
|
||||||
|
|
||||||
|
# ── stage 2: pack (pure: snapshot + image bytes -> atlas) ─────────────────────────────────
|
||||||
|
def pack(args):
|
||||||
|
from PIL import Image
|
||||||
|
snap = json.load(open(args.snapshot))
|
||||||
|
shop, items = snap["shop"], snap["items"]
|
||||||
|
out_dir = os.path.join(STOCK_DIR, str(shop["godverse_id"]))
|
||||||
|
os.makedirs(out_dir, exist_ok=True)
|
||||||
|
|
||||||
|
idx_items, atlases = [], []
|
||||||
|
for ai in range((len(items) + PER_ATLAS - 1) // PER_ATLAS):
|
||||||
|
chunk = items[ai * PER_ATLAS:(ai + 1) * PER_ATLAS]
|
||||||
|
W, H = GRID * CELL_W, (ATLAS_PX // CELL_H) * CELL_H
|
||||||
|
sheet = Image.new("RGB", (W, H), (28, 28, 32))
|
||||||
|
aname = f"stock_record_atlas_{ai:02d}.webp"
|
||||||
|
for j, it in enumerate(chunk):
|
||||||
|
gx, gy = j % GRID, j // GRID
|
||||||
|
path = os.path.join(IMG_CACHE, it["cover"][len("/img/"):])
|
||||||
|
with open(path, "rb") as fh:
|
||||||
|
raw = fh.read()
|
||||||
|
if hashlib.sha256(raw).hexdigest() != it["cover_sha256"]:
|
||||||
|
sys.exit(f"cover drift: {it['sku']} {it['cover']} — snapshot sha mismatch, re-snapshot")
|
||||||
|
import io
|
||||||
|
cov = Image.open(io.BytesIO(raw)).convert("RGB").resize((CELL_W, CELL_H), Image.LANCZOS)
|
||||||
|
sheet.paste(cov, (gx * CELL_W, gy * CELL_H))
|
||||||
|
u0, v0 = gx * CELL_W / W, gy * CELL_H / H
|
||||||
|
idx_items.append({
|
||||||
|
"id": f"rec_{len(idx_items):04d}", "title": it["title"], "artist": it["artist"],
|
||||||
|
"price": int(round(it["price"])), "price_band": price_band(it["price"]),
|
||||||
|
"atlas": aname, "uv": [round(u0, 4), round(v0, 4),
|
||||||
|
round(u0 + CELL_W / W, 4), round(v0 + CELL_H / H, 4)],
|
||||||
|
})
|
||||||
|
sheet.save(os.path.join(out_dir, aname), "WEBP", quality=88, method=6)
|
||||||
|
atlases.append(aname)
|
||||||
|
|
||||||
|
with open(args.snapshot, "rb") as fh:
|
||||||
|
snap_sha = hashlib.sha256(fh.read()).hexdigest()
|
||||||
|
index = {
|
||||||
|
"version": 1, "kind": "record", "tier": 1,
|
||||||
|
"atlas_px": ATLAS_PX, "cell_w": CELL_W, "cell_h": CELL_H,
|
||||||
|
"count": len(idx_items), "atlases": atlases,
|
||||||
|
"shop": {k: shop[k] for k in ("godverse_id", "slug", "name", "suburb", "state")},
|
||||||
|
"crate": snap["crate"],
|
||||||
|
"provenance": {
|
||||||
|
"generator": "pipeline/godverse_stock.py",
|
||||||
|
"source": snap["source"], "licence": snap["licence"],
|
||||||
|
"snapshot": os.path.basename(args.snapshot), "snapshot_sha256": snap_sha,
|
||||||
|
"determinism": "pure bake — same snapshot + same cover bytes => byte-identical atlas; "
|
||||||
|
"covers pinned by sha256, no RNG, no clock",
|
||||||
|
},
|
||||||
|
"items": idx_items,
|
||||||
|
}
|
||||||
|
dst = os.path.join(out_dir, "stock_record_index.json")
|
||||||
|
with open(dst, "w") as fh:
|
||||||
|
json.dump(index, fh, indent=1)
|
||||||
|
fh.write("\n")
|
||||||
|
print(f"pack: {len(idx_items)} real sleeves -> {len(atlases)} atlas(es) in {out_dir}")
|
||||||
|
print(f" shop {shop['godverse_id']} ({shop['slug']}) · crate "
|
||||||
|
f"{snap['crate']['id']} '{snap['crate']['label'] or snap['crate']['name']}'")
|
||||||
|
|
||||||
|
|
||||||
|
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||||
|
sub = ap.add_subparsers(dest="cmd", required=True)
|
||||||
|
s1 = sub.add_parser("snapshot", help="POS + image map -> frozen snapshot (runs where the DBs are)")
|
||||||
|
s1.add_argument("--crate", type=int, required=True)
|
||||||
|
s1.add_argument("--image-map", required=True, help="TSV: release_id, image_url, local_image, ...")
|
||||||
|
s1.add_argument("--max-items", type=int, default=0, help="cap (atlas budget — charter risk #2)")
|
||||||
|
s1.add_argument("--pos-dump", default="", help="recordgod dump filename, for provenance")
|
||||||
|
s1.add_argument("--image-map-dump", default="", help="dealgod dump filename, for provenance")
|
||||||
|
s1.add_argument("--captured", default="", help="snapshot date (YYYY-MM-DD), for provenance")
|
||||||
|
s1.set_defaults(func=snapshot)
|
||||||
|
s2 = sub.add_parser("pack", help="snapshot -> atlas + index (pure; needs PIL, no DB)")
|
||||||
|
s2.add_argument("snapshot")
|
||||||
|
s2.set_defaults(func=pack)
|
||||||
|
args = ap.parse_args()
|
||||||
|
args.func(args)
|
||||||
BIN
web/assets/stock_godverse/3962749/stock_record_atlas_00.webp
Normal file
BIN
web/assets/stock_godverse/3962749/stock_record_atlas_00.webp
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 644 KiB |
BIN
web/assets/stock_godverse/3962749/stock_record_atlas_01.webp
Normal file
BIN
web/assets/stock_godverse/3962749/stock_record_atlas_01.webp
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 580 KiB |
1728
web/assets/stock_godverse/3962749/stock_record_index.json
Normal file
1728
web/assets/stock_godverse/3962749/stock_record_index.json
Normal file
File diff suppressed because it is too large
Load Diff
Loading…
Reference in New Issue
Block a user