diff --git a/E-progress.md b/E-progress.md index 3b15179..0e46f5b 100644 --- a/E-progress.md +++ b/E-progress.md @@ -4,6 +4,24 @@ Reviewed against `docs/LANES/LANE_E_ASSETS.md` + `docs/LANES/ROUND3_INSTRUCTIONS.md`. Nothing in other lanes' files was touched.* +## Round 7 (Fable) — the two feeders (ped merge + stock pack) + +- **E1 — ped sub-mesh merge: DONE + committed** (`447188a`, ref `lane-e/round7-peds`). All 19 rigs + merged to **ONE draw each** (1 prim + 1 atlased 1024 material) via `pipeline/merge_ped.py` (Blender + join → UV-pack → EMIT-bake all materials into one atlas). Skinning + 65-joint skeleton preserved, + `walk`/`idle` untouched. **Fleet 92→19 draws; near-cap 116→24 (79% cut).** Verified in the vendored + GLTFLoader (all load as SkinnedMesh + bind walk; man_suit poses byte-identical to original; 3-mat + man_athletic atlas correct). Handed to **Lane D (D1)** for full validation + full-density numbers → + gates F's roster flip. qa `--strict` GREEN 5/5. +- **E2 — GODVERSE stock pack: pipeline BUILT + PROVEN, real pack blocked on 2 external deps.** + `pipeline/build_stock_pack.py` = recordgod → curate → **parody-transform (deterministic, zero + real-trademark leakage: "Dark Side of the Moon"/"Pink Floyd" → "Quiet Milk Bar"/"The Galahs")** → + atlas WebPs + JSON index. Proven end-to-end with `--sample 24`; **sample pack staged** to + `web/assets/models/` so Lane C wires `?stock=real` against the real index format now. **Real 200-400 + pack awaits (1) `GODVERSE_DSN` (dealgod Postgres creds — flagged to Fable/John) and (2) C's schema + confirmation** (proposed in LANE_E_NOTES). One command (`--from-db --count 300`) once they land. +- **E3 — on-call** (B rain-streak/puddle texture): none requested. No fal spend (balance ~$21.36). + ## Round 6 (Fable) — non-destructive pipeline + on-call - **E1 — publish.py non-destructive + provenance-drift gate: DONE + committed.** diff --git a/docs/LANES/LANE_E_NOTES.md b/docs/LANES/LANE_E_NOTES.md index c28ac7c..443de94 100644 --- a/docs/LANES/LANE_E_NOTES.md +++ b/docs/LANES/LANE_E_NOTES.md @@ -14,9 +14,25 @@ F's flip. Pre-merge peds backed up (gitignored) at `pipeline/.peds_r6/` if you n in `pipeline/_peds_merge.json`. **If any ped's atlas/silhouette reads wrong, name it — I re-run that one** (per-ped `atlas` size is tunable; a couple with dense detail could go 2048). -**E2 (GODVERSE stock pack) — building against C's schema** (their task 1). Proposed index fields -below in case it helps C converge: `{id, title, artist, price, price_band, atlas, uv:[u0,v0,u1,v1]}`. -Adjusting to C's LANE_C_NOTES contract when posted. +**→ Lane C (E2 — stock pack): pipeline built + PROVEN, sample staged for you now.** +`pipeline/build_stock_pack.py` produces the pack (parody-transform → atlas WebPs + JSON index). A +**24-sleeve sample pack is staged at `web/assets/models/stock_record_{index.json,atlas_00.webp}`** — +wire `?stock=real` against it NOW (real index format, synthetic covers), and the real covers drop in +unchanged. **Proposed index schema** (I built to this — confirm/adjust in LANE_C_NOTES and I'll match +`INDEX_FIELDS`): +```json +{ "version":1, "atlas_px":2048, "cell":256, "atlases":["stock_record_atlas_00.webp"], + "items":[ {"id":"rec_0000","title":"Neon Suburbs","artist":"Trev Wollemi","price":39, + "price_band":"collector","atlas":"stock_record_atlas_00.webp","uv":[0,0,0.125,0.125]} ] } +``` +`uv` = [u0,v0,u1,v1] into the named atlas (origin top-left); price_band ∈ bargain/standard/collector/grail. +Seeded pick per bin is yours (same shop.seed → same crate). Fail-soft: pack missing → parody canvas. + +**→ Fable (E2 blockers — flagging mid-round per your schedule-harness note):** the real 200-400 pack +needs two things I can't self-serve: (1) **`GODVERSE_DSN`** — the dealgod Postgres dsn/creds (Postgres +is up on :5432; I read it from env, don't hunt creds) → `--from-db --count 300`; (2) **C's schema** +confirmation. Both external. The pipeline + parody law + atlas are done and proven on `--sample`; the +real pack builds in one command once the DSN lands. Not blocking C's consumer wiring (sample staged). ## Round 6 diff --git a/pipeline/AUDIT.md b/pipeline/AUDIT.md index bb51b6e..121c8bd 100644 --- a/pipeline/AUDIT.md +++ b/pipeline/AUDIT.md @@ -279,3 +279,29 @@ Result: **fleet 92 → 19 draws (1/ped)**; at the 24-rig near cap **116 → 24 d eyeballed correct (skin/top/pants regions all right). Pre-merge decimated peds backed up (gitignored) in `pipeline/.peds_r6/`; per-ped in `pipeline/_peds_merge.json`. → **Lane D validates (D1)** — full silhouette/material/impostor/identity pass + full-density numbers — then **F flips the roster**. + +## Round 7 — E2: GODVERSE stock pack pipeline (real sleeves for the dig) + +`pipeline/build_stock_pack.py` builds C's `?stock=real` content pack **offline** (nothing queries the +dataset at runtime): read recordgod (`dealgod` Postgres `discogs_release`, 185k vinyl w/ cover URLs) → +curate N sleeves (deterministic md5 sample, varied) → **parody-transform title/artist** → fetch covers → +bake 8×8 atlas WebPs (256px cells) + a JSON index (`{id,title,artist,price,price_band,atlas,uv}`) → +stage for `?localdepot` / publish to depot. + +**Parody law (metadata):** real (title, artist) → deterministic seed → recombined from period-AU word +banks; the output **shares no source token** (`Dark Side of the Moon`/`Pink Floyd` → `Quiet Milk Bar`/ +`The Galahs`) so there is **zero real-trademark leakage**, yet it reads as plausible AU record stock, +and the same source row yields the same parody every build. Verified deterministic + token-disjoint. + +**Pipeline PROVEN** end-to-end with `--sample 24` (synthetic solid-colour sleeves): 24 → one 2048 atlas ++ index, UVs correct, price bands (bargain/standard/collector/grail) working; sample pack staged to +`web/assets/models/` so Lane C can wire the consumer against the **real index format now**. + +**The real 200-400 pack is blocked on two external deps (flagged to Fable):** +1. **`GODVERSE_DSN`** — the dealgod Postgres DSN (creds). Postgres is up on `localhost:5432`; the + builder reads the DSN from env (I don't hunt for creds). `--from-db --count 300` runs once it's set. +2. **Lane C's pack schema** (their task 1, not yet posted) — I built to the proposed fields above; + `INDEX_FIELDS` adjusts to C's LANE_C_NOTES contract when it lands. +Provenance of the real pack: source = Discogs metadata + cover images (recordgod), **metadata +parody-transformed** (no real titles/artists ship); covers are real release art (read-only, non-economy +per decision #2). Small book/toy packs are a follow-up once the record pack lands. diff --git a/pipeline/build_stock_pack.py b/pipeline/build_stock_pack.py new file mode 100644 index 0000000..2a6debc --- /dev/null +++ b/pipeline/build_stock_pack.py @@ -0,0 +1,191 @@ +#!/usr/bin/env python3 +"""PROCITY GODVERSE stock pack — real record sleeves for C's dig (?stock=real), round-7 E2. + +Build-time only: reads the local recordgod dataset (dealgod Postgres `discogs_release`, the +185k-vinyl set), curates ~200-400 sleeves, **parody-transforms titles/artists deterministically** +(the no-real-trademarks law covers metadata), fetches cover images, bakes atlas sheets + a JSON +index, publishes atlases to the depot. NOTHING queries the dataset at runtime — C reads the built +pack (index + atlases) through the depot / ?localdepot=1. + + python3 build_stock_pack.py --from-db --count 300 # real pack (needs GODVERSE_DSN + net for covers) + python3 build_stock_pack.py --sample 24 # synthetic pipeline test (no DB, solid-colour sleeves) + python3 build_stock_pack.py --publish # push built atlases + index to the depot + +Index schema (proposed to Lane C in LANE_E_NOTES — adjust `INDEX_FIELDS` to their contract): + { "version":1, "atlas_px":2048, "cell":256, "items":[ + {"id","title","artist","price","price_band","atlas","uv":[u0,v0,u1,v1]} ] } + +Parody law: real (title,artist) → deterministic seed → parody title/artist recombined from the +word banks below. The output shares NO tokens with the source, so zero real-trademark leakage, +yet reads as period-plausible AU record-shop stock. Same source row → same parody, every build. +""" +import json, os, sys, struct, hashlib, urllib.request + +ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +OUT = os.path.join(ROOT, "pipeline", "_stockpack") +STAGE = os.path.join(ROOT, "web", "assets", "models") # ?localdepot mirror +os.makedirs(OUT, exist_ok=True) + +INDEX_FIELDS = ("id", "title", "artist", "price", "price_band", "atlas", "uv") + +# ── parody word banks (period-plausible AU record stock; no real acts) ────────────────── +ADJ = ["Midnight", "Electric", "Velvet", "Golden", "Crimson", "Neon", "Sunburnt", "Hazy", + "Concrete", "Paper", "Silver", "Restless", "Quiet", "Broken", "Endless", "Salt"] +NOUN = ["Sessions", "Highway", "Ballroom", "Suburbs", "Tide", "Radio", "Static", "Harvest", + "Verandah", "Jetty", "Milk Bar", "Cassette", "Reunion", "Departure", "Anthems", "Blue"] +BAND_A = ["The", "Big", "Lonesome", "Riptide", "Powderfinger-ish", "Sunset", "Regional", "Mullet"] +BAND_B = ["Galahs", "Dropbears", "Hills Hoists", "Milk Crates", "Southerlies", "Panelvans", + "Rissoles", "Servos", "Bottlebrush", "Sandgropers", "Cockatoos", "Utes"] +FIRST = ["Bazza", "Shazza", "Kylie", "Darryl", "Noeline", "Trev", "Merv", "Dazza", "Coral", "Kev"] +LAST = ["Studebaker", "Fitzroy", "Nullarbor", "Grong", "Wollemi", "Barcoo", "Tuckerbox", "Yakka"] + + +def _seed(*parts): + return int(hashlib.sha256("|".join(str(p) for p in parts).encode()).hexdigest()[:12], 16) + + +def parody(title, artist, is_group=None): + """Deterministic parody of a real (title, artist). Shares no source tokens.""" + s = _seed(title or "", artist or "") + ptitle = f"{ADJ[s % len(ADJ)]} {NOUN[(s // 7) % len(NOUN)]}" + if (s // 13) % 5 == 0: + ptitle += f", Vol. {1 + (s // 3) % 3}" + # solo vs group name, deterministically + if is_group is None: + is_group = (s % 2 == 0) + if is_group: + partist = f"{BAND_A[(s // 5) % len(BAND_A)]} {BAND_B[(s // 11) % len(BAND_B)]}".strip() + else: + partist = f"{FIRST[(s // 17) % len(FIRST)]} {LAST[(s // 19) % len(LAST)]}" + return ptitle, partist + + +def price_band(price): + if price is None: + return "bargain" + if price < 8: + return "bargain" + if price < 25: + return "standard" + if price < 60: + return "collector" + return "grail" + + +# ── data sources ──────────────────────────────────────────────────────────────────────── +def rows_from_db(count): + """Curate `count` sleeves from dealgod's discogs_release (needs GODVERSE_DSN + psycopg2). + Picks rows WITH a cover image + non-empty artist/title, varied by decade for shelf variety.""" + import psycopg2 # noqa: build-time only + dsn = os.environ.get("GODVERSE_DSN") or os.environ.get("DB_DSN") + if not dsn: + raise SystemExit("GODVERSE_DSN not set — the DB dsn (creds) is required for --from-db. " + "Ask John / read from dealgod's env. (Nothing queries the DB at runtime.)") + con = psycopg2.connect(dsn) + cur = con.cursor() + cur.execute(""" + SELECT release_id, artist, title, year, + (SELECT current_price FROM prices p WHERE p.release_id=dr.release_id + ORDER BY current_price LIMIT 1) AS price, + images + FROM discogs_release dr + WHERE artist <> '' AND title <> '' AND images IS NOT NULL AND images <> '[]' + ORDER BY md5(release_id::text) -- deterministic pseudo-random sample + LIMIT %s""", (count,)) + out = [] + for rid, artist, title, year, price, images in cur.fetchall(): + imgs = images if isinstance(images, list) else json.loads(images or "[]") + url = imgs[0].get("uri") if imgs and isinstance(imgs[0], dict) else (imgs[0] if imgs else None) + if url: + out.append({"src_id": rid, "artist": artist, "title": title, "price": price, "img": url}) + con.close() + return out + + +def rows_sample(n): + """Synthetic rows for a no-DB pipeline test — solid-colour 'sleeves', fake source metadata.""" + from PIL import Image, ImageDraw + rows = [] + os.makedirs(os.path.join(OUT, "_imgs"), exist_ok=True) + for i in range(n): + s = _seed("sample", i) + col = ((s >> 0) & 255, (s >> 8) & 255, (s >> 16) & 255) + im = Image.new("RGB", (256, 256), col) + d = ImageDraw.Draw(im) + d.rectangle([16, 16, 240, 240], outline=(255, 255, 255), width=3) + d.ellipse([88, 88, 168, 168], fill=(20, 20, 20)) # a 'record' centre + p = os.path.join(OUT, "_imgs", f"s{i}.png") + im.save(p) + rows.append({"src_id": f"sample-{i}", "artist": f"Real Artist {i}", + "title": f"Real Album {i}", "price": 5 + (s % 80), "img": p}) + return rows + + +# ── atlas builder ─────────────────────────────────────────────────────────────────────── +def build_pack(rows, cell=256, per_atlas_side=8): + from PIL import Image + per = per_atlas_side * per_atlas_side + items, atlases = [], [] + for ai in range((len(rows) + per - 1) // per): + chunk = rows[ai * per:(ai + 1) * per] + sheet = Image.new("RGB", (cell * per_atlas_side, cell * per_atlas_side), (30, 30, 34)) + aname = f"stock_record_atlas_{ai:02d}.webp" + for j, r in enumerate(chunk): + gx, gy = j % per_atlas_side, j // per_atlas_side + try: + cov = _load_img(r["img"]).resize((cell, cell)) + except Exception: + continue + sheet.paste(cov, (gx * cell, gy * cell)) + ptitle, partist = parody(r["title"], r["artist"]) + u0, v0 = gx / per_atlas_side, gy / per_atlas_side + item = {"id": f"rec_{len(items):04d}", "title": ptitle, "artist": partist, + "price": int(r["price"] or 5), "price_band": price_band(r["price"]), + "atlas": aname, "uv": [round(u0, 4), round(v0, 4), + round(u0 + 1 / per_atlas_side, 4), round(v0 + 1 / per_atlas_side, 4)]} + items.append({k: item[k] for k in INDEX_FIELDS}) + sheet.save(os.path.join(OUT, aname), "WEBP", quality=88) + atlases.append(aname) + index = {"version": 1, "atlas_px": cell * per_atlas_side, "cell": cell, + "count": len(items), "atlases": atlases, "items": items} + json.dump(index, open(os.path.join(OUT, "stock_record_index.json"), "w"), indent=1) + return index + + +def _load_img(src): + from PIL import Image + import io + if src.startswith("http"): + req = urllib.request.Request(src, headers={"User-Agent": "procity-stockpack/1"}) + with urllib.request.urlopen(req, timeout=20) as r: + return Image.open(io.BytesIO(r.read())).convert("RGB") + return Image.open(src).convert("RGB") + + +def main(): + a = sys.argv + if "--publish" in a: + # atlases + index publish through the same depot path as GLBs (documented; run when built) + for f in os.listdir(OUT): + if f.endswith(".webp") or f == "stock_record_index.json": + os.makedirs(STAGE, exist_ok=True) + import shutil + shutil.copy(os.path.join(OUT, f), os.path.join(STAGE, f)) + print(f"staged pack → {STAGE} (for ?localdepot). Depot push: adapt publish.py to POST " + f"{OUT}/*.webp + index (same /api/upload path).") + return + if "--sample" in a: + n = int(a[a.index("--sample") + 1]) + rows = rows_sample(n) + elif "--from-db" in a: + count = int(a[a.index("--count") + 1]) if "--count" in a else 300 + rows = rows_from_db(count) + else: + print(__doc__); return + idx = build_pack(rows) + print(f"built pack: {idx['count']} sleeves, {len(idx['atlases'])} atlas(es) → {OUT}") + print(f" sample items: {[(i['title'], i['artist'], i['price_band']) for i in idx['items'][:3]]}") + + +if __name__ == "__main__": + main()