5815dfbd58
THE STREET (ledger #3). redhill_real: Red Hill, Brisbane — 36 shops (9 heroes + 27 texture, -29 subsampled), 884 roads, median NN 73 m. **Musgrave Road is in roads[] (41 ways, primary + residential)** — that was the point: Monster Robot Party's real shop now has its real street. One bounded Overpass fetch under the standing authorization; raw snapshots in _raw/, ODbL, C's mapping. This is G's donor for godverse_town.py -> redhill_godverse. TWO HAZARDS HANDED TO G/A, both measured against the real geometry (notes R24 §1): (a) The shop is NOT in OSM — no shop=music node, no name match anywhere in the bbox (checked the raw snapshot, not the emitted cache). It cannot arrive via the fetch; G must inject id 3962749 from the census or the town ships without the shop the release is about. (b) THE SEATING HAZARD: A's lift seats on the NEAREST edge, and from G's geocode three streets are nearer than the addressed one — Zig Zag Street 38.6 m (residential), Craig Street 46.1 m, Musgrave Road 51.6 m. The shop would seat on Zig Zag St and the headline would quietly become false. Measured fix (G's call): the point on the real Musgrave centreline nearest G's geocode is -27.455169, 153.006902 — 0 m to Musgrave, next candidate 43 m. Snaps the street, does NOT verify the house number; I have no address data for where #147 sits along the road. THE HONEST GATE (ledger #4). F's catch was right and there were FOUR bugs: the glob used the published-contract name the runtime can never load (0 matches -> returned 0: it passed identically whether atlases were absent, misnamed, or perfect); the code read provenance top-level while my own docstring said nested (G nested it correctly — the gate would have failed G's real artifact); licence/license; and a stale ceiling. Rebuilt on the "runtime is the authority" ruling: globs stock_godverse/<id>/stock_<type>_index.json, type in {record,book,toy}. Vacuous-gate law implemented literally — 0 matches + stray files FAIL (listing them); 0 matches + empty dir prints "atlas-QA SKIP: no atlases present (0 files matched glob ...)"; plus an id equality (shop.godverse_id == dir) so it proves it touched its subject. Proved on four cases: empty -> loud SKIP, misnamed (F's exact bug) -> FAIL, unparseable -> FAIL, G's real atlas -> PASS (120 items, 2 atlases, licence printed). My own test found a fifth hole — the glob still matched the old stock_shop_* name, so the misnamed case failed only incidentally; added a STOCK_TYPES name check so it fails for the RIGHT reason. A gate that fails for the wrong reason will pass for the wrong reason later. BALLARAT — RETIRED, counted, and a DIFFERENT illness from Toowoomba (notes R24 §3). Toowoomba was dead everywhere. Ballarat is alive exactly where the game isn't: 103 POIs, densest 400 m disc = 47 of them (Toowoomba's was 3) — but that disc is cafes/clothes/bakeries while the op-shops sit 854 m, 1042 m and 1709 m out. Heroes and density don't coincide, so no bbox holds both. Real builds at every span: 1.0 km -> 8 shops/184 m/hub 4-8 but ZERO op-shops (a thrift town with no thrift shop); 2.0 km -> 12/186 m/3-12; 3.0 km as shipped -> 20/255 m/4-20. Every row worse than any shipped town. THE BINDING CONSTRAINT WAS MINE, NOT OSM'S: with 2 heroes the 3x texture cap keeps 6 of 47 texture POIs and the subsample is spatially blind, so survivors scatter at any span — part of the 255 m A's warn caught was my own cap's fingerprint. Seeding texture near heroes fixes the class but moves every town's golden -> filed as a v5.x proposal, not done in a release round. I OVERSTATED MY R23 ADVICE TO A: the hub fraction is NOT a standalone discriminator — glebe (shipped, healthy) reads 13% and redhill 14%; a hub threshold would false-flag both. Median NN is the robust primary; hub is only comparable at equal shop counts (toowoomba 2/12 vs darwin 7/12 was fair because both had 12). The raw-vs-seated calibration point stands. METHODOLOGY, recorded because it nearly shipped: I picked Ballarat's span from an offline simulation that predicted 68 m / hub 5-8, and had already written those numbers into a config comment as fact. The real build returned 184 m / 4-8 — same counts, different sample. Never trust a re-implementation of the pipeline's own selection; run the pipeline. -> A (batched pins): ADD REAL_TOWN_GOLDENS['redhill_real']=0x6046700f and REAL_TOWN_GIG_GOLDENS['redhill_real']=0x673785ee (the headline town currently only WARNS as UNPINNED — ungated until you pin it). REMOVE ballarat_real 0xf0a9e8a1 / 0xf097c331 — dead config by your own R23 note. -> F: tools/qa/tour_shots.py still shoots town=toowoomba_real, which I deleted in R23 — the v4 tour breaks on a missing town. Your file, your call. A retirement isn't done when the cache is deleted; it's done when the consumers are swept. -> C: built to the runtime + G's real artifact per the ruling; LANE_C_PUB §7 line 271 still names the contradictory stock_shop_<id>_index.json. I reconcile when your corrected contract lands. Determinism receipt: the retirement forced a full 21-town rebuild from cached raw (no refetches) and every existing cache came back byte-identical — only ballarat, redhill and index.json moved. Pack: 21 real towns · 1196 shops · 299 heroes (22 rostered with G's newtown_godverse). Selfcheck ALL GREEN 149,696, synthetic golden 0x3fa36874 unchanged. qa.sh --strict --matrix: 7 passed / 0 failed, MATRIX GREEN 10x7. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| _godverse | ||
| packs | ||
| _audit_stats.json | ||
| _gig_results.json | ||
| _normalize_results.json | ||
| _peds_decim.json | ||
| _peds_merge.json | ||
| _props_results.json | ||
| _published.json | ||
| AUDIT.md | ||
| bake_lowpoly.py | ||
| batch_bake.json | ||
| batch_counter.json | ||
| batch_e2.json | ||
| batch_fal.json | ||
| batch_peds.json | ||
| batch_props.json | ||
| batch.json | ||
| build_manifest.py | ||
| build_stock_pack.py | ||
| build_towns.py | ||
| decimate_props.py | ||
| fbx_to_clip.py | ||
| fix_glb_textures.py | ||
| gen_audio.py | ||
| gen_props_fal.py | ||
| gen_props.py | ||
| gen_skins.py | ||
| gig_instruments.json | ||
| gig_norm_batch.json | ||
| glb_stat.py | ||
| godverse_stock.py | ||
| godverse_town.py | ||
| merge_ped.py | ||
| meshgod_batch.json | ||
| MESHGOD_BATCH.md | ||
| normalize.py | ||
| publish.py | ||
| README.md | ||
| SKIN_PROMPTS.md | ||
| stage_local_depot.py | ||
| validate_atlas.py | ||
| validate_manifest.py | ||
| validate_pack.py | ||
PROCITY pipeline (Lane E) — how to re-run everything
The asset pipeline: audit ultra's library → normalize the good bits to house GLB law → publish
to the 3GOD depot → ship web/assets/manifest.json (the contract Lanes B/C/D read) → fill facade
- prop gaps. Everything is optional at runtime — the game runs asset-free; this lane makes it gorgeous, it doesn't make it work.
Where things live
| what | where |
|---|---|
| ultra (library + Blender) | ssh johnking@100.91.239.7 — key auth, no coreutils/timeout, py3.9, Blender at /Applications/Blender.app/Contents/MacOS/Blender (not on PATH) |
| model library | ultra ~/Documents/3D=models/{shop-fittings,street-furniture,furniture,props-scenes} |
| 3GOD depot | https://digalot.fyi/3god — GET /a/<f>, GET /api/list, POST /api/upload?name=<f> |
| OpenRouter key (skins) | env OPENROUTER_KEY or ~/Documents/dealgod/lore/keys.env (never committed) |
| depot password (publish) | env GOD3_PW (never committed; ask John) — cookie = hmac(GOD3_PW,"3god-ok") |
| Ollama (free LLM) | http://100.89.131.57:11434 (qwen3) — for names/lore/cataloguing, unused so far |
| MODELBEAST (free local gen) | this box, ~/Documents/MODELBEAST + its own queue on :8777. flux_local = FLUX.2-klein-4B text→image on MPS (the facade skins — ungated, weights cached). sf3d/trellis_mac = image→GLB, local + PROVEN, weights already cached — trellis_mac (TRELLIS.2 1024_cascade) went 15/15 on thriftgod's Tier-1 props (Lane G/ultra, R1); sf3d = seconds-fast draft. Also live on the queue: bg_remove_local, mflux_image_edit, seedvr2_upscale. (The old "HF-gated / setup needed" note here was stale — flagged by ultra, corrected R22.) |
The steps (each is idempotent / resumable)
1. Audit — AUDIT.md (free)
scp pipeline/glb_stat.py johnking@100.91.239.7:/tmp/
ssh johnking@100.91.239.7 'cd ~/Documents/3D=models && python3 /tmp/glb_stat.py --json shop-fittings street-furniture furniture props-scenes' > pipeline/_audit_stats.json
glb_stat.py is a dependency-free GLB parser (tris, bbox, texture sizes, Draco flag). Findings +
licensing triage are written up in AUDIT.md.
2. Normalize — normalize.py → 16 GLBs + thumbs (free, runs on ultra)
scp pipeline/normalize.py pipeline/batch.json johnking@100.91.239.7:/tmp/
BL=/Applications/Blender.app/Contents/MacOS/Blender
ssh johnking@100.91.239.7 "\$BL --background --python /tmp/normalize.py -- /tmp/batch.json /tmp/procity_norm"
scp johnking@100.91.239.7:/tmp/procity_norm/_results.json pipeline/_normalize_results.json
scp 'johnking@100.91.239.7:/tmp/procity_norm/thumbs/*.png' web/assets/thumbs/
scp 'johnking@100.91.239.7:/tmp/procity_norm/*.glb' pipeline/_normalized/
Enforces the house GLB law (metres, +Y up, base origin, ≤5k tris, WebP ≤1024, no Draco), renders a
256px thumbnail each. Extends MESHGOD/scripts/finish_glb.py (solidify/decimate/bright/base-origin)
- adds texture-webp, height-normalize, outlier cleanup, per-asset up/yaw. Edit
batch.jsonto add assets; always eyeball the thumbnails — node-transform-distorted sources (see AUDIT rejects) look fine in-frame but have wrong footprints.
3. Manifest — build_manifest.py + validate_manifest.py (free)
python3 pipeline/build_manifest.py # → web/assets/manifest.json
python3 pipeline/validate_manifest.py # Lane F gate: 0 errors = green
python3 pipeline/validate_manifest.py --depot # stricter: require GLBs live on the depot
build_manifest.py derives skins from web/assets/gen/ + fittings from _normalize_results.json;
the curated facade→shop-type maps live at the top of that file (edit there when adding skins).
4. Publish — publish.py → 3GOD depot (the one outward-facing step)
python3 pipeline/publish.py --dry-run # free: list what would upload
export GOD3_PW=... # ask John
python3 pipeline/publish.py # POST each GLB + thumb to the depot (idempotent)
5. Facade skin gaps — gen_skins.py --local (DONE, free, on-device)
The 23-skin gap pack is already generated + harvested + mapped. To re-run or extend:
python3 pipeline/gen_skins.py --local --dry-run # free: list the pack, no GPU
python3 pipeline/gen_skins.py --local # MODELBEAST flux2-klein-4b on MPS → .genraw/ (~5.7s/img)
python3 pipeline/gen_skins.py --harvest # → web/assets/gen/facade-*.jpg, then re-run build_manifest.py
--local drives ~/Documents/MODELBEAST/server/operators/flux_local/run.py directly (no server,
no key, weights cached). It runs one GPU job at a time. Add skins by editing PROMPTS in
gen_skins.py and their shop-type map in build_manifest.py (FACADE_TYPES). Always eyeball
.genraw/ and delete rejects before --harvest. Cloud fallback (drop --local): OpenRouter
google/gemini-3.1-flash-image, ~$0.004/img, needs OPENROUTER_KEY. Manual flow: SKIN_PROMPTS.md.
6. Bespoke hero props — gen_props.py (MODELBEAST) (round-3: DONE, free, on-device)
The meshgod_batch.json gap list now goes through MODELBEAST instead of fal.ai — free, local:
python3 pipeline/gen_props.py --dry-run # list props + what's done
python3 pipeline/gen_props.py --concepts # flux_local concept images → .genprops/*.png (eyeball!)
python3 pipeline/gen_props.py --mesh --only glass-case # trellis_mac 1024_cascade image→GLB (~5-8 min)
python3 pipeline/gen_props.py --mesh # all props with a concept, no glb
# then normalize (uses local Blender), stage, catalogue:
BL=/Applications/Blender.app/Contents/MacOS/Blender
"$BL" --background --python pipeline/normalize.py -- pipeline/batch_props.json pipeline/_normalized web/assets/thumbs
cp pipeline/_normalized/_results.json pipeline/_props_results.json && python3 pipeline/build_manifest.py
5 shipped (arcade_cabinet, listening_booth, drinks_fridge, milkshake_mixer, novelty_record) + cash-register
reused from depot. magazine-rack → primitive (thin-wire gotcha); glass-case/bus-shelter → fal.ai
fallback (TRELLIS can't do transparent glass; sf3d --draft gives a featureless blob). TRELLIS meshes run
heavy (25–77k tris) — see AUDIT provenance. Eyeball every thumbnail; ≤2 local attempts then fal (John's go).
⚠️ Finishing a trellis_mac mesh — ultra's post-mortem (Lane G, R1) — folded in verbatim
⚠️ Do NOT run trellis_mac output through the MESHGOD finish farm — its fill_holes+decimate shreds 200k-tri marching-cubes meshes (proven on ultra, post-mortem in dealgod memory). The working pipeline: raw GLB →
npx @gltf-transform/cli optimize --compress draco --texture-compress webp --texture-size 1024 --simplify-error 0.01→ 0.4–1.8 MB at raw-quality look.
Credit: Lane G / ultra (G-progress.md), proven post-mortem — it cost them a shredded batch so it
shouldn't cost us one. One amendment, per Fable's Ruling 1 (docs/LANES/FABLE_TO_LANE_G.md):
NO-DRACO WINS — the depot speaks uncompressed GLB and PROCITY's loader stays decoder-free. So drop
--compress draco; everything else in ultra's command stands (WebP ≤1024 is already our law):
npx @gltf-transform/cli optimize IN.glb OUT.glb \
--texture-compress webp --texture-size 1024 --simplify-error 0.01 # NO --compress draco
Files land ~2–3× bigger than the Draco'd version and that's fine — the depot holds ONE law.
Note our own in-house paths remain valid for their own cases and are NOT the finish farm:
bake_lowpoly.py (voxel-remesh + bake, for shell-soup that must hit a low budget) and
decimate_props.py (collapse, detail-preserving — see the R13 instrument diet).
7. Local-depot mode — loaders.js ?localdepot=1 + stage_local_depot.py (round-3: DONE)
Lets Lanes B/C/F validate GLB loading with no network / before publish:
python3 pipeline/stage_local_depot.py # mirror _normalized/*.glb → web/assets/models/ (git-ignored)
# then load the game with ?localdepot=1 — depot:<file> resolves to assets/models/<file> instead of the CDN
8. Audio pack — gen_audio.py (round-11: DONE, free, on-device)
Tool choice (Fable asked E to pick + document): pure procedural synthesis in numpy, not a neural model. MODELBEAST has no audio operator, and for this pack synthesis wins outright — $0, no model download, deterministic/seeded (house law), perfectly seamless loops (neural gens don't loop), parody-safe by construction (every sample is an original oscillator/noise render — nothing to imitate), and precise ≤25 MB budget control. Pipeline: numpy synth → WAV → ffmpeg OGG/Opus (+ M4A/AAC).
PY=~/Documents/MODELBEAST/venvs/mflux/bin/python # numpy lives in the mflux venv
$PY pipeline/gen_audio.py --dry-run # list the 23-asset pack + budget
$PY pipeline/gen_audio.py # render all → web/assets/audio/*.ogg + *.m4a
$PY pipeline/gen_audio.py --only music # just the music beds
python3 pipeline/build_manifest.py # catalogue → manifest.json audio:{ambience,sfx,music}
23 assets, 8.1 MB shipped (opus+aac): 6 ambience beds (street day/night, rain, 3 roomtones), 13
sfx (doorbell, door, till, riffle, toast, tram bell/rumble, 6 footsteps), 4 instrumental 90s music
loops. Loudness: beds/music loudnorm -16 LUFS, sfx peak-normalised. Ear-ball = spectrogram sheet
docs/shots/laneE/audio_spectrograms.png + numeric loop-seam/peak check (I can't subjectively listen —
final ear-check is John's/F's). Add assets by editing PACK in gen_audio.py + the AUDIO_* maps in
build_manifest.py. WAV scratch pipeline/.genaudio/ is git-ignored; OGG/M4A ship committed.
9. Gig layer — round-12 (v3.0-alpha, ?gigs=1) (DONE, free, on-device)
Three asset streams for one-pub-end-to-end, all $0:
# instruments (5 GLBs): flux concept -> trellis -> normalize, via the --batch override
python3 pipeline/gen_props.py --batch pipeline/gig_instruments.json --concepts # eyeball first
python3 pipeline/gen_props.py --batch pipeline/gig_instruments.json --mesh # ~3-4 min each
BL=/Applications/Blender.app/Contents/MacOS/Blender
"$BL" --background --python pipeline/normalize.py -- pipeline/gig_norm_batch.json pipeline/_normalized web/assets/thumbs
# posters (4 skins): portrait flyers with a blank name-band (B/C overprint like signboards)
python3 pipeline/gen_skins.py --posters && python3 pipeline/gen_skins.py --harvest
# gig audio (3): pub-rock live bed + crowd-walla + applause — already in gen_audio.py PACK
~/Documents/MODELBEAST/venvs/mflux/bin/python pipeline/gen_audio.py --only music # etc.
python3 pipeline/build_manifest.py # catalogues skins.poster + audio gig assets + fittings
Instruments: amp/guitar/bass reconstruct well; thin objects (mic-stand pole, drum hardware) may fall
to D's primitive (round-3 thin-wire lesson — spec allows "primitives when absent"). Posters carry
nameZone [x,y,w,h] in skins.poster. Gig audio is flagged gig:true in manifest.audio.
Cost summary
| step | cost |
|---|---|
| audit, normalize, manifest, validate, publish | free (compute + bandwidth) |
| facade skin gap-fill (23 imgs) | free — done on-device (MODELBEAST flux_local); cloud fallback ~9¢ |
| hero props (round-3) | free — 5 via MODELBEAST trellis_mac; glass-case/bus-shelter on fal.ai list (≤$2.70, John's go) |
| audio pack (round-11, 23 assets) | free — 100% procedural synthesis (numpy → ffmpeg), $0, no model |
Artifacts (not committed — regenerated from the steps)
pipeline/_normalized/ (staging GLBs, live on depot), pipeline/.genraw/ (raw skin gen).
Committed: the scripts, *.md, batch.json, meshgod_batch.json, _audit_stats.json,
_normalize_results.json, web/assets/manifest.json, web/assets/thumbs/, docs/shots/laneE/.