PROCITY/tools/qa.sh
m3ultra d14cf7fa6e Lane F R41 §41.6: 18 gates green — the deny-list, the wardrobe switch, and no byte-hash goldens
qa.sh --strict: 18 passed · 0 failed · 0 warn · 0 skipped. selfcheck 157,647/157,647,
fingerprint 0x5f76e76 unmoved. No tag — John's playtest session rules the epoch.

THE FIVE ASKS. [C] wardrobe wired via exported WARDROBE_BASE + stockBaseFor(shop), preloaded
inside the existing STOCK_REAL gate: ON under ?stock=real (52 pack items, 69 garment ids on
rendered meshes, 1 atlas, exactly 2 requests) and CHEAPER (59->55 draws, 111->100 textures);
OFF is what ships by default, A/B'd against a reverted control tree — default arm identical
on every counter (160 URLs same hash, 131/18,018 street, 59/28,505 interior). [D] three gates
wired (r41_shots.py writes to TMPDIR so gate runs never rewrite committed shots). [E]
clips_verify wired — 46 clips / 6 groups / 3,498,124 B.

[B's bookmark finding] CONFIRMED AND WIDER: street_noon, shopfront_detail, crossroads_busy
AND market_square all give 3 distinct hashes across 3 boots; only night_neon is stable.
RULING: no gate pins a screenshot byte-hash on any bookmark — everything R41 asserts is a
counter. Filed to B: one await document.fonts.ready in buildings.js.

[C's drawSweep] CONFIRMED INDEPENDENTLY with F's own sweep (never calls C's): GLB-on worst
123 @ dept/auto. The phantom control (one stale room left in scene) reads 197 — +74 on every
reading — decomposing R39/R40's '188' as 116 real + ~72 phantom.

NEW GATES, controls demonstrated. r41_denylist.mjs (ruling 3): 738 files / 197.3 MiB scanned
by path AND bytes, 7 banned names, 0 hits; the control plants a banned manifest row every run
-> RED on 2 names -> removed -> GREEN. r41_integration.py: wardrobe both arms · interior <=350
on both instruments · ?noassets=1 zero across six fetch classes over a 3-shop walk (control
fetches 5/6) · ?clips=0 142 URLs and ?classic=1 125 URLs with zero R41 cargo.

BUDGETS. Street: noon 282 · NIGHT 291/120,093 — margin 9, INTACT · classic 269 byte-exact ·
?clips=0 NIGHT 291, delta 0. No lane spent a street draw. Interior: true pre-R41 116, R41 123
@ dept/auto, margin 227; over 33 real shops worst 110; pub 49 quiet / 77 gig night.

FILED BACK: flags_check's _enter_record_shop went RED on 8 smokes because §41.4's bins put
every one of the first six record-shop counters inside 2.6 m — widened to the whole open set,
verified not a game defect. smoke_djdance read _actions[0] and D's per-instance clip swapping
made the mixer multi-action, so a healthy dancer was called a stall — now follows the clip by
name. D's r41_shots.py browse arm is deterministically red here (re-enters the winner after a
~3 s/candidate scan, by which time the occupant has left) — wired WARN-level with the reason
at the call site, fix handed to D. F's own r39_transmission.py had an unmeasured 30 s goto
default that flaked once under a full strict run; pinned to 90 s.

11 proof frames in docs/shots/laneF_r41/ with sidecars: street_postures (5 rigs, 3 clips
playing) against its ?clips=0 control (1 clip) · street_lean · browse_interior · pub_furnished
+ the honest kit before/after pair · record_dj_booth · opshop_wardrobe on/off · credits_panel
with ODbL on screen. Every humanoid frame carries its R10 human-sized line.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-07 18:11:58 +10:00

269 lines
17 KiB
Bash
Executable File
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

#!/usr/bin/env bash
# PROCITY Lane F — the QA gate runner (CITY_SPEC / LANE_F acceptance).
#
# tools/qa.sh run every gate whose inputs have landed; skip the rest
# tools/qa.sh --strict also FAIL on any still-pending lane deliverable (use at v1 sign-off)
#
# Design: F runs before AE finish. A gate whose target file does not exist yet is reported
# SKIPPED (yellow), not failed — so this is safe to run continuously as lanes land. Once every
# lane is in, `--strict` turns the skips into hard failures: that is the v1 readiness gate.
#
# Gates:
# 1. scaffold_check.mjs — scaffold + PRNG determinism law + lane-readiness matrix (F-owned)
# 2. citygen selfcheck — node web/js/citygen/selfcheck.js (Lane A ships it)
# 3. manifest validator — python3 pipeline/validate_manifest.py (Lane E ships it)
# 8. the R39 (v9 wave 1) gates — tools/qa/r39_address.mjs · r39_transmission.py · r39_runtime.py
# 9. the R40 gates — tools/qa/door_footpath_check.mjs · r40_lane_b.py · r40_playtest.py
# 10. the R41 (THE LIBRARY LANDS) gates — pipeline/clips_verify.py · tools/qa/r41_postures.mjs ·
# r41_denylist.mjs · r41_citizens.py · r41_shots.py · r41_integration.py
#
# Browser-driven gates (determinism-PNG, 10-min soak, budget-HUD, ?noassets run) need the running
# game; they live in tools/soak.md + tools/shots.md and run once Lane B's index.html lands.
set -uo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$ROOT"
STRICT=0
[ "${1:-}" = "--strict" ] && STRICT=1
pass=0; failn=0; skip=0
c_red=$'\033[31m'; c_grn=$'\033[32m'; c_yel=$'\033[33m'; c_bold=$'\033[1m'; c_off=$'\033[0m'
hr() { printf '%s\n' "────────────────────────────────────────────────────────"; }
run_gate() { # run_gate "name" cmd...
local name="$1"; shift
printf '%s▶ %s%s\n' "$c_bold" "$name" "$c_off"
if "$@"; then printf '%s ✓ PASS%s %s\n\n' "$c_grn" "$c_off" "$name"; pass=$((pass+1));
else printf '%s ✗ FAIL%s %s\n\n' "$c_red" "$c_off" "$name"; failn=$((failn+1)); fi
}
skip_gate() { # skip_gate "name" "reason"
printf '%s▶ %s%s\n' "$c_bold" "$1" "$c_off"
printf '%s ⊘ SKIP%s %s\n\n' "$c_yel" "$c_off" "$2"
skip=$((skip+1))
}
warnn=0
soft_skip() { # non-blocking skip for a warn-level gate — does NOT trip --strict readiness
printf '%s▶ %s%s\n' "$c_bold" "$1" "$c_off"
printf '%s ⊘ SKIP%s %s (warn-level, non-blocking)\n\n' "$c_yel" "$c_off" "$2"
}
warn_gate() { # warn_gate "name" cmd... — runs, but a failure is a WARN not a FAIL (round-6 policy)
local name="$1"; shift
printf '%s▶ %s%s (warn-level)\n' "$c_bold" "$name" "$c_off"
if "$@"; then printf '%s ✓ PASS%s %s\n\n' "$c_grn" "$c_off" "$name"; pass=$((pass+1));
else printf '%s ! WARN%s %s (non-blocking this round — strict in r7)\n\n' "$c_yel" "$c_off" "$name"; warnn=$((warnn+1)); fi
}
printf '%sPROCITY QA GATES%s (%s)\n' "$c_bold" "$c_off" "$([ $STRICT = 1 ] && echo strict || echo lenient)"
hr
# ── Gate 1: scaffold + determinism + readiness (always) ──────────────────────
run_gate "scaffold_check (scaffold · PRNG determinism · readiness matrix)" \
node tools/qa/scaffold_check.mjs
# ── Gate 1b: cross-lane consistency (plan ↔ registry ↔ assets ↔ manifest) ────
# Self-skips (exit 0) until Lane A's plan.js + registry.js exist.
run_gate "consistency (plan ↔ registry ↔ assets ↔ manifest)" \
node tools/qa/consistency_check.mjs
# ── Gate 2: Lane A citygen selfcheck ─────────────────────────────────────────
if [ -f web/js/citygen/selfcheck.js ]; then
run_gate "citygen selfcheck (determinism · <100ms · lots/shops integrity)" \
node web/js/citygen/selfcheck.js
else
skip_gate "citygen selfcheck" "web/js/citygen/selfcheck.js not landed yet (Lane A)"
fi
# ── Gate 3: Lane E manifest validator ────────────────────────────────────────
# [Lane F R3] --depot: the 16 GLBs are published + live on the 3GOD depot (Fable, 2026-07-14), so the
# gate now REQUIRES them reachable (was a soft local-only check). Override the endpoint with GOD3_DEPOT
# (tailnet path) if digalot.fyi is unreachable; the validator falls back to the manifest's depot field.
if [ -f pipeline/validate_manifest.py ]; then
run_gate "manifest validator (files exist · footprints sane · GLBs live on depot · JSON parses)" \
python3 pipeline/validate_manifest.py --depot
else
skip_gate "manifest validator" "pipeline/validate_manifest.py not landed yet (Lane E)"
fi
# ── Gate 4 (warn-level, round 6): v2 flag enforcement harness ────────────────
# Flags-off regression (prime-law) + per-flag + all-on-combo smokes. Needs the Playwright venv +
# a browser (self-contained server), auto-skips where absent. STRICT as of R7 (F2): a harness FAIL
# now fails qa (flags-off regression + the four existing flags + all-on combo). New flags (stock,
# weather) report at warn inside the harness, so they never trip this gate until they graduate.
# Skip explicitly with --no-flags (fast determinism-only runs).
FLAGS_SKIP=0; for a in "$@"; do [ "$a" = "--no-flags" ] && FLAGS_SKIP=1; done
if [ "$FLAGS_SKIP" = 1 ]; then
soft_skip "v2 flags harness" "--no-flags"
elif [ -x tools/.venv/bin/python ] && tools/.venv/bin/python -c "import playwright" 2>/dev/null; then
run_gate "flags harness (classic regression · default-boot gate · per-flag · smokes · STRICT · R16 flip)" \
tools/.venv/bin/python tools/flags_check.py
else
soft_skip "v2 flags harness" "Playwright venv absent (python3 -m venv tools/.venv && …/pip install playwright)"
fi
# ── Gate 5 (Lane D, round 10): interior figure-scale regression guard ────────
# Enters a sample of open shops and asserts every interior rig figure's crown is human-sized
# [1.4,2.0] m AND under the room ceiling — so the R9 giant blocker (buildFigure normalised off
# head-above-hips → ~2× too tall) can never pass a gate again. run_gate ⇒ a giant FAILS qa.
# Needs the Playwright venv (auto-skips where absent, like the flags harness). Wired per ROUND10 §D.
SCALE_SKIP=0; for a in "$@"; do [ "$a" = "--no-scale" ] && SCALE_SKIP=1; done
if [ "$SCALE_SKIP" = 1 ]; then
soft_skip "interior figure-scale gate" "--no-scale"
elif [ -x tools/.venv/bin/python ] && tools/.venv/bin/python -c "import playwright" 2>/dev/null; then
run_gate "interior figure-scale (no giants: crown ∈ [1.4,2.0] m, under ceiling · Lane D R10)" \
tools/.venv/bin/python tools/qa/interior_scale_check.py
else
soft_skip "interior figure-scale gate" "Playwright venv absent (python3 -m venv tools/.venv && …/pip install playwright)"
fi
# ── Gate 6 (Lane F, round 15): full-week gig soak — OPT-IN (--soak), NON-STRICT ─
# The v3.0 release soak: 7 nights × every playing venue in one context — latch state per plan.gigs,
# cover stamp per venue per night, roster clears per night, leak geo/tex to a warmed baseline across
# ≥20 enter/exit cycles. Slow (~90s) + diagnostic, so it is OPT-IN and warn-level (never blocks a tag);
# the core 6 gates stay a fast strict run. Wired per ROUND15 §Lane F.3.
SOAK_RUN=0; for a in "$@"; do [ "$a" = "--soak" ] && SOAK_RUN=1; done
if [ "$SOAK_RUN" = 1 ]; then
if [ -x tools/.venv/bin/python ] && tools/.venv/bin/python -c "import playwright" 2>/dev/null; then
warn_gate "full-week gig soak (7 nights · cover/roster per night · leak-free · Lane F R15)" \
tools/.venv/bin/python tools/qa/week_soak.py
else
soft_skip "full-week gig soak" "Playwright venv absent"
fi
fi
# ── Gate 7 (Lane F, round 17): town-matrix smoke — OPT-IN (--matrix), NON-STRICT ─
# The v4 real-map scout evidence: boot / determinism / budget / district / noassets across the full town
# matrix — synthetic hero seeds + the 3 osm fixtures + Lane E's 5 real AU town caches. One town × gate
# table. Slow (~20 boots) + diagnostic, so OPT-IN + warn-level. Wired per ROUND17 §Lane F (ledger #7).
MATRIX_RUN=0; for a in "$@"; do [ "$a" = "--matrix" ] && MATRIX_RUN=1; done
if [ "$MATRIX_RUN" = 1 ]; then
if [ -x tools/.venv/bin/python ] && tools/.venv/bin/python -c "import playwright" 2>/dev/null; then
warn_gate "town-matrix (synthetic + fixtures + real caches × 5 gates · Lane F R17)" \
tools/.venv/bin/python tools/qa/town_matrix.py
else
soft_skip "town-matrix" "Playwright venv absent"
fi
fi
# ── Gate 8 (Lane F, round 39): the v9 wave-1 gates ───────────────────────────
# 39.1 THE ADDRESS LAYER (node, no browser) · 39.4 THE TRANSMISSION PRE-PASS (validator control +
# the interior draw sweep against a fabricated glass asset) · 39.2 THE FOG / THE SIGN / THE RUMMAGE
# BIN (browser). Each ships its falsifiability control in the same run — see LANE_F_NOTES §39.
# --no-r39 skips the browser half on a fast determinism-only run; the node half always runs.
run_gate "R39 address layer (>=97% corpus · degrade-to-district · the wrong-shift red arm)" \
node tools/qa/r39_address.mjs
R39_SKIP=0; for a in "$@"; do [ "$a" = "--no-r39" ] && R39_SKIP=1; done
if [ "$R39_SKIP" = 1 ]; then
soft_skip "R39 transmission + runtime gates" "--no-r39"
elif [ -x tools/.venv/bin/python ] && tools/.venv/bin/python -c "import playwright" 2>/dev/null; then
run_gate "R39 transmission pre-pass (validator hard-fail control · interior sweep vs a glass asset)" \
tools/.venv/bin/python tools/qa/r39_transmission.py --validator
run_gate "R39 runtime (THE SIGN · THE FOG both arms · THE RUMMAGE BIN, contents unarmed)" \
tools/.venv/bin/python tools/qa/r39_runtime.py
else
soft_skip "R39 transmission + runtime gates" "Playwright venv absent"
fi
# ── Gate 9 (Lane F, round 40): the R40 gates ─────────────────────────────────
# [Lane F R40] Three gates, wired per each lane's §40 ask:
# • D §40.5 — door→footpath (node, ~2s, zero deps): walkable-front ≥95% on all four towns, the
# back-point control non-vacuous, the kerb clamp FIX-ONLY. rc 0/1.
# • B §40.3 — the ?r= budget law (default 300 / diagnostic 420 + warn, measured stepwalk ≤ own
# ceiling), classic's town selector (RULED R40), fog signage both arms. Own no-store server.
# • F §40.6 — the playtest harness: off-boot byte-identical (no fetch, no DOM, no storage), the
# F8 note round-trip (drop → export → schema-validate → reload persists → corrupt-reject), and
# ?bugtour=1 advancing through every stop with zero console errors.
run_gate "R40 door→footpath (D §40.5: walkable ≥95% · back-point control · fix-only clamp)" \
node tools/qa/door_footpath_check.mjs
R40_SKIP=0; for a in "$@"; do [ "$a" = "--no-r40" ] && R40_SKIP=1; done
if [ "$R40_SKIP" = 1 ]; then
soft_skip "R40 browser gates (budget law · selector ruling · playtest harness)" "--no-r40"
elif [ -x tools/.venv/bin/python ] && tools/.venv/bin/python -c "import playwright" 2>/dev/null; then
run_gate "R40 ?r= budget law + classic selector + fog signage (B §40.3, both arms each)" \
tools/.venv/bin/python tools/qa/r40_lane_b.py
run_gate "R40 playtest harness (F §40.6: off byte-identical · F8 round-trip · bugtour clean)" \
tools/.venv/bin/python tools/qa/r40_playtest.py
else
soft_skip "R40 browser gates (budget law · selector ruling · playtest harness)" "Playwright venv absent"
fi
# ── Gate 10 (Lane F, round 41): THE LIBRARY LANDS ────────────────────────────
# [Lane F R41 §41.6] Six gates, wired per each lane's §41 ask. The node/stdlib half always runs
# (fast, zero deps); the browser half rides the Playwright venv like every other class here.
#
# • E §41.1 — pipeline/clips_verify.py: CLIP-MANIFEST INTEGRITY. Every clip id in
# web/assets/motion_manifest.json resolves to a named animation in its group GLB, sizes match,
# all six groups are zero-draw and Draco-free, all six carry the identical 66-node skeleton,
# every channel targets a real node, every declared duration equals the keyframes in the bytes.
# Ships with the library, so a renamed clip or a re-packed group fails HERE, not in the game.
# • D §41.3 — r41_postures.mjs: the posture table resolves, is deterministic (two module
# instances byte-equal), stream-isolated from the 12 pre-R41 keys, spread across all 10 idles,
# and loop-safe. Five controls in the same run.
# • F §41.6 — r41_denylist.mjs: RULING 3 AS REPO LAW. Nothing under web/ may reference OR CONTAIN
# a banned name — paths AND bytes, binaries included (a GLB carries its source node names). The
# control plants a real banned reference, asserts RED, removes it, asserts GREEN: this gate can
# never go quietly vacuous.
# • D §41.3 — r41_citizens.py: 7 browser arms (boot ledger · lazy loading · determinism incl. a
# 2 s forced clip stall · the draw table both arms · ?noassets/?classic · bench binding + its
# 2 m-offset control · leak · gig widening).
# • D §41.3 — r41_shots.py: the acceptance shots, camera chosen by measurement + occlusion
# raycast, every figure measured for stature (the R10 no-giants line). Written to a scratch
# outdir by default so a gate run never rewrites the round's committed shots; set
# PROCITY_SHOT_OUT=docs/shots/laneD to regenerate them deliberately. WARN-LEVEL — see the
# measured reason at the call site (its browse arm can lose its browser to the sim mid-scan).
# • F §41.6 — r41_integration.py: the seams F wired and the budgets F answers for — the wardrobe
# both arms (ON resolves the pack, OFF is the control and is what ships default), the interior
# ≤350 law on F'S OWN sweep plus the PHANTOM control that reproduces the pre-R41 instrument bug,
# ?noassets=1 clean over a three-shop walk, ?clips=0 / ?classic=1 carrying no R41 cargo, and the
# deny-list checked again from the network side.
if [ -f pipeline/clips_verify.py ]; then
run_gate "R41 clip-manifest integrity (E §41.1: 46 clips resolve · zero-draw · one skeleton · durations in the bytes)" \
python3 pipeline/clips_verify.py
else
skip_gate "R41 clip-manifest integrity" "pipeline/clips_verify.py not landed yet (Lane E)"
fi
run_gate "R41 postures (D §41.3: manifest resolution · determinism · stream isolation · pool spread · loop safety)" \
node tools/qa/r41_postures.mjs
run_gate "R41 deny-list (F §41.6 / ruling 3: web/ clean by path AND bytes · plant-and-remove control)" \
node tools/qa/r41_denylist.mjs
R41_SKIP=0; for a in "$@"; do [ "$a" = "--no-r41" ] && R41_SKIP=1; done
if [ "$R41_SKIP" = 1 ]; then
soft_skip "R41 browser gates (citizens · acceptance shots · integration)" "--no-r41"
elif [ -x tools/.venv/bin/python ] && tools/.venv/bin/python -c "import playwright" 2>/dev/null; then
run_gate "R41 citizens (D §41.3: boot ledger · lazy · determinism under a 2s stall · bench binding + control · leak)" \
tools/.venv/bin/python tools/qa/r41_citizens.py
# WARN-LEVEL, and the reason is measured, not a shrug (R6 policy for a gate that can go red for a
# reason unrelated to the property it asserts). Its street arm is solid — 8 near-tier rigs, 3 distinct
# clips, every stature clean, reproducible run to run. Its BROWSE arm is not: it scans up to 8 shops
# for browsers, keeps the best, then RE-ENTERS the winner — and a patronage occupant is free to walk
# out during the ~3 s per candidate that scan costs, so the re-entry spawns 0 browsers and the arm
# fails on an empty room. Measured on this tree: 3 consecutive standalone runs all land on `T & R
# Books` with figures 1 (keeper only), byte-identical (34 draws / 36,751 tris), while the same gate
# passed with figures 2 earlier the same day under different machine load. Nothing in R41's shipped
# code moved between those runs. FILED TO D (F-progress §Round 41): re-check `k.browse` after the
# re-entry and re-pick, or keep the winner entered instead of re-entering it.
warn_gate "R41 acceptance shots (D §41.3: measured camera · occlusion raycast · R10 stature line per figure)" \
tools/.venv/bin/python tools/qa/r41_shots.py --outdir "${PROCITY_SHOT_OUT:-${TMPDIR:-/tmp}/procity-r41-shots}"
run_gate "R41 integration (F §41.6: wardrobe both arms · interior ≤350 on F's sweep + phantom control · noassets · clip gates · deny-list live)" \
tools/.venv/bin/python tools/qa/r41_integration.py
else
soft_skip "R41 browser gates (citizens · acceptance shots · integration)" "Playwright venv absent"
fi
# ── Summary ──────────────────────────────────────────────────────────────────
hr
printf '%sSUMMARY%s %s%d passed%s · %s%d failed%s · %s%d warn%s · %s%d skipped%s\n' \
"$c_bold" "$c_off" "$c_grn" "$pass" "$c_off" "$c_red" "$failn" "$c_off" "$c_yel" "$warnn" "$c_off" "$c_yel" "$skip" "$c_off"
if [ "$failn" -gt 0 ]; then
printf '%s● QA RED%s — fix the failing gate(s) above.\n' "$c_red" "$c_off"; exit 1
fi
if [ "$STRICT" = 1 ] && [ "$skip" -gt 0 ]; then
printf '%s● QA NOT READY%s — %d gate(s) still pending; not v1 yet.\n' "$c_yel" "$c_off" "$skip"; exit 2
fi
printf '%s● QA GREEN%s — every landed gate passed.\n' "$c_grn" "$c_off"; exit 0