076b972f71
FIVE GATES, FIVE CONTROLS, each demonstrated in the same run. New F-owned tools wired into qa.sh as Gate 8: r39_address.mjs · r39_transmission.py · r39_runtime.py · r39_shots.py (--no-r39 skips the browser half). Lane A's selfcheck reproduces exactly: 157,647/157,647 ALL GREEN, fingerprint 0x5f76e76 — ZERO GOLDENS MOVED, in a round that added an address layer, a fog ledger, a spawn move and a new interior fitting. THE ROUND'S BIGGEST NUMBER IS NOT THE ONE THE LEDGER ASKED FOR. Two port-isolated no-store roots, the same inode for every byte except one file — the shipped bookshelf GLB vs a copy with transmissionFactor:1 re-inserted (BIN sha1 IDENTICAL, only the JSON chunk moved) — both through interior_test.html's own drawSweep: GLB-on worst 329 (opshop/hall) -> 188 (dept/auto) -141 GLB-off worst 162 (opshop/hall) -> 162 (opshop/hall) 0 — the control's own control margin vs <=350 21 -> 162 +141 opshop 329->174 (-47%) · book 247->125 (-49%) · every other type 0, exactly the shape Lane C predicted from an isolated room, at sweep scale, by a harness C did not write. The worst interior room in the game is now dept/auto — a room with no bookshelf in it. The default boot fetches these GLBs from the public depot, so all four R39 GLBs were fetched off digalot.fyi and scanned: sha1 identical to the local tree, 0 transmissive materials, 4/4. Binding fact #3's third and final value. 39.1 ADDRESS — GREEN. 1,192/1,219 shops (97.8%), 23/23 'ways', 857/857 membership agree / 0 wrong (and the control is asserted to have CHECKED 857 edges). No cache -> supplier 'district', 0 names, still labels 72/72 from district+block. RED ARM: a 3 m shift error RAISES adelaide 89 -> 98 resolved shops (Lane A's number to the digit) while INVENTING 9 names; stats().shiftCheck reads DISAGREE on all 72 perturbations. And the honest limit: at 3 m the membership control reads 0 disagreements, because those shops front ways OSM leaves unnamed — only the shift cross-derivation catches it. 39.2 FOG — GREEN, both arms on one boot. HIDES 4/72 at spawn, 0 of 61 distant shops known. REVEALS 2,125 m -> 58 shops. FRONT TEST: probe from 5 m BEHIND each of 59 unknown shopfronts, 0 learned. +0 DRAWS AND +0 TRIS, four freshly launched browsers, both orders, drained, 3 reads each, with the arms asserted distinguishable first. 8/8 corrupt saves rejected, state untouched. Null store on classic/game=0/fog=0; game=0 and fog=0 pixel-identical, default differs. 39.3 THE SIGN — GREEN. Shops in front of the player at spawn: 4/4 · 4/4 · 4/4 · 1/1, and ZERO on all four with the pre-R39 sign re-derived on the same tree. The live spawn is asserted equal to the +sin derivation, so the shell is what is scored. Re-baselined WITHOUT falsifying a release record: the v7 tour and v7-beta frames are kept as history; docs/shots/laneF_r39/ ships the new pair with its own BEFORE. The R35 gate itself is pose-independent and was green either side. 39.4 RUMMAGE BIN — GREEN. 120 rooms: tub 60/60 both arms, kind:'bin' 0/60 default vs 60/60 armed, 0 carves, 60/60 deterministic, reachable 0.80-1.03 m by a flood fill written here. THE CONTENTS DID NOT SHIP: on the default street boot the tub is kind:'prop', 0 dig targets, E opens nothing. 39.5 TRANSMISSION VALIDATOR — GREEN and it FIRES: hard-fails rc 1 on the fabricated glass asset AND passes on the shipped tree. THREE CORRECTIONS, ALL MEASURED. A's "a 30 m shift returns 0 street names" — it returns 40/12/47 wrong ones; a big shift lands you on the next street, it does not degrade to null. B's "a fresh export has no `known` key" — true of ?fog=0, no longer of a default boot, because B's own +sin fix means you spawn in front of four shopfronts. And my own: I first aimed the transmission control at web/assets/models/ when validate_manifest.py reads pipeline/_normalized/, and nearly filed a defect against Lane E on the strength of it. Aim the control at the code path, not the plausible path. HELD -> LANE E: check_transmission() skips any GLB with no local _normalized copy and never scans web/assets/models/. Demonstrated with a live control — remove the local copy and the same glass asset passes rc 0. A depot-only transmissive GLB (the exact shape of longbench and streetlight) would ship unseen. No tag. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
176 lines
10 KiB
Bash
Executable File
176 lines
10 KiB
Bash
Executable File
#!/usr/bin/env bash
|
||
# PROCITY Lane F — the QA gate runner (CITY_SPEC / LANE_F acceptance).
|
||
#
|
||
# tools/qa.sh run every gate whose inputs have landed; skip the rest
|
||
# tools/qa.sh --strict also FAIL on any still-pending lane deliverable (use at v1 sign-off)
|
||
#
|
||
# Design: F runs before A–E finish. A gate whose target file does not exist yet is reported
|
||
# SKIPPED (yellow), not failed — so this is safe to run continuously as lanes land. Once every
|
||
# lane is in, `--strict` turns the skips into hard failures: that is the v1 readiness gate.
|
||
#
|
||
# Gates:
|
||
# 1. scaffold_check.mjs — scaffold + PRNG determinism law + lane-readiness matrix (F-owned)
|
||
# 2. citygen selfcheck — node web/js/citygen/selfcheck.js (Lane A ships it)
|
||
# 3. manifest validator — python3 pipeline/validate_manifest.py (Lane E ships it)
|
||
# 8. the R39 (v9 wave 1) gates — tools/qa/r39_address.mjs · r39_transmission.py · r39_runtime.py
|
||
#
|
||
# Browser-driven gates (determinism-PNG, 10-min soak, budget-HUD, ?noassets run) need the running
|
||
# game; they live in tools/soak.md + tools/shots.md and run once Lane B's index.html lands.
|
||
|
||
set -uo pipefail
|
||
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||
cd "$ROOT"
|
||
|
||
STRICT=0
|
||
[ "${1:-}" = "--strict" ] && STRICT=1
|
||
|
||
pass=0; failn=0; skip=0
|
||
c_red=$'\033[31m'; c_grn=$'\033[32m'; c_yel=$'\033[33m'; c_bold=$'\033[1m'; c_off=$'\033[0m'
|
||
hr() { printf '%s\n' "────────────────────────────────────────────────────────"; }
|
||
run_gate() { # run_gate "name" cmd...
|
||
local name="$1"; shift
|
||
printf '%s▶ %s%s\n' "$c_bold" "$name" "$c_off"
|
||
if "$@"; then printf '%s ✓ PASS%s %s\n\n' "$c_grn" "$c_off" "$name"; pass=$((pass+1));
|
||
else printf '%s ✗ FAIL%s %s\n\n' "$c_red" "$c_off" "$name"; failn=$((failn+1)); fi
|
||
}
|
||
skip_gate() { # skip_gate "name" "reason"
|
||
printf '%s▶ %s%s\n' "$c_bold" "$1" "$c_off"
|
||
printf '%s ⊘ SKIP%s %s\n\n' "$c_yel" "$c_off" "$2"
|
||
skip=$((skip+1))
|
||
}
|
||
warnn=0
|
||
soft_skip() { # non-blocking skip for a warn-level gate — does NOT trip --strict readiness
|
||
printf '%s▶ %s%s\n' "$c_bold" "$1" "$c_off"
|
||
printf '%s ⊘ SKIP%s %s (warn-level, non-blocking)\n\n' "$c_yel" "$c_off" "$2"
|
||
}
|
||
warn_gate() { # warn_gate "name" cmd... — runs, but a failure is a WARN not a FAIL (round-6 policy)
|
||
local name="$1"; shift
|
||
printf '%s▶ %s%s (warn-level)\n' "$c_bold" "$name" "$c_off"
|
||
if "$@"; then printf '%s ✓ PASS%s %s\n\n' "$c_grn" "$c_off" "$name"; pass=$((pass+1));
|
||
else printf '%s ! WARN%s %s (non-blocking this round — strict in r7)\n\n' "$c_yel" "$c_off" "$name"; warnn=$((warnn+1)); fi
|
||
}
|
||
|
||
printf '%sPROCITY QA GATES%s (%s)\n' "$c_bold" "$c_off" "$([ $STRICT = 1 ] && echo strict || echo lenient)"
|
||
hr
|
||
|
||
# ── Gate 1: scaffold + determinism + readiness (always) ──────────────────────
|
||
run_gate "scaffold_check (scaffold · PRNG determinism · readiness matrix)" \
|
||
node tools/qa/scaffold_check.mjs
|
||
|
||
# ── Gate 1b: cross-lane consistency (plan ↔ registry ↔ assets ↔ manifest) ────
|
||
# Self-skips (exit 0) until Lane A's plan.js + registry.js exist.
|
||
run_gate "consistency (plan ↔ registry ↔ assets ↔ manifest)" \
|
||
node tools/qa/consistency_check.mjs
|
||
|
||
# ── Gate 2: Lane A citygen selfcheck ─────────────────────────────────────────
|
||
if [ -f web/js/citygen/selfcheck.js ]; then
|
||
run_gate "citygen selfcheck (determinism · <100ms · lots/shops integrity)" \
|
||
node web/js/citygen/selfcheck.js
|
||
else
|
||
skip_gate "citygen selfcheck" "web/js/citygen/selfcheck.js not landed yet (Lane A)"
|
||
fi
|
||
|
||
# ── Gate 3: Lane E manifest validator ────────────────────────────────────────
|
||
# [Lane F R3] --depot: the 16 GLBs are published + live on the 3GOD depot (Fable, 2026-07-14), so the
|
||
# gate now REQUIRES them reachable (was a soft local-only check). Override the endpoint with GOD3_DEPOT
|
||
# (tailnet path) if digalot.fyi is unreachable; the validator falls back to the manifest's depot field.
|
||
if [ -f pipeline/validate_manifest.py ]; then
|
||
run_gate "manifest validator (files exist · footprints sane · GLBs live on depot · JSON parses)" \
|
||
python3 pipeline/validate_manifest.py --depot
|
||
else
|
||
skip_gate "manifest validator" "pipeline/validate_manifest.py not landed yet (Lane E)"
|
||
fi
|
||
|
||
# ── Gate 4 (warn-level, round 6): v2 flag enforcement harness ────────────────
|
||
# Flags-off regression (prime-law) + per-flag + all-on-combo smokes. Needs the Playwright venv +
|
||
# a browser (self-contained server), auto-skips where absent. STRICT as of R7 (F2): a harness FAIL
|
||
# now fails qa (flags-off regression + the four existing flags + all-on combo). New flags (stock,
|
||
# weather) report at warn inside the harness, so they never trip this gate until they graduate.
|
||
# Skip explicitly with --no-flags (fast determinism-only runs).
|
||
FLAGS_SKIP=0; for a in "$@"; do [ "$a" = "--no-flags" ] && FLAGS_SKIP=1; done
|
||
if [ "$FLAGS_SKIP" = 1 ]; then
|
||
soft_skip "v2 flags harness" "--no-flags"
|
||
elif [ -x tools/.venv/bin/python ] && tools/.venv/bin/python -c "import playwright" 2>/dev/null; then
|
||
run_gate "flags harness (classic regression · default-boot gate · per-flag · smokes · STRICT · R16 flip)" \
|
||
tools/.venv/bin/python tools/flags_check.py
|
||
else
|
||
soft_skip "v2 flags harness" "Playwright venv absent (python3 -m venv tools/.venv && …/pip install playwright)"
|
||
fi
|
||
|
||
# ── Gate 5 (Lane D, round 10): interior figure-scale regression guard ────────
|
||
# Enters a sample of open shops and asserts every interior rig figure's crown is human-sized
|
||
# [1.4,2.0] m AND under the room ceiling — so the R9 giant blocker (buildFigure normalised off
|
||
# head-above-hips → ~2× too tall) can never pass a gate again. run_gate ⇒ a giant FAILS qa.
|
||
# Needs the Playwright venv (auto-skips where absent, like the flags harness). Wired per ROUND10 §D.
|
||
SCALE_SKIP=0; for a in "$@"; do [ "$a" = "--no-scale" ] && SCALE_SKIP=1; done
|
||
if [ "$SCALE_SKIP" = 1 ]; then
|
||
soft_skip "interior figure-scale gate" "--no-scale"
|
||
elif [ -x tools/.venv/bin/python ] && tools/.venv/bin/python -c "import playwright" 2>/dev/null; then
|
||
run_gate "interior figure-scale (no giants: crown ∈ [1.4,2.0] m, under ceiling · Lane D R10)" \
|
||
tools/.venv/bin/python tools/qa/interior_scale_check.py
|
||
else
|
||
soft_skip "interior figure-scale gate" "Playwright venv absent (python3 -m venv tools/.venv && …/pip install playwright)"
|
||
fi
|
||
|
||
# ── Gate 6 (Lane F, round 15): full-week gig soak — OPT-IN (--soak), NON-STRICT ─
|
||
# The v3.0 release soak: 7 nights × every playing venue in one context — latch state per plan.gigs,
|
||
# cover stamp per venue per night, roster clears per night, leak geo/tex to a warmed baseline across
|
||
# ≥20 enter/exit cycles. Slow (~90s) + diagnostic, so it is OPT-IN and warn-level (never blocks a tag);
|
||
# the core 6 gates stay a fast strict run. Wired per ROUND15 §Lane F.3.
|
||
SOAK_RUN=0; for a in "$@"; do [ "$a" = "--soak" ] && SOAK_RUN=1; done
|
||
if [ "$SOAK_RUN" = 1 ]; then
|
||
if [ -x tools/.venv/bin/python ] && tools/.venv/bin/python -c "import playwright" 2>/dev/null; then
|
||
warn_gate "full-week gig soak (7 nights · cover/roster per night · leak-free · Lane F R15)" \
|
||
tools/.venv/bin/python tools/qa/week_soak.py
|
||
else
|
||
soft_skip "full-week gig soak" "Playwright venv absent"
|
||
fi
|
||
fi
|
||
|
||
# ── Gate 7 (Lane F, round 17): town-matrix smoke — OPT-IN (--matrix), NON-STRICT ─
|
||
# The v4 real-map scout evidence: boot / determinism / budget / district / noassets across the full town
|
||
# matrix — synthetic hero seeds + the 3 osm fixtures + Lane E's 5 real AU town caches. One town × gate
|
||
# table. Slow (~20 boots) + diagnostic, so OPT-IN + warn-level. Wired per ROUND17 §Lane F (ledger #7).
|
||
MATRIX_RUN=0; for a in "$@"; do [ "$a" = "--matrix" ] && MATRIX_RUN=1; done
|
||
if [ "$MATRIX_RUN" = 1 ]; then
|
||
if [ -x tools/.venv/bin/python ] && tools/.venv/bin/python -c "import playwright" 2>/dev/null; then
|
||
warn_gate "town-matrix (synthetic + fixtures + real caches × 5 gates · Lane F R17)" \
|
||
tools/.venv/bin/python tools/qa/town_matrix.py
|
||
else
|
||
soft_skip "town-matrix" "Playwright venv absent"
|
||
fi
|
||
fi
|
||
|
||
# ── Gate 8 (Lane F, round 39): the v9 wave-1 gates ───────────────────────────
|
||
# 39.1 THE ADDRESS LAYER (node, no browser) · 39.4 THE TRANSMISSION PRE-PASS (validator control +
|
||
# the interior draw sweep against a fabricated glass asset) · 39.2 THE FOG / THE SIGN / THE RUMMAGE
|
||
# BIN (browser). Each ships its falsifiability control in the same run — see LANE_F_NOTES §39.
|
||
# --no-r39 skips the browser half on a fast determinism-only run; the node half always runs.
|
||
run_gate "R39 address layer (>=97% corpus · degrade-to-district · the wrong-shift red arm)" \
|
||
node tools/qa/r39_address.mjs
|
||
|
||
R39_SKIP=0; for a in "$@"; do [ "$a" = "--no-r39" ] && R39_SKIP=1; done
|
||
if [ "$R39_SKIP" = 1 ]; then
|
||
soft_skip "R39 transmission + runtime gates" "--no-r39"
|
||
elif [ -x tools/.venv/bin/python ] && tools/.venv/bin/python -c "import playwright" 2>/dev/null; then
|
||
run_gate "R39 transmission pre-pass (validator hard-fail control · interior sweep vs a glass asset)" \
|
||
tools/.venv/bin/python tools/qa/r39_transmission.py --validator
|
||
run_gate "R39 runtime (THE SIGN · THE FOG both arms · THE RUMMAGE BIN, contents unarmed)" \
|
||
tools/.venv/bin/python tools/qa/r39_runtime.py
|
||
else
|
||
soft_skip "R39 transmission + runtime gates" "Playwright venv absent"
|
||
fi
|
||
|
||
# ── Summary ──────────────────────────────────────────────────────────────────
|
||
hr
|
||
printf '%sSUMMARY%s %s%d passed%s · %s%d failed%s · %s%d warn%s · %s%d skipped%s\n' \
|
||
"$c_bold" "$c_off" "$c_grn" "$pass" "$c_off" "$c_red" "$failn" "$c_off" "$c_yel" "$warnn" "$c_off" "$c_yel" "$skip" "$c_off"
|
||
|
||
if [ "$failn" -gt 0 ]; then
|
||
printf '%s● QA RED%s — fix the failing gate(s) above.\n' "$c_red" "$c_off"; exit 1
|
||
fi
|
||
if [ "$STRICT" = 1 ] && [ "$skip" -gt 0 ]; then
|
||
printf '%s● QA NOT READY%s — %d gate(s) still pending; not v1 yet.\n' "$c_yel" "$c_off" "$skip"; exit 2
|
||
fi
|
||
printf '%s● QA GREEN%s — every landed gate passed.\n' "$c_grn" "$c_off"; exit 0
|