bitmax/docs/handbook/03-qa-and-postmortems.md

QA Playbook & Post-Mortems

How five idle games are actually tested, and the eight real bugs that taught us the two laws of the form: idle must reach equilibrium, never death — and a save must never brick boot.

Philosophy of testing an idle game

Testing a normal app asks does the button do the thing. Testing an idle game asks does the game keep its promise to a player who isn't here. The whole genre is a contract with an absent user: leave, come back, be rewarded — and meanwhile the simulation runs honestly in a tab you forgot about. Almost every bug we caught lived in that seam, between the foreground loop you can watch and the offline, backgrounded, throttled, or freshly-loaded state you can't.

So the QA here is less "click everything" than "verify the loop survives contact with time." Two failure modes dominate, and they are the two laws the Design Laws (law 5, respect the idle contract) only hint at:

1. Idle must reach equilibrium, never death. A simulation that runs unattended must converge to a steady state, not a terminal one. If any quantity can grow without bound while the player is gone — bad sectors, inbox depth, heat — the contract is broken and you return to a corpse.
2. A save must never brick boot. Every volume round-trips its entire G state object through localStorage as JSON. A partial, hand-edited, version-skewed, or simply unlucky save must boot anyway. sanitizeState() is the load-bearing wall, and most of our worst bugs were holes in it.

Everything below catches violations of those two laws before a player does.

· · ·

Part A — The verification playbook

We do not have a unit-test suite. These are single self-contained .html files with no build step (the Shared Technical DNA), so the harness is a real browser and a disciplined checklist, in order.

1. Live smoke-test in a real browser

Open the file; watch it boot. The single most valuable signal is the console — these games are "use strict" throughout, and a thrown exception inside the tick is silent to the player but loud in the console. A clean console across thirty seconds of idle is the baseline gate. (Post-mortem #1 shows what an uncaught throw inside tick() looks like from the player's chair: nothing, then nothing forever.)

2. Drive the real loop, by hand

Click the thing the game is actually about. Add a torrent and watch torrentDownSpeed(t) resolve, then flip to seeding and mint Data. Drag-select a DEFRAG region and watch the head teleport. Drag-fill a column in MACRO_VIRUS. Type spawn miner in UPLINK and watch a row appear in the top table. The loop has to feel right before any number means anything.

3. Defeat beforeunload to get a truly fresh state

Every volume wires window.addEventListener('beforeunload', save). That is correct for players and a menace for testers: you cannot get a clean first-boot by reloading, because the reload saves on the way out. To test the genuine new-player experience you must clear the key first:

localStorage.removeItem("qbittorrz_save_v1"); location.reload();

(Each volume exposes this as File ▸ Delete Everything…, the confirm-reset path, but during QA we hit the key directly.) Half our balance bugs only manifested on a true fresh boot, because a developer's own save was already past the rough patch. The opening minutes are the hardest thing to keep testing — you stop being a new player on day one.

4. Numeric balance sweeps across tiers

The dangerous numbers are the ones you never personally reach — a bug at the Singularity tier or on the 1 PB Array drive never surfaces in casual play. So we sweep: with the console open, push state to a target tier and read the derived values directly. In qBitTorrz the central balance function is one line —

function clientDown(){ return BASE_DOWN * Math.sqrt(maxUnlockedSize()/MIN_SIZE) * downMult(); }

— so sanity-check clientDown() against the size of the most expensive unlocked CATALOG entry. If fmtTime(size / clientDown()) reads in hours at the start of a tier, the tier has a wall (post-mortem #3). Same drill for DEFRAG's clusterValue() across the DRIVES ladder and INBOX's arrivalRate() against inboxCap().

5. The offline / visibility checks

Three distinct time-paths must each be exercised, because they share a simulation core (simulateAway(dt)) but reach it differently:

Path	Trigger	Code
Offline catch-up	boot after time away	applyOffline() → simulateAway(dt), capped at MAX_OFFLINE_S (8h)
Foreground tick	every TICK_MS (100ms)	tick() → step(dt), dt capped at 5s
Refocus catch-up	tab becomes visible	visibilitychange → simulateAway(min(MAX_OFFLINE_S, gap))

The refocus path is the subtle one. Background tabs get setInterval throttled to roughly once a second or slower, so tick()'s 5-second dt cap would silently discard most of the elapsed time. The visibilitychange handler credits the gap through the offline sim, then resets lastTick so the next foreground tick doesn't double-count it. Test it by switching tabs for a minute and confirming the welcome-back math (post-mortem #4 is the version that didn't).

DEV NOTE — When you test offline progress, test both a 30-second gap and an 8-hour gap. simulateAway runs a coarse fixed-step loop — const steps = Math.min(240, Math.max(20, Math.ceil(dt/30))) — so the step size differs wildly between a short refocus and a long sleep. A torrent that completes mid-gap has to keep seeding for the remainder, which only works if the step count is fine enough. Eight hours over 240 steps is a two-minute step; verify completions still credit.

6. The canvas-resize gotcha (DEFRAG only)

DEFRAG renders 960-plus clusters to a <canvas>, not DOM nodes (per the DEFRAG technical notes — never 960 DOM cells). resizeCanvas() recomputes dpr, cellPx, and the centering offsets ox/oy from the frame's client size; drawGrid() paints into device pixels. The trap: the grid is only correct if resizeCanvas() has run before the first drawGrid(), and again on every window resize. Boot order matters —

resizeCanvas();
renderAll();
drawGrid();
setInterval(tick, TICK_MS);
window.addEventListener('resize',()=>{ resizeCanvas(); drawGrid(); renderProgress(); });

— and the mouse-to-grid mapping in the drag-select handler rescales by canvas.width/rect.width, so a stale canvas size means your drag-box lands on the wrong clusters. QA step: resize the window mid-drag and confirm the selection still tracks the cursor.

· · ·

Part B — Post-mortems

Eight real bugs, each a case study. Not hypotheticals — every one was in a build, found, and fixed, and the fix is in the source you can read now.

1. qBitTorrz — the status bar that killed the loop

Symptom. After exactly one render, the game froze. No numbers ticked. The console showed a TypeError once per tick(), forever.

Root cause. An early renderStatus() replaced the #sb-conn element's contents via innerHTML and then tried to read a child of the old subtree — the sb-dht node it had just orphaned. The read threw. Because renderStatus() runs inside tick(), the throw aborted the rest of the tick after the first pass, and the simulation never advanced again. The cruel part: nothing told the player. An idle game that stops idling looks identical to one that's merely quiet.

Fix. renderStatus() now rebuilds #sb-conn in one innerHTML assignment per branch and reads nothing back out of it:

const dht = hasUnlock('dht') ? Math.floor(120+lvl('bandwidth')*37+G.totalUp%500) : 0;
const conn = document.getElementById('sb-conn');
if(now()<G.throttleUntil && !hasUnlock('quantum')){ conn.innerHTML='…Throttled by ISP…'; }
else if(now()<G.burstUntil){ conn.innerHTML='…Swarm burst ×10…'; }
else { conn.innerHTML='…DHT: <b>'+fmtInt(dht)+'</b> nodes…'; }

Lesson. Never read from a node after you've replaced its parent's innerHTML in the same function — the reference is dangling. More broadly: any function reachable from the tick must not throw, or it takes the whole simulation down with it.

2. qBitTorrz — fakeHash() and the all-zero info hashes

Symptom. Every torrent's "Info Hash" in the General tab read as mostly zeros, and the procedurally generated peer IPs in the Peers tab clustered around 0.0.0.x. Cosmetic, but it broke the illusion of fidelity instantly.

Root cause. The original fakeHash() built its 40 hex digits from the low bits of a linear congruential generator. LCGs are notoriously weak in exactly that place — the low-order bits have short periods and tiny entropy — so digit after digit came up 0. A torrent client whose hashes are all zeros is a tell.

Fix. Replaced the LCG with an FNV-1a seed feeding an xorshift32 stream, sampling a rotating high nibble each iteration:

let x = 0x811c9dc5;
for(let i=0;i<s.length;i++){ x = (Math.imul(x,16777619) ^ s.charCodeAt(i)) >>> 0; }
if(x===0) x = 0x9e3779b9;
let h='';
for(let i=0;i<40;i++){
  x^=x<<13; x>>>=0; x^=x>>>17; x^=x<<5; x>>>=0;   // xorshift32
  h += hex[(x>>>(4*(i%7)))&15];
}

Lesson. Even for display-only randomness, bit-quality matters. xorshift32 is three shifts and three XORs — no more expensive than the LCG — and it doesn't visibly fail. Sampling (x>>>(4*(i%7))) instead of x&15 also matters: take from the top, not the bottom.

3. qBitTorrz — the unlock-gate "dead zone"

Symptom. Buying a tier unlock (Cat6, Private Tracker Invite, Fiber-Optic Trunk Line) felt like a punishment. Each new tier opened onto a file an order of magnitude larger than the last, but download speed was flat, so the first torrent of every tier was a multi-hour wall before any seeding income resumed.

Root cause. Line speed was a constant times your bandwidth multipliers. File sizes in CATALOG span seventeen orders of magnitude — from 4e3 bytes to 3e20. With flat line speed, every tier transition multiplied download time by the same factor it multiplied file size. The unlock you just paid for was a wall, not a door.

Fix. Line speed now scales with the square root of the largest unlocked file size, so unlocking a tier also grants the throughput to actually download it:

const BASE_DOWN = 1000;   // B/s anchor before scaling/multipliers
const MIN_SIZE  = 4e3;    // smallest catalog file (Tier A)
function clientDown(){ return BASE_DOWN * Math.sqrt(maxUnlockedSize()/MIN_SIZE) * downMult(); }

The sqrt halves the exponent the player must cover with upgrades — download times stay sane across the whole ladder while the biggest late-tier files remain a genuine sink for the bandwidth and overclock upgrades. The fiction even cooperates: "Cat6 unlocks MB-tier speeds."

Lesson. When a cost and a reward both scale exponentially, scale the capacity between them — sqrt is the natural midpoint. A gate should open onto a slope, not a cliff.

DEV NOTE — The sqrt trick recurs across the anthology. INBOX's prestige payout is Math.pow(p/2000, 0.42); DEFRAG's migration reward uses Math.sqrt(G.runReclaimed/40). Whenever the headline number runs to scientific notation, the prestige and capacity curves want a fractional exponent to stay legible. If a sweep shows a tier feeling like a wall, reach for the exponent before the base values.

4. qBitTorrz — backgrounded-tab progress loss

Symptom. Leave the tab in the background for ten minutes, come back, and you'd earned roughly five seconds of Data. The game "ran" the whole time — the tick fired — but almost nothing accrued.

Root cause. Two faults compounded. First, tick() caps dt at 5 seconds to absorb a throttled tab (if(dt>5) dt=5;) — correct for a single tick, but a tab throttled to one tick per minute throws away 55 of every 60 seconds. Second, the periodic autosave kept writing G.lastSave = Date.now() from the throttled ticks, so even the offline path on a later reload saw almost no gap — the elapsed time had been poisoned out of the save.

Fix. A visibilitychange handler that credits the real wall-clock gap through simulateAway and resets lastTick so the next tick can't double-count it:

document.addEventListener('visibilitychange',()=>{
  if(document.visibilityState!=='visible') return;
  const gap=(Date.now()-lastTick)/1000;
  if(gap>10){ simulateAway(Math.min(MAX_OFFLINE_S,gap)); save(); renderAll(); }
  lastTick=Date.now();
});

Lesson. The dt cap and the offline sim are two halves of one mechanism: the cap protects a single tick from a huge jump, and the catch-up sim is what's supposed to absorb that jump instead — so anything you cap, you must catch up elsewhere. DEFRAG and INBOX carry the identical handler now; it's part of the DNA.

5. DEFRAG — the entropy death-spiral

Symptom. Walk away from a freshly migrated drive and come back to a tombstone. In a live idle test the grid went from healthy to roughly 95% bad sectors in about three minutes of doing nothing. The disk ate itself. This is the canonical violation of the idle contract, and it's why the Design Laws cite it by name.

Root cause. Entropy spread bad sectors to orthogonal neighbors on a timer, with no cap. Each new bad sector became a new spreading source, so the bad-sector count grew geometrically. There was no equilibrium — only a fixed point at the entire disk is dead. Idle reached death, exactly as forbidden.

Fix. An equilibrium cap, badCap(), that both the spawn roll and the spread loop respect:

function badCap(){ return Math.floor(totalCells() * Math.min(0.34, 0.18 + 0.02*G.driveTier)); }

rollEntropy() only spawns a bad sector if(badNow() < badCap()), and spreadBad() halts at the same ceiling. Bad sectors never grow past the cap passively — the player repairs to push the count below it, but absence can never be lethal. Bigger, older drives rot a little more (the 0.02*G.driveTier term, clamped at 34%), but the disk always converges.

Lesson. Any spreading or compounding process in an idle sim needs a ceiling, full stop. The test is mechanical: walk away for the offline cap (8h via simulateAway) and confirm every gauge converges rather than pinning. If a quantity can reach 100% bad while you're gone, you have a death-spiral, not a game.

6. INBOX ZERO — the inert penalty and the cruel opening

Symptom. Two bugs in one feature. The UI and the help text both promised "Overwhelmed — income halved until you dig out," but overflowing the cap did nothing to income; the penalty was a no-op. And separately, a fresh player was overwhelmed roughly nine seconds after load — the flood outran any possible manual triage before the loop was even legible.

Root cause. The penalty was computed (isOverwhelmed() flipped true, the status bar flashed) but never applied — no income path multiplied by it. Meanwhile base inbox capacity was too low for the opening arrival rate, so a new save crossed the cap almost immediately.

Fix. Two changes. First, overwhelmPenalty() is now wired into every income multiplier:

function overwhelmPenalty(){ return isOverwhelmed() ? 0.5 : 1; }
function manualMult(){ return plane().ppMult * … * perkMult('clarity') * overwhelmPenalty(); }
function ruleMult(){ return plane().ppMult * … * perkMult('enlightened') * overwhelmPenalty(); }

Both the manual-triage and rule-automation income paths now actually halve while overwhelmed — the promise the UI was already making. Second, base inboxCap() was raised from 12 to 18, giving a new player about twenty seconds of grace before the flood:

function inboxCap(){ return 18 + lvl('cap')*UP_BY_ID.cap.add + perkAdd('serenity') + G.ascensions*0; }

Lesson. A penalty you display but don't apply is worse than no penalty — it teaches the player a lie about how the game works. And tune the opening against a true fresh boot, never against your own seasoned save (Part A, step 3). The first twenty seconds are the only twenty seconds every player sees.

7. MACRO_VIRUS — the output-column soft-lock

Symptom. Some saves loaded into a permanently dead economy: Net Worth pinned at zero, Cash never accrued, no purchase could ever fix it. The board looked owned and full, but earned nothing forever.

Root cause. Net Worth is SUM of the output column, OUT_COL (column H), and Cash accrues from the rate of that column's growth. If a save — cleared, hand-edited, or pathological — owned cells but had no producing cell in column H, there was no income and no way to buy your way to one: buying needs Cash and Cash needs an output cell. A dependency deadlock — you needed money to make the thing that makes money.

Fix. sanitizeState() now guarantees a producing output cell on every load:

let hasOut=false;
for(let r=0;r<ROWS_N;r++){ const oc=G.cells[cid(OUT_COL,r)]; if(oc&&oc.owned&&oc.formula){ hasOut=true; break; } }
if(!hasOut){
  ensureCell('H1'); G.cells['H1'].owned=true;
  if(!G.cells['H1'].formula){ G.cells['H1'].formula='prev'; G.cells['H1'].lvl=plvl('startTier'); if(!(G.cells['H1'].value>0)) G.cells['H1'].value=1; }
}

Combined with the existing guarantee that A1 is always owned with a compounding prev formula, this makes a productive economy a load-time invariant rather than something the player can lose.

Lesson. This is law 2 in its purest form — a save must never brick boot. "Brick" doesn't only mean a crash; an economy that can't restart itself is just as bricked as one that throws. sanitizeState() must guarantee not just valid state but playable state: every volume needs at least one source of income that no save can ever take away.

8. UPLINK — kill -9 <pid> targeted the wrong process

Symptom. Typing the most natural kill command in the world — kill -9 1337 — failed to kill PID 1337, or killed nothing, or errored. Plain kill 1337 worked, but the muscle-memory form did not.

Root cause. The command parser read args[0] as the target PID. With kill -9 1337, args[0] is -9 — the signal flag, not the target. The game tried to interpret the signal as the process to kill. In a terminal game, breaking kill -9 breaks the one command every user knows by heart, and fidelity to a real shell is the entire point of UPLINK.

Fix. Strip any leading-dash flags before reading the target:

case 'kill': {
  // accept an optional signal flag (e.g. `kill -9 1337`) before the target
  const kargs = args.filter(x=>!/^-/.test(x));
  const a = (kargs[0]||'').toLowerCase();
  …
  const pid = parsePid(kargs[0]);
  if(pid==null){ logLine('c-err','kill: '+esc(kargs[0])+': arguments must be process or job IDs'); break; }
  killPid(pid,false); break;
}

Now kill 1337, kill -9 1337, and kill -SIGKILL 1337 all resolve to PID 1337, matching real-shell behavior, while kill all and kill zombies still work because they aren't dash-prefixed.

Lesson. When you imitate a real command-line, you inherit its argument grammar whether you implement it or not — players type the real syntax. Flags and positionals are different things; parse them as different things. The clutter line cuts both ways: the more convincingly boring your frame, the more exactly users expect it to behave like the real tool.

· · ·

The save-key registry

A recurring near-miss: every volume must use a unique localStorage key, or two games silently clobber each other's saves on a shared origin. We verify this on every new volume.

Volume	File	Save key
I — qBitTorrz	index.html	qbittorrz_save_v1
II — DEFRAG.EXE	defrag.html	boringsoft_defrag_v1
III — MACRO_VIRUS.XLS	spreadsheet.html	boringsoft_xls_v1
IV — UPLINK	terminal.html	boringsoft_uplink_v1
V — INBOX ZERO	inbox.html	boringsoft_inbox_v1

The comment in inbox.html says it plainly: // UNIQUE per volume — do not collide.

· · ·

Gotchas — if you change this

• If you touch any function reachable from tick(), audit it for throws and dangling-node reads. A single uncaught exception in the tick path stops the simulation silently (post-mortem #1). The player gets no error — just a game that quietly stopped being a game.
• If you add a spreading, compounding, or accumulating quantity, give it a cap in the same commit. Then run the 8-hour offline test and confirm it converges. No exceptions; this is law 1 (post-mortem #5).
• If you change sanitizeState(), re-confirm it produces playable state, not merely valid state — at minimum one guaranteed source of income that survives a cleared or hostile save (post-mortems #6, #7). Test it by loading {} and a save with every field set to null.
• If you change the dt cap, autosave cadence, or lastSave writes, re-test all three time-paths from Part A step 5 together — they are one mechanism. Capping without catching up loses time; catching up without resetting lastTick double-counts it (post-mortem #4).
• If you add a balance number at a tier you don't personally play to, sweep it in the console against the relevant derived function (clientDown(), clusterValue(), arrivalRate() vs inboxCap()). A wall at the Singularity is invisible until a player hits it (post-mortem #3).
• If you imitate a real command, formula, or file format, you've signed up for its grammar. Parse flags as flags (post-mortem #8). The boring frame is a promise of fidelity, and players will test it by reflex.