bitmax/docs/handbook/11-vol-II-defrag.md

# Vol II Internals — DEFRAG.EXE

*A maintainer's tour of `defrag.html`: the cluster grid, the head sweep, the entropy engine that almost killed the volume, and the equilibrium that saved it.*

This chapter is the engineering bible for **Vol II — DEFRAG.EXE** (`defrag.html`, save key `boringsoft_defrag_v1`). Everything here is drawn from the source; the identifiers, constants, and formulas are real. The hook — *you steer a defrag* — is a spatial simulation laid over the shared scaffolding from `index.html` (*see* **Shared Technical DNA**), so this chapter spends most of its words on the parts that are genuinely new: the grid, the sweep, the rot, and the balance hack that keeps the rot survivable.

## The cell model

The entire playfield is one flat `Int8Array` named `grid`, indexed in reading order (`i = y*cols + x`). Six cell-state constants are packed in at the top of the script:

```js
/* cell state codes (flat Int8Array) */
const FREE=0, FRAG=1, CONT=2, YOU=3, SYS=4, BAD=5;
```

| Const | Value | Color (`COLORS[]`) | Meaning | Movable? |
|-------|-------|--------------------|---------|----------|
| `FREE` | 0 | `#ffffff` white | unused cluster — a relocation target | — |
| `FRAG` | 1 | `#d83a2a` red | scattered data; what the head hunts | yes |
| `CONT` | 2 | `#1038b0` blue | consolidated data; pays passive | yes (but packed) |
| `YOU`  | 3 | `#18c8d8` cyan | FRAG's own consciousness fragments | yes |
| `SYS`  | 4 | `#e8c020` yellow | boot sector / MFT — unmovable | no |
| `BAD`  | 5 | `#101010` black | corruption; spreads | no |

There are no per-cell objects. A cell *is* its `Int8Array` slot, which is what makes a 96×52 = 4,992-cluster grid cheap to simulate and repaint every tick. Two parallel typed arrays carry the rest of the per-cell state that doesn't fit in one byte: `badAge` (a `Float64Array` of `Date.now()`-relative spread deadlines) and the head/queue structures, which live outside the grid entirely.

The grid dimensions live in module-scope `cols`/`rows`, seeded to `40`/`24` but overwritten on every build and load. Always read them from the globals; never assume the starting size.

### buildGrid()

`buildGrid()` is the deterministic-shape, random-fill generator. It pulls `cols`/`rows`/`value`/`decay` from the current drive tier, allocates `new Int8Array(cols*rows).fill(FREE)`, then layers content in a fixed order:

```js
const sysCount=Math.floor(total*0.05);          // 5% SYS band, top-left
const dataCount=Math.floor(total*0.55);          // 55% FRAG, scattered
let youCount=Math.max(4, Math.floor(total*0.012)); // ~1.2% YOU
let badCount=Math.max(1, Math.floor(total*0.004)); // ~0.4% starting BAD
```

`SYS` is written as a contiguous run from index 0 — a yellow band in the top-left, exactly where a real boot sector and master file table would sit. Everything else is scattered with a rejection-sampling loop bounded by a `guard` counter (`guard++ < total*4`), so a near-full grid can never spin forever. `YOU` cells are only ever placed *onto existing `FRAG`*, so your consciousness starts embedded in the noise. After the fill, the **Compression Seed** perk runs `preConsolidate(seedPct)`, then `badQueue` is cleared and `initBadAge()` / `initHeads()` seed the auxiliary arrays.

> **DEV NOTE** — the `guard` ceilings are not paranoia. The seed perk (`pseed`) and a high drive tier can leave very few `FREE`/`FRAG` cells; without the guards, the placement `while` loops would hang the boot thread on a starved grid. The cost of an occasional under-fill (a few clusters short of target) is invisible; a hung tab is not. Trade accepted.

### The grid ↔ G.grid sync

`grid` is the live typed array; `G.grid` is its serialized cousin for `localStorage`. They are deliberately *not* the same object. The bridge is two functions.

On save, `serialize()` snapshots the live array into the state object (`defrag.html:1443`):

```js
G.grid=grid?Array.from(grid):null;
G.heads=heads.map(h=>({x:h.x,y:h.y,focus:h.focus||null}));
G.badQueue=badQueue.slice();
```

On load, `rebuildFromSave()` reverses it — but only if the shape still matches the current tier (`defrag.html:1481`):

```js
if(Array.isArray(G.grid) && G.grid.length===want){
  grid=new Int8Array(want);
  for(let i=0;i<want;i++){ const v=G.grid[i]; grid[i]=(v>=FREE&&v<=BAD)?v:FREE; }
  initBadAge();
} else {
  buildGrid();
}
```

The `want = cols*rows` length check is load-bearing: if a save predates a balance change to the tier table, or is otherwise the wrong size, the restore silently falls back to a fresh `buildGrid()` rather than reading off the end of a too-short array. The per-cell clamp `(v>=FREE&&v<=BAD)?v:FREE` is the second guard — any garbage byte becomes harmless `FREE`. Note that `badAge` is *never* persisted; it is regenerated from scratch by `initBadAge()`, so a reload resets every bad sector's spread clock. That is intentional — it's one byte of mercy on every return.

· · ·

## The head sweep

`heads` is an array of `{x, y, focus}` cursors. The simulation walks each one every tick inside `step(dt)`.

The per-head budget is throughput in clusters, computed fresh each tick:

```js
const perHead = headSpeed()*cacheWidth()*dt;
```

`headSpeed()` is `BASE_HEAD * Math.pow(1.5, lvl('speed'))` with `BASE_HEAD = 4` clusters/sec at level 0; `cacheWidth()` is `1 + lvl('cache')`. A fractional `_carry` accumulator per head banks sub-cluster budget between ticks so slow early-game heads still make progress. When a head holds a `focus` region (from the hook), its budget is multiplied by `burstPower()*1.4` and its mint multiplier becomes `burstPower()`.

`stepHead(h, budget, mintMult)` is the inner loop. It scans forward in reading order via `advanceHead()`, wrapping at the region or grid edge, until it finds a `FRAG` (or an unpacked `YOU`), then calls `consolidateAt()`. The relocation is the heart of the whole game:

```js
function consolidateAt(idx, intoLeft, mintMult){
  const wasYou = grid[idx]===YOU;
  let target=idx;
  if(intoLeft){
    const f=firstFreeBefore(idx);
    if(f>=0 && f<idx){ target=f; grid[idx]=FREE; }
  }
  grid[target]= wasYou?YOU:CONT;
  const pay = clusterValue() * (mintMult||1);
  G.rb += pay; G.totalRb += pay; G.runReclaimed += 1;
  return pay;
}
```

`firstFreeBefore(idx)` finds the leftmost `FREE` slot before `idx`, so data is packed toward index 0 — visibly draining red toward a clean blue band on the left. Each consolidation mints `clusterValue()` Reclaimed Bytes and increments `runReclaimed` (the platter-calculation counter, *see* **Prestige**). The `isPacked()` predicate stops the head from endlessly re-relocating a `YOU` cell that already has no free space to its left.

### clusterValue, algoMult, globalMult

The per-cluster payout is a product of three terms, each a clean power-law:

| Function | Formula | Driver |
|----------|---------|--------|
| `clusterValue()` | `driveDef(tier).value * algoMult() * globalMult()` | the whole stack |
| `algoMult()` | `Math.pow(2.2, lvl('algo'))` | Consolidation Algorithm upgrade |
| `globalMult()` | `Math.pow(1.6, lvl('compact')) * (1 + 0.3*plvl('pmult'))` | Aggressive Compaction + Cold Storage perk |

`globalMult()` is the one term that folds in a *persistent* perk (`pmult`), which is why it appears in both the active mint and the passive-income path — it must apply to every byte the drive ever earns. The base `value` comes straight from the tier table and spans `64` (40 MB) to `1.3e8` (1 PB), so the headline number climbs through SI suffixes purely from the hardware ladder.

### The automation layer

Two one-time upgrades steer heads without the player. The **Defragmentation Daemon** (`daemon`) gently biases an idle head's `y` toward `fragHotspot()` — the row with the most `FRAG` — but only with a 4% per-tick probability, so the steer feels like drift, not teleportation. The **Auto-Repair Daemon** (`autorepair`) is covered under **Repair**, below.

· · ·

## Passive income from contiguous runs

Beyond per-cluster mints, contiguous blue pays rent. `passiveRate()` scans each row, accumulates run lengths of `CONT`/`YOU`, and pays each run through `runPay()`:

```js
function runPay(run){
  return run * driveDef(G.driveTier).value * 0.06 * (1 + run*0.04) * algoMult();
}
```

The `(1 + run*0.04)` term is super-linear: a run of 20 contiguous clusters earns more per cluster than two runs of 10. That is the entire design argument for packing left into long unbroken bands rather than scattering consolidations. `passiveRate()` then multiplies the row total by `streakMult()` (`1 + 0.35*lvl('streak')`) and `globalMult()`, and `step()` banks `passiveRate()*dt` into both `rb` and `totalRb` every tick.

Two derived gauges read the same grid:

```js
function consolidatedPct(s){
  const movable=s.frag+s.cont+s.you;
  if(movable<=0) return 100;
  return clamp((s.cont+s.you)/movable*100, 0, 100);
}
function diskHealth(s){
  return clamp(100 - s.bad/totalCells()*100, 0, 100);
}
```

`consolidatedPct()` is the progress bar and the prestige gate — it counts *only movable cells*, so `SYS` and `BAD` don't poison the denominator. `diskHealth()` is the SMART gauge, a pure function of bad-sector fraction. When health drops to ≤0.5%, `step()` sets `G._readonly` and the head stops consolidating — the soft cap that pushes a dying drive toward migration.

> **DEV NOTE** — `consolidatedPct()` returning `100` when `movable<=0` is not a rounding accident. A grid that has been fully consolidated (or pre-seeded into oblivion) has zero movable cells, and dividing by zero there would paint `NaN%` on the progress bar and break the migrate gate. "Nothing left to move" is, correctly, "100% done."

· · ·

## Entropy — the engine that almost shipped a death spiral

Entropy is the antagonist, and it is the one system that was redesigned after a post-mortem. The driver is `driveDecay()`:

```js
function driveDecay(){ return driveDef(G.driveTier).decay * entropyMult(); }
```

`entropyMult()` is `Math.pow(0.91, lvl('cooling')) * Math.pow(0.92, plvl('pcool'))` — the only thing the player can do to slow the clock. `entropyTick(dt)` accumulates real time and fires `rollEntropy()` once per `period = 2.4 / driveDecay()` seconds, so a higher-tier drive (decay up to `6.0`) rolls the dice far more often.

`rollEntropy()` is an 80/20 split:

```js
function rollEntropy(){
  const r=Math.random();
  if(r<0.80){
    const i=findRefragTarget();
    if(i>=0){ if(grid[i]===CONT) grid[i]=FRAG; else if(grid[i]===FREE) grid[i]=FRAG; }
  } else {
    if(badNow() < badCap()){
      const i=findBadTarget();
      if(i>=0){ grid[i]=BAD; badAge[i]=now()+rand(5000,10000)/driveDecay(); }
    }
  }
}
```

80% of rolls **re-fragment** — `findRefragTarget()` prefers flipping a `CONT` cell back to `FRAG` (visible decay of your work) and falls back to spawning fresh `FRAG` in free space. 20% **spawn a bad sector**, and — critically — *only if `badNow() < badCap()`*. Separately, `spreadBad()` runs every tick: each `BAD` cell whose `badAge` deadline has passed infects one random orthogonal neighbor (skipping `SYS` and other `BAD`), then resets its own deadline. Queued sectors are frozen — `if(badQueue.indexOf(i)>=0) continue;` — so corruption you've already targeted can't metastasize while it waits.

### badCap() / badNow() — the equilibrium

Here is the fix, and the reason the comment in the source reads like a confession:

```js
// Entropy reaches an EQUILIBRIUM instead of devouring the whole disk while you idle —
// bad sectors never grow past this cap passively (you repair to push below it). Bigger,
// older drives rot a little more, but it is never lethal.
function badCap(){ return Math.floor(totalCells() * Math.min(0.34, 0.18 + 0.02*G.driveTier)); }
function badNow(){ let b=0; for(let i=0;i<grid.length;i++) if(grid[i]===BAD) b++; return b; }
```

`badCap()` is a hard ceiling on passive corruption: 18% of the grid at tier 0, climbing 2 points per tier, clamped at 34%. **Both** entropy paths — the `rollEntropy()` spawn *and* the `spreadBad()` infection — check `badNow() < badCap()` before adding a black cell. The player's repair charges are the only thing that can push bad sectors *below* the cap; entropy can never push them above it on its own.

This exists because the first cut didn't have it. Without a cap, `spreadBad()` is exponential — bad sectors spawn bad sectors — and an idle drive would wake up at 100% black, health zero, read-only, unrecoverable. That violates Design Law 5: *idle must reach equilibrium, never death.* The cap converts a runaway feedback loop into a stable setpoint. You can walk away from a defrag and come back to a drive that's a little rotten and entirely playable. The `min(0.34, …)` clamp guarantees that even the 1 PB array, decaying at `6.0`, tops out at roughly a third corrupted — annoying, never terminal.

> **DEV NOTE** — the death spiral shipped, briefly, in a build with no cap and `spreadBad()` running on every tick. An eight-hour offline session would return a solid black grid and a save you couldn't dig out of: every repair you bought was immediately re-corrupted faster than you could spend charges. The lesson became canon. A cap is cheaper than a comeback mechanic, and equilibrium is a feature you can feel even when you can't see it. While paused, entropy still runs — `entropyTick(G.running?dt:dt*0.4)` — at 40% rate, so pausing slows the rot but never stops it. The tension is always on.

· · ·

## Repair — charges, queue, daemon

Repair is the player's lever against `BAD`. `maxCharges()` is `BASE_CHARGES + lvl('ecc') + 2*plvl('pcharge')` (base 3), and charges regenerate one at a time over `regenTime()` seconds (`REGEN_BASE/Math.pow(1.4, lvl('regen'))/(1+0.25*plvl('pcharge'))`, base 8s).

Clicking a black cell calls `queueRepair(idx)`, which spends a charge and pushes the index onto `badQueue`. `processRepairs(dt)` works the queue head-first: a queued sector accrues `G._repairProg` and flips to `FREE` after 0.6s. The **Auto-Repair Daemon** automates targeting:

```js
function autoRepair(){
  if(!has('autorepair')) return;
  if(G.charges<1) return;
  if(badQueue.length>=2) return;
  // …find nearest BAD to any head by Manhattan distance, then queueRepair(best)
}
```

It deliberately caps its own queue at 2 and bails when out of charges, so it assists rather than trivializes containment.

· · ·

## The hook — drag-select and click

The mouse handlers live in one block. `mousedown` records `downCell` and starts a `dragSel` rubber-band; `mousemove` extends it and flips `dragStarted` once the box exceeds one cell; `mouseup` decides intent by area:

```js
if(dragStarted && area>1){
  focusBurst(r);
} else if(downCell){
  clickCell(downCell);
}
```

`focusBurst(r)` assigns the normalized region as a `focus` to the one or two nearest heads (two if the region holds >40 `FRAG`, for a satisfying swoop) and snaps them to its top-left corner. A focused head gets the `burstPower()` mint and `burstPower()*1.4` speed bonus from `step()`. `clickCell(c)` branches on the cell: a `BAD` cell queues for repair; anything else builds a small 5×3 focus box around the click and steers the nearest head there for a micro-defrag. `cellFromEvent()` maps screen pixels to grid coordinates using the same `ox`/`oy`/`cellPx` the renderer computes, so clicks land where they look.

· · ·

## The drive ladder

`DRIVES` is the prestige tier table. `driveDef(i)` clamps the index to the array bounds. Bigger drives pay more and rot faster — the central tension of every migration.

| Tier | Label | `name` | cols × rows | clusters | `value` | `decay` |
|------|-------|--------|-------------|----------|---------|---------|
| 0 | 40 MB | C: | 40 × 24 | 960 | 64 | 1.00 |
| 1 | 540 MB | D: | 48 × 28 | 1,344 | 512 | 1.35 |
| 2 | 6 GB | E: | 56 × 32 | 1,792 | 4,096 | 1.75 |
| 3 | 40 GB | F: | 64 × 36 | 2,304 | 3.3e4 | 2.25 |
| 4 | 120 GB | G: | 72 × 40 | 2,880 | 2.6e5 | 2.9 |
| 5 | 2 TB | H: | 80 × 44 | 3,520 | 2.1e6 | 3.7 |
| 6 | 16 TB | S: | 88 × 48 | 4,224 | 1.7e7 | 4.7 |
| 7 | 1 PB Array | Z: | 96 × 52 | 4,992 | 1.3e8 | 6.0 |

### Migration

The prestige gate is `PRESTIGE_PCT = 60` percent consolidated *and* at least one Platter earned. The payout is sub-linear in clusters reclaimed:

```js
function plattersFor(){
  const def=driveDef(G.driveTier);
  const reclaimedFrac = Math.min(1, G.peakConsolidated/100);
  const base = Math.sqrt(G.runReclaimed/40) * (1+G.driveTier*0.6) * reclaimedFrac;
  return Math.floor(base);
}
```

The `Math.sqrt(runReclaimed/40)` term means Platters scale with the *square root* of effort — diminishing returns within a run, which is what pushes you to migrate up rather than farm one drive forever. `doMigrate()` banks the Platters, increments `driveTier` (capped at the last tier), wipes the run (`rb`, `runReclaimed`, `peakConsolidated`, `bestRun`, and critically `G.upgrades={}`), preserves `perks`, then calls `buildGrid()` on the new, larger platter. Perks persist; hardware upgrades do not. That is the reincarnation contract.

· · ·

## Upgrades and perks

Upgrades are bought with Reclaimed Bytes in the "Defragmentation Options" dialog and reset on migration. Cost is `base * growth^level` for leveled upgrades, flat for one-time installs.

| id | Name | tab | kind | base | growth | max | Effect |
|----|------|-----|------|------|--------|-----|--------|
| `speed` | Head Speed | head | lvl | 48 | 1.55 | 60 | head throughput ×1.5/lvl |
| `cache` | Read-Ahead Cache | head | lvl | 120 | 1.7 | 24 | sweep width `1+lvl` |
| `heads` | Multiple Heads | head | lvl | 800 | 2.15 | 11 | `+1` parallel head/lvl |
| `algo` | Consolidation Algorithm | head | lvl | 300 | 2.4 | 7 | $/cluster ×2.2/lvl |
| `streak` | Contiguity Bonus | head | lvl | 420 | 1.95 | 20 | streak income `+35%`/lvl |
| `ecc` | ECC / Repair Passes | repair | lvl | 260 | 1.85 | 16 | max charges `+1`/lvl |
| `regen` | Repair Regen Rate | repair | lvl | 340 | 1.8 | 20 | regen ×1.4/lvl |
| `autorepair` | Auto-Repair Daemon | repair | once | 9000 | — | — | auto-targets nearest BAD |
| `cooling` | Cooling / Anti-Entropy | repair | lvl | 500 | 1.9 | 18 | entropy ×0.91/lvl |
| `daemon` | Defragmentation Daemon | auto | once | 6000 | — | — | idle heads steer to hotspot |
| `prefetch` | Smart Prefetch | auto | lvl | 1500 | 2.0 | 12 | burst power `+50%`/lvl |
| `compact` | Aggressive Compaction | auto | lvl | 2200 | 2.1 | 14 | all income ×1.6/lvl |

The `algo` upgrade also advances the cosmetic `ALGO_NAMES` ladder: `BubbleSort → InsertionSort → QuickSort → HeapSort → RadixSort → TimSort → "I wrote my own" → Singularity-Sort`.

Perks are bought with Platters in the migration wizard and persist across every drive forever.

| id | Name | base | growth | max | Effect |
|----|------|------|--------|-----|--------|
| `pmult` | Cold Storage Dividend | 1 | 1.7 | 40 | `+30%` all income/lvl |
| `pcool` | Cryogenic Platters | 1 | 1.8 | 30 | base entropy ×0.92/lvl |
| `pheads` | Spare Head Assembly | 2 | 2.0 | 10 | `+1` starting head/lvl |
| `pcharge` | Hardened Sectors | 2 | 1.9 | 12 | `+2` charges & faster regen/lvl |
| `pseed` | Compression Seed | 3 | 2.2 | 10 | `8%` pre-consolidated/lvl |

· · ·

## Rendering

The grid is a single `<canvas>`, repainted whole every tick by `drawGrid()` — not 4,992 DOM nodes. `resizeCanvas()` computes a square `cellPx` that fits the frame and centers the grid with `ox`/`oy` offsets, capping device-pixel-ratio at 2 to keep large drives cheap. The draw order is: white background, then every cell as a `cellPx-1` square (the `-1` leaves a one-pixel grout line for the grid texture), then `YOU` glow rings, then orange outlines on `badQueue` entries, then green head highlights with their faint vertical guides, then the dashed navy rubber-band if a drag is live. `image-rendering: pixelated` on the canvas keeps the cells crisp.

The HUD is plain DOM, throttled to repaint at most every 0.2s (`railAccum`). `renderRail()`, `renderProgress()`, and `renderStatus()` read the cached `lastRailStates` snapshot rather than rescanning the grid on every call. The status line cycles through a 2.5-second `statusFlash` override, then read-only / paused warnings, then the live "Compacting cluster N…" message — the same diegetic chatter a real defragmenter prints.

## If you change this

- **The entropy/bad-cap balance is the load-bearing wall.** If you touch `badCap()`, `rollEntropy()`, or `spreadBad()`, re-derive the worst case: simulate an 8-hour offline session (`MAX_OFFLINE_S`) on the **top tier** (`decay: 6.0`) with *zero* `cooling`/`pcool`. The grid must come back below 34% bad and above read-only. Both spawn paths must keep their `badNow() < badCap()` guard, or the death spiral returns. This is not a knob; it is the contract.
- **`DRIVES` tier shapes feed the save sync.** Changing any `cols`/`rows` changes `cols*rows`, which is the exact length `rebuildFromSave()` checks against `G.grid.length`. Existing saves on a resized tier will silently regenerate via `buildGrid()` — acceptable, but know it happens. Never reorder or remove tiers without a save-version bump.
- **`value` and `decay` move together.** The whole feel is "bigger pays more, rots faster." If you raise a tier's `value` without raising `decay`, you flatten the difficulty curve; the migration decision stops being a trade-off.
- **`badAge` is not persisted.** If you ever need bad-sector spread timing to survive reload, you must add it to `serialize()`/`rebuildFromSave()` *and* re-clamp it in `sanitizeState()`. Today it's regenerated by `initBadAge()` on purpose — don't half-migrate it.
- **`globalMult()` is in two hot paths.** It's folded into both `clusterValue()` (active mint) and `passiveRate()` (streak income). Add a new global multiplier to `globalMult()` itself, never to one call site, or active and passive income will drift apart and the numbers stop being legible (Design Law 4).
- **`consolidatedPct()` counts only movable cells.** If you change what "consolidated" means, check the prestige gate (`PRESTIGE_PCT`), the progress bar, and the `movable<=0 → 100` early return together. They all read this one function.