Completes every Apple Silicon GPU generation ever shipped (M1 2020 -> M5).
Headline: tiled 768/64+compile keys a full 2048 frame in ~2.2GB on EVERY
machine with IDENTICAL alpha (MAE 0.00849) — M3 Ultra 1949ms ... M1 mini 8GB
14269ms. Full-frame needs 26GB and is unreachable for most Macs ever sold, so
tiled should arguably be the default, not the fallback. The README's 'might
even work on your old MacBook Pro' undersells it.
Also: real CorridorKey ablation on M1 Max (stage_gc harmful at 32GB too:
0.81x @512), and the OG 8-core M1 mini confirms the hdim-56 fallback (1.00x
x3, padding 1.9x) — now 8/8 machines.
hdim-56 fallback is textbook here (1.00x x3); padding = 2.0x. Fits the
corrected fused-kernel-quality theory as a prediction, not a fit: 0.50x
fused/unfused -> 2.0x padding win, exactly between M3 Air (0.40x->2.6x) and
M1 Max (0.87x->1.3x). At 10c matched width the ladder is 138.9/48.8/44.0ms at
hdim56 but 54.3/24.4/8.9ms at hdim64 — M5's gains live almost entirely in the
fused kernel, making the head-dim fix most valuable on the newest silicon.
Padding 56->64 helps everywhere (1.3x-4.9x) and the hdim-56 silent fallback
reproduces on all six machines. But the M1 Max (24c) gains LEAST of all,
breaking our 'narrower GPU => bigger win' reading (M5 10c gained most).
Real predictor: fused-kernel quality vs that chip's own matmul throughput —
M1 Max's fused path is only 0.87x unfused at every size, so missing it costs
little; M5's is 0.14x, so missing it costs a lot. Corrected in place rather
than left as a tidy-but-wrong story.
Fifth machine. Two additions:
- M3 Air (10c) vs M5 (10c): same GPU width, 2 generations apart -> M5 is 7.7x
at seq 4096. Isolates architecture from width (the Ultras can't). head_dim-56
fallback reproduces here too (padding = 2.6x), so it now holds on every
Apple GPU generation shipping today.
- Fanless sustained load: NO meaningful throttle over 5 min (1.01x at 300s).
We predicted a decay curve and found none — reporting the negative, since it
validates that the burst numbers aren't short-run artifacts.
M5 MacBook Pro (10-core GPU, 16GB) reproduces the head_dim=56 silent fallback
and shows the fix matters far MORE on narrow GPUs: padding 56->64 gives 4.9x
(1,16,4096), 5.1x (1,16,2048), 3.2x (4096,2,64). Raw fused-vs-unfused at
4096/fp32 is 0.14x on M5 vs ~0.53x on M3 Ultra — an M5 laptop matches an M1
Ultra on fused attention but is ~4x slower unfused. This reframes the fix as a
laptop optimization, not a big-iron one.
Shape-spy on a real 2048 forward + exact-shape sweeps on M1/M3 Ultra:
fused==unfused at hdim 56 (silent fallback) on both machines, so use_sdpa
buys nothing and its transpose choreography costs ~4.5x on M1. Padding to
hdim 64 engages the fast kernel: 2.2x (M3) / 2.7x (M1) on the dominant
global-attention shape despite 14% extra FLOPs.
Ran scripts/bench_optimizations.py unmodified across three machines + an
engine-level tiled×compile extension with ground-truth alpha scoring.
Headlines: sdpa = 4.5x regression on M1 at 2048 (bf16 innocent — isolated
per-toggle on both Ultras); stage_gc harmful on Ultra-class; tiled 768/64
+ compile = best 2048 config on both Ultras; 24GB Macs swap-bound at 2048
full-frame (tiled required).
