monster/corridorkey-mrp-mlx
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: M1 Max (6th machine) refutes the width theory — report the correction

Browse Source 
Padding 56->64 helps everywhere (1.3x-4.9x) and the hdim-56 silent fallback
reproduces on all six machines. But the M1 Max (24c) gains LEAST of all,
breaking our 'narrower GPU => bigger win' reading (M5 10c gained most).
Real predictor: fused-kernel quality vs that chip's own matmul throughput —
M1 Max's fused path is only 0.87x unfused at every size, so missing it costs
little; M5's is 0.14x, so missing it costs a lot. Corrected in place rather
than left as a tidy-but-wrong story.
This commit is contained in:
modelbeast 2026-07-17 14:02:54 +10:00
parent 6b651f6f8e
commit 5f87aec717
1 changed files with 26 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

33
docs/2026-07-16-m-series-fleet-ablation-results.md
View File

@ -1,7 +1,7 @@
---
title: "M-series fleet ablation: M1 / M3 / M4 / M5 across 5 machines (Ultra → fanless Air)"
title: "M-series fleet ablation: 6 machines, M1 → M5 (8GB mini → 256GB Ultra)"
date: 2026-07-16
device: "M1 Ultra 128GB · M3 Ultra 256GB · M3 Air 16GB (fanless) · M4 Pro mini 24GB · M5 MacBook Pro 16GB"
device: "M1 Max 32GB (24c) · M1 Ultra 128GB (64c) · M3 Air 16GB (10c, fanless) · M3 Ultra 256GB (80c) · M4 Pro mini 24GB · M5 MacBook Pro 16GB (10c)"
script: scripts/bench_optimizations.py (unmodified) + engine-level tiled×compile extension
---
@ -56,11 +56,30 @@ happy to run follow-ups on any of them.
| (1, 16, 2048) | 11.44 ms *(1.06×)* | **2.23 ms** *(0.22×)* | **5.1×** |
| (4096, 2, 64) | 16.72 ms *(0.97×)* | **5.29 ms** *(0.25×)* | **3.2×** |
Why bigger on the laptop: the fused kernel's advantage over unfused scales inversely
with GPU width. At seq 4096 fp32 (hdim 64) fused-vs-unfused is **7.1× on M5** vs
~1.9× on M3 Ultra — a 10-core GPU cannot brute-force the materialized N×N path the
way an 80-core Ultra can. So the head_dim-56 fallback costs Studio owners ~2× and
**MacBook owners ~5×**. Padding is not a big-iron optimization; it's a laptop one.
**The fallback is universal; the size of the win is not.** Measured on six machines
(padding 56→64 at `(1,16,4096)` fp32). Every machine shows the fallback at hdim 56
(fused/unfused 0.97–1.06×, i.e. the fast kernel never engages) — but the payoff varies
~4× and **does not track GPU width monotonically**:
| machine | GPU cores | hdim 56 | hdim 64 | **padding speedup** | fused/unfused @4096 |
|---|---|---|---|---|---|
| M5 MacBook Pro | 10c | 44.00 ms | 8.91 ms | **4.9×** | 0.14× |
| M1 Ultra | 64c | 150.5 ms | 53.8 ms | **2.7×** | 0.59× |
| M3 Air | 10c | 138.9 ms | 54.3 ms | **2.6×** | 0.40× |
| M3 Ultra | 80c | 63.2 ms | 28.6 ms | **2.2×** | 0.53× |
| **M1 Max** | **24c** | 20.16 ms | 15.79 ms | **1.3×** | **0.87×** |
We initially read this as "narrower GPU ⇒ bigger win" (M5 10c gains most, M3 Ultra 80c
least). **The M1 Max refutes that**: it is mid-width (24c) yet gains the *least* of all.
The actual predictor is each chip's **fused-kernel quality relative to its own raw matmul
throughput** — the M1 Max's fused path is only ~0.87× its unfused path at every size
(i.e. MLX's SDPA kernel barely beats a plain matmul there), whereas the M5's is 0.14×.
Where the fused kernel is strong, missing it is expensive; where it's weak, missing it
barely matters.
Practical upshot is unchanged and *strictly positive*: padding to 64 helps on **every
Apple GPU generation tested** — by 1.3× (M1 Max) to 4.9× (M5) — and hurts nowhere,
despite ~14 % more FLOPs.
Suggested changes: (a) pad qkv projections to head_dim 64 (at minimum for the global
blocks) so sdpa's fast kernel actually engages — M1 benefits *more* than M3; (b) until