diff --git a/docs/2026-07-16-m-series-fleet-ablation-results.md b/docs/2026-07-16-m-series-fleet-ablation-results.md index 6486023..77a18dc 100644 --- a/docs/2026-07-16-m-series-fleet-ablation-results.md +++ b/docs/2026-07-16-m-series-fleet-ablation-results.md @@ -8,11 +8,11 @@ script: scripts/bench_optimizations.py (unmodified) + engine-level tiled×compil # M-series Fleet Ablation Benchmarks Same methodology as the 2026-03-09 wave2 doc (`--sweep ablation --resolution 512 1024 2048`, -checkpoint weights, 3 warmup + 10 bench runs), executed across **five machines** spanning the -Apple Silicon range — **M1 Ultra (64c) · M3 Ultra (80c) · M3 Air 16GB (10c, fanless) · M4 Pro -mini 24GB · M5 MacBook Pro 16GB (10c)** — i.e. every current GPU generation, Studio→fanless -laptop, and a matched-width M3-vs-M5 pair. Contributed from a multi-Mac render-farm setup; -happy to run follow-ups on any of them. +checkpoint weights, 3 warmup + 10 bench runs), executed across **six machines** spanning the +Apple Silicon range — **M1 Max 32GB (24c) · M1 Ultra 128GB (64c) · M3 Air 16GB (10c, fanless) · +M3 Ultra 256GB (80c) · M4 Pro mini 24GB · M5 MacBook Pro 16GB (10c)** — i.e. every current GPU +generation, Studio→fanless laptop, an M1 width ladder (24c→64c) and a matched-width M3-vs-M5 +pair. Contributed from a multi-Mac render-farm setup; happy to run follow-ups on any of them. ## Headline findings @@ -47,8 +47,8 @@ happy to run follow-ups on any of them. | (1, 16, 4096, ·) | 3.8 / 7.2 ms → 1.9× | 7.5 / 11.1 ms → 1.5× | | (4096, 2, 64, ·) | 1.3 / 2.5 ms → 1.9× | 1.5 / 2.6 ms → 1.7× | - **…and on M5 it's worth ~5×, i.e. this fix matters MOST for laptop users.** Same - experiment on an M5 MacBook Pro (10-core GPU, 16 GB) — hdim 56 vs 64, fp32: + **On M5 it's worth ~5× — the largest win we measured.** M5 MacBook Pro (10-core GPU, + 16 GB), hdim 56 vs 64, fp32: | shape | hdim 56 (today) | hdim 64 (padded) | **speedup from padding** | |---|---|---|---| @@ -135,8 +135,9 @@ ratio (lower = fused is winning by more): Absolute fused times at 4096/fp32: M3 3.86 ms · M1 7.19 ms · **M5 7.22 ms** — an M5 laptop matches an M1 Ultra on *fused* attention while being ~4× slower unfused (51.5 ms -vs 12.2 ms). The narrower the GPU, the more the fast path is worth. (M5 numbers taken on -a machine in active desktop use; treat as indicative, not lab-clean.) +vs 12.2 ms). **How much the fast path is worth is chip-specific, not width-specific** +(M1 Max 24c: 0.87×; M5 10c: 0.14×) — see the six-machine table above. (M5 numbers taken +on a machine in active desktop use; treat as indicative, not lab-clean.) ### Generational delta at *matched* GPU width — M3 Air vs M5, both 10-core