From ed2ee8dc161c85b0b96e02eaad25db96430cc6f4 Mon Sep 17 00:00:00 2001 From: m3ultra Date: Sun, 19 Jul 2026 00:31:55 +1000 Subject: [PATCH] =?UTF-8?q?benchmarks:=206-machine=20fleet=20MLX=20matrix?= =?UTF-8?q?=20=E2=80=94=20bf16-on-M1=20answered,=20head=5Fdim-64=20padding?= =?UTF-8?q?=20critical=20on=20M4/M5?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 --- BENCHMARKS.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/BENCHMARKS.md b/BENCHMARKS.md index 14fbebb..a88ddfa 100644 --- a/BENCHMARKS.md +++ b/BENCHMARKS.md @@ -147,3 +147,24 @@ regression suite (12 checks, ~30s). can make a node look 5–11x slower and it is invisible to load average (the M3 Ultra reads load 2.45 with its GPU pinned). Anything measured here should use min-of-runs and check `ioreg … IOAccelerator` "Device Utilization %" *before* touching the GPU. See PERFCHECK.md. + +## 2026-07-19 — fleet MLX capability matrix (mlx 0.32.0 everywhere) + +Probe: 4096² matmul TFLOPS (fp16/bf16), 256MB elementwise GB/s, sdpa head_dim 56 vs 64 (B1·H8·L2048). + +| host | chip | RAM | fp16 TF | bf16 TF | ~GB/s | sdpa d56/d64 | +|---|---|---|---|---|---|---| +| m3ultra | M3 Ultra | 256 | 23.48 | 23.54 | 400 | 1.51× | +| ultra | M1 Ultra | 128 | 16.31 | 13.16 | 297 | 0.94× | +| JING5 | M5 | 16 | 8.61 | 8.99 | 43* | **4.72×** | +| m4pro | M4 Pro | 24 | 5.24 | 3.86 | 225 | 1.29× | +| m4mini | M4 | 16 | 3.65 | 3.65 | 98 | 1.55× | +| AIRPRO | M3 | 16 | 1.55 | 1.64 | 87 | 0.95× | + +*JING5 bandwidth measured under interactive load — retest idle before trusting. + +Verdicts: +- **bf16-on-M1 question answered**: works, ~20% slower than fp16 (16.3→13.2). Ship fp16 on M1-class, bf16 free on M3 Ultra/M4/M5. +- **head_dim pad-to-64 matters MOST on newest chips** (M5 4.7×!, M4-class ~1.3-1.6×, M3 Ultra 1.5×) and ~zero on M1-class (no fused fast path there anyway — matches the corridorkey ablation). +- Fleet total ≈ 59 TFLOPS fp16; the two Ultras are 2/3 of it. TRELLIS.2-MLX lane assignment: m3ultra primary, ultra second (fp16), M4 Pro light ops. +- Probe script: reusable one-liner via `uv run --with mlx` — mlxprobe.py (this bench).