diff --git a/BENCHMARKS.md b/BENCHMARKS.md index 1fd7f21..b2ef3a2 100644 --- a/BENCHMARKS.md +++ b/BENCHMARKS.md @@ -92,6 +92,8 @@ Corridor Digital's keyer (`vendor/corridorkey`, 14.4k★) with the native MLX ba **Fleet verdict:** best-quality config = `tile_size=512, overlap=64` + our patch: M3 ~0.40 fps, M1 ~0.24 fps at 2048², IoU 0.932, 2.3 GB — runs on every node including the M4 24GB. Throughput mode: full-1024 (1.65/1.34 fps, IoU 0.908). MLX gaps: blue-screen checkpoint + despill/despeckle not yet on MLX (torch backend covers those). License: CC BY-NC-SA (non-commercial). +**8-machine matrix (2026-07-17) — tiled keys 2048 on EVERY Mac, same accuracy.** `tile_size=768, overlap=64` + our compile patch, full 2048² input, **peak ~2.2GB and alpha MAE 0.00849 identical on all**: M3 Ultra 1949ms · M1 Ultra 3275ms · M1 Max(24c/32GB) 4062ms · M4 mini(10c/16GB) 7284ms · **OG M1 mini (2020, 8c/8GB) 14269ms**. Full-frame 2048 needs 26GB → unreachable on most Macs ever sold; **tiled isn't the fallback, it's the answer**. Also: hdim-56 fast-path fallback confirmed on all 8 (padding win 1.3×–4.9×, tracks fused-kernel quality **not** core count — the M1 Max refuted our width theory, the M4 mini then fit the corrected one as a prediction); M5's MLX gains live almost entirely in the fused kernel; fanless M3 Air does **not** throttle (1.01× at 5 min). + **Fleet ablation follow-up (2026-07-16, full report in the fork: `monster/corridorkey-mrp-mlx` → `docs/2026-07-16-m-series-fleet-ablation-results.md`):** ran upstream's own 6-toggle benchmark matrix on M3+M1+M4. New best config on BOTH Ultras = **tiled 768/64 + compile** (M3 1949ms, M1 3275ms per 2048² raster — vs 2788/5373 full-frame) at 2.4GB. Big discovery: **`sdpa` is a 4.5× regression on M1-class GPUs at 2048 (18.6s vs 4.1s) and free on M3; bf16 is neutral on both** — gate sdpa by GPU generation, the "M1 bf16 danger" assumption is wrong for this workload. `stage_gc` harmful on Ultras (0.53–0.66×). M4 24GB swaps at full-frame 2048 (~20–24s all configs) → tiled mandatory there; healthy ≤1024. ### Phase D — hunyuan Studio tuning (2026-07-16, M3 Ultra) → new operator defaults