corridorkey-mrp-mlx/docs/benchmarks/2026-03-09-wave2-ablation-results.md

title	date	device	script
Wave 2 optimization ablation benchmarks	2026-03-09	Apple Silicon (unified memory)	scripts/bench_optimizations.py

Wave 2 Optimization Ablation Benchmarks

Ablation Sweep (baseline = all optimizations off)

Toggle flags: slim, stage_gc, sdpa, bf16, fused_decode, gpu_preprocess

512x512

Config	Median (ms)	Min (ms)	Peak Mem (MB)	vs Baseline
baseline	119.6	118.3	2419	1.00x
slim+sdpa+bf16+fused_decode+gpu_preprocess	120.0	119.2	2413	1.00x
slim+stage_gc+bf16+fused_decode+gpu_preprocess	147.8	146.7	2306	0.81x
slim+stage_gc+sdpa+bf16+fused_decode+gpu_preprocess	149.2	147.9	2344	0.80x
slim+stage_gc+sdpa+bf16+fused_decode	149.3	147.6	2344	0.80x
stage_gc+sdpa+bf16+fused_decode+gpu_preprocess	150.5	148.7	2344	0.79x
slim+stage_gc+sdpa+bf16+gpu_preprocess	151.6	148.5	2344	0.79x
slim+stage_gc+sdpa+fused_decode+gpu_preprocess	151.9	149.1	2344	0.79x

1024x1024

Config	Median (ms)	Min (ms)	Peak Mem (MB)	vs Baseline
slim+sdpa+bf16+fused_decode+gpu_preprocess	583.3	579.9	3819	1.05x
baseline	610.7	606.2	3673	1.00x
stage_gc+sdpa+bf16+fused_decode+gpu_preprocess	655.5	650.6	3673	0.93x
slim+stage_gc+sdpa+bf16+fused_decode+gpu_preprocess	655.9	650.6	3673	0.93x
slim+stage_gc+bf16+fused_decode+gpu_preprocess	657.5	650.6	3673	0.93x
slim+stage_gc+sdpa+bf16+gpu_preprocess	675.3	667.7	3673	0.90x
slim+stage_gc+sdpa+fused_decode+gpu_preprocess	686.0	682.7	3673	0.89x
slim+stage_gc+sdpa+bf16+fused_decode	687.5	680.4	3673	0.89x

2048x2048

Config	Median (ms)	Min (ms)	Peak Mem (MB)	vs Baseline
baseline	4984.7	4954.6	26689	1.00x
stage_gc+sdpa+bf16+fused_decode+gpu_preprocess	5016.8	4840.4	26661	0.99x
slim+stage_gc+sdpa+bf16+fused_decode+gpu_preprocess	5048.9	4848.1	26661	0.99x
slim+sdpa+bf16+fused_decode+gpu_preprocess	5175.5	5065.2	27245	0.96x
slim+stage_gc+sdpa+bf16+fused_decode	5375.2	5083.1	26661	0.93x
slim+stage_gc+sdpa+bf16+gpu_preprocess	5396.3	5357.0	26661	0.92x
slim+stage_gc+sdpa+fused_decode+gpu_preprocess	5683.3	5546.8	26661	0.88x
slim+stage_gc+bf16+fused_decode+gpu_preprocess	5771.3	5643.0	26689	0.86x

Tiled vs Full-Frame at 2048x2048

Config	Median (ms)	Min (ms)	Peak Mem (MB)	Speed vs FF	Mem vs FF
full-frame 2048	5031	4838	26281	1.0x	1.0x
tiled 768/64	3344	3311	2302	1.5x	11.4x less
tiled 512/64	3978	3963	2133	1.3x	12.3x less
tiled 512/128	4055	4025	2133	1.2x	12.3x less
tiled 1024/64	6144	6125	3425	0.8x	7.7x less

Key Findings

1. stage_gc hurts at 512/1024, breaks even at 2048. GC pauses cost more than memory savings at small resolutions. Only at 2048 does the computation graph get large enough to justify intermediate materialization.

2. No single optimization consistently beats baseline across resolutions. At 512, baseline wins. At 1024, slim+sdpa (no stage_gc) wins by 5%. At 2048, nothing reliably beats baseline.

3. Memory scales super-linearly: 2.4GB → 3.7GB → 26.7GB for 512 → 1024 → 2048. The 7x jump from 1024→2048 indicates lazy graph accumulation across 24 backbone blocks dominates at high resolution.

4. Tiled 768/64 is the clear winner at 2048 — 1.5x faster AND 11.4x less memory than full-frame. The smaller per-tile computation graph avoids the massive intermediate state buildup.

5. bf16 can hurt at 2048 — possible promotion overhead in large graphs.

6. gpu_preprocess is the most impactful single flag — dropping it consistently hurts latency.

7. Phase 6 (GPU tile accumulators) is low priority — numpy accumulator transfer overhead is negligible vs compute. The memory problem is entirely graph-side.

Recommendations

• Default to tiled 768/64 for 2048x2048 production use
• For ≤1024, use full-frame with slim+sdpa+bf16+fused_decode+gpu_preprocess
• stage_gc should be off by default, enabled only if memory-constrained at ≥2048