The flex_gemm_sparse_attn backend is now flash-attention-v2 with
simdgroup_matrix_multiply_accumulate for Q@K^T and P@V plus simd-shuffle
softmax row reductions, and wins 5–15× over SDPA-padded-CPU-bounce at
every measured shape including max_seqlen=2048. Production decoder block
on M3 Max: 5.04× wall-clock vs all-SDPA-padded baseline (3.49 ms vs
17.57 ms), entirely from this change being the default.
The existing __flex_gemm_works_on_mps() probe covers the same package
the attention path lives in, so the gate is identical: if conv probe
passes, set both CONV='flex_gemm' and ATTN='flex_gemm_sparse_attn'.
SPARSE_ATTN_BACKEND= (or ATTN_BACKEND=) env override is unchanged.
Also fix benchmarks/e2e_decoder.py to inject the repo root into
sys.path so it runs as `python benchmarks/e2e_decoder.py` from any cwd.
End-to-end micro-bench for the production decoder block at res=32 ch=64
with seqlens=[256, 192, 128, 64]. Three stages reported (convs-only /
attn-only / combined block) at fp16, vs the all-SDPA-padded baseline.
Used to track the cumulative effect of the mtlgemm flash-attention-v2
fwd + bwd work on the actual decoder hot path.
