Commit Graph

3 Commits

Author SHA1 Message Date
Pedro Augusto
c869eb995a sparse/attention: add 'flex_gemm_sparse_attn' backend option
Wires the new mtlgemm fused sparse attention kernel through the ATTN
backend selector. Dispatches to the fused Metal kernel when
max(max_q, max_kv) <= 256 (where the naive per-thread-serial-KV kernel
beats SDPA-padded on M3 Max), and falls through to an inline SDPA-padded
path for larger max_seqlen. Opt in via ATTN_BACKEND=flex_gemm_sparse_attn
or SPARSE_ATTN_BACKEND=flex_gemm_sparse_attn. Default on Darwin stays
'sdpa' — the threshold-based fallback doesn't yet prove a universal
win across the pipeline's attention shape distribution.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-07-17 07:51:05 +03:00
Pedro Augusto
9cd27537d9 add mlx backend for apple silicon with metal gpu acceleration 2026-07-17 07:51:05 +03:00
JeffreyXiang
de38fdd408 Init Release 2025-12-16 19:00:42 +00:00