lato.2_mrp_mlx/lato_mlx/sparse
John 95fd4496da Sparse layer set: linear, norms, activations, resblock, attention, transformer blocks
All verified against torch, which IS available here - only spconv was not. So unlike
the submanifold conv these compare against upstream's real semantics rather than a
hand-written paraphrase. 6/6 pass, max err 9.5e-7.

The norm distinction is the trap: LayerNorm32 is applied to x.feats (per-voxel over
channels) while SparseGroupNorm32 reshapes [N_b,C] -> [1,C,N_b] per batch item, so its
statistics span channels-in-group AND voxels. Both produce identical shapes, so a mixup
is numerically silent - there is an explicit test asserting the group norm does NOT
match a per-voxel group_norm.

Architecture confirmed from the converted weights rather than constructor defaults:
no rope, no qk_rms_norm, transformer norms non-affine, ResBlock norm1 affine/norm2 not.
2026-08-02 10:18:38 +10:00
..
__init__.py MLX sparse core: SubMConv3d + SparseTensor + weight converter 2026-08-02 10:04:24 +10:00
conv.py Cache the prepared device gather index (5.9x total on the hot op) 2026-08-02 10:09:57 +10:00
ops.py Sparse layer set: linear, norms, activations, resblock, attention, transformer blocks 2026-08-02 10:18:38 +10:00
tensor.py MLX sparse core: SubMConv3d + SparseTensor + weight converter 2026-08-02 10:04:24 +10:00