80991a8fe9
Every SparseConv3d in the Pixal3D model is (channels, out, 3) - stride=1/padding=None, so spconv dispatches to SubMConv3d, which trellis_sparse_mlx already implements and benchmarks. attn_mode is 'full' throughout. The container split (VarLenTensor / SparseTensor) and the get/register_spatial_cache spelling are already in the shared core. Remaining gap is two ops: SparseUpsample and SparseSpatial2Channel, both cache-paired with a matching downsample rather than recomputing structure. |
||
|---|---|---|
| .gitignore | ||
| CLAUDE.md | ||
| README.md | ||
pixal3d_mrp_mlx
MLX port of Pixal3D (TencentARC + Tsinghua, SIGGRAPH 2026, MIT) — pixel-aligned single-image 3D generation — for Apple Silicon.
Built on trellis_sparse_mlx, the shared TRELLIS-lineage
sparse core. Pixal3D and LATO.2 inherit the same sparse module from TRELLIS.2, so the
expensive part — submanifold sparse convolution, which has no Metal implementation — is
already done and tested there.
Why Pixal3D
It back-projects pixel features directly into 3D rather than injecting them through attention, so silhouettes stay exact to the source image. Different failure mode from TRELLIS/Hunyuan, and complementary to them.
Upstream needs ~24 GB VRAM, which is a wall on consumer Nvidia and a non-issue on a 128 GB+ Ultra.
Scope, measured against the shared core
| Need | Status |
|---|---|
SparseConv3d — every call is (c, out, 3), i.e. stride=1/padding=None → SubMConv3d |
✅ in shared core |
attn_mode='full' (the only mode used) |
✅ in shared core |
SparseLinear, norms, activations, ResBlock, transformer blocks |
✅ in shared core |
SparseDownsample(2) |
✅ in shared core |
VarLenTensor / SparseTensor split + get/register_spatial_cache |
✅ added to shared core |
dense nn.Conv3d(.., 2, stride=2) in sparse_structure_vae |
✅ maps to mlx.nn.Conv3d |
SparseUpsample(2) |
❌ to do — cache-paired inverse of a downsample |
SparseSpatial2Channel(2) |
❌ to do — sparse pixel-shuffle, spatial→channel |
So the gap is two ops, both of which work through the spatial cache (they pair with a matching downsample rather than recomputing structure). Everything else is already covered by work done for LATO.2.
Model surface
pixal3d/models/
sparse_structure_vae.py dense Conv3d — voxel structure
sparse_structure_flow.py structure flow (SS)
structured_latent_flow.py SLAT flow
sc_vaes/sparse_unet_vae.py the only file using sparse conv
Weights: 24.04 GB across 19 files (1.3B DiTs at 512/1024 + shape/tex decoders).
Status
- Scoped against the shared core — gap is two ops
- Weights downloaded
SparseUpsample,SparseSpatial2Channel- Weight converter
- Model graphs
- End-to-end