THE DECIMATION FLOOR WAS MISDIAGNOSED. I attributed it to ~180k boundary edges. It is
non-manifold edges. Measured on the shipped 500k mesh:
boundary edges 32,370
NON-MANIFOLD 81,112 <- the actual blocker, 2.5x more
Quadric decimation cannot collapse an edge shared by more than two faces. Upstream's
own fix (fill_holes, via CUDA-only cumesh) targets boundaries and caps at
max_hole_perimeter=3e-2, so it was never going to help: trimesh's equivalent moved
boundaries 32,370 -> 30,990 and the floor only 214k -> 210k. That falsified it.
--manifold: voxelise -> fill -> marching cubes. Removes BOTH classes at once and so
closes three of the four items in one change:
as shipped 499,984 faces bnd 32,370 nonmani 81,112 watertight=F IoU 0.969
remeshed 1,178,142 faces bnd 0 nonmani 0 watertight=T IoU 0.949
-> 20k 19,998 faces bnd 0 winding consistent IoU 0.956
25x smaller, fully manifold, consistent winding, for 1.3% silhouette IoU. Lossy by
design - it gives up the dual grid's open-surface representation - so it is opt-in.
UV BAKE is unblocked by the same change: its cost is driven by face count, not by
remesh. 5.0s at 20k faces against >20min at 214k. No longer offline-only when paired
with manifold.
THE SCALING TRAP, worth knowing: marching_cubes returns vertices in VOXEL INDEX space.
Translating without apply_scale(pitch) leaves the mesh ~292x too large. It still
exports and renders as a plausible object; it silhouettes at IoU 0.08. That is how it
was caught.
MoGe-2 CAMERA is now wired and is the default, matching upstream; --fixed-fov keeps
the old constant. It runs once per image in torch/MPS, ~0.4s after load.
Reporting this one straight: it did NOT improve the samples. On 1_img, fixed 49.1 deg
scored 0.893 and MoGe's 29.7 deg scored 0.883. Two caveats keep it as the default
anyway - the silhouette metric projects with the SAME FOV used to generate, so a
wrong-but-consistent camera can still score well and the metric cannot fully arbitrate
camera correctness; and the bundled samples are synthetic renders, not the photographs
MoGe reads. Real photos are the intended input here, and upstream estimates too. But
the constant is one flag away and the measurement is on record rather than assumed.
Operator gains manifold, divisions, fixed_fov. README and PROFILE.md corrected where
they repeated the boundary-edge claim.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The status list still said the pipeline was unwired. It runs end to end now:
proj conditioning (all four extractors at corr 1.00000000), the NAF branch without
natten, the full 32^3 -> refine -> 64^3 cascade at silhouette IoU 0.969, cleanup,
and the MODELBEAST operator. Textures remain the one open item.
Also records the decimation table, because the floor is a real constraint and not
obvious: 500k faces is effectively lossless (IoU 0.965) but ~214k cannot be beaten,
and reaching it costs fidelity (0.823). Cause is the ~180k boundary edges the dual
grid emits for open surfaces - quadric decimation will not collapse them at any
setting, and o_voxel's remesh ran >20min on 214k faces before being killed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
All seven Pixal3D models now load and run. shape_dec 292/292, tex_dec 284/284, both
with zero missing/unmapped/mismatched keys - the 8-param difference between them is
exactly the four to_subdiv layers, since tex_dec has pred_subdiv=False.
Behaviour is right: 12 latent voxels grow SELECTIVELY through four stages
(12 -> 91 -> 193 -> 358 -> 1150) rather than x8 each time, which would have reached
49,152. Scale lands at 1/16. shape_dec emits the hardcoded 7 channels and its vertex
head produces offsets inside the [-0.5, 1.5] band its sigmoid+voxel_margin allows.
tex_dec, guided by shape_dec's masks, reproduces exactly the same voxel count.
VERIFICATION CAVEAT, recorded in the README: these two are the only models using sparse
conv, so upstream cannot run here and there is no numerical oracle. Unlike the five
models verified at correlation 1.0, these are checked structurally and behaviourally
only. Weaker evidence, and labelled as such rather than presented alongside the
verified results.
74/74 params, max abs diff 4.6e-4 on values around -163 (~3e-6 relative), 60ms for a
16^3 latent -> 64^3 occupancy grid.
Fully dense, so no sparse ops involved - but MLX's Conv3d is channels-LAST where torch
is NCDHW, so tensors are carried channels-last throughout and transposed only at the
boundaries. The converter already emits [O,kz,ky,kx,I] to match. pixel_shuffle_3d had
to be rewritten for that layout: the (H,s)(W,s)(D,s) interleave order is what matters
and getting it wrong scrambles the grid while preserving its shape.
This completes the structure stage end to end: image -> ss_flow -> latent -> ss_dec.
slat_flow joins ss_flow: max abs diff 9.3e-6, 700/700 params, against upstream running
on CPU torch. All three SLAT checkpoints load clean and run (img2shape 512/1024 and
imgshape2tex, the last taking 64 in-channels since shape is concatenated).
The SLAT flows differ from ss_flow in two ways, both handled in the shared core:
tokens are a SparseTensor's voxels so attention runs per batch item, and RoPE phases
are NOT shipped - positions are the input's own coordinates, so they are derived at
call time by rope_phases_from_coords.
tests/oracle_slat.py keeps the CPU-torch oracle harness: it patches both the dense and
sparse flash-attn kernels with SDPA equivalents. Use it rather than reasoning about
correctness - it has already caught three bugs a perfect 700/700 key match did not.
700/700 params, max abs diff 1.2e-5 on the real 1.3B checkpoint.
Key realisation: the flow models have no sparse conv, so upstream RUNS on CPU torch
with flash-attn swapped for SDPA. That gives a real numerical oracle - unavailable for
the sparse path, where spconv cannot be installed at all.
It was needed. Three bugs survived a loader reporting a perfect 700/700 with zero
missing and zero unmapped keys:
- a parameterless final LayerNorm (no params -> no checkpoint trace) that the output
was 200x too large without
- rope_phases being complex64, so the rotation is a complex multiply
- qk_rms_norm belonging before rope rather than after
Weight-key matching is necessary but nowhere near sufficient for a port.
Converter is light because Pixal3D ships safetensors with sibling .json configs, so
there is no architecture to infer and no pickle to unpack. Only rank-5 tensors are
rearranged; dtype is preserved (upcasting fp16->fp32 doubled 24GB for no benefit) and
the KRSC remap is verified a pure permutation of values.
Measured: the four flow models (~20GB) have ZERO 5-D tensors - pure transformers that
never touch sparse conv. shape_dec/tex_dec carry 40 KRSC kernels each, ss_dec 20 dense
Conv3d, and classify_5d separates them correctly on the real weights.
Corrects the earlier 'gap is two ops' claim: that was right about modules/sparse but
undercounted the model blocks. The configs show all four flow models need RoPE,
qk_rms_norm and AdaLN modulation, and the decoders need SparseConvNeXtBlock3d and
SparseResBlockC2S3d.
Every SparseConv3d in the Pixal3D model is (channels, out, 3) - stride=1/padding=None,
so spconv dispatches to SubMConv3d, which trellis_sparse_mlx already implements and
benchmarks. attn_mode is 'full' throughout. The container split (VarLenTensor /
SparseTensor) and the get/register_spatial_cache spelling are already in the shared core.
Remaining gap is two ops: SparseUpsample and SparseSpatial2Channel, both cache-paired
with a matching downsample rather than recomputing structure.
