The raw cascade output is ~4M verts / 8M faces and is not usable as-is. cleanup.py is
ordinary mesh hygiene, kept out of the model code, and the ORDER is the whole point:
weld -> strip floaters -> decimate -> strip again -> fix normals
CORRECTION TO THE FLOATER COUNT. The health pass reported 52,855 components with
52,838 fragments under 100 faces, and I took those for stray shells. They were mostly
NOT: the decoder emits per-voxel vertices, so coincident corners are duplicated and
the same continuous surface reads as tens of thousands of islands. Welding FIRST
collapses it to a single component, and only 6,332 faces are genuinely stray. Ordering
the pass the other way round removes 250k faces of real geometry and calls it cleaning.
Two performance fixes, both because an operator runs this every job:
- Component labelling is a scipy union-find over the VERTEX graph, not
trimesh.face_adjacency. Same answer, ~240s -> ~1s on this mesh.
- fix_normals runs LAST, on the decimated mesh. It walks face adjacency, so on the
raw 7.99M-face mesh it costs minutes and the result is then thrown away by
decimation. Cleanup went ~243s -> ~20s.
Decimation is iterative. A single fast_simplification call will not reduce past
roughly 4.4% of its input whatever target_reduction (or agg) is asked for: from 7.99M
faces, targets of 200k, 50k and 20k ALL returned 351,535. Repeated smaller passes get
further because each re-evaluates quadrics on the collapsed mesh.
MEASURED, on the sample:
target 500,000 -> 499,984 faces silhouette IoU 0.965 19.3s
target 200,000 -> 214,322 faces silhouette IoU 0.823 34.8s
target 100,000 -> 214,322 faces silhouette IoU 0.823
target 20,000 -> 214,322 faces silhouette IoU 0.823
HONEST LIMITATION: ~214k is a hard floor, and reaching it costs real fidelity
(0.965 -> 0.823). The cause is the ~180,000 BOUNDARY edges the Flexible Dual Grid
produces for open surfaces - quadric decimation will not collapse those, and no
aggressiveness setting changes it. Below ~214k needs a REMESH, not a decimator.
500k is effectively lossless and is the setting to use; anything under 214k is not
currently reachable and the loop stops rather than spinning.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
image_to_mesh() now runs the real cascade, not the single-stage shortcut:
structure 3048 voxels @32^3 (64^3 occupancy, MAX-POOLED DOWN)
LR SLAT 3048 x 32 shape_512 extractor
refine 13147 coords @64^3 four decoder stages -> coords -> quantise
HR SLAT 13147 x 32 shape_1024 extractor
mesh 3988052 verts, 7996876 faces @1024^3
TOTAL 258.4s, peak 27.9GB with every model resident silhouette IoU 0.969
Three things the cascade needed:
1. occupied_coords_at() - ss_dec always decodes 64^3 but the cascade STARTS at 32^3.
Upstream max-pools the boolean grid down by the ratio (a voxel survives if ANY of
its eight children was occupied). I had been feeding the raw 64^3 set to the HR flow.
2. decoder.upsample() - pushes the LR latent four stages in and returns COORDS, not
features. The predicted subdivisions grow the occupied set; those coords quantise
onto the HR flow's grid. Stops BEFORE stage `upsample_times`, as upstream does;
one stage further doubles the resolution and misplaces every voxel.
3. grid_resolution override on ProjConditioner - upstream backs the HR grid off in
128-unit steps while the token count exceeds max_num_tokens, so a dense object
degrades instead of exploding. refine_coords() implements that loop.
I WAS WRONG ABOUT THE HALO. The previous commit blamed the single-stage shortcut for a
0.639 silhouette IoU and predicted the cascade would fix it. The cascade measured
0.640 - no change. The real fault was in my VERIFICATION, not the pipeline: o_voxel
returns vertices in the voxel-grid frame, while ProjGrid rotates its lattice by
_BLENDER_ROT before projecting. Rotating the mesh the same way scores 0.969 on the
same geometry the earlier commit had already produced. Added mesh.to_camera_frame()
so the trap is named where it bites; the earlier mesh was correct all along.
The cascade is still the right thing - it is the shipped path, and staged loading
halves peak memory (12.8GB vs 22.6GB) when models are released between stages.
Also adds models.load_all(), so a server builds all five models plus both conditioners
ONCE. Warmup is ~71s against ~17s of compute, so an operator must never fork per job.
Holding everything resident costs 27.9GB peak - nothing on a 256GB box.
scripts/image_to_mesh.py exits non-zero if IoU < 0.85: a run that completes with a bad
reconstruction has failed even though nothing raised.
27/27 green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The whole geometry chain now runs on a real photograph:
[1] occupancy 12948 voxels 19.7s
[2] cond proj (1, 262144, 2048) @ 64^3 2.6s
gathered proj (12948, 2048)
[3] SLAT (12948, 32) 88.7s
[4] MESH 3556515 verts, 7071196 faces 11.0s grid 1024^3
peak 22.6 GB, bounds inside the unit cube
Two bugs fixed on the way:
1. "global" must be FLAT [M,C] for the sparse blocks, not the dense stage's [B,T,C].
The sparse cross-attention takes a token stack plus an explicit layout, so the
dense shape dies inside to_kv's reshape rather than anywhere informative. Gathering
now reshapes it, and refuses batch > 1 rather than silently mislabelling a layout.
2. o_voxel needs the decoder OUTPUT grid, not its configured resolution. The shape
decoder applies four 2x upsamples, so a res-64 latent decodes into 1024^3, while
the config says 256 (upstream overrides it per run via set_resolution). Passing 256
raised an opaque out-of-bounds inside o_voxel's hashmap insert. Added
output_resolution() and a guard that names the real cause.
HONEST LIMITATION - this is NOT yet the shipped cascade. Upstream's
sample_shape_slat_cascade runs the 512 flow (res 32) first, denormalises, UPSAMPLES
THE COORDINATE SET through the shape decoder, then runs the 1024 flow on the refined
coords. Running the HR flow straight off the 64^3 occupancy set yields a complete,
exportable mesh whose silhouette IoU is 0.639 - against 0.842 for the occupancy grid
that seeded it. The gap is a halo of geometry outside the true silhouette, exactly
what the missing coordinate refinement would prune. Do not read the current mesh
quality as the model's; wiring the cascade is the next step.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every SparseConv3d in the Pixal3D model is (channels, out, 3) - stride=1/padding=None,
so spconv dispatches to SubMConv3d, which trellis_sparse_mlx already implements and
benchmarks. attn_mode is 'full' throughout. The container split (VarLenTensor /
SparseTensor) and the get/register_spatial_cache spelling are already in the shared core.
Remaining gap is two ops: SparseUpsample and SparseSpatial2Channel, both cache-paired
with a matching downsample rather than recomputing structure.
