The texture flow is imgshape2tex - it denoises 32 PBR channels while SEEING the shape
latent, so in_channels is 64 against out_channels 32. Upstream feeds the shape latent
as concat_cond and the model does sparse_cat([x, concat_cond], dim=-1); both share
coords, so it reduces to a channel concat. Added to slat_flow and carried on the
sampler (it is fixed for the whole trajectory and must reach BOTH CFG branches).
Verified running on the real checkpoints: 3,988,052 PBR voxels x 6 channels in 65.8s
(base_color 0:3, metallic 3:4, roughness 4:5, alpha 5:6).
Two things here fail SILENTLY rather than loudly, so both are asserted in comments:
1. shape_slat arrives DENORMALISED - the shape stage un-standardises it for the
decoder - but the texture flow was trained against the standardised form. It is
re-normalised before use as concat_cond. Skipping that gives a plausible mesh with
wrong colours, not an error.
2. tex_dec has pred_subdiv=False: it cannot invent subdivisions and must be handed the
shape decoder's subs as guides, so texture voxels land on the geometry that was
actually built.
The decoder's output is mapped * 0.5 + 0.5 into [0,1], the range o_voxel expects.
BAKE ORDER. Handing o_voxel the raw ~8M-face mesh hangs - the same wall the standalone
remesh test hit (killed at 20min), and the trellis2 lane's own operator note says the
uncapped bake peaks at 75GB. So the mesh is welded, stripped of floaters and decimated
BEFORE baking; the baker samples the attribute VOLUME at mesh positions, so a
decimated mesh still gets correct colours. Measured on the way through:
welded 3,988,052 -> 3,983,672 verts
floaters 12 components -> 1 kept, 6,332 faces dropped
decimated 7,996,876 -> 214,322 faces
pre-bake 34.6s
That floater count is worth noting: 12 components, not the 52,855 the first health
pass reported. Welding first is what makes the difference.
remesh now defaults OFF in to_glb, unlike upstream. Upstream runs on CUDA; this is the
CPU/Metal build and its remesher took >20 minutes on a 214k-face mesh. It is also
handed an already-clean mesh, so there is far less for it to fix.
Operator gains texture + texture_size params; geometry-only stays the default because
it is ~3min against the textured path's extra flow and bake.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
base_color 0:3, metallic 3:4, roughness 4:5, alpha 5:6 - the pipeline's own
pbr_attr_layout. o_voxel's baker indexes this dict BY NAME and raises KeyError deep
inside to_glb on any missing slot rather than at the call, so a partial layout looks
like a baker bug. Needed by the texture stage; recorded now while it is in hand.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
image_to_mesh() now runs the real cascade, not the single-stage shortcut:
structure 3048 voxels @32^3 (64^3 occupancy, MAX-POOLED DOWN)
LR SLAT 3048 x 32 shape_512 extractor
refine 13147 coords @64^3 four decoder stages -> coords -> quantise
HR SLAT 13147 x 32 shape_1024 extractor
mesh 3988052 verts, 7996876 faces @1024^3
TOTAL 258.4s, peak 27.9GB with every model resident silhouette IoU 0.969
Three things the cascade needed:
1. occupied_coords_at() - ss_dec always decodes 64^3 but the cascade STARTS at 32^3.
Upstream max-pools the boolean grid down by the ratio (a voxel survives if ANY of
its eight children was occupied). I had been feeding the raw 64^3 set to the HR flow.
2. decoder.upsample() - pushes the LR latent four stages in and returns COORDS, not
features. The predicted subdivisions grow the occupied set; those coords quantise
onto the HR flow's grid. Stops BEFORE stage `upsample_times`, as upstream does;
one stage further doubles the resolution and misplaces every voxel.
3. grid_resolution override on ProjConditioner - upstream backs the HR grid off in
128-unit steps while the token count exceeds max_num_tokens, so a dense object
degrades instead of exploding. refine_coords() implements that loop.
I WAS WRONG ABOUT THE HALO. The previous commit blamed the single-stage shortcut for a
0.639 silhouette IoU and predicted the cascade would fix it. The cascade measured
0.640 - no change. The real fault was in my VERIFICATION, not the pipeline: o_voxel
returns vertices in the voxel-grid frame, while ProjGrid rotates its lattice by
_BLENDER_ROT before projecting. Rotating the mesh the same way scores 0.969 on the
same geometry the earlier commit had already produced. Added mesh.to_camera_frame()
so the trap is named where it bites; the earlier mesh was correct all along.
The cascade is still the right thing - it is the shipped path, and staged loading
halves peak memory (12.8GB vs 22.6GB) when models are released between stages.
Also adds models.load_all(), so a server builds all five models plus both conditioners
ONCE. Warmup is ~71s against ~17s of compute, so an operator must never fork per job.
Holding everything resident costs 27.9GB peak - nothing on a 256GB box.
scripts/image_to_mesh.py exits non-zero if IoU < 0.85: a run that completes with a bad
reconstruction has failed even though nothing raised.
27/27 green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The whole geometry chain now runs on a real photograph:
[1] occupancy 12948 voxels 19.7s
[2] cond proj (1, 262144, 2048) @ 64^3 2.6s
gathered proj (12948, 2048)
[3] SLAT (12948, 32) 88.7s
[4] MESH 3556515 verts, 7071196 faces 11.0s grid 1024^3
peak 22.6 GB, bounds inside the unit cube
Two bugs fixed on the way:
1. "global" must be FLAT [M,C] for the sparse blocks, not the dense stage's [B,T,C].
The sparse cross-attention takes a token stack plus an explicit layout, so the
dense shape dies inside to_kv's reshape rather than anywhere informative. Gathering
now reshapes it, and refuses batch > 1 rather than silently mislabelling a layout.
2. o_voxel needs the decoder OUTPUT grid, not its configured resolution. The shape
decoder applies four 2x upsamples, so a res-64 latent decodes into 1024^3, while
the config says 256 (upstream overrides it per run via set_resolution). Passing 256
raised an opaque out-of-bounds inside o_voxel's hashmap insert. Added
output_resolution() and a guard that names the real cause.
HONEST LIMITATION - this is NOT yet the shipped cascade. Upstream's
sample_shape_slat_cascade runs the 512 flow (res 32) first, denormalises, UPSAMPLES
THE COORDINATE SET through the shape decoder, then runs the 1024 flow on the refined
coords. Running the HR flow straight off the 64^3 occupancy set yields a complete,
exportable mesh whose silhouette IoU is 0.639 - against 0.842 for the occupancy grid
that seeded it. The gap is a halo of geometry outside the true silhouette, exactly
what the missing coordinate refinement would prune. Do not read the current mesh
quality as the model's; wiring the cascade is the next step.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The shape decoder's 7 channels are not an SDF — O-Voxel solves a QEF over a Flexible
Dual Grid, which is what lets it carry open and non-manifold surfaces:
0:3 vertex offset in-voxel, (1+2m)*sigmoid(v)-m so it may sit OUTSIDE its own cell
3:6 per-axis intersection logits, thresholded at 0
6:7 quad split weight through softplus
mesh.py is the MLX->torch boundary for export. o_voxel's convert/postprocess are
native (C++/Metal) and deliberately NOT ported: o-voxel builds a CPU CppExtension when
CUDA is absent, and the trellis-2 lane on this fleet already runs it with a Metal
baker, so reusing that build beats reimplementing a QEF solver in MLX. Installed into
the shared venv from ~/Documents/trellis-2-mrp-mlx/o-voxel; it needs cv2 and xatlas,
and NOT utils3d (which drags in open3d, with no cp312 wheel).
Verified against the REAL shape_dec (292/292 params, resolution 256):
decoded 5954 voxels x 7ch -> 5954 vertices, 6886 faces -> GLB written
Not watertight, correctly: the input was a random latent, and FlexiDualGrid represents
open surfaces by design. Vertices land inside the octant of the unit cube matching the
sparse coords fed in, which is the check that the grid indexing is right.
Still to wire: the SLAT stage itself (sparse latents seeded from the occupancy coords,
with proj features gathered at those coords).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
