bf8b1a35d5
7 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
m3ultra
|
bf8b1a35d5 |
Close all four open items: MoGe camera, manifold remesh, winding, UV bake
THE DECIMATION FLOOR WAS MISDIAGNOSED. I attributed it to ~180k boundary edges. It is non-manifold edges. Measured on the shipped 500k mesh: boundary edges 32,370 NON-MANIFOLD 81,112 <- the actual blocker, 2.5x more Quadric decimation cannot collapse an edge shared by more than two faces. Upstream's own fix (fill_holes, via CUDA-only cumesh) targets boundaries and caps at max_hole_perimeter=3e-2, so it was never going to help: trimesh's equivalent moved boundaries 32,370 -> 30,990 and the floor only 214k -> 210k. That falsified it. --manifold: voxelise -> fill -> marching cubes. Removes BOTH classes at once and so closes three of the four items in one change: as shipped 499,984 faces bnd 32,370 nonmani 81,112 watertight=F IoU 0.969 remeshed 1,178,142 faces bnd 0 nonmani 0 watertight=T IoU 0.949 -> 20k 19,998 faces bnd 0 winding consistent IoU 0.956 25x smaller, fully manifold, consistent winding, for 1.3% silhouette IoU. Lossy by design - it gives up the dual grid's open-surface representation - so it is opt-in. UV BAKE is unblocked by the same change: its cost is driven by face count, not by remesh. 5.0s at 20k faces against >20min at 214k. No longer offline-only when paired with manifold. THE SCALING TRAP, worth knowing: marching_cubes returns vertices in VOXEL INDEX space. Translating without apply_scale(pitch) leaves the mesh ~292x too large. It still exports and renders as a plausible object; it silhouettes at IoU 0.08. That is how it was caught. MoGe-2 CAMERA is now wired and is the default, matching upstream; --fixed-fov keeps the old constant. It runs once per image in torch/MPS, ~0.4s after load. Reporting this one straight: it did NOT improve the samples. On 1_img, fixed 49.1 deg scored 0.893 and MoGe's 29.7 deg scored 0.883. Two caveats keep it as the default anyway - the silhouette metric projects with the SAME FOV used to generate, so a wrong-but-consistent camera can still score well and the metric cannot fully arbitrate camera correctness; and the bundled samples are synthetic renders, not the photographs MoGe reads. Real photos are the intended input here, and upstream estimates too. But the constant is one flag away and the measurement is on record rather than assumed. Operator gains manifold, divisions, fixed_fov. README and PROFILE.md corrected where they repeated the boundary-edge claim. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
m3ultra
|
5f14517553 |
Fix the silhouette gate: sample the surface, not the vertex list
Two bugs in the acceptance test, both found by running it for real.
1. THE GATE DID NOT APPLY TO TEXTURED RUNS. The texture path returned before the
check, so the one path most likely to be used for real assets was the only one that
could ship a blob silently. The check is now a shared gate() called from both.
2. THE METRIC WAS TESSELLATION-DEPENDENT. It projected the VERTEX LIST, so a decimated
mesh sampled its own silhouette more sparsely and scored lower for an identical
shape - holes appear inside the outline and count as misses. Measured on two real
assets:
asset verts vertex-proj surface-sampled
1_img (gate FAILED) 133,842 0.790 0.911
0_img (gate passed) 227,546 0.965 0.969
The dense mesh barely moves; the sparse one jumps 0.12. That is the metric
measuring tessellation, not accuracy - and at min_iou 0.85 it had just rejected a
good reconstruction. Overlay confirmed it: the "missing" region was speckle inside
the silhouette, not a wrong shape.
Now samples 3M points uniformly over the surface, so density is a constant of the
metric rather than a property of the mesh.
Worth stating plainly: the gate caught a real problem on its first live failure - just
not the one it reported. A quality gate that is itself unvalidated is a liability, and
this one needed the same "measure it, do not reason about it" treatment as the model.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
m3ultra
|
d338eca925 |
Vertex-colour baker: 0.2s against the UV path's >20 minutes
The texture stage produced correct PBR voxels, but getting them ONTO a mesh through
o_voxel's UV path is not viable on this build. Measured, on an already welded,
floater-free, decimated 214k-face mesh:
o_voxel to_glb, remesh=True killed at 20min
o_voxel to_glb, remesh=False >20min CPU, killed
bake_vertex_colors 0.2s
xatlas scales badly and 214k is the decimation floor, so it cannot be fed a smaller
mesh either. The trellis-2 lane reached the same conclusion independently and ships
--baker vertex as its fast path; this now matches.
bake_vertex_colors samples the PBR attribute volume at each vertex and writes COLOR_0.
Positions map to voxel indices by the same linear aabb relation fdg_to_mesh uses, so
nothing is resampled; lookup is a sorted-key searchsorted, and misses keep neutral
grey rather than black.
Verified on the real pipeline output:
bake vertex colours, 96.1% of vertices hit 0.2s
result 90,093 verts / 214,322 faces
57,925 unique colours, mean RGB [105 100 87], std [40 35 38]
3.9% still default grey (matches the 4% miss rate)
TOTAL 266.2s end to end, peak 32.6GB
What this costs: no metallic/roughness maps, base colour only. That is the honest
trade and it is stated in the operator description rather than buried - remesh also
now defaults OFF in to_glb, since upstream's remesh=True assumes CUDA.
Operator gains a baker param (vertex default, uv opt-in and flagged offline-only).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
m3ultra
|
b6f0d76619 |
Texture stage: tex SLAT + PBR decode, and the bake order that makes it tractable
The texture flow is imgshape2tex - it denoises 32 PBR channels while SEEING the shape latent, so in_channels is 64 against out_channels 32. Upstream feeds the shape latent as concat_cond and the model does sparse_cat([x, concat_cond], dim=-1); both share coords, so it reduces to a channel concat. Added to slat_flow and carried on the sampler (it is fixed for the whole trajectory and must reach BOTH CFG branches). Verified running on the real checkpoints: 3,988,052 PBR voxels x 6 channels in 65.8s (base_color 0:3, metallic 3:4, roughness 4:5, alpha 5:6). Two things here fail SILENTLY rather than loudly, so both are asserted in comments: 1. shape_slat arrives DENORMALISED - the shape stage un-standardises it for the decoder - but the texture flow was trained against the standardised form. It is re-normalised before use as concat_cond. Skipping that gives a plausible mesh with wrong colours, not an error. 2. tex_dec has pred_subdiv=False: it cannot invent subdivisions and must be handed the shape decoder's subs as guides, so texture voxels land on the geometry that was actually built. The decoder's output is mapped * 0.5 + 0.5 into [0,1], the range o_voxel expects. BAKE ORDER. Handing o_voxel the raw ~8M-face mesh hangs - the same wall the standalone remesh test hit (killed at 20min), and the trellis2 lane's own operator note says the uncapped bake peaks at 75GB. So the mesh is welded, stripped of floaters and decimated BEFORE baking; the baker samples the attribute VOLUME at mesh positions, so a decimated mesh still gets correct colours. Measured on the way through: welded 3,988,052 -> 3,983,672 verts floaters 12 components -> 1 kept, 6,332 faces dropped decimated 7,996,876 -> 214,322 faces pre-bake 34.6s That floater count is worth noting: 12 components, not the 52,855 the first health pass reported. Welding first is what makes the difference. remesh now defaults OFF in to_glb, unlike upstream. Upstream runs on CUDA; this is the CPU/Metal build and its remesher took >20 minutes on a 214k-face mesh. It is also handed an already-clean mesh, so there is far less for it to fix. Operator gains texture + texture_size params; geometry-only stays the default because it is ~3min against the textured path's extra flow and bake. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
m3ultra
|
516e3e4457 |
Mesh cleanup: weld, strip floaters, iterative decimation
The raw cascade output is ~4M verts / 8M faces and is not usable as-is. cleanup.py is ordinary mesh hygiene, kept out of the model code, and the ORDER is the whole point: weld -> strip floaters -> decimate -> strip again -> fix normals CORRECTION TO THE FLOATER COUNT. The health pass reported 52,855 components with 52,838 fragments under 100 faces, and I took those for stray shells. They were mostly NOT: the decoder emits per-voxel vertices, so coincident corners are duplicated and the same continuous surface reads as tens of thousands of islands. Welding FIRST collapses it to a single component, and only 6,332 faces are genuinely stray. Ordering the pass the other way round removes 250k faces of real geometry and calls it cleaning. Two performance fixes, both because an operator runs this every job: - Component labelling is a scipy union-find over the VERTEX graph, not trimesh.face_adjacency. Same answer, ~240s -> ~1s on this mesh. - fix_normals runs LAST, on the decimated mesh. It walks face adjacency, so on the raw 7.99M-face mesh it costs minutes and the result is then thrown away by decimation. Cleanup went ~243s -> ~20s. Decimation is iterative. A single fast_simplification call will not reduce past roughly 4.4% of its input whatever target_reduction (or agg) is asked for: from 7.99M faces, targets of 200k, 50k and 20k ALL returned 351,535. Repeated smaller passes get further because each re-evaluates quadrics on the collapsed mesh. MEASURED, on the sample: target 500,000 -> 499,984 faces silhouette IoU 0.965 19.3s target 200,000 -> 214,322 faces silhouette IoU 0.823 34.8s target 100,000 -> 214,322 faces silhouette IoU 0.823 target 20,000 -> 214,322 faces silhouette IoU 0.823 HONEST LIMITATION: ~214k is a hard floor, and reaching it costs real fidelity (0.965 -> 0.823). The cause is the ~180,000 BOUNDARY edges the Flexible Dual Grid produces for open surfaces - quadric decimation will not collapse those, and no aggressiveness setting changes it. Below ~214k needs a REMESH, not a decimator. 500k is effectively lossless and is the setting to use; anything under 214k is not currently reachable and the loop stops rather than spinning. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
m3ultra
|
165de26db5 |
The shipped cascade: image -> GLB at silhouette IoU 0.969
image_to_mesh() now runs the real cascade, not the single-stage shortcut: structure 3048 voxels @32^3 (64^3 occupancy, MAX-POOLED DOWN) LR SLAT 3048 x 32 shape_512 extractor refine 13147 coords @64^3 four decoder stages -> coords -> quantise HR SLAT 13147 x 32 shape_1024 extractor mesh 3988052 verts, 7996876 faces @1024^3 TOTAL 258.4s, peak 27.9GB with every model resident silhouette IoU 0.969 Three things the cascade needed: 1. occupied_coords_at() - ss_dec always decodes 64^3 but the cascade STARTS at 32^3. Upstream max-pools the boolean grid down by the ratio (a voxel survives if ANY of its eight children was occupied). I had been feeding the raw 64^3 set to the HR flow. 2. decoder.upsample() - pushes the LR latent four stages in and returns COORDS, not features. The predicted subdivisions grow the occupied set; those coords quantise onto the HR flow's grid. Stops BEFORE stage `upsample_times`, as upstream does; one stage further doubles the resolution and misplaces every voxel. 3. grid_resolution override on ProjConditioner - upstream backs the HR grid off in 128-unit steps while the token count exceeds max_num_tokens, so a dense object degrades instead of exploding. refine_coords() implements that loop. I WAS WRONG ABOUT THE HALO. The previous commit blamed the single-stage shortcut for a 0.639 silhouette IoU and predicted the cascade would fix it. The cascade measured 0.640 - no change. The real fault was in my VERIFICATION, not the pipeline: o_voxel returns vertices in the voxel-grid frame, while ProjGrid rotates its lattice by _BLENDER_ROT before projecting. Rotating the mesh the same way scores 0.969 on the same geometry the earlier commit had already produced. Added mesh.to_camera_frame() so the trap is named where it bites; the earlier mesh was correct all along. The cascade is still the right thing - it is the shipped path, and staged loading halves peak memory (12.8GB vs 22.6GB) when models are released between stages. Also adds models.load_all(), so a server builds all five models plus both conditioners ONCE. Warmup is ~71s against ~17s of compute, so an operator must never fork per job. Holding everything resident costs 27.9GB peak - nothing on a 256GB box. scripts/image_to_mesh.py exits non-zero if IoU < 0.85: a run that completes with a bad reconstruction has failed even though nothing raised. 27/27 green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
m3ultra
|
79ef81988c |
Real image -> occupancy grid, with a silhouette check and honest timings
image_to_occupancy() runs the structure stage on an actual photo: preprocess ->
DINOv3 -> proj back-projection -> ss_flow -> ss_dec -> 64^3 occupancy.
VERIFICATION THAT MATTERS: scripts/run_structure.py re-projects the occupied voxels
through the same camera and compares against the input alpha matte. On the upstream
sample that is silhouette IoU 0.842 with 12948 voxels occupied (4.94% of 64^3). This
is the model's own headline claim, so it is the right thing to assert — 'it ran
without crashing' would pass just as happily on a generic blob.
Two real bugs this phase found, neither visible without reading the shipped configs:
1. THE SAMPLER WAS MISSING guidance_rescale. The checkpoint's own pipeline.json sets
0.7 for the structure stage and 0.5 for shape_slat, so this fires at the model's
DEFAULT settings — omitting it silently overcooks every structure prediction. Now
implemented (Lin et al. CFG rescale) and diffed against upstream's
ClassifierFreeGuidanceSamplerMixin, run directly rather than reimplemented.
2. The sampler defaults were wrong: the real ss stage is steps=12 / rescale_t=5.0 /
guidance 7.5 / interval [0.6,1.0], not the steps=25 / rescale_t=3.0 the smoke test
assumed. All three stages' real params now live in pipeline.py, read from
pipeline.json rather than guessed.
TIMINGS, measured with interleaved reps after warmup (the first pass attributed the
same 11s of residual warmup to both 'rescale' and 'torch contention'; it was neither):
cold run 89.3s
warm, full settings 16.5s
warm, CFG off 9.2s -> CFG costs 1.80x, as expected for 10/12
steps falling inside the guidance interval
guidance_rescale ~0s -> free
torch/MPS contention ~0s -> DINOv3 can stay resident
peak memory 6.8GB
THE FINDING THAT SHAPES THE OPERATOR: warmup is ~71s against ~17s of actual compute,
i.e. 4x the work. A MODELBEAST operator MUST hold the models resident across jobs
rather than fork per job — the trellis2 lane shows the same shape (47.9s cold vs 2.5s
warm pipeline_load). Cost this in before optimising any kernel.
17/17 tests green (12 proj + 5 sampler).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|