Hunyuan3D-2.2-mrp-MLX/README.md

Apple MLX Port (Apple Silicon)

This fork adds native MLX inference for the full Hunyuan3D-2.1 pipeline on Apple Silicon Macs — both shape generation (image → mesh) and PBR texture synthesis (mesh + reference image → textured GLB).

Weights

Pre-converted MLX weights are hosted on HuggingFace and auto-downloaded on first use:

dgrauet/hunyuan3d-2.1-mlx

If you'd rather convert from the original PyTorch checkpoints yourself:

pip install mlx-forge
mlx-forge convert hunyuan3d-2.1 --output ./models/hunyuan3d-2.1-mlx
# or quantized for 16GB Macs:
mlx-forge convert hunyuan3d-2.1 --quantize --bits 8 --output ./models/hunyuan3d-2.1-mlx

Override the default HF lookup by setting HUNYUAN3D_MLX_WEIGHTS_DIR=/path/to/local/models.

Install

pip install mlx mlx-arsenal safetensors Pillow trimesh scikit-image PyMCubes scipy
pip install huggingface_hub xatlas opencv-python  # for Stage 2

Stage 1 — Shape Generation (image → mesh)

from hy3dshape.hy3dshape.pipeline_mlx import ShapePipeline

pipe = ShapePipeline.from_pretrained("dgrauet/hunyuan3d-2.1-mlx")
mesh = pipe("your_image.png", num_inference_steps=50, guidance_scale=7.5, octree_resolution=256)
mesh.export("output.glb")

Stage 2 — PBR Texture Synthesis (mesh + reference → textured GLB)

from textureGenPipeline_mlx import Hunyuan3DPaintConfigMLX, Hunyuan3DPaintPipelineMLX

cfg = Hunyuan3DPaintConfigMLX(max_num_view=6, resolution=512)
pipe = Hunyuan3DPaintPipelineMLX(cfg)
pipe(
    mesh_path="mesh.glb",
    image_path="reference.png",
    output_mesh_path="textured.obj",
    save_glb=True,
)

Output: textured.glb with 2048² PBR albedo + metallic-roughness textures baked onto the mesh UVs (~9 min for 6 views at 512px on an M2 Pro).

Defaults match the PyTorch reference config exactly, with one documented exception: texture_size=2048 (PT uses 4096). The MLX Metal rasterizer has no tiling and would exceed the GPU command-buffer budget at 4096² on laptop GPUs (kIOGPUCommandBufferCallbackErrorImpactingInteractivity). Every other knob — max_num_view, render_size, num_inference_steps, guidance_scale, bake mode, inpaint method — is PT-identical.

End-to-end (image → textured GLB)

tests/test_stage1_to_stage2.py chains both stages on a single reference image. Stage 1's dense marching-cubes mesh is automatically remeshed to ~40k faces before Stage 2's bake (the Metal rasterizer can't handle Stage 1's raw ~500k faces at texture_size=2048).

Memory Requirements

Precision	DiT Size	Peak Memory	Recommended Mac
FP16	5.7 GB	~10 GB	32 GB+
INT8	3.0 GB	~6 GB	16 GB+
INT4	1.6 GB	~4 GB	16 GB

Stage 2 (paint) adds ~6 GB peak for the UNet + VAE + DINOv2 at fp32.

Scope

• ✅ Stage 1 (shape generation): fully ported to MLX, validated numerically against PyTorch (1e-5)
• ✅ Stage 2 (PBR texture synthesis): fully ported to MLX, full UNet match within 1.17e-5 vs PyTorch
  • Metal rasterizer, UNet 2.5D with dual-stream reference attention, VAE, DINOv2, v_prediction scheduler
  • Cosine-weighted bake blend (PT-parity fast_bake_texture) smooths across face seams
  • Mesh-aware vertex-color propagation + face barycentric raster, cv2.INPAINT_NS (PT-parity), plus an EDT nearest-fill post-pass that pads UV gutters so 3D viewers doing bilinear sampling across island boundaries never pull in the atlas background
  • RealESRGAN x4 super-resolution in MLX (512² → 2048² per view before bake)
  • GLB export: PBRMaterial with sRGB baseColorTexture, metallicRoughnessTexture (glTF channel order), doubleSided=true
  • All pipeline defaults now match the PyTorch reference exactly (see porting principle in docs/forward_pass.md)

Texture quality notes

• Reference image must match the mesh content (e.g. mermaid image on mermaid mesh). Cross-pairing produces fragmented atlases since the diffusion's albedo views can't coherently project onto an unrelated 3D layout.
• Tested on assets/case_1/mesh.glb (fox) and assets/case_2/mesh.glb (mermaid) — all 6/6 views clean with PT-aligned defaults.

---

🔥 News

• Jul 26, 2025: 🤗 We release the first open-source, simulation-capable, immersive 3D world generation model, HunyuanWorld-1.0!
• Jun 19, 2025: 👋 We present the technical report of Hunyuan3D-2.1, please check out the details and spark some discussion!
• Jun 13, 2025: 🤗 We release the first production-ready 3D asset generation model, Hunyuan3D-2.1!

Join our Wechat and Discord group to discuss and find help from us.

Wechat Group	Xiaohongshu	X	Discord

🤗 Community Contribution Leaderboard

1. By @visualbruno

• ComfyUI-Hunyuan3d-2-1: https://github.com/visualbruno/ComfyUI-Hunyuan3d-2-1

2. By @VR-Jobs

• Hunyuan3d-2-1 Unity Support: https://github.com/VR-Jobs/Hunyuan3D-2.1-Unity-XR-PC-Phone

☯️ Hunyuan3D 2.1

Architecture

Tencent Hunyuan3D-2.1 is a scalable 3D asset creation system that advances state-of-the-art 3D generation through two pivotal innovations: Fully Open-Source Framework and Physically-Based Rendering (PBR) Texture Synthesis. For the first time, the system releases full model weights and training code, enabling community developers to directly fine-tune and extend the model for diverse downstream applications. This transparency accelerates academic research and industrial deployment. Moreover, replacing the prior RGB-based texture model, the upgraded PBR pipeline leverages physics-grounded material simulation to generate textures with photorealistic light interaction (e.g., metallic reflections, subsurface scattering).

Performance

We have evaluated Hunyuan3D 2.1 with other open-source as well as close-source 3d-generation methods. The numerical results indicate that Hunyuan3D 2.1 surpasses all baselines in the quality of generated textured 3D assets and the condition following ability.

Model	ULIP-T(⬆)	ULIP-I(⬆)	Uni3D-T(⬆)	Uni3D-I(⬆)
Michelangelo	0.0752	0.1152	0.2133	0.2611
Craftsman	0.0745	0.1296	0.2375	0.2987
TripoSG	0.0767	0.1225	0.2506	0.3129
Step1X-3D	0.0735	0.1183	0.2554	0.3195
Trellis	0.0769	0.1267	0.2496	0.3116
Direct3D-S2	0.0706	0.1134	0.2346	0.2930
Hunyuan3D-Shape-2.1	0.0774	0.1395	0.2556	0.3213

Model	CLIP-FiD(⬇)	CMMD(⬇)	CLIP-I(⬆)	LPIPS(⬇)
SyncMVD-IPA	28.39	2.397	0.8823	0.1423
TexGen	28.24	2.448	0.8818	0.1331
Hunyuan3D-2.0	26.44	2.318	0.8893	0.1261
Hunyuan3D-Paint-2.1	24.78	2.191	0.9207	0.1211

🎁 Models Zoo

It takes 10 GB VRAM for shape generation, 21GB for texture generation and 29GB for shape and texture generation in total.

Model	Description	Date	Size	Huggingface
Hunyuan3D-Shape-v2-1	Image to Shape Model	2025-06-14	3.3B	Download
Hunyuan3D-Paint-v2-1	Texture Generation Model	2025-06-14	2B	Download

🤗 Get Started with Hunyuan3D 2.1

Hunyuan3D 2.1 supports Macos, Windows, Linux. You may follow the next steps to use Hunyuan3D 2.1 via:

Install Requirements

We test our model with Python 3.10 and PyTorch 2.5.1+cu124.

pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu124
pip install -r requirements.txt

cd hy3dpaint/custom_rasterizer
pip install -e .
cd ../..
cd hy3dpaint/DifferentiableRenderer
bash compile_mesh_painter.sh
cd ../..

wget https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth -P hy3dpaint/ckpt

Code Usage

We designed a diffusers-like API to use our shape generation model - Hunyuan3D-Shape and texture synthesis model - Hunyuan3D-Paint.

import sys
sys.path.insert(0, './hy3dshape')
sys.path.insert(0, './hy3dpaint')
from textureGenPipeline import Hunyuan3DPaintPipeline
from textureGenPipeline import Hunyuan3DPaintPipeline, Hunyuan3DPaintConfig
from hy3dshape.pipelines import Hunyuan3DDiTFlowMatchingPipeline

# let's generate a mesh first
shape_pipeline = Hunyuan3DDiTFlowMatchingPipeline.from_pretrained('tencent/Hunyuan3D-2.1')
mesh_untextured = shape_pipeline(image='assets/demo.png')[0]

paint_pipeline = Hunyuan3DPaintPipeline(Hunyuan3DPaintConfig(max_num_view=6, resolution=512))
mesh_textured = paint_pipeline(mesh_path, image_path='assets/demo.png')

Gradio App

You could also host a Gradio App in your own computer via:

python3 gradio_app.py \
  --model_path tencent/Hunyuan3D-2.1 \
  --subfolder hunyuan3d-dit-v2-1 \
  --texgen_model_path tencent/Hunyuan3D-2.1 \
  --low_vram_mode

🔗 BibTeX

If you found this repository helpful, please cite our reports:

@misc{hunyuan3d2025hunyuan3d,
    title={Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Material},
    author={Tencent Hunyuan3D Team},
    year={2025},
    eprint={2506.15442},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

@misc{hunyuan3d22025tencent,
    title={Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation},
    author={Tencent Hunyuan3D Team},
    year={2025},
    eprint={2501.12202},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

@misc{yang2024hunyuan3d,
    title={Hunyuan3D 1.0: A Unified Framework for Text-to-3D and Image-to-3D Generation},
    author={Tencent Hunyuan3D Team},
    year={2024},
    eprint={2411.02293},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

Acknowledgements

We would like to thank the contributors to the TripoSG, Trellis, DINOv2, Stable Diffusion, FLUX, diffusers, HuggingFace, CraftsMan3D, Michelangelo, Hunyuan-DiT, and HunyuanVideo repositories, for their open research and exploration.

Star History