monster/Hunyuan3D-2.2-mrp-MLX
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

README: MODELBEAST build overview (provenance, our changes, quick start, hardware, license); upstream README -> README_UPSTREAM.md

Browse Source
This commit is contained in:
modelbeast 2026-07-16 15:00:25 +10:00
parent e4cfa9d1e9
commit 04fd1ceb28
2 changed files with 327 additions and 256 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

301
README.md
View File

@ -1,282 +1,71 @@
# Hunyuan3D-2.2-mrp-MLX
<p align="center"> **One-command image → textured 3D on Apple Silicon.** A tuned, packaged build of Tencent's Hunyuan3D 2.1, running natively on Mac GPUs via Apple MLX — no NVIDIA, no CUDA, no cloud.
<img src="assets/images/teaser.jpg">
</p>
<div align="center">
<a href=https://3d.hunyuan.tencent.com target="_blank"><img src=https://img.shields.io/badge/Official%20Site-333399.svg?logo=homepage height=22px></a>
<a href=https://huggingface.co/spaces/tencent/Hunyuan3D-2.1 target="_blank"><img src=https://img.shields.io/badge/%F0%9F%A4%97%20Demo-276cb4.svg height=22px></a>
<a href=https://huggingface.co/tencent/Hunyuan3D-2.1 target="_blank"><img src=https://img.shields.io/badge/%F0%9F%A4%97%20Models-d96902.svg height=22px></a>
<a href=https://3d-models.hunyuan.tencent.com/ target="_blank"><img src= https://img.shields.io/badge/Page-bb8a2e.svg?logo=github height=22px></a>
<a href=https://discord.gg/dNBrdrGGMa target="_blank"><img src= https://img.shields.io/badge/Discord-white.svg?logo=discord height=22px></a>
<a href=https://arxiv.org/pdf/2506.15442 target="_blank"><img src=https://img.shields.io/badge/Report-b5212f.svg?logo=arxiv height=22px></a>
<a href=https://x.com/TencentHunyuan target="_blank"><img src=https://img.shields.io/badge/Hunyuan-black.svg?logo=x height=22px></a>
<a href="#community-resources" target="_blank"><img src=https://img.shields.io/badge/Community-lavender.svg?logo=homeassistantcommunitystore height=22px></a>
</div>
[//]: # ( <a href=# target="_blank"><img src=https://img.shields.io/badge/Report-b5212f.svg?logo=arxiv height=22px></a>)
[//]: # ( <a href=# target="_blank"><img src= https://img.shields.io/badge/Colab-8f2628.svg?logo=googlecolab height=22px></a>)
[//]: # ( <a href="#"><img alt="PyPI - Downloads" src="https://img.shields.io/pypi/v/mulankit?logo=pypi" height=22px></a>)
<br>
## Apple MLX Port (Apple Silicon)
This fork adds native **MLX inference** for the full Hunyuan3D-2.1 pipeline on Apple Silicon Macs — both shape generation (image → mesh) and PBR texture synthesis (mesh + reference image → textured GLB).
### Weights
Pre-converted MLX weights are hosted on HuggingFace and auto-downloaded on first use:
> [`dgrauet/hunyuan3d-2.1-mlx`](https://huggingface.co/dgrauet/hunyuan3d-2.1-mlx)
If you'd rather convert from the original PyTorch checkpoints yourself:
```bash ```bash
pip install mlx-forge python generate_e2e.py your_image.png --output out/mymesh
mlx-forge convert hunyuan3d-2.1 --output ./models/hunyuan3d-2.1-mlx # → out/mymesh.glb (PBR-textured mesh) + out/mymesh_shape.glb (untextured)
# or quantized for 16GB Macs:
mlx-forge convert hunyuan3d-2.1 --quantize --bits 8 --output ./models/hunyuan3d-2.1-mlx
``` ```
Override the default HF lookup by setting `HUNYUAN3D_MLX_WEIGHTS_DIR=/path/to/local/models`.
### Install
```bash
pip install mlx mlx-arsenal safetensors Pillow trimesh scikit-image PyMCubes scipy
pip install huggingface_hub xatlas opencv-python # for Stage 2
```
### Stage 1 — Shape Generation (image → mesh)
```python
from hy3dshape.hy3dshape.pipeline_mlx import ShapePipeline
pipe = ShapePipeline.from_pretrained("dgrauet/hunyuan3d-2.1-mlx")
mesh = pipe("your_image.png", num_inference_steps=50, guidance_scale=7.5, octree_resolution=256)
mesh.export("output.glb")
```
### Stage 2 — PBR Texture Synthesis (mesh + reference → textured GLB)
```python
from textureGenPipeline_mlx import Hunyuan3DPaintConfigMLX, Hunyuan3DPaintPipelineMLX
cfg = Hunyuan3DPaintConfigMLX(max_num_view=6, resolution=512)
pipe = Hunyuan3DPaintPipelineMLX(cfg)
pipe(
mesh_path="mesh.glb",
image_path="reference.png",
output_mesh_path="textured.obj",
save_glb=True,
)
```
Output: `textured.glb` with 2048² PBR albedo + metallic-roughness textures baked onto the mesh UVs (~9 min for 6 views at 512px on an M2 Pro).
Defaults match the PyTorch reference config exactly, with one documented exception: `texture_size=2048` (PT uses 4096). The MLX Metal rasterizer has no tiling and would exceed the GPU command-buffer budget at 4096² on laptop GPUs (`kIOGPUCommandBufferCallbackErrorImpactingInteractivity`). Every other knob — `max_num_view`, `render_size`, `num_inference_steps`, `guidance_scale`, bake mode, inpaint method — is PT-identical.
### End-to-end (image → textured GLB)
`tests/test_stage1_to_stage2.py` chains both stages on a single reference image. Stage 1's dense marching-cubes mesh is automatically remeshed to ~40k faces before Stage 2's bake (the Metal rasterizer can't handle Stage 1's raw ~500k faces at `texture_size=2048`).
### Memory Requirements
| Precision | DiT Size | Peak Memory | Recommended Mac |
|-----------|----------|-------------|-----------------|
| FP16 | 5.7 GB | ~10 GB | 32 GB+ |
| INT8 | 3.0 GB | ~6 GB | 16 GB+ |
| INT4 | 1.6 GB | ~4 GB | 16 GB |
Stage 2 (paint) adds ~6 GB peak for the UNet + VAE + DINOv2 at fp32.
### Scope
- ✅ **Stage 1** (shape generation): fully ported to MLX, validated numerically against PyTorch (1e-5)
- ✅ **Stage 2** (PBR texture synthesis): fully ported to MLX, full UNet match within 1.17e-5 vs PyTorch
- Metal rasterizer, UNet 2.5D with dual-stream reference attention, VAE, DINOv2, v_prediction scheduler
- **Cosine-weighted bake blend** (PT-parity `fast_bake_texture`) smooths across face seams
- **Mesh-aware vertex-color propagation + face barycentric raster**, `cv2.INPAINT_NS` (PT-parity), plus an EDT nearest-fill post-pass that pads UV gutters so 3D viewers doing bilinear sampling across island boundaries never pull in the atlas background
- **RealESRGAN x4 super-resolution** in MLX (512² → 2048² per view before bake)
- **GLB export**: PBRMaterial with sRGB `baseColorTexture`, `metallicRoughnessTexture` (glTF channel order), `doubleSided=true`
- All pipeline defaults now match the PyTorch reference exactly (see porting principle in `docs/forward_pass.md`)
### Texture quality notes
- Reference image must match the mesh content (e.g. mermaid image on mermaid mesh). Cross-pairing produces fragmented atlases since the diffusion's albedo views can't coherently project onto an unrelated 3D layout.
- Tested on `assets/case_1/mesh.glb` (fox) and `assets/case_2/mesh.glb` (mermaid) — all 6/6 views clean with PT-aligned defaults.
--- ---
## 🔥 News ## What this is (provenance & credit)
- Jul 26, 2025: 🤗 We release the first open-source, simulation-capable, immersive 3D world generation model, [HunyuanWorld-1.0](https://github.com/Tencent-Hunyuan/HunyuanWorld-1.0)! A **fork of a fork** — three layers, credited honestly:
- Jun 19, 2025: 👋 We present the [technical report](https://arxiv.org/pdf/2506.15442) of Hunyuan3D-2.1, please check out the details and spark some discussion!
- Jun 13, 2025: 🤗 We release the first production-ready 3D asset generation model, Hunyuan3D-2.1!
> Join our **[Wechat](#)** and **[Discord](https://discord.gg/dNBrdrGGMa)** group to discuss and find help from us. 1. **Tencent Hunyuan3D 2.1** — the actual models: `Hunyuan3D-Shape-v2-1` (3.3B, image→mesh) + `Hunyuan3D-Paint-v2-1` (2B, PBR texture). Originally **CUDA/NVIDIA only**.
2. **[dgrauet/Hunyuan3D-2.1-mlx](https://github.com/dgrauet/Hunyuan3D-2.1-mlx)** — the hard part: a native **MLX (Metal)** port of *both* stages. fp16, no bf16 dependency → runs on **any** Apple Silicon Mac (M1–M5). This is what made it work on a Mac at all.
3. **This build (MODELBEAST / `mrp`)** — the usability + tuning layer on top of dgrauet (below).
| Wechat Group | Xiaohongshu | X | Discord | ## What *we* added over upstream
|--------------------------------------------------|-------------------------------------------------------|---------------------------------------------|---------------------------------------------------|
| <img src="assets/qrcode/wechat.png" height=140> | <img src="assets/qrcode/xiaohongshu.png" height=140> | <img src="assets/qrcode/x.png" height=140> | <img src="assets/qrcode/discord.png" height=140> |
## 🤗 Community Contribution Leaderboard - **`generate_e2e.py`** — a single command chaining Stage 1 (shape) → Stage 2 (PBR texture). Upstream only shipped a test script. Flags: `--steps --guidance --octree-resolution --max-num-view --view-resolution --texture-size --remesh-faces --seed --no-texture`; prints per-stage timings + peak memory.
1. By [@visualbruno](https://github.com/visualbruno) - **Env-tunable bake mesh** — `HY3D_REMESH_FACES` (upstream hard-coded 40k). The **single biggest quality lever** — it's what un-melts faces.
- ComfyUI-Hunyuan3d-2-1: https://github.com/visualbruno/ComfyUI-Hunyuan3d-2-1 - **Studio-quality defaults** — `octree 384 / texture 4096 / remesh 120k`, **verified watchdog-free on M3 *and* M1 Ultra**. Upstream capped texture at 2048 because laptop GPUs hit a Metal command-buffer limit; Studio-class GPUs don't, and the existing tiling handles 4096².
2. By [@VR-Jobs](https://github.com/VR-Jobs) - **[HARDWARE.md](HARDWARE.md)** — exact RAM needs per precision and per Mac.
- Hunyuan3d-2-1 Unity Support: https://github.com/VR-Jobs/Hunyuan3D-2.1-Unity-XR-PC-Phone - **Slimmed for inference** — upstream training data (`mini_trainset`) + demo images stripped (~165 MB → ~9 MB). Recover any time via `git fetch upstream` (github.com/dgrauet).
## ☯️ **Hunyuan3D 2.1** ## Quick start
### Architecture
Tencent Hunyuan3D-2.1 is a scalable 3D asset creation system that advances state-of-the-art 3D generation through two pivotal innovations: Fully Open-Source Framework and Physically-Based Rendering (PBR) Texture Synthesis. For the first time, the system releases full model weights and training code, enabling community developers to directly fine-tune and extend the model for diverse downstream applications. This transparency accelerates academic research and industrial deployment. Moreover, replacing the prior RGB-based texture model, the upgraded PBR pipeline leverages physics-grounded material simulation to generate textures with photorealistic light interaction (e.g., metallic reflections, subsurface scattering).
<p align="left">
<img src="assets/images/pipeline.png">
</p>
### Performance
We have evaluated Hunyuan3D 2.1 with other open-source as well as close-source 3d-generation methods.
The numerical results indicate that Hunyuan3D 2.1 surpasses all baselines in the quality of generated textured 3D assets
and the condition following ability.
| Model | ULIP-T(⬆) | ULIP-I(⬆) | Uni3D-T(⬆) | Uni3D-I(⬆) |
|-------------------------|-----------|-------------|-------------|---------------|
| Michelangelo | 0.0752 | 0.1152 | 0.2133 | 0.2611 |
| Craftsman | 0.0745 | 0.1296 | 0.2375 | 0.2987 |
| TripoSG | 0.0767 | 0.1225 | 0.2506 | 0.3129 |
| Step1X-3D | 0.0735 | 0.1183 | 0.2554 | 0.3195 |
| Trellis | 0.0769 | 0.1267 | 0.2496 | 0.3116 |
| Direct3D-S2 | 0.0706 | 0.1134 | 0.2346 | 0.2930 |
| Hunyuan3D-Shape-2.1 | **0.0774** | **0.1395** | **0.2556** | **0.3213** |
| Model | CLIP-FiD(⬇) | CMMD(⬇) | CLIP-I(⬆) | LPIPS(⬇) |
|-------------------------|-----------|-------------|-------------|---------------|
| SyncMVD-IPA | 28.39 | 2.397 | 0.8823 | 0.1423 |
| TexGen | 28.24 | 2.448 | 0.8818 | 0.1331 |
| Hunyuan3D-2.0 | 26.44 | 2.318 | 0.8893 | 0.1261 |
| Hunyuan3D-Paint-2.1 | **24.78** | **2.191** | **0.9207** | **0.1211** |
## 🎁 Models Zoo
It takes 10 GB VRAM for shape generation, 21GB for texture generation and 29GB for shape and texture generation in total.
| Model | Description | Date | Size | Huggingface |
|----------------------------|-----------------------------|------------|------|-------------------------------------------------------------------------------------------|
| Hunyuan3D-Shape-v2-1 | Image to Shape Model | 2025-06-14 | 3.3B | [Download](https://huggingface.co/tencent/Hunyuan3D-2.1/tree/main/hunyuan3d-dit-v2-1) |
| Hunyuan3D-Paint-v2-1 | Texture Generation Model | 2025-06-14 | 2B | [Download](https://huggingface.co/tencent/Hunyuan3D-2.1/tree/main/hunyuan3d-paintpbr-v2-1) |
## 🤗 Get Started with Hunyuan3D 2.1
Hunyuan3D 2.1 supports Macos, Windows, Linux. You may follow the next steps to use Hunyuan3D 2.1 via:
### Install Requirements
We test our model with Python 3.10 and PyTorch 2.5.1+cu124.
```bash
pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu124
pip install -r requirements.txt
cd hy3dpaint/custom_rasterizer
pip install -e .
cd ../..
cd hy3dpaint/DifferentiableRenderer
bash compile_mesh_painter.sh
cd ../..
wget https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth -P hy3dpaint/ckpt
```
### Code Usage
We designed a diffusers-like API to use our shape generation model - Hunyuan3D-Shape and texture synthesis model -
Hunyuan3D-Paint.
```python
import sys
sys.path.insert(0, './hy3dshape')
sys.path.insert(0, './hy3dpaint')
from textureGenPipeline import Hunyuan3DPaintPipeline
from textureGenPipeline import Hunyuan3DPaintPipeline, Hunyuan3DPaintConfig
from hy3dshape.pipelines import Hunyuan3DDiTFlowMatchingPipeline
# let's generate a mesh first
shape_pipeline = Hunyuan3DDiTFlowMatchingPipeline.from_pretrained('tencent/Hunyuan3D-2.1')
mesh_untextured = shape_pipeline(image='assets/demo.png')[0]
paint_pipeline = Hunyuan3DPaintPipeline(Hunyuan3DPaintConfig(max_num_view=6, resolution=512))
mesh_textured = paint_pipeline(mesh_path, image_path='assets/demo.png')
```
### Gradio App
You could also host a [Gradio](https://www.gradio.app/) App in your own computer via:
Python **3.11** (the ML stack pins old numpy; 3.12+ won't build it).
```bash ```bash
python3 gradio_app.py \ uv venv --python 3.11 .venv && source .venv/bin/activate # or: python3.11 -m venv .venv
--model_path tencent/Hunyuan3D-2.1 \ uv pip install mlx mlx-arsenal torch torchvision transformers timm \
--subfolder hunyuan3d-dit-v2-1 \ huggingface_hub safetensors numpy scipy scikit-image PyMCubes \
--texgen_model_path tencent/Hunyuan3D-2.1 \ trimesh pygltflib xatlas opencv-python pillow pymeshlab \
--low_vram_mode einops omegaconf pyyaml tqdm torchdiffeq diffusers accelerate fast_simplification
export HF_HUB_DISABLE_XET=1
python generate_e2e.py your_image.png --output out/mymesh
``` ```
Weights (~13 GB, fp16 MLX, **public — no HF login**) auto-download on first run from `dgrauet/hunyuan3d-2.1-mlx`. Override with `HUNYUAN3D_MLX_WEIGHTS_DIR`. Best results: feed a **background-removed cutout**.
## 🔗 BibTeX ## Quality vs speed (generate_e2e.py flags)
If you found this repository helpful, please cite our reports: | | fast draft | **default (studio)** | max |
|---|---|---|---|
| `--texture-size` | 2048 | **4096** | 4096 |
| `--remesh-faces` | 40000 | **120000** | 200000 |
| `--octree-resolution` | 256 | **384** | 384 |
| `--max-num-view` | 6 | 6 | 8–12 |
| time (M3 Ultra) | ~260 s | ~380 s | longer |
| output | 7.6 MB / 40k faces | 21.5 MB / 120k faces | larger |
```bibtex `--no-texture` = Stage 1 only (geometry, ~2.5 min, ~10 GB RAM).
@misc{hunyuan3d2025hunyuan3d,
title={Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Material},
author={Tencent Hunyuan3D Team},
year={2025},
eprint={2506.15442},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
@misc{hunyuan3d22025tencent, ## Hardware (full table in [HARDWARE.md](HARDWARE.md))
title={Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation},
author={Tencent Hunyuan3D Team},
year={2025},
eprint={2501.12202},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
@misc{yang2024hunyuan3d, Peak **~20 GB** fp16 for the full textured pipeline (measured). **Min 16 GB** (INT8/INT4) or **24 GB** (fp16, tight) · **32 GB recommended**. Any M1→M5 — fp16-MLX path, no bf16 gotcha.
title={Hunyuan3D 1.0: A Unified Framework for Text-to-3D and Image-to-3D Generation},
author={Tencent Hunyuan3D Team},
year={2024},
eprint={2411.02293},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```
## Acknowledgements Measured full textured gen: **M3 Ultra ~380 s**, **M1 Ultra ~750 s** (4096 / 120k).
We would like to thank the contributors to ## ⚠️ License
the [TripoSG](https://github.com/VAST-AI-Research/TripoSG), [Trellis](https://github.com/microsoft/TRELLIS), [DINOv2](https://github.com/facebookresearch/dinov2), [Stable Diffusion](https://github.com/Stability-AI/stablediffusion), [FLUX](https://github.com/black-forest-labs/flux), [diffusers](https://github.com/huggingface/diffusers), [HuggingFace](https://huggingface.co), [CraftsMan3D](https://github.com/wyysf-98/CraftsMan3D), [Michelangelo](https://github.com/NeuralCarver/Michelangelo/tree/main), [Hunyuan-DiT](https://github.com/Tencent-Hunyuan/HunyuanDiT), and [HunyuanVideo](https://github.com/Tencent-Hunyuan/HunyuanVideo) repositories, for their open research and
exploration.
## Star History Inherits the **Tencent Hunyuan Non-Commercial License** (see `LICENSE` / `Notice.txt`). **Non-commercial use only** — share and use accordingly.
<a href="https://star-history.com/#Tencent-Hunyuan/Hunyuan3D-2.1&Date"> ## Credits
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=Tencent-Hunyuan/Hunyuan3D-2.1&type=Date&theme=dark" /> Tencent Hunyuan3D team (models) · [@dgrauet](https://github.com/dgrauet) (MLX port) · MODELBEAST (this build).
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=Tencent-Hunyuan/Hunyuan3D-2.1&type=Date" /> Original upstream README preserved as [`README_UPSTREAM.md`](README_UPSTREAM.md).
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=Tencent-Hunyuan/Hunyuan3D-2.1&type=Date" />
</picture>
</a>

282
README_UPSTREAM.md Normal file
View File

@ -0,0 +1,282 @@
<p align="center">
<img src="assets/images/teaser.jpg">
</p>
<div align="center">
<a href=https://3d.hunyuan.tencent.com target="_blank"><img src=https://img.shields.io/badge/Official%20Site-333399.svg?logo=homepage height=22px></a>
<a href=https://huggingface.co/spaces/tencent/Hunyuan3D-2.1 target="_blank"><img src=https://img.shields.io/badge/%F0%9F%A4%97%20Demo-276cb4.svg height=22px></a>
<a href=https://huggingface.co/tencent/Hunyuan3D-2.1 target="_blank"><img src=https://img.shields.io/badge/%F0%9F%A4%97%20Models-d96902.svg height=22px></a>
<a href=https://3d-models.hunyuan.tencent.com/ target="_blank"><img src= https://img.shields.io/badge/Page-bb8a2e.svg?logo=github height=22px></a>
<a href=https://discord.gg/dNBrdrGGMa target="_blank"><img src= https://img.shields.io/badge/Discord-white.svg?logo=discord height=22px></a>
<a href=https://arxiv.org/pdf/2506.15442 target="_blank"><img src=https://img.shields.io/badge/Report-b5212f.svg?logo=arxiv height=22px></a>
<a href=https://x.com/TencentHunyuan target="_blank"><img src=https://img.shields.io/badge/Hunyuan-black.svg?logo=x height=22px></a>
<a href="#community-resources" target="_blank"><img src=https://img.shields.io/badge/Community-lavender.svg?logo=homeassistantcommunitystore height=22px></a>
</div>
[//]: # ( <a href=# target="_blank"><img src=https://img.shields.io/badge/Report-b5212f.svg?logo=arxiv height=22px></a>)
[//]: # ( <a href=# target="_blank"><img src= https://img.shields.io/badge/Colab-8f2628.svg?logo=googlecolab height=22px></a>)
[//]: # ( <a href="#"><img alt="PyPI - Downloads" src="https://img.shields.io/pypi/v/mulankit?logo=pypi" height=22px></a>)
<br>
## Apple MLX Port (Apple Silicon)
This fork adds native **MLX inference** for the full Hunyuan3D-2.1 pipeline on Apple Silicon Macs — both shape generation (image → mesh) and PBR texture synthesis (mesh + reference image → textured GLB).
### Weights
Pre-converted MLX weights are hosted on HuggingFace and auto-downloaded on first use:
> [`dgrauet/hunyuan3d-2.1-mlx`](https://huggingface.co/dgrauet/hunyuan3d-2.1-mlx)
If you'd rather convert from the original PyTorch checkpoints yourself:
```bash
pip install mlx-forge
mlx-forge convert hunyuan3d-2.1 --output ./models/hunyuan3d-2.1-mlx
# or quantized for 16GB Macs:
mlx-forge convert hunyuan3d-2.1 --quantize --bits 8 --output ./models/hunyuan3d-2.1-mlx
```
Override the default HF lookup by setting `HUNYUAN3D_MLX_WEIGHTS_DIR=/path/to/local/models`.
### Install
```bash
pip install mlx mlx-arsenal safetensors Pillow trimesh scikit-image PyMCubes scipy
pip install huggingface_hub xatlas opencv-python # for Stage 2
```
### Stage 1 — Shape Generation (image → mesh)
```python
from hy3dshape.hy3dshape.pipeline_mlx import ShapePipeline
pipe = ShapePipeline.from_pretrained("dgrauet/hunyuan3d-2.1-mlx")
mesh = pipe("your_image.png", num_inference_steps=50, guidance_scale=7.5, octree_resolution=256)
mesh.export("output.glb")
```
### Stage 2 — PBR Texture Synthesis (mesh + reference → textured GLB)
```python
from textureGenPipeline_mlx import Hunyuan3DPaintConfigMLX, Hunyuan3DPaintPipelineMLX
cfg = Hunyuan3DPaintConfigMLX(max_num_view=6, resolution=512)
pipe = Hunyuan3DPaintPipelineMLX(cfg)
pipe(
mesh_path="mesh.glb",
image_path="reference.png",
output_mesh_path="textured.obj",
save_glb=True,
)
```
Output: `textured.glb` with 2048² PBR albedo + metallic-roughness textures baked onto the mesh UVs (~9 min for 6 views at 512px on an M2 Pro).
Defaults match the PyTorch reference config exactly, with one documented exception: `texture_size=2048` (PT uses 4096). The MLX Metal rasterizer has no tiling and would exceed the GPU command-buffer budget at 4096² on laptop GPUs (`kIOGPUCommandBufferCallbackErrorImpactingInteractivity`). Every other knob — `max_num_view`, `render_size`, `num_inference_steps`, `guidance_scale`, bake mode, inpaint method — is PT-identical.
### End-to-end (image → textured GLB)
`tests/test_stage1_to_stage2.py` chains both stages on a single reference image. Stage 1's dense marching-cubes mesh is automatically remeshed to ~40k faces before Stage 2's bake (the Metal rasterizer can't handle Stage 1's raw ~500k faces at `texture_size=2048`).
### Memory Requirements
| Precision | DiT Size | Peak Memory | Recommended Mac |
|-----------|----------|-------------|-----------------|
| FP16 | 5.7 GB | ~10 GB | 32 GB+ |
| INT8 | 3.0 GB | ~6 GB | 16 GB+ |
| INT4 | 1.6 GB | ~4 GB | 16 GB |
Stage 2 (paint) adds ~6 GB peak for the UNet + VAE + DINOv2 at fp32.
### Scope
- ✅ **Stage 1** (shape generation): fully ported to MLX, validated numerically against PyTorch (1e-5)
- ✅ **Stage 2** (PBR texture synthesis): fully ported to MLX, full UNet match within 1.17e-5 vs PyTorch
- Metal rasterizer, UNet 2.5D with dual-stream reference attention, VAE, DINOv2, v_prediction scheduler
- **Cosine-weighted bake blend** (PT-parity `fast_bake_texture`) smooths across face seams
- **Mesh-aware vertex-color propagation + face barycentric raster**, `cv2.INPAINT_NS` (PT-parity), plus an EDT nearest-fill post-pass that pads UV gutters so 3D viewers doing bilinear sampling across island boundaries never pull in the atlas background
- **RealESRGAN x4 super-resolution** in MLX (512² → 2048² per view before bake)
- **GLB export**: PBRMaterial with sRGB `baseColorTexture`, `metallicRoughnessTexture` (glTF channel order), `doubleSided=true`
- All pipeline defaults now match the PyTorch reference exactly (see porting principle in `docs/forward_pass.md`)
### Texture quality notes
- Reference image must match the mesh content (e.g. mermaid image on mermaid mesh). Cross-pairing produces fragmented atlases since the diffusion's albedo views can't coherently project onto an unrelated 3D layout.
- Tested on `assets/case_1/mesh.glb` (fox) and `assets/case_2/mesh.glb` (mermaid) — all 6/6 views clean with PT-aligned defaults.
---
## 🔥 News
- Jul 26, 2025: 🤗 We release the first open-source, simulation-capable, immersive 3D world generation model, [HunyuanWorld-1.0](https://github.com/Tencent-Hunyuan/HunyuanWorld-1.0)!
- Jun 19, 2025: 👋 We present the [technical report](https://arxiv.org/pdf/2506.15442) of Hunyuan3D-2.1, please check out the details and spark some discussion!
- Jun 13, 2025: 🤗 We release the first production-ready 3D asset generation model, Hunyuan3D-2.1!
> Join our **[Wechat](#)** and **[Discord](https://discord.gg/dNBrdrGGMa)** group to discuss and find help from us.
| Wechat Group | Xiaohongshu | X | Discord |
|--------------------------------------------------|-------------------------------------------------------|---------------------------------------------|---------------------------------------------------|
| <img src="assets/qrcode/wechat.png" height=140> | <img src="assets/qrcode/xiaohongshu.png" height=140> | <img src="assets/qrcode/x.png" height=140> | <img src="assets/qrcode/discord.png" height=140> |
## 🤗 Community Contribution Leaderboard
1. By [@visualbruno](https://github.com/visualbruno)
- ComfyUI-Hunyuan3d-2-1: https://github.com/visualbruno/ComfyUI-Hunyuan3d-2-1
2. By [@VR-Jobs](https://github.com/VR-Jobs)
- Hunyuan3d-2-1 Unity Support: https://github.com/VR-Jobs/Hunyuan3D-2.1-Unity-XR-PC-Phone
## ☯️ **Hunyuan3D 2.1**
### Architecture
Tencent Hunyuan3D-2.1 is a scalable 3D asset creation system that advances state-of-the-art 3D generation through two pivotal innovations: Fully Open-Source Framework and Physically-Based Rendering (PBR) Texture Synthesis. For the first time, the system releases full model weights and training code, enabling community developers to directly fine-tune and extend the model for diverse downstream applications. This transparency accelerates academic research and industrial deployment. Moreover, replacing the prior RGB-based texture model, the upgraded PBR pipeline leverages physics-grounded material simulation to generate textures with photorealistic light interaction (e.g., metallic reflections, subsurface scattering).
<p align="left">
<img src="assets/images/pipeline.png">
</p>
### Performance
We have evaluated Hunyuan3D 2.1 with other open-source as well as close-source 3d-generation methods.
The numerical results indicate that Hunyuan3D 2.1 surpasses all baselines in the quality of generated textured 3D assets
and the condition following ability.
| Model | ULIP-T(⬆) | ULIP-I(⬆) | Uni3D-T(⬆) | Uni3D-I(⬆) |
|-------------------------|-----------|-------------|-------------|---------------|
| Michelangelo | 0.0752 | 0.1152 | 0.2133 | 0.2611 |
| Craftsman | 0.0745 | 0.1296 | 0.2375 | 0.2987 |
| TripoSG | 0.0767 | 0.1225 | 0.2506 | 0.3129 |
| Step1X-3D | 0.0735 | 0.1183 | 0.2554 | 0.3195 |
| Trellis | 0.0769 | 0.1267 | 0.2496 | 0.3116 |
| Direct3D-S2 | 0.0706 | 0.1134 | 0.2346 | 0.2930 |
| Hunyuan3D-Shape-2.1 | **0.0774** | **0.1395** | **0.2556** | **0.3213** |
| Model | CLIP-FiD(⬇) | CMMD(⬇) | CLIP-I(⬆) | LPIPS(⬇) |
|-------------------------|-----------|-------------|-------------|---------------|
| SyncMVD-IPA | 28.39 | 2.397 | 0.8823 | 0.1423 |
| TexGen | 28.24 | 2.448 | 0.8818 | 0.1331 |
| Hunyuan3D-2.0 | 26.44 | 2.318 | 0.8893 | 0.1261 |
| Hunyuan3D-Paint-2.1 | **24.78** | **2.191** | **0.9207** | **0.1211** |
## 🎁 Models Zoo
It takes 10 GB VRAM for shape generation, 21GB for texture generation and 29GB for shape and texture generation in total.
| Model | Description | Date | Size | Huggingface |
|----------------------------|-----------------------------|------------|------|-------------------------------------------------------------------------------------------|
| Hunyuan3D-Shape-v2-1 | Image to Shape Model | 2025-06-14 | 3.3B | [Download](https://huggingface.co/tencent/Hunyuan3D-2.1/tree/main/hunyuan3d-dit-v2-1) |
| Hunyuan3D-Paint-v2-1 | Texture Generation Model | 2025-06-14 | 2B | [Download](https://huggingface.co/tencent/Hunyuan3D-2.1/tree/main/hunyuan3d-paintpbr-v2-1) |
## 🤗 Get Started with Hunyuan3D 2.1
Hunyuan3D 2.1 supports Macos, Windows, Linux. You may follow the next steps to use Hunyuan3D 2.1 via:
### Install Requirements
We test our model with Python 3.10 and PyTorch 2.5.1+cu124.
```bash
pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu124
pip install -r requirements.txt
cd hy3dpaint/custom_rasterizer
pip install -e .
cd ../..
cd hy3dpaint/DifferentiableRenderer
bash compile_mesh_painter.sh
cd ../..
wget https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth -P hy3dpaint/ckpt
```
### Code Usage
We designed a diffusers-like API to use our shape generation model - Hunyuan3D-Shape and texture synthesis model -
Hunyuan3D-Paint.
```python
import sys
sys.path.insert(0, './hy3dshape')
sys.path.insert(0, './hy3dpaint')
from textureGenPipeline import Hunyuan3DPaintPipeline
from textureGenPipeline import Hunyuan3DPaintPipeline, Hunyuan3DPaintConfig
from hy3dshape.pipelines import Hunyuan3DDiTFlowMatchingPipeline
# let's generate a mesh first
shape_pipeline = Hunyuan3DDiTFlowMatchingPipeline.from_pretrained('tencent/Hunyuan3D-2.1')
mesh_untextured = shape_pipeline(image='assets/demo.png')[0]
paint_pipeline = Hunyuan3DPaintPipeline(Hunyuan3DPaintConfig(max_num_view=6, resolution=512))
mesh_textured = paint_pipeline(mesh_path, image_path='assets/demo.png')
```
### Gradio App
You could also host a [Gradio](https://www.gradio.app/) App in your own computer via:
```bash
python3 gradio_app.py \
--model_path tencent/Hunyuan3D-2.1 \
--subfolder hunyuan3d-dit-v2-1 \
--texgen_model_path tencent/Hunyuan3D-2.1 \
--low_vram_mode
```
## 🔗 BibTeX
If you found this repository helpful, please cite our reports:
```bibtex
@misc{hunyuan3d2025hunyuan3d,
title={Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Material},
author={Tencent Hunyuan3D Team},
year={2025},
eprint={2506.15442},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
@misc{hunyuan3d22025tencent,
title={Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation},
author={Tencent Hunyuan3D Team},
year={2025},
eprint={2501.12202},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
@misc{yang2024hunyuan3d,
title={Hunyuan3D 1.0: A Unified Framework for Text-to-3D and Image-to-3D Generation},
author={Tencent Hunyuan3D Team},
year={2024},
eprint={2411.02293},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```
## Acknowledgements
We would like to thank the contributors to
the [TripoSG](https://github.com/VAST-AI-Research/TripoSG), [Trellis](https://github.com/microsoft/TRELLIS), [DINOv2](https://github.com/facebookresearch/dinov2), [Stable Diffusion](https://github.com/Stability-AI/stablediffusion), [FLUX](https://github.com/black-forest-labs/flux), [diffusers](https://github.com/huggingface/diffusers), [HuggingFace](https://huggingface.co), [CraftsMan3D](https://github.com/wyysf-98/CraftsMan3D), [Michelangelo](https://github.com/NeuralCarver/Michelangelo/tree/main), [Hunyuan-DiT](https://github.com/Tencent-Hunyuan/HunyuanDiT), and [HunyuanVideo](https://github.com/Tencent-Hunyuan/HunyuanVideo) repositories, for their open research and
exploration.
## Star History
<a href="https://star-history.com/#Tencent-Hunyuan/Hunyuan3D-2.1&Date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=Tencent-Hunyuan/Hunyuan3D-2.1&type=Date&theme=dark" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=Tencent-Hunyuan/Hunyuan3D-2.1&type=Date" />
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=Tencent-Hunyuan/Hunyuan3D-2.1&type=Date" />
</picture>
</a>