diff --git a/README.md b/README.md index 053c4a3..96f86d0 100644 --- a/README.md +++ b/README.md @@ -1,282 +1,71 @@ +# Hunyuan3D-2.2-mrp-MLX -

- -

- - -
- - - - - - - - -
- -[//]: # ( ) - -[//]: # ( ) - -[//]: # ( PyPI - Downloads) -
- -## Apple MLX Port (Apple Silicon) - -This fork adds native **MLX inference** for the full Hunyuan3D-2.1 pipeline on Apple Silicon Macs — both shape generation (image → mesh) and PBR texture synthesis (mesh + reference image → textured GLB). - -### Weights - -Pre-converted MLX weights are hosted on HuggingFace and auto-downloaded on first use: - -> [`dgrauet/hunyuan3d-2.1-mlx`](https://huggingface.co/dgrauet/hunyuan3d-2.1-mlx) - -If you'd rather convert from the original PyTorch checkpoints yourself: +**One-command image → textured 3D on Apple Silicon.** A tuned, packaged build of Tencent's Hunyuan3D 2.1, running natively on Mac GPUs via Apple MLX — no NVIDIA, no CUDA, no cloud. ```bash -pip install mlx-forge -mlx-forge convert hunyuan3d-2.1 --output ./models/hunyuan3d-2.1-mlx -# or quantized for 16GB Macs: -mlx-forge convert hunyuan3d-2.1 --quantize --bits 8 --output ./models/hunyuan3d-2.1-mlx +python generate_e2e.py your_image.png --output out/mymesh +# → out/mymesh.glb (PBR-textured mesh) + out/mymesh_shape.glb (untextured) ``` -Override the default HF lookup by setting `HUNYUAN3D_MLX_WEIGHTS_DIR=/path/to/local/models`. - -### Install - -```bash -pip install mlx mlx-arsenal safetensors Pillow trimesh scikit-image PyMCubes scipy -pip install huggingface_hub xatlas opencv-python # for Stage 2 -``` - -### Stage 1 — Shape Generation (image → mesh) - -```python -from hy3dshape.hy3dshape.pipeline_mlx import ShapePipeline - -pipe = ShapePipeline.from_pretrained("dgrauet/hunyuan3d-2.1-mlx") -mesh = pipe("your_image.png", num_inference_steps=50, guidance_scale=7.5, octree_resolution=256) -mesh.export("output.glb") -``` - -### Stage 2 — PBR Texture Synthesis (mesh + reference → textured GLB) - -```python -from textureGenPipeline_mlx import Hunyuan3DPaintConfigMLX, Hunyuan3DPaintPipelineMLX - -cfg = Hunyuan3DPaintConfigMLX(max_num_view=6, resolution=512) -pipe = Hunyuan3DPaintPipelineMLX(cfg) -pipe( - mesh_path="mesh.glb", - image_path="reference.png", - output_mesh_path="textured.obj", - save_glb=True, -) -``` - -Output: `textured.glb` with 2048² PBR albedo + metallic-roughness textures baked onto the mesh UVs (~9 min for 6 views at 512px on an M2 Pro). - -Defaults match the PyTorch reference config exactly, with one documented exception: `texture_size=2048` (PT uses 4096). The MLX Metal rasterizer has no tiling and would exceed the GPU command-buffer budget at 4096² on laptop GPUs (`kIOGPUCommandBufferCallbackErrorImpactingInteractivity`). Every other knob — `max_num_view`, `render_size`, `num_inference_steps`, `guidance_scale`, bake mode, inpaint method — is PT-identical. - -### End-to-end (image → textured GLB) - -`tests/test_stage1_to_stage2.py` chains both stages on a single reference image. Stage 1's dense marching-cubes mesh is automatically remeshed to ~40k faces before Stage 2's bake (the Metal rasterizer can't handle Stage 1's raw ~500k faces at `texture_size=2048`). - -### Memory Requirements - -| Precision | DiT Size | Peak Memory | Recommended Mac | -|-----------|----------|-------------|-----------------| -| FP16 | 5.7 GB | ~10 GB | 32 GB+ | -| INT8 | 3.0 GB | ~6 GB | 16 GB+ | -| INT4 | 1.6 GB | ~4 GB | 16 GB | - -Stage 2 (paint) adds ~6 GB peak for the UNet + VAE + DINOv2 at fp32. - -### Scope - -- ✅ **Stage 1** (shape generation): fully ported to MLX, validated numerically against PyTorch (1e-5) -- ✅ **Stage 2** (PBR texture synthesis): fully ported to MLX, full UNet match within 1.17e-5 vs PyTorch - - Metal rasterizer, UNet 2.5D with dual-stream reference attention, VAE, DINOv2, v_prediction scheduler - - **Cosine-weighted bake blend** (PT-parity `fast_bake_texture`) smooths across face seams - - **Mesh-aware vertex-color propagation + face barycentric raster**, `cv2.INPAINT_NS` (PT-parity), plus an EDT nearest-fill post-pass that pads UV gutters so 3D viewers doing bilinear sampling across island boundaries never pull in the atlas background - - **RealESRGAN x4 super-resolution** in MLX (512² → 2048² per view before bake) - - **GLB export**: PBRMaterial with sRGB `baseColorTexture`, `metallicRoughnessTexture` (glTF channel order), `doubleSided=true` - - All pipeline defaults now match the PyTorch reference exactly (see porting principle in `docs/forward_pass.md`) - -### Texture quality notes - -- Reference image must match the mesh content (e.g. mermaid image on mermaid mesh). Cross-pairing produces fragmented atlases since the diffusion's albedo views can't coherently project onto an unrelated 3D layout. -- Tested on `assets/case_1/mesh.glb` (fox) and `assets/case_2/mesh.glb` (mermaid) — all 6/6 views clean with PT-aligned defaults. - --- -## 🔥 News +## What this is (provenance & credit) -- Jul 26, 2025: 🤗 We release the first open-source, simulation-capable, immersive 3D world generation model, [HunyuanWorld-1.0](https://github.com/Tencent-Hunyuan/HunyuanWorld-1.0)! -- Jun 19, 2025: 👋 We present the [technical report](https://arxiv.org/pdf/2506.15442) of Hunyuan3D-2.1, please check out the details and spark some discussion! -- Jun 13, 2025: 🤗 We release the first production-ready 3D asset generation model, Hunyuan3D-2.1! +A **fork of a fork** — three layers, credited honestly: -> Join our **[Wechat](#)** and **[Discord](https://discord.gg/dNBrdrGGMa)** group to discuss and find help from us. +1. **Tencent Hunyuan3D 2.1** — the actual models: `Hunyuan3D-Shape-v2-1` (3.3B, image→mesh) + `Hunyuan3D-Paint-v2-1` (2B, PBR texture). Originally **CUDA/NVIDIA only**. +2. **[dgrauet/Hunyuan3D-2.1-mlx](https://github.com/dgrauet/Hunyuan3D-2.1-mlx)** — the hard part: a native **MLX (Metal)** port of *both* stages. fp16, no bf16 dependency → runs on **any** Apple Silicon Mac (M1–M5). This is what made it work on a Mac at all. +3. **This build (MODELBEAST / `mrp`)** — the usability + tuning layer on top of dgrauet (below). -| Wechat Group | Xiaohongshu | X | Discord | -|--------------------------------------------------|-------------------------------------------------------|---------------------------------------------|---------------------------------------------------| -| | | | | +## What *we* added over upstream -## 🤗 Community Contribution Leaderboard -1. By [@visualbruno](https://github.com/visualbruno) - - ComfyUI-Hunyuan3d-2-1: https://github.com/visualbruno/ComfyUI-Hunyuan3d-2-1 -2. By [@VR-Jobs](https://github.com/VR-Jobs) - - Hunyuan3d-2-1 Unity Support: https://github.com/VR-Jobs/Hunyuan3D-2.1-Unity-XR-PC-Phone +- **`generate_e2e.py`** — a single command chaining Stage 1 (shape) → Stage 2 (PBR texture). Upstream only shipped a test script. Flags: `--steps --guidance --octree-resolution --max-num-view --view-resolution --texture-size --remesh-faces --seed --no-texture`; prints per-stage timings + peak memory. +- **Env-tunable bake mesh** — `HY3D_REMESH_FACES` (upstream hard-coded 40k). The **single biggest quality lever** — it's what un-melts faces. +- **Studio-quality defaults** — `octree 384 / texture 4096 / remesh 120k`, **verified watchdog-free on M3 *and* M1 Ultra**. Upstream capped texture at 2048 because laptop GPUs hit a Metal command-buffer limit; Studio-class GPUs don't, and the existing tiling handles 4096². +- **[HARDWARE.md](HARDWARE.md)** — exact RAM needs per precision and per Mac. +- **Slimmed for inference** — upstream training data (`mini_trainset`) + demo images stripped (~165 MB → ~9 MB). Recover any time via `git fetch upstream` (github.com/dgrauet). -## ☯️ **Hunyuan3D 2.1** - -### Architecture - -Tencent Hunyuan3D-2.1 is a scalable 3D asset creation system that advances state-of-the-art 3D generation through two pivotal innovations: Fully Open-Source Framework and Physically-Based Rendering (PBR) Texture Synthesis. For the first time, the system releases full model weights and training code, enabling community developers to directly fine-tune and extend the model for diverse downstream applications. This transparency accelerates academic research and industrial deployment. Moreover, replacing the prior RGB-based texture model, the upgraded PBR pipeline leverages physics-grounded material simulation to generate textures with photorealistic light interaction (e.g., metallic reflections, subsurface scattering). - -

- -

- -### Performance - -We have evaluated Hunyuan3D 2.1 with other open-source as well as close-source 3d-generation methods. -The numerical results indicate that Hunyuan3D 2.1 surpasses all baselines in the quality of generated textured 3D assets -and the condition following ability. - -| Model | ULIP-T(⬆) | ULIP-I(⬆) | Uni3D-T(⬆) | Uni3D-I(⬆) | -|-------------------------|-----------|-------------|-------------|---------------| -| Michelangelo | 0.0752 | 0.1152 | 0.2133 | 0.2611 | -| Craftsman | 0.0745 | 0.1296 | 0.2375 | 0.2987 | -| TripoSG | 0.0767 | 0.1225 | 0.2506 | 0.3129 | -| Step1X-3D | 0.0735 | 0.1183 | 0.2554 | 0.3195 | -| Trellis | 0.0769 | 0.1267 | 0.2496 | 0.3116 | -| Direct3D-S2 | 0.0706 | 0.1134 | 0.2346 | 0.2930 | -| Hunyuan3D-Shape-2.1 | **0.0774** | **0.1395** | **0.2556** | **0.3213** | - - -| Model | CLIP-FiD(⬇) | CMMD(⬇) | CLIP-I(⬆) | LPIPS(⬇) | -|-------------------------|-----------|-------------|-------------|---------------| -| SyncMVD-IPA | 28.39 | 2.397 | 0.8823 | 0.1423 | -| TexGen | 28.24 | 2.448 | 0.8818 | 0.1331 | -| Hunyuan3D-2.0 | 26.44 | 2.318 | 0.8893 | 0.1261 | -| Hunyuan3D-Paint-2.1 | **24.78** | **2.191** | **0.9207** | **0.1211** | - - - -## 🎁 Models Zoo - -It takes 10 GB VRAM for shape generation, 21GB for texture generation and 29GB for shape and texture generation in total. - - -| Model | Description | Date | Size | Huggingface | -|----------------------------|-----------------------------|------------|------|-------------------------------------------------------------------------------------------| -| Hunyuan3D-Shape-v2-1 | Image to Shape Model | 2025-06-14 | 3.3B | [Download](https://huggingface.co/tencent/Hunyuan3D-2.1/tree/main/hunyuan3d-dit-v2-1) | -| Hunyuan3D-Paint-v2-1 | Texture Generation Model | 2025-06-14 | 2B | [Download](https://huggingface.co/tencent/Hunyuan3D-2.1/tree/main/hunyuan3d-paintpbr-v2-1) | - - -## 🤗 Get Started with Hunyuan3D 2.1 - -Hunyuan3D 2.1 supports Macos, Windows, Linux. You may follow the next steps to use Hunyuan3D 2.1 via: - -### Install Requirements -We test our model with Python 3.10 and PyTorch 2.5.1+cu124. -```bash -pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu124 -pip install -r requirements.txt - -cd hy3dpaint/custom_rasterizer -pip install -e . -cd ../.. -cd hy3dpaint/DifferentiableRenderer -bash compile_mesh_painter.sh -cd ../.. - -wget https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth -P hy3dpaint/ckpt -``` - -### Code Usage - -We designed a diffusers-like API to use our shape generation model - Hunyuan3D-Shape and texture synthesis model - -Hunyuan3D-Paint. - -```python -import sys -sys.path.insert(0, './hy3dshape') -sys.path.insert(0, './hy3dpaint') -from textureGenPipeline import Hunyuan3DPaintPipeline -from textureGenPipeline import Hunyuan3DPaintPipeline, Hunyuan3DPaintConfig -from hy3dshape.pipelines import Hunyuan3DDiTFlowMatchingPipeline - -# let's generate a mesh first -shape_pipeline = Hunyuan3DDiTFlowMatchingPipeline.from_pretrained('tencent/Hunyuan3D-2.1') -mesh_untextured = shape_pipeline(image='assets/demo.png')[0] - -paint_pipeline = Hunyuan3DPaintPipeline(Hunyuan3DPaintConfig(max_num_view=6, resolution=512)) -mesh_textured = paint_pipeline(mesh_path, image_path='assets/demo.png') -``` - - -### Gradio App - -You could also host a [Gradio](https://www.gradio.app/) App in your own computer via: +## Quick start +Python **3.11** (the ML stack pins old numpy; 3.12+ won't build it). ```bash -python3 gradio_app.py \ - --model_path tencent/Hunyuan3D-2.1 \ - --subfolder hunyuan3d-dit-v2-1 \ - --texgen_model_path tencent/Hunyuan3D-2.1 \ - --low_vram_mode +uv venv --python 3.11 .venv && source .venv/bin/activate # or: python3.11 -m venv .venv +uv pip install mlx mlx-arsenal torch torchvision transformers timm \ + huggingface_hub safetensors numpy scipy scikit-image PyMCubes \ + trimesh pygltflib xatlas opencv-python pillow pymeshlab \ + einops omegaconf pyyaml tqdm torchdiffeq diffusers accelerate fast_simplification + +export HF_HUB_DISABLE_XET=1 +python generate_e2e.py your_image.png --output out/mymesh ``` +Weights (~13 GB, fp16 MLX, **public — no HF login**) auto-download on first run from `dgrauet/hunyuan3d-2.1-mlx`. Override with `HUNYUAN3D_MLX_WEIGHTS_DIR`. Best results: feed a **background-removed cutout**. -## 🔗 BibTeX +## Quality vs speed (generate_e2e.py flags) -If you found this repository helpful, please cite our reports: +| | fast draft | **default (studio)** | max | +|---|---|---|---| +| `--texture-size` | 2048 | **4096** | 4096 | +| `--remesh-faces` | 40000 | **120000** | 200000 | +| `--octree-resolution` | 256 | **384** | 384 | +| `--max-num-view` | 6 | 6 | 8–12 | +| time (M3 Ultra) | ~260 s | ~380 s | longer | +| output | 7.6 MB / 40k faces | 21.5 MB / 120k faces | larger | -```bibtex -@misc{hunyuan3d2025hunyuan3d, - title={Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Material}, - author={Tencent Hunyuan3D Team}, - year={2025}, - eprint={2506.15442}, - archivePrefix={arXiv}, - primaryClass={cs.CV} -} +`--no-texture` = Stage 1 only (geometry, ~2.5 min, ~10 GB RAM). -@misc{hunyuan3d22025tencent, - title={Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation}, - author={Tencent Hunyuan3D Team}, - year={2025}, - eprint={2501.12202}, - archivePrefix={arXiv}, - primaryClass={cs.CV} -} +## Hardware (full table in [HARDWARE.md](HARDWARE.md)) -@misc{yang2024hunyuan3d, - title={Hunyuan3D 1.0: A Unified Framework for Text-to-3D and Image-to-3D Generation}, - author={Tencent Hunyuan3D Team}, - year={2024}, - eprint={2411.02293}, - archivePrefix={arXiv}, - primaryClass={cs.CV} -} -``` +Peak **~20 GB** fp16 for the full textured pipeline (measured). **Min 16 GB** (INT8/INT4) or **24 GB** (fp16, tight) · **32 GB recommended**. Any M1→M5 — fp16-MLX path, no bf16 gotcha. -## Acknowledgements +Measured full textured gen: **M3 Ultra ~380 s**, **M1 Ultra ~750 s** (4096 / 120k). -We would like to thank the contributors to -the [TripoSG](https://github.com/VAST-AI-Research/TripoSG), [Trellis](https://github.com/microsoft/TRELLIS), [DINOv2](https://github.com/facebookresearch/dinov2), [Stable Diffusion](https://github.com/Stability-AI/stablediffusion), [FLUX](https://github.com/black-forest-labs/flux), [diffusers](https://github.com/huggingface/diffusers), [HuggingFace](https://huggingface.co), [CraftsMan3D](https://github.com/wyysf-98/CraftsMan3D), [Michelangelo](https://github.com/NeuralCarver/Michelangelo/tree/main), [Hunyuan-DiT](https://github.com/Tencent-Hunyuan/HunyuanDiT), and [HunyuanVideo](https://github.com/Tencent-Hunyuan/HunyuanVideo) repositories, for their open research and -exploration. +## ⚠️ License -## Star History +Inherits the **Tencent Hunyuan Non-Commercial License** (see `LICENSE` / `Notice.txt`). **Non-commercial use only** — share and use accordingly. - - - - - Star History Chart - - +## Credits + +Tencent Hunyuan3D team (models) · [@dgrauet](https://github.com/dgrauet) (MLX port) · MODELBEAST (this build). +Original upstream README preserved as [`README_UPSTREAM.md`](README_UPSTREAM.md). diff --git a/README_UPSTREAM.md b/README_UPSTREAM.md new file mode 100644 index 0000000..053c4a3 --- /dev/null +++ b/README_UPSTREAM.md @@ -0,0 +1,282 @@ + +

+ +

+ + +
+ + + + + + + + +
+ +[//]: # ( ) + +[//]: # ( ) + +[//]: # ( PyPI - Downloads) +
+ +## Apple MLX Port (Apple Silicon) + +This fork adds native **MLX inference** for the full Hunyuan3D-2.1 pipeline on Apple Silicon Macs — both shape generation (image → mesh) and PBR texture synthesis (mesh + reference image → textured GLB). + +### Weights + +Pre-converted MLX weights are hosted on HuggingFace and auto-downloaded on first use: + +> [`dgrauet/hunyuan3d-2.1-mlx`](https://huggingface.co/dgrauet/hunyuan3d-2.1-mlx) + +If you'd rather convert from the original PyTorch checkpoints yourself: + +```bash +pip install mlx-forge +mlx-forge convert hunyuan3d-2.1 --output ./models/hunyuan3d-2.1-mlx +# or quantized for 16GB Macs: +mlx-forge convert hunyuan3d-2.1 --quantize --bits 8 --output ./models/hunyuan3d-2.1-mlx +``` + +Override the default HF lookup by setting `HUNYUAN3D_MLX_WEIGHTS_DIR=/path/to/local/models`. + +### Install + +```bash +pip install mlx mlx-arsenal safetensors Pillow trimesh scikit-image PyMCubes scipy +pip install huggingface_hub xatlas opencv-python # for Stage 2 +``` + +### Stage 1 — Shape Generation (image → mesh) + +```python +from hy3dshape.hy3dshape.pipeline_mlx import ShapePipeline + +pipe = ShapePipeline.from_pretrained("dgrauet/hunyuan3d-2.1-mlx") +mesh = pipe("your_image.png", num_inference_steps=50, guidance_scale=7.5, octree_resolution=256) +mesh.export("output.glb") +``` + +### Stage 2 — PBR Texture Synthesis (mesh + reference → textured GLB) + +```python +from textureGenPipeline_mlx import Hunyuan3DPaintConfigMLX, Hunyuan3DPaintPipelineMLX + +cfg = Hunyuan3DPaintConfigMLX(max_num_view=6, resolution=512) +pipe = Hunyuan3DPaintPipelineMLX(cfg) +pipe( + mesh_path="mesh.glb", + image_path="reference.png", + output_mesh_path="textured.obj", + save_glb=True, +) +``` + +Output: `textured.glb` with 2048² PBR albedo + metallic-roughness textures baked onto the mesh UVs (~9 min for 6 views at 512px on an M2 Pro). + +Defaults match the PyTorch reference config exactly, with one documented exception: `texture_size=2048` (PT uses 4096). The MLX Metal rasterizer has no tiling and would exceed the GPU command-buffer budget at 4096² on laptop GPUs (`kIOGPUCommandBufferCallbackErrorImpactingInteractivity`). Every other knob — `max_num_view`, `render_size`, `num_inference_steps`, `guidance_scale`, bake mode, inpaint method — is PT-identical. + +### End-to-end (image → textured GLB) + +`tests/test_stage1_to_stage2.py` chains both stages on a single reference image. Stage 1's dense marching-cubes mesh is automatically remeshed to ~40k faces before Stage 2's bake (the Metal rasterizer can't handle Stage 1's raw ~500k faces at `texture_size=2048`). + +### Memory Requirements + +| Precision | DiT Size | Peak Memory | Recommended Mac | +|-----------|----------|-------------|-----------------| +| FP16 | 5.7 GB | ~10 GB | 32 GB+ | +| INT8 | 3.0 GB | ~6 GB | 16 GB+ | +| INT4 | 1.6 GB | ~4 GB | 16 GB | + +Stage 2 (paint) adds ~6 GB peak for the UNet + VAE + DINOv2 at fp32. + +### Scope + +- ✅ **Stage 1** (shape generation): fully ported to MLX, validated numerically against PyTorch (1e-5) +- ✅ **Stage 2** (PBR texture synthesis): fully ported to MLX, full UNet match within 1.17e-5 vs PyTorch + - Metal rasterizer, UNet 2.5D with dual-stream reference attention, VAE, DINOv2, v_prediction scheduler + - **Cosine-weighted bake blend** (PT-parity `fast_bake_texture`) smooths across face seams + - **Mesh-aware vertex-color propagation + face barycentric raster**, `cv2.INPAINT_NS` (PT-parity), plus an EDT nearest-fill post-pass that pads UV gutters so 3D viewers doing bilinear sampling across island boundaries never pull in the atlas background + - **RealESRGAN x4 super-resolution** in MLX (512² → 2048² per view before bake) + - **GLB export**: PBRMaterial with sRGB `baseColorTexture`, `metallicRoughnessTexture` (glTF channel order), `doubleSided=true` + - All pipeline defaults now match the PyTorch reference exactly (see porting principle in `docs/forward_pass.md`) + +### Texture quality notes + +- Reference image must match the mesh content (e.g. mermaid image on mermaid mesh). Cross-pairing produces fragmented atlases since the diffusion's albedo views can't coherently project onto an unrelated 3D layout. +- Tested on `assets/case_1/mesh.glb` (fox) and `assets/case_2/mesh.glb` (mermaid) — all 6/6 views clean with PT-aligned defaults. + +--- + +## 🔥 News + +- Jul 26, 2025: 🤗 We release the first open-source, simulation-capable, immersive 3D world generation model, [HunyuanWorld-1.0](https://github.com/Tencent-Hunyuan/HunyuanWorld-1.0)! +- Jun 19, 2025: 👋 We present the [technical report](https://arxiv.org/pdf/2506.15442) of Hunyuan3D-2.1, please check out the details and spark some discussion! +- Jun 13, 2025: 🤗 We release the first production-ready 3D asset generation model, Hunyuan3D-2.1! + +> Join our **[Wechat](#)** and **[Discord](https://discord.gg/dNBrdrGGMa)** group to discuss and find help from us. + +| Wechat Group | Xiaohongshu | X | Discord | +|--------------------------------------------------|-------------------------------------------------------|---------------------------------------------|---------------------------------------------------| +| | | | | + +## 🤗 Community Contribution Leaderboard +1. By [@visualbruno](https://github.com/visualbruno) + - ComfyUI-Hunyuan3d-2-1: https://github.com/visualbruno/ComfyUI-Hunyuan3d-2-1 +2. By [@VR-Jobs](https://github.com/VR-Jobs) + - Hunyuan3d-2-1 Unity Support: https://github.com/VR-Jobs/Hunyuan3D-2.1-Unity-XR-PC-Phone + +## ☯️ **Hunyuan3D 2.1** + +### Architecture + +Tencent Hunyuan3D-2.1 is a scalable 3D asset creation system that advances state-of-the-art 3D generation through two pivotal innovations: Fully Open-Source Framework and Physically-Based Rendering (PBR) Texture Synthesis. For the first time, the system releases full model weights and training code, enabling community developers to directly fine-tune and extend the model for diverse downstream applications. This transparency accelerates academic research and industrial deployment. Moreover, replacing the prior RGB-based texture model, the upgraded PBR pipeline leverages physics-grounded material simulation to generate textures with photorealistic light interaction (e.g., metallic reflections, subsurface scattering). + +

+ +

+ +### Performance + +We have evaluated Hunyuan3D 2.1 with other open-source as well as close-source 3d-generation methods. +The numerical results indicate that Hunyuan3D 2.1 surpasses all baselines in the quality of generated textured 3D assets +and the condition following ability. + +| Model | ULIP-T(⬆) | ULIP-I(⬆) | Uni3D-T(⬆) | Uni3D-I(⬆) | +|-------------------------|-----------|-------------|-------------|---------------| +| Michelangelo | 0.0752 | 0.1152 | 0.2133 | 0.2611 | +| Craftsman | 0.0745 | 0.1296 | 0.2375 | 0.2987 | +| TripoSG | 0.0767 | 0.1225 | 0.2506 | 0.3129 | +| Step1X-3D | 0.0735 | 0.1183 | 0.2554 | 0.3195 | +| Trellis | 0.0769 | 0.1267 | 0.2496 | 0.3116 | +| Direct3D-S2 | 0.0706 | 0.1134 | 0.2346 | 0.2930 | +| Hunyuan3D-Shape-2.1 | **0.0774** | **0.1395** | **0.2556** | **0.3213** | + + +| Model | CLIP-FiD(⬇) | CMMD(⬇) | CLIP-I(⬆) | LPIPS(⬇) | +|-------------------------|-----------|-------------|-------------|---------------| +| SyncMVD-IPA | 28.39 | 2.397 | 0.8823 | 0.1423 | +| TexGen | 28.24 | 2.448 | 0.8818 | 0.1331 | +| Hunyuan3D-2.0 | 26.44 | 2.318 | 0.8893 | 0.1261 | +| Hunyuan3D-Paint-2.1 | **24.78** | **2.191** | **0.9207** | **0.1211** | + + + +## 🎁 Models Zoo + +It takes 10 GB VRAM for shape generation, 21GB for texture generation and 29GB for shape and texture generation in total. + + +| Model | Description | Date | Size | Huggingface | +|----------------------------|-----------------------------|------------|------|-------------------------------------------------------------------------------------------| +| Hunyuan3D-Shape-v2-1 | Image to Shape Model | 2025-06-14 | 3.3B | [Download](https://huggingface.co/tencent/Hunyuan3D-2.1/tree/main/hunyuan3d-dit-v2-1) | +| Hunyuan3D-Paint-v2-1 | Texture Generation Model | 2025-06-14 | 2B | [Download](https://huggingface.co/tencent/Hunyuan3D-2.1/tree/main/hunyuan3d-paintpbr-v2-1) | + + +## 🤗 Get Started with Hunyuan3D 2.1 + +Hunyuan3D 2.1 supports Macos, Windows, Linux. You may follow the next steps to use Hunyuan3D 2.1 via: + +### Install Requirements +We test our model with Python 3.10 and PyTorch 2.5.1+cu124. +```bash +pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu124 +pip install -r requirements.txt + +cd hy3dpaint/custom_rasterizer +pip install -e . +cd ../.. +cd hy3dpaint/DifferentiableRenderer +bash compile_mesh_painter.sh +cd ../.. + +wget https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth -P hy3dpaint/ckpt +``` + +### Code Usage + +We designed a diffusers-like API to use our shape generation model - Hunyuan3D-Shape and texture synthesis model - +Hunyuan3D-Paint. + +```python +import sys +sys.path.insert(0, './hy3dshape') +sys.path.insert(0, './hy3dpaint') +from textureGenPipeline import Hunyuan3DPaintPipeline +from textureGenPipeline import Hunyuan3DPaintPipeline, Hunyuan3DPaintConfig +from hy3dshape.pipelines import Hunyuan3DDiTFlowMatchingPipeline + +# let's generate a mesh first +shape_pipeline = Hunyuan3DDiTFlowMatchingPipeline.from_pretrained('tencent/Hunyuan3D-2.1') +mesh_untextured = shape_pipeline(image='assets/demo.png')[0] + +paint_pipeline = Hunyuan3DPaintPipeline(Hunyuan3DPaintConfig(max_num_view=6, resolution=512)) +mesh_textured = paint_pipeline(mesh_path, image_path='assets/demo.png') +``` + + +### Gradio App + +You could also host a [Gradio](https://www.gradio.app/) App in your own computer via: + + +```bash +python3 gradio_app.py \ + --model_path tencent/Hunyuan3D-2.1 \ + --subfolder hunyuan3d-dit-v2-1 \ + --texgen_model_path tencent/Hunyuan3D-2.1 \ + --low_vram_mode +``` + + +## 🔗 BibTeX + +If you found this repository helpful, please cite our reports: + +```bibtex +@misc{hunyuan3d2025hunyuan3d, + title={Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Material}, + author={Tencent Hunyuan3D Team}, + year={2025}, + eprint={2506.15442}, + archivePrefix={arXiv}, + primaryClass={cs.CV} +} + +@misc{hunyuan3d22025tencent, + title={Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation}, + author={Tencent Hunyuan3D Team}, + year={2025}, + eprint={2501.12202}, + archivePrefix={arXiv}, + primaryClass={cs.CV} +} + +@misc{yang2024hunyuan3d, + title={Hunyuan3D 1.0: A Unified Framework for Text-to-3D and Image-to-3D Generation}, + author={Tencent Hunyuan3D Team}, + year={2024}, + eprint={2411.02293}, + archivePrefix={arXiv}, + primaryClass={cs.CV} +} +``` + +## Acknowledgements + +We would like to thank the contributors to +the [TripoSG](https://github.com/VAST-AI-Research/TripoSG), [Trellis](https://github.com/microsoft/TRELLIS), [DINOv2](https://github.com/facebookresearch/dinov2), [Stable Diffusion](https://github.com/Stability-AI/stablediffusion), [FLUX](https://github.com/black-forest-labs/flux), [diffusers](https://github.com/huggingface/diffusers), [HuggingFace](https://huggingface.co), [CraftsMan3D](https://github.com/wyysf-98/CraftsMan3D), [Michelangelo](https://github.com/NeuralCarver/Michelangelo/tree/main), [Hunyuan-DiT](https://github.com/Tencent-Hunyuan/HunyuanDiT), and [HunyuanVideo](https://github.com/Tencent-Hunyuan/HunyuanVideo) repositories, for their open research and +exploration. + +## Star History + + + + + + Star History Chart + +