a148b4a543
- autorig: landmark detection from vertex cloud (markers JSON override), mixamorig 22-bone skeleton fit, bone-heat weights + nearest-bone rescue - retarget: world-space quaternion delta transfer, hip-height scale compensation, fuzzy/explicit bone mapping (CMU map bundled) - smoke: procedural test humanoid + synthetic BVH -> rig -> retarget -> verify - deploy: push to gitea, pull + smoke on m3ultra and m1ultra Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
96 lines
3.9 KiB
Markdown
96 lines
3.9 KiB
Markdown
# MIRPAMO
|
|
|
|
**A local, offline Mixamo.** Auto-rigging + animation retargeting as a
|
|
headless-Blender pipeline — no cloud, no Adobe account, no upload limits.
|
|
Built to run on the studio Macs (M3 Ultra / M1 Ultra) and any laptop with
|
|
Blender installed.
|
|
|
|
```
|
|
raw mesh (.glb/.fbx/.obj)
|
|
│
|
|
▼
|
|
[1] autorig landmark detection → mixamorig skeleton fit → bone-heat weights
|
|
│
|
|
▼
|
|
rigged.glb ──► [2] retarget ◄── motion clip (.bvh/.fbx/.glb)
|
|
│ quaternion delta transfer + hip-height scale compensation
|
|
▼
|
|
character@clip.glb (drop straight into three.js / game engines)
|
|
```
|
|
|
|
## Requirements
|
|
|
|
- Blender 4.2+ (tested on 5.0.1 and 5.1.2) at
|
|
`/Applications/Blender.app` — or set `MIRPAMO_BLENDER=/path/to/blender`
|
|
- Python 3.9+ (only for the thin CLI; all bpy work runs inside Blender)
|
|
|
|
No pip installs. No GPU requirements — bone heat and retargeting are CPU.
|
|
|
|
## Quickstart
|
|
|
|
```bash
|
|
make smoke # end-to-end self test: proves the machine can run the pipeline
|
|
|
|
# rig a raw mesh (landmarks are auto-detected; T- or A-pose, upright, on the floor)
|
|
./bin/mirpamo rig mymesh.glb -o rigged.glb
|
|
|
|
# fix a bad joint guess with Mixamo-style markers (world-space Z-up meters)
|
|
./bin/mirpamo rig mymesh.glb -o rigged.glb --markers markers.json
|
|
# {"chin": [0,0,1.55], "l_wrist": [0.7,0,1.4], "groin": [0,0,0.9], ...}
|
|
|
|
# retarget one clip
|
|
./bin/mirpamo anim rigged.glb walk.bvh -o walking.glb
|
|
|
|
# retarget an entire clip folder (e.g. the existing Mixamo bank)
|
|
./bin/mirpamo batch rigged.glb ~/Documents/mixamo-fetch/out -o out/clips/
|
|
|
|
# CMU MotionBuilder-friendly BVH needs the bundled bone map
|
|
./bin/mirpamo anim rigged.glb cmu_01_01.bvh -o out.glb \
|
|
--bonemap bonemaps/cmu_mb_to_mixamo.json
|
|
```
|
|
|
|
## How it works (the Mixamo parts, replicated)
|
|
|
|
| Mixamo (cloud) | MIRPAMO (local) |
|
|
|---|---|
|
|
| Colored marker dots (chin/wrists/elbows/knees/groin) | Automatic landmark detection from the vertex cloud; optional `--markers` JSON override ([bpy/autorig.py](bpy/autorig.py)) |
|
|
| Master skeleton template snapped to markers | mixamorig-named 22-bone humanoid fitted to landmarks — existing Mixamo clips retarget 1:1 |
|
|
| Spatial-proximity skin weighting with smooth falloff | Blender bone-heat automatic weights |
|
|
| Rotational (quaternion) data transfer | World-space rotation-delta transfer per bone per frame ([bpy/retarget.py](bpy/retarget.py)) |
|
|
| Scale compensation for different proportions | Root translation scaled by target/source hip-height ratio |
|
|
|
|
Same caveats as Mixamo apply: loose clothing/hair confuses proximity
|
|
weighting, and meshes should be clean, symmetric, T/A-pose.
|
|
|
|
## Machines
|
|
|
|
| host | hw | blender | role |
|
|
|---|---|---|---|
|
|
| laptop | dev | 5.1.2 | build here, push |
|
|
| `m3ultra@100.89.131.57` | M3 Ultra, 256GB | 5.1.2 | heavy batch retargets, big meshes |
|
|
| `johnking@100.91.239.7` | M1 Ultra, 128GB | 5.0.1 | second test target |
|
|
|
|
`make deploy` pushes to Gitea, pulls on both hosts, and runs the remote
|
|
smoke test on each.
|
|
|
|
## Clip libraries
|
|
|
|
- **Existing Mixamo bank**: machines with `~/Documents/mixamo-fetch/out/`
|
|
can batch-retarget it directly (mixamorig names match natively).
|
|
- `python3 library/fetch.py` downloads the sources in
|
|
[library/manifest.json](library/manifest.json) — check each entry's
|
|
license before shipping clips in a web game (see the asset-pipeline
|
|
licensing zones: research-only datasets stay local).
|
|
|
|
## Roadmap
|
|
|
|
- [ ] Finger bones (Mixamo's 65-bone variant) + finger weighting
|
|
- [ ] In-place conversion flag per clip in `batch`
|
|
- [ ] Multi-clip single-GLB export (one character, N animations) — the
|
|
retargeter already stores each clip as a named Action; needs an
|
|
export pass
|
|
- [ ] MLX texture generation stage (M3 Ultra: PBR texture synthesis
|
|
alongside rigging — separate `texgen/` stage)
|
|
- [ ] Optional ML landmark detection (replace heuristics for non-standard
|
|
bodies)
|