diff --git a/docs/CUDA-WORKER.md b/docs/CUDA-WORKER.md index 2891f1c..9c197ab 100644 --- a/docs/CUDA-WORKER.md +++ b/docs/CUDA-WORKER.md @@ -1,63 +1,71 @@ # CUDA Worker Setup — T490s + 2080 Ti eGPU on the tailnet -Goal: a headless Linux box with the RTX 2080 Ti (11GB) eGPU, live on Tailscale, that +Goal: a headless box with the RTX 2080 Ti (11GB) eGPU, live on Tailscale, that MODELBEAST (on the M3 Ultra) offloads the CUDA-only jobs the Mac can't run — gaussian-splat → mesh extraction (SuGaR/2DGS), research-grade video mocap (GVHMR/WHAM/TRAM), NeRF/tiny-cuda-nn, nvdiffrast/PyTorch3D. -**Why a ThinkPad and not the Intel iMac:** no T2 chip = no Secure-Boot/Thunderbolt +**Why the T490s and not the Intel iMac:** no T2 chip = no Secure-Boot/Thunderbolt fight; laptop idles ~6–10W (+ ~20W eGPU) vs ~55–75W for the iMac; battery = built-in -UPS. The 2080 Ti is EOL-ish (Turing: no bf16/fp8, 11GB) — it's a *compatibility* -box, not a performance one. It will not beat the M3 Ultra at anything the Mac -already does; it only unlocks the CUDA-locked tail. +UPS. The 2080 Ti is EOL-ish (Turing: no bf16/fp8, 11GB) — a *compatibility* box, not +a performance one. It won't beat the M3 Ultra at anything the Mac already does; it +only unlocks the CUDA-locked tail. -## Do this at the shop (once) +**A physical GPU can't be passed to a VM on a Mac/Windows consumer hypervisor** — so +the CUDA OS must have direct hardware access. On this ThinkPad that's either native +Windows (already installed) or WSL2 (a real Linux env Windows gives GPU access to). +NOT a Linux VM on macOS. -1. **Ubuntu 24.04 LTS**, bare metal (dual-boot Win11 or wipe — bare-metal, NOT a VM; - macOS/Windows hypervisors can't pass a physical GPU through, so a Linux VM can't - do CUDA on the eGPU). +--- -2. **Thunderbolt authorize the eGPU** (plug in the Razer Core first): - ```bash - sudo apt install bolt - boltctl list # find the enclosure UUID - boltctl enroll --policy auto +## Path A — Keep Windows 11 + WSL2 (RECOMMENDED: Win11 already runs, no reinstall) + +Windows drives the eGPU; WSL2 gives a clean Ubuntu where the Linux-first 3D repos +build properly (native-Windows builds of their custom CUDA kernels are a pain). + +1. **NVIDIA Windows driver** (Studio or Game Ready) with the 2080 Ti plugged in. + Confirm Device Manager → Display adapters shows the 2080 Ti. This one driver is + all WSL2 needs — do NOT install a driver inside WSL. +2. **WSL2 + Ubuntu** (admin PowerShell): `wsl --install -d Ubuntu` → reboot. +3. **CUDA toolkit INSIDE WSL** (the `wsl-ubuntu` package — toolkit only, no driver), + per NVIDIA's "CUDA on WSL" guide. Then verify: `nvidia-smi` inside Ubuntu lists + the 2080 Ti. (If empty: update the Windows driver; ensure eGPU present at boot.) +4. **Reach WSL from the tailnet cleanly** — create `C:\Users\\.wslconfig`: ``` - -3. **NVIDIA driver + CUDA:** - ```bash - sudo ubuntu-drivers autoinstall # or the latest official driver branch - sudo apt install nvidia-cuda-toolkit - sudo reboot - nvidia-smi # must list the 2080 Ti over Thunderbolt + [wsl2] + networkingMode=mirrored ``` - If it doesn't appear: confirm the eGPU was connected at boot, re-check `boltctl`, - ensure the `thunderbolt` module loads before `nvidia_drm`. + (Win11 22H2+; makes WSL share the host's network so Tailscale/SSH reach it.) +5. **Tailscale** on the Windows host: install, sign in, `tailscale ip -4` → note it. +6. **Headless Windows settings:** Power → "when I close the lid: do nothing" + never + sleep; enable **OpenSSH Server** (Settings → Optional Features); set Windows Update + active hours wide / pause auto-reboots so it never reboots mid-job. +7. Access model: MODELBEAST SSHes to the Windows host over Tailscale and runs CUDA + tools via `wsl bash -lc "..."`, or SSH straight into WSL with mirrored networking. -4. **Tailscale:** - ```bash - curl -fsSL https://tailscale.com/install.sh | sh - sudo tailscale up --ssh # --ssh lets the M3 Ultra SSH in over the tailnet - tailscale ip -4 # note this IP → put it in MODELBEAST settings - ``` +## Path B — Bare-metal Ubuntu (leaner/lower-power, if you'd rather wipe Win11) -5. **Headless / low-power:** run lid-closed → - `sudo sed -i 's/#HandleLidSwitch=suspend/HandleLidSwitch=ignore/' /etc/systemd/logind.conf` - then `sudo systemctl restart systemd-logind`. Enable Wake-on-LAN in BIOS + `ethtool` - if you want it to sleep and wake on demand. +1. Ubuntu 24.04 LTS bare metal. +2. `sudo ubuntu-drivers autoinstall && sudo apt install nvidia-cuda-toolkit && sudo reboot`; `nvidia-smi` must show the 2080 Ti. +3. `curl -fsSL https://tailscale.com/install.sh | sh && sudo tailscale up --ssh`. +4. Lid-closed headless: set `HandleLidSwitch=ignore` in `/etc/systemd/logind.conf`. + (No Thunderbolt-authorization dance needed on a non-Mac — it enumerates on boot.) -## MODELBEAST integration (I'll build this when the box exists) +--- + +## MODELBEAST integration (I'll build this once the box is on the tailnet) The operator framework already shells out to tools; a **remote-CUDA operator** wraps -that pattern: rsync the input to `cuda_host:~/work/`, `ssh cuda_host` to run the CUDA -tool in its venv, rsync results back, register as assets. Planned operators: -- `sugar_mesh` — splat `.ply` → textured mesh (closes the Brush→mesh gap; the Mac can't extract meshes from splats). -- `gvhmr_mocap` — video → world-grounded SMPL → FBX (research-grade, replaces the cloud/manual path). +that: rsync the input to the worker, `ssh` (→ `wsl` on Path A) to run the CUDA tool +in its env, rsync results back, register as assets. Add a `cuda_host` setting (tailnet +IP + user) to the vault. Planned: +- `sugar_mesh` — splat `.ply` → textured mesh (closes the Brush→mesh gap; the Mac can't extract meshes from splats). ~5–6GB, fits 11GB. +- `gvhmr_mocap` — video → world-grounded SMPL → FBX (research-grade; replaces the cloud/manual path). Light. -Add a `cuda_host` setting (the tailnet IP + user) to the vault so these operators know where to reach it. Fits in 11GB: SuGaR/2DGS/gsplat (~5-6GB), GVHMR/WHAM (light). Does NOT fit: FLUX, 20B+ models — those stay on the Mac. +Does NOT fit 11GB: FLUX, 20B+ models — those stay on the Mac. ## Reality check -Only worth it because the card's already owned. If you were buying, a used RTX 3090 -(24GB, bf16) would be strictly better and unlock FLUX/video too. Turing's runway is -shrinking (newer 3D wheels drop 20-series), so treat this as "use the free card for -the CUDA-only leftovers," not a long-term foundation. +Worth it only because the card's already owned. If buying, a used RTX 3090 (24GB, +bf16) is strictly better and unlocks FLUX/video too. Turing's runway is shrinking +(newer 3D wheels drop 20-series). Use the free card for the CUDA-only leftovers, not +as a foundation.