modelbeast/docs/CUDA-WORKER.md

CUDA Worker Setup — T490s + 2080 Ti eGPU on the tailnet

Goal: a headless box with the RTX 2080 Ti (11GB) eGPU, live on Tailscale, that MODELBEAST (on the M3 Ultra) offloads the CUDA-only jobs the Mac can't run — gaussian-splat → mesh extraction (SuGaR/2DGS), research-grade video mocap (GVHMR/WHAM/TRAM), NeRF/tiny-cuda-nn, nvdiffrast/PyTorch3D.

Why the T490s and not the Intel iMac: no T2 chip = no Secure-Boot/Thunderbolt fight; laptop idles ~6–10W (+ ~20W eGPU) vs ~55–75W for the iMac; battery = built-in UPS. The 2080 Ti is EOL-ish (Turing: no bf16/fp8, 11GB) — a compatibility box, not a performance one. It won't beat the M3 Ultra at anything the Mac already does; it only unlocks the CUDA-locked tail.

A physical GPU can't be passed to a VM on a Mac/Windows consumer hypervisor — so the CUDA OS must have direct hardware access. On this ThinkPad that's either native Windows (already installed) or WSL2 (a real Linux env Windows gives GPU access to). NOT a Linux VM on macOS.

---

Path A — Keep Windows 11 + WSL2 (RECOMMENDED: Win11 already runs, no reinstall)

Windows drives the eGPU; WSL2 gives a clean Ubuntu where the Linux-first 3D repos build properly (native-Windows builds of their custom CUDA kernels are a pain).

1. NVIDIA Windows driver (Studio or Game Ready) with the 2080 Ti plugged in. Confirm Device Manager → Display adapters shows the 2080 Ti. This one driver is all WSL2 needs — do NOT install a driver inside WSL.
2. WSL2 + Ubuntu (admin PowerShell): wsl --install -d Ubuntu → reboot.
3. CUDA toolkit INSIDE WSL (the wsl-ubuntu package — toolkit only, no driver), per NVIDIA's "CUDA on WSL" guide. Then verify: nvidia-smi inside Ubuntu lists the 2080 Ti. (If empty: update the Windows driver; ensure eGPU present at boot.)
4. Reach WSL from the tailnet cleanly — create C:\Users\<you>\.wslconfig:

   [wsl2]
   networkingMode=mirrored
   

   (Win11 22H2+; makes WSL share the host's network so Tailscale/SSH reach it.)
5. Tailscale on the Windows host: install, sign in, tailscale ip -4 → note it.
6. Headless Windows settings: Power → "when I close the lid: do nothing" + never sleep; enable OpenSSH Server (Settings → Optional Features); set Windows Update active hours wide / pause auto-reboots so it never reboots mid-job.
7. Access model: MODELBEAST SSHes to the Windows host over Tailscale and runs CUDA tools via wsl bash -lc "...", or SSH straight into WSL with mirrored networking.

Path B — Bare-metal Ubuntu (leaner/lower-power, if you'd rather wipe Win11)

1. Ubuntu 24.04 LTS bare metal.
2. sudo ubuntu-drivers autoinstall && sudo apt install nvidia-cuda-toolkit && sudo reboot; nvidia-smi must show the 2080 Ti.
3. curl -fsSL https://tailscale.com/install.sh | sh && sudo tailscale up --ssh.
4. Lid-closed headless: set HandleLidSwitch=ignore in /etc/systemd/logind.conf. (No Thunderbolt-authorization dance needed on a non-Mac — it enumerates on boot.)

---

MODELBEAST integration (I'll build this once the box is on the tailnet)

The operator framework already shells out to tools; a remote-CUDA operator wraps that: rsync the input to the worker, ssh (→ wsl on Path A) to run the CUDA tool in its env, rsync results back, register as assets. Add a cuda_host setting (tailnet IP + user) to the vault. Planned:

• sugar_mesh — splat .ply → textured mesh (closes the Brush→mesh gap; the Mac can't extract meshes from splats). ~5–6GB, fits 11GB.
• gvhmr_mocap — video → world-grounded SMPL → FBX (research-grade; replaces the cloud/manual path). Light.

Does NOT fit 11GB: FLUX, 20B+ models — those stay on the Mac.

Reality check

Worth it only because the card's already owned. If buying, a used RTX 3090 (24GB, bf16) is strictly better and unlocks FLUX/video too. Turing's runway is shrinking (newer 3D wheels drop 20-series). Use the free card for the CUDA-only leftovers, not as a foundation.