Unified worker pool: gpu lane dispatches across M3 + M1 nodes (HANDOFF2 phase B)
The gpu lane is now a NODE POOL (this Mac + remote workers from nodes.json) instead of a single-Metal semaphore. A gpu job runs on whichever node is free: - server/remote.py: ssh+rsync dispatch — mkdir remote dirs, rsync inputs out, run the operator's run.py over ssh with repo-relative paths (HF_TOKEN sourced from the node's .env.remote, off the process table), rsync outputs back, clean up. Cached health checks; per-node operator allowlist. - runner: _acquire_gpu_node picks the first free node that supports the op (local runs anything; a remote must list it + be reachable → auto-fallback to local). _run_job branches local/remote; the M1 never touches the M3's DB. - sysinfo/Dashboard: gpu lane limit = pool size; per-node status cards. - nodes.json (gitignored, primary-only): M1 worker, allowlist excludes trellis_mac (bf16 unverified on M1) + brush_train (not installed there). Verified: two FLUX jobs from one queue split M3(local, 10.3s) + M1(remote via ssh, 21.2s), both images rsync'd back and registered on the M3. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
5f46939321
commit
b8c333a93e
3
.gitignore
vendored
3
.gitignore
vendored
@ -37,3 +37,6 @@ target/
|
||||
# Brush prebuilt binary (reinstall via scripts/install_brush.sh)
|
||||
bin/brush
|
||||
bin/brush-app-aarch64-apple-darwin/
|
||||
|
||||
# GPU worker-pool config — primary only (a leaf worker must NOT have this)
|
||||
nodes.json
|
||||
|
||||
@ -83,6 +83,17 @@ export default function Dashboard({ jobs }) {
|
||||
))}
|
||||
</div>
|
||||
|
||||
{sys.nodes && sys.nodes.length > 1 && (
|
||||
<div className="lane-strip">
|
||||
{sys.nodes.map((n) => (
|
||||
<div key={n.name} className={"lane-card" + (n.busy ? " busy" : "")}>
|
||||
<b>🖥 {n.name}{n.remote ? "" : " (this Mac)"}</b>
|
||||
<span>GPU node · {n.busy ? "running a job" : "idle"}</span>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
|
||||
<h2>Now running / queued</h2>
|
||||
<table className="dash-table">
|
||||
<tbody>
|
||||
|
||||
Loading…
Reference in New Issue
Block a user