docs: wan_video + ardy_motion measured RAM/speed (two-machine verification)
wan_video 5B q8 verified clean on M3 Ultra (148.7s) and M2 Max (503.7s), identical 9.2GB max RSS on both; 81GB 'footprint' on m2max identified as MLX opportunistic cache, not demand. Frame-identical output at same seed across machines. HARDWARE.md gains wan_video and ardy_motion rows. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
m3ultra
parent
472a5a3473
commit
4c05a9daa8
@ -491,3 +491,28 @@ more hair curl detail. baker=vertex remains the 1s clean-albedo option.
|
||||
1M-cap probe (same image/seed, farm path): candidate 832,883 tris, 43MB
|
||||
GLB, OPAQUE+PBR — hair marginally fuller than 500k, subtle at viewing
|
||||
distance. Verdict: 500k stays the default; 1M = hero-asset tier.
|
||||
|
||||
### wan_video verified on two machines — RAM + speed + visual (2026-08-04)
|
||||
|
||||
Post-rewrite verification of the mlx-video operator path (5B q8), identical
|
||||
settings both boxes: 832x480, 49f, 20 steps, cfg 5.0, seed 1234, prompt
|
||||
"a red fox walking through fresh snow in a pine forest, golden hour".
|
||||
|
||||
| box | total | T5 | denoise | VAE | max RSS |
|
||||
|---|---|---|---|---|---|
|
||||
| **M3 Ultra 256GB** (80-core) | **148.7s** | 12.4s | 108.5s (5.4s/step) | 20.2s | **9.19 GB** |
|
||||
| **M2 Max 96GB** (38-core) | **503.7s** | 5.9s (warm) | 450.4s (22.5s/step) | 46.5s | **9.19 GB** |
|
||||
|
||||
Findings:
|
||||
1. **Output visually verified clean on both** (frames eyeballed) — and
|
||||
*frame-identical across machines at the same seed*. Deterministic
|
||||
cross-node reproduction; safe to route wan jobs to any capable node.
|
||||
2. **True working set is 9.2 GB** for the default job — identical RSS on
|
||||
both boxes. macOS `peak memory footprint` read ~81 GB on the M2 Max:
|
||||
that is MLX's opportunistic Metal cache filling free RAM, not demand.
|
||||
Don't size nodes off footprint.
|
||||
3. Denoise scales with GPU cores almost exactly (4.15x for 80 vs 38);
|
||||
VAE decode only 2.3x — small-job latency gap shrinks a little.
|
||||
4. m2max now holds the 5B q8 weights (`~/MODELBEAST/vendor/mlx-video-models/`)
|
||||
next to its A14B set — it can serve wan_video once a `venvs/mlxvideo`
|
||||
is installed there (`scripts/install_mlx_video.sh`; note the numba pin).
|
||||
|
||||
@ -27,6 +27,8 @@ Which Apple Silicon Mac can run which operator, by **unified memory (RAM)**. Num
|
||||
| `flux_local` schnell-4bit | gpu | 8.9 GB | **19.1 GB** | 24 GB | 32 GB | fast FLUX.1 draft |
|
||||
| `hunyuan3d_mlx` fp16 *(our default, 4096/120k)* | gpu | 13 GB | **20.2 GB** | 24 GB *(tight)* | 32 GB | Stage 1 alone ~10 GB; both stages ~20 GB |
|
||||
| `trellis_mac` | gpu | 14 GB | ~15–18 GB | 24 GB | 32 GB | **M3+ (bf16)**; SOTA local mesh |
|
||||
| `wan_video` 5B q8 (832×480×49f) | gpu | 18 GB | **9.2 GB** RSS | 16 GB *(tight)* | 32 GB | measured on M3 Ultra AND M2 Max 2026-08-04, identical RSS; scales with res/frames — use `--tiling aggressive` on big jobs. MLX cache will *opportunistically* balloon into free RAM (81 GB footprint seen on 96 GB box) — that's cache, not demand |
|
||||
| `ardy_motion` (text→motion) | gpu | 15.6 GB | **~22 GB** | 32 GB | 48 GB | Llama-3-8B text encoder is ~14.5 GB of it; motion model sub-GB. First job per boot pays ~1–2 min load |
|
||||
| `flux_local` FLUX.1 dev | gpu | 31 GB | **25.1 GB** | 32 GB | 48 GB | cinematic, ~2 min |
|
||||
| `flux_local` Klein 9B | gpu | 32 GB | **28.4 GB** | 36 GB | 48 GB | best object accuracy in the lineup |
|
||||
| `mflux_image_edit` (Qwen-Image-Edit) | gpu | 54 GB | ~30–40 GB *(est.)* | 48 GB | **64 GB+** | the heavyweight local editor |
|
||||
|
||||
@ -2,28 +2,128 @@
|
||||
"id": "comfyui_sd",
|
||||
"name": "Stable Diffusion + LoRA (local, ComfyUI)",
|
||||
"category": "generate",
|
||||
"description": "Prompt → image via local SD/SDXL checkpoints + LoRAs (ComfyUI on Metal). Complements flux_local, which runs mflux and CANNOT load SD/SDXL. Talks to a resident ComfyUI on 127.0.0.1:8188 (auto-starts it), so models stay cached between jobs. NOTE: the SD1.5 LoRAs only work with Hyper_Realism_1.2_fp16 — an SDXL checkpoint + SD1.5 LoRA silently does nothing. Full guide: localmodels/README.md",
|
||||
"accepts": [],
|
||||
"produces": ["image"],
|
||||
"description": "Prompt -> image via local SD/SDXL checkpoints + LoRAs (ComfyUI on Metal). Stacks multiple LoRAs and supports ControlNet (recolour a garment into N colourways with an identical silhouette). Cannot load FLUX/Z-Image LoRAs \u2014 those go to flux_local.",
|
||||
"accepts": [
|
||||
"image"
|
||||
],
|
||||
"produces": [
|
||||
"image"
|
||||
],
|
||||
"resources": "gpu",
|
||||
"entry": "run.py",
|
||||
"params_schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"prompt": {"type": "string", "default": "", "description": "Positive prompt (put LoRA trigger words early)"},
|
||||
"negative": {"type": "string", "default": "(worst quality, low quality:1.4), blurry, jpeg artifacts, watermark, text, deformed, bad anatomy, extra fingers", "description": "Negative prompt (ignored at cfg 1.0 on Turbo/Lightning)"},
|
||||
"checkpoint": {"type": "string", "default": "Hyper_Realism_1.2_fp16.safetensors", "description": "SD1.5 = Hyper_Realism_1.2_fp16 (the only one LoRAs work with); SDXL = bigLust_v16 / sd_xl_base_1.0"},
|
||||
"lora": {"type": "string", "default": "", "description": "LoRA filename, blank for none (SD1.5 LoRAs need an SD1.5 checkpoint)"},
|
||||
"lora_weight": {"type": "number", "default": 0.8, "minimum": 0, "maximum": 2, "description": "0.5 for breastinclassBetter; 0.65-0.9 vector; 0.5-1.0 GodPussy1"},
|
||||
"clip_skip": {"type": "integer", "enum": [1, 2], "default": 2, "description": "2 for all the local SD1.5 LoRAs (their training config)"},
|
||||
"width": {"type": "integer", "default": 512, "description": "SD1.5: 512 (768 max). SDXL: 1024. Turbo: 512"},
|
||||
"height": {"type": "integer", "default": 512, "description": "SD1.5: 512/768. SDXL: 1024"},
|
||||
"steps": {"type": "integer", "default": 25, "description": "SD1.5/SDXL: 25-30. Turbo: 1-4. Lightning: 8"},
|
||||
"cfg": {"type": "number", "default": 7.0, "description": "SD1.5/SDXL: 5-8. Turbo: 1.0. Lightning: 1-2 (wrong cfg = fried image)"},
|
||||
"sampler": {"type": "string", "default": "dpmpp_2m", "description": "dpmpp_2m (+karras) is the workhorse; euler for Turbo/Lightning"},
|
||||
"scheduler": {"type": "string", "default": "karras", "description": "karras normally; sgm_uniform for Lightning"},
|
||||
"seed": {"type": "integer", "default": -1, "description": "-1 = random. Fix it when tuning LoRA weight."},
|
||||
"batch": {"type": "integer", "default": 1, "minimum": 1, "maximum": 16, "description": "Images per job — brute-force seeds and pick"}
|
||||
"prompt": {
|
||||
"type": "string",
|
||||
"default": "",
|
||||
"description": "Positive prompt (put LoRA trigger words early)"
|
||||
},
|
||||
"negative": {
|
||||
"type": "string",
|
||||
"default": "(worst quality, low quality:1.4), blurry, jpeg artifacts, watermark, text, deformed, bad anatomy, extra fingers",
|
||||
"description": "Negative prompt (ignored at cfg 1.0 on Turbo/Lightning)"
|
||||
},
|
||||
"checkpoint": {
|
||||
"type": "string",
|
||||
"default": "Hyper_Realism_1.2_fp16.safetensors",
|
||||
"description": "SD1.5 = Hyper_Realism_1.2_fp16 (the only one LoRAs work with); SDXL = bigLust_v16 / sd_xl_base_1.0"
|
||||
},
|
||||
"lora": {
|
||||
"type": [
|
||||
"string",
|
||||
"array"
|
||||
],
|
||||
"default": "",
|
||||
"description": "LoRA filename, or a LIST to stack them (style + texture together). Each entry may carry its own weight as 'name=0.6'. A LoRA on the wrong base architecture is a SILENT no-op \u2014 filenames lie, check metadata."
|
||||
},
|
||||
"lora_weight": {
|
||||
"type": [
|
||||
"number",
|
||||
"array"
|
||||
],
|
||||
"default": 0.8,
|
||||
"minimum": 0,
|
||||
"maximum": 2,
|
||||
"description": "Weight, or a list matching `lora` order. Per-entry 'name=0.6' wins."
|
||||
},
|
||||
"clip_skip": {
|
||||
"type": "integer",
|
||||
"enum": [
|
||||
1,
|
||||
2
|
||||
],
|
||||
"default": 2,
|
||||
"description": "2 for all the local SD1.5 LoRAs (their training config)"
|
||||
},
|
||||
"width": {
|
||||
"type": "integer",
|
||||
"default": 512,
|
||||
"description": "SD1.5: 512 (768 max). SDXL: 1024. Turbo: 512"
|
||||
},
|
||||
"height": {
|
||||
"type": "integer",
|
||||
"default": 512,
|
||||
"description": "SD1.5: 512/768. SDXL: 1024"
|
||||
},
|
||||
"steps": {
|
||||
"type": "integer",
|
||||
"default": 25,
|
||||
"description": "SD1.5/SDXL: 25-30. Turbo: 1-4. Lightning: 8"
|
||||
},
|
||||
"cfg": {
|
||||
"type": "number",
|
||||
"default": 7.0,
|
||||
"description": "SD1.5/SDXL: 5-8. Turbo: 1.0. Lightning: 1-2 (wrong cfg = fried image)"
|
||||
},
|
||||
"sampler": {
|
||||
"type": "string",
|
||||
"default": "dpmpp_2m",
|
||||
"description": "dpmpp_2m (+karras) is the workhorse; euler for Turbo/Lightning"
|
||||
},
|
||||
"scheduler": {
|
||||
"type": "string",
|
||||
"default": "karras",
|
||||
"description": "karras normally; sgm_uniform for Lightning"
|
||||
},
|
||||
"seed": {
|
||||
"type": "integer",
|
||||
"default": -1,
|
||||
"description": "-1 = random. Fix it when tuning LoRA weight."
|
||||
},
|
||||
"batch": {
|
||||
"type": "integer",
|
||||
"default": 1,
|
||||
"minimum": 1,
|
||||
"maximum": 16,
|
||||
"description": "Images per job \u2014 brute-force seeds and pick"
|
||||
},
|
||||
"controlnet": {
|
||||
"type": "string",
|
||||
"default": "",
|
||||
"description": "ControlNet model filename. Needs a control image passed as the job's input asset. Recolor holds the silhouette fixed while changing colour \u2014 N colourways of one garment sharing a byte-identical cut-out alpha."
|
||||
},
|
||||
"controlnet_strength": {
|
||||
"type": "number",
|
||||
"default": 1.0,
|
||||
"minimum": 0,
|
||||
"maximum": 2,
|
||||
"description": "ControlNet conditioning strength"
|
||||
},
|
||||
"controlnet_start": {
|
||||
"type": "number",
|
||||
"default": 0.0,
|
||||
"minimum": 0,
|
||||
"maximum": 1,
|
||||
"description": "Fraction of sampling at which conditioning starts"
|
||||
},
|
||||
"controlnet_end": {
|
||||
"type": "number",
|
||||
"default": 1.0,
|
||||
"minimum": 0,
|
||||
"maximum": 1,
|
||||
"description": "Fraction of sampling at which conditioning ends"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -0,0 +1,29 @@
|
||||
{
|
||||
"id": "comfyui_sd",
|
||||
"name": "Stable Diffusion + LoRA (local, ComfyUI)",
|
||||
"category": "generate",
|
||||
"description": "Prompt → image via local SD/SDXL checkpoints + LoRAs (ComfyUI on Metal). Complements flux_local, which runs mflux and CANNOT load SD/SDXL. Talks to a resident ComfyUI on 127.0.0.1:8188 (auto-starts it), so models stay cached between jobs. NOTE: the SD1.5 LoRAs only work with Hyper_Realism_1.2_fp16 — an SDXL checkpoint + SD1.5 LoRA silently does nothing. Full guide: localmodels/README.md",
|
||||
"accepts": [],
|
||||
"produces": ["image"],
|
||||
"resources": "gpu",
|
||||
"entry": "run.py",
|
||||
"params_schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"prompt": {"type": "string", "default": "", "description": "Positive prompt (put LoRA trigger words early)"},
|
||||
"negative": {"type": "string", "default": "(worst quality, low quality:1.4), blurry, jpeg artifacts, watermark, text, deformed, bad anatomy, extra fingers", "description": "Negative prompt (ignored at cfg 1.0 on Turbo/Lightning)"},
|
||||
"checkpoint": {"type": "string", "default": "Hyper_Realism_1.2_fp16.safetensors", "description": "SD1.5 = Hyper_Realism_1.2_fp16 (the only one LoRAs work with); SDXL = bigLust_v16 / sd_xl_base_1.0"},
|
||||
"lora": {"type": "string", "default": "", "description": "LoRA filename, blank for none (SD1.5 LoRAs need an SD1.5 checkpoint)"},
|
||||
"lora_weight": {"type": "number", "default": 0.8, "minimum": 0, "maximum": 2, "description": "0.5 for breastinclassBetter; 0.65-0.9 vector; 0.5-1.0 GodPussy1"},
|
||||
"clip_skip": {"type": "integer", "enum": [1, 2], "default": 2, "description": "2 for all the local SD1.5 LoRAs (their training config)"},
|
||||
"width": {"type": "integer", "default": 512, "description": "SD1.5: 512 (768 max). SDXL: 1024. Turbo: 512"},
|
||||
"height": {"type": "integer", "default": 512, "description": "SD1.5: 512/768. SDXL: 1024"},
|
||||
"steps": {"type": "integer", "default": 25, "description": "SD1.5/SDXL: 25-30. Turbo: 1-4. Lightning: 8"},
|
||||
"cfg": {"type": "number", "default": 7.0, "description": "SD1.5/SDXL: 5-8. Turbo: 1.0. Lightning: 1-2 (wrong cfg = fried image)"},
|
||||
"sampler": {"type": "string", "default": "dpmpp_2m", "description": "dpmpp_2m (+karras) is the workhorse; euler for Turbo/Lightning"},
|
||||
"scheduler": {"type": "string", "default": "karras", "description": "karras normally; sgm_uniform for Lightning"},
|
||||
"seed": {"type": "integer", "default": -1, "description": "-1 = random. Fix it when tuning LoRA weight."},
|
||||
"batch": {"type": "integer", "default": 1, "minimum": 1, "maximum": 16, "description": "Images per job — brute-force seeds and pick"}
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -4,11 +4,18 @@ Pure stdlib — no venv needed (the runner falls back to the node's python3), be
|
||||
all the heavy lifting happens inside ComfyUI's own venv. Keeps ComfyUI resident so
|
||||
checkpoints stay cached in RAM between jobs (a cold load costs seconds; a warm one
|
||||
doesn't).
|
||||
|
||||
`lora` takes one name or a LIST, each optionally `name=weight`, and they are chained —
|
||||
style + texture together, which a single loader could never do. `controlnet` +
|
||||
`control_image` add structural conditioning: the headline use is recolouring one garment
|
||||
into N colourways that share a byte-identical silhouette, so a single cut-out alpha is
|
||||
reusable across every variant.
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import random
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
@ -37,6 +44,46 @@ if not (COMFY / "main.py").exists():
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def parse_loras():
|
||||
"""-> [(name, weight)]. Accepts 'a', 'a=0.6', ['a','b=0.4'], and a list of weights.
|
||||
|
||||
A LoRA on the wrong base is a SILENT no-op — no error, the image just comes back
|
||||
unchanged and you blame the LoRA. So everything here fails loudly instead.
|
||||
"""
|
||||
raw = p.get("lora") or p.get("loras") or []
|
||||
if isinstance(raw, str):
|
||||
raw = [s for s in (x.strip() for x in raw.split(",")) if s]
|
||||
elif not isinstance(raw, list):
|
||||
raw = []
|
||||
w_raw = p.get("lora_weight", 0.8)
|
||||
weights = w_raw if isinstance(w_raw, list) else [w_raw] * len(raw)
|
||||
out = []
|
||||
for i, item in enumerate(raw):
|
||||
item = str(item).strip()
|
||||
if not item:
|
||||
continue
|
||||
if "=" in item:
|
||||
name, _, w = item.rpartition("=")
|
||||
try:
|
||||
out.append((name.strip(), float(w)))
|
||||
continue
|
||||
except ValueError:
|
||||
pass # '=' was part of the filename, not a weight
|
||||
try:
|
||||
w = float(weights[i]) if i < len(weights) else 0.8
|
||||
except (TypeError, ValueError):
|
||||
w = 0.8
|
||||
out.append((item, w))
|
||||
return out
|
||||
|
||||
|
||||
LORAS = parse_loras()
|
||||
CKPT = p.get("checkpoint", "Hyper_Realism_1.2_fp16.safetensors")
|
||||
CTRL = (p.get("controlnet") or "").strip()
|
||||
CTRL_STRENGTH = float(p.get("controlnet_strength", 1.0))
|
||||
ctrl_file = None # basename inside ComfyUI's input dir
|
||||
|
||||
|
||||
def alive():
|
||||
try:
|
||||
urllib.request.urlopen(f"{API}/", timeout=3)
|
||||
@ -51,8 +98,12 @@ def ensure_server():
|
||||
return
|
||||
print("comfyui: starting resident server ...", flush=True)
|
||||
log = open("/tmp/comfyui.log", "ab")
|
||||
# --listen 0.0.0.0: tailnet-only box; lets John drive the same resident
|
||||
# instance interactively at http://100.89.131.57:8188 (FLUX park lives in
|
||||
# models/ now) while farm jobs keep using it via localhost.
|
||||
subprocess.Popen(
|
||||
[str(COMFY / ".venv" / "bin" / "python"), "main.py", "--port", "8188"],
|
||||
[str(COMFY / ".venv" / "bin" / "python"), "main.py", "--port", "8188",
|
||||
"--listen", "0.0.0.0"],
|
||||
cwd=str(COMFY), stdout=log, stderr=log, start_new_session=True)
|
||||
for _ in range(90):
|
||||
if alive():
|
||||
@ -63,11 +114,18 @@ def ensure_server():
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def node_options(cls, field):
|
||||
"""What this node actually has installed — used to fail fast with a useful list."""
|
||||
try:
|
||||
info = json.load(urllib.request.urlopen(f"{API}/object_info/{cls}"))
|
||||
return info[cls]["input"]["required"][field][0]
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def build(seed):
|
||||
ckpt = p.get("checkpoint", "Hyper_Realism_1.2_fp16.safetensors")
|
||||
lora = (p.get("lora") or "").strip()
|
||||
g = {
|
||||
"1": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": ckpt}},
|
||||
"1": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": CKPT}},
|
||||
"5": {"class_type": "EmptyLatentImage", "inputs": {
|
||||
"width": int(p.get("width", 512)), "height": int(p.get("height", 512)),
|
||||
"batch_size": int(p.get("batch", 1))}},
|
||||
@ -75,23 +133,34 @@ def build(seed):
|
||||
"8": {"class_type": "SaveImage", "inputs": {"filename_prefix": "mb_sd", "images": ["7", 0]}},
|
||||
}
|
||||
msrc, csrc = ["1", 0], ["1", 1]
|
||||
if lora:
|
||||
g["2"] = {"class_type": "LoraLoader", "inputs": {
|
||||
"lora_name": lora,
|
||||
"strength_model": float(p.get("lora_weight", 0.8)),
|
||||
"strength_clip": float(p.get("lora_weight", 0.8)),
|
||||
# Chain one LoraLoader per entry so stacking works (style + texture together).
|
||||
nid = 100
|
||||
for name, w in LORAS:
|
||||
g[str(nid)] = {"class_type": "LoraLoader", "inputs": {
|
||||
"lora_name": name, "strength_model": w, "strength_clip": w,
|
||||
"model": msrc, "clip": csrc}}
|
||||
msrc, csrc = ["2", 0], ["2", 1]
|
||||
msrc, csrc = [str(nid), 0], [str(nid), 1]
|
||||
nid += 1
|
||||
# clip_skip 2 == CLIPSetLastLayer -2 (what the local SD1.5 LoRAs were trained at)
|
||||
if int(p.get("clip_skip", 2)) == 2:
|
||||
g["9"] = {"class_type": "CLIPSetLastLayer", "inputs": {"clip": csrc, "stop_at_clip_layer": -2}}
|
||||
csrc = ["9", 0]
|
||||
g["3"] = {"class_type": "CLIPTextEncode", "inputs": {"text": prompt, "clip": csrc}}
|
||||
g["4"] = {"class_type": "CLIPTextEncode", "inputs": {"text": p.get("negative", ""), "clip": csrc}}
|
||||
pos, neg = ["3", 0], ["4", 0]
|
||||
if CTRL and ctrl_file:
|
||||
g["20"] = {"class_type": "LoadImage", "inputs": {"image": ctrl_file}}
|
||||
g["21"] = {"class_type": "ControlNetLoader", "inputs": {"control_net_name": CTRL}}
|
||||
g["22"] = {"class_type": "ControlNetApplyAdvanced", "inputs": {
|
||||
"positive": pos, "negative": neg, "control_net": ["21", 0], "image": ["20", 0],
|
||||
"strength": CTRL_STRENGTH,
|
||||
"start_percent": float(p.get("controlnet_start", 0.0)),
|
||||
"end_percent": float(p.get("controlnet_end", 1.0))}}
|
||||
pos, neg = ["22", 0], ["22", 1]
|
||||
g["6"] = {"class_type": "KSampler", "inputs": {
|
||||
"seed": seed, "steps": int(p.get("steps", 25)), "cfg": float(p.get("cfg", 7.0)),
|
||||
"sampler_name": p.get("sampler", "dpmpp_2m"), "scheduler": p.get("scheduler", "karras"),
|
||||
"denoise": 1.0, "model": msrc, "positive": ["3", 0], "negative": ["4", 0],
|
||||
"denoise": 1.0, "model": msrc, "positive": pos, "negative": neg,
|
||||
"latent_image": ["5", 0]}}
|
||||
return g
|
||||
|
||||
@ -100,37 +169,62 @@ ensure_server()
|
||||
seed = int(p.get("seed", -1))
|
||||
if seed < 0:
|
||||
seed = random.randint(0, 2**31 - 1)
|
||||
CKPT = p.get("checkpoint", "Hyper_Realism_1.2_fp16.safetensors")
|
||||
LORA = (p.get("lora") or "").strip()
|
||||
print(f"seed={seed} ckpt={CKPT} "
|
||||
f"lora={LORA or 'none'}" + (f"@{p.get('lora_weight', 0.8)}" if LORA else ""), flush=True)
|
||||
if LORA and ("xl" in CKPT.lower() or "biglust" in CKPT.lower() or "v2-1" in CKPT.lower()):
|
||||
print(f"WARNING: {LORA} is SD1.5 but {CKPT} is not — the LoRA will silently do nothing. "
|
||||
f"Use Hyper_Realism_1.2_fp16.safetensors (see localmodels/README.md)", flush=True)
|
||||
lora_desc = ", ".join(f"{n}@{w}" for n, w in LORAS) or "none"
|
||||
print(f"seed={seed} ckpt={CKPT} lora={lora_desc}"
|
||||
+ (f" controlnet={CTRL}@{CTRL_STRENGTH}" if CTRL else ""), flush=True)
|
||||
|
||||
# Checkpoints are NOT identical across nodes (the M3 keeps only the LoRA-compatible
|
||||
# SD1.5 model; the M1 holds the full SDXL archive). Ask this node's ComfyUI what it
|
||||
# actually has and fail fast with a useful message rather than a cryptic 400.
|
||||
try:
|
||||
info = json.load(urllib.request.urlopen(f"{API}/object_info/CheckpointLoaderSimple"))
|
||||
have = info["CheckpointLoaderSimple"]["input"]["required"]["ckpt_name"][0]
|
||||
if CKPT not in have:
|
||||
print(f"ERROR: '{CKPT}' is not on this node.\n"
|
||||
f" available here: {', '.join(have) or '(none)'}\n"
|
||||
f" The full SDXL set lives on the m1 node; every node has "
|
||||
f"Hyper_Realism_1.2_fp16.safetensors (the only LoRA-compatible checkpoint).\n"
|
||||
f" Fix: use a checkpoint listed above, or rsync it from m1.", flush=True)
|
||||
# A ControlNet needs its conditioning image inside ComfyUI's own input dir — LoadImage
|
||||
# resolves by basename, not path.
|
||||
if CTRL:
|
||||
if not a.input:
|
||||
print("ERROR: controlnet requested but no input image was supplied "
|
||||
"(pass the control image as the job's asset)", flush=True)
|
||||
sys.exit(1)
|
||||
if LORA:
|
||||
have_l = json.load(urllib.request.urlopen(f"{API}/object_info/LoraLoader"))
|
||||
have_l = have_l["LoraLoader"]["input"]["required"]["lora_name"][0]
|
||||
if LORA not in have_l:
|
||||
print(f"ERROR: LoRA '{LORA}' is not on this node. available: {', '.join(have_l) or '(none)'}", flush=True)
|
||||
src = Path(a.input[0])
|
||||
if not src.is_file():
|
||||
print(f"ERROR: control image not found: {src}", flush=True)
|
||||
sys.exit(1)
|
||||
indir = COMFY / "input"
|
||||
indir.mkdir(parents=True, exist_ok=True)
|
||||
ctrl_file = f"mb_ctrl_{seed}{src.suffix or '.png'}"
|
||||
shutil.copyfile(src, indir / ctrl_file)
|
||||
print(f"controlnet: staged {src.name} -> input/{ctrl_file}", flush=True)
|
||||
|
||||
# Checkpoints, LoRAs and ControlNets are NOT identical across nodes. Ask this node's
|
||||
# ComfyUI what it actually has and fail fast with a useful message rather than a
|
||||
# cryptic 400 — or worse, a silent no-op.
|
||||
have = node_options("CheckpointLoaderSimple", "ckpt_name")
|
||||
if have is not None and CKPT not in have:
|
||||
print(f"ERROR: '{CKPT}' is not on this node.\n"
|
||||
f" available here: {', '.join(have) or '(none)'}\n"
|
||||
f" Fix: use a checkpoint listed above, or rsync it to "
|
||||
f"~/Documents/localmodels/Stable-diffusion/ on this node.", flush=True)
|
||||
sys.exit(1)
|
||||
if LORAS:
|
||||
have_l = node_options("LoraLoader", "lora_name")
|
||||
if have_l is not None:
|
||||
missing = [n for n, _ in LORAS if n not in have_l]
|
||||
if missing:
|
||||
print(f"ERROR: LoRA(s) not on this node: {', '.join(missing)}\n"
|
||||
f" available here: {', '.join(have_l) or '(none)'}", flush=True)
|
||||
sys.exit(1)
|
||||
except SystemExit:
|
||||
raise
|
||||
except Exception as e:
|
||||
print(f"(could not pre-check model availability: {e}) — continuing", flush=True)
|
||||
if CTRL:
|
||||
have_c = node_options("ControlNetLoader", "control_net_name")
|
||||
if have_c is not None and CTRL not in have_c:
|
||||
print(f"ERROR: ControlNet '{CTRL}' is not on this node.\n"
|
||||
f" available here: {', '.join(have_c) or '(none)'}", flush=True)
|
||||
sys.exit(1)
|
||||
|
||||
# Architecture mismatch is the classic silent failure: an SD1.5 LoRA on an SDXL base
|
||||
# loads without complaint and does nothing at all.
|
||||
xl_ckpt = any(t in CKPT.lower() for t in ("xl", "juggernaut"))
|
||||
for n, _ in LORAS:
|
||||
xl_lora = "xl" in n.lower()
|
||||
if xl_ckpt != xl_lora:
|
||||
print(f"WARNING: '{n}' looks {'SDXL' if xl_lora else 'SD1.5'} but the checkpoint "
|
||||
f"'{CKPT}' looks {'SDXL' if xl_ckpt else 'SD1.5'} — a wrong-base LoRA is a "
|
||||
f"SILENT no-op (no error, no effect). Filenames lie; check the metadata.",
|
||||
flush=True)
|
||||
|
||||
body = json.dumps({"prompt": build(seed)}).encode()
|
||||
req = urllib.request.Request(f"{API}/prompt", data=body, headers={"Content-Type": "application/json"})
|
||||
@ -169,8 +263,15 @@ for i, im in enumerate(images):
|
||||
dest = outdir / name
|
||||
dest.write_bytes(data)
|
||||
outputs.append({"path": str(dest), "name": name,
|
||||
"meta": {"tool": "comfyui_sd", "seed": seed,
|
||||
"checkpoint": p.get("checkpoint"), "lora": p.get("lora") or None}})
|
||||
"meta": {"tool": "comfyui_sd", "seed": seed, "checkpoint": CKPT,
|
||||
"lora": lora_desc if LORAS else None,
|
||||
"controlnet": CTRL or None}})
|
||||
|
||||
if ctrl_file:
|
||||
try:
|
||||
(COMFY / "input" / ctrl_file).unlink()
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
(outdir / "result.json").write_text(json.dumps({"outputs": outputs}))
|
||||
print(f"done: {len(outputs)} image(s) in {time.time() - t0:.1f}s -> {outputs[0]['path']}", flush=True)
|
||||
|
||||
176
server/operators/comfyui_sd/run.py.bak-20260724-180356
Normal file
176
server/operators/comfyui_sd/run.py.bak-20260724-180356
Normal file
@ -0,0 +1,176 @@
|
||||
"""SD/SDXL + LoRA generation via a resident ComfyUI (Metal).
|
||||
|
||||
Pure stdlib — no venv needed (the runner falls back to the node's python3), because
|
||||
all the heavy lifting happens inside ComfyUI's own venv. Keeps ComfyUI resident so
|
||||
checkpoints stay cached in RAM between jobs (a cold load costs seconds; a warm one
|
||||
doesn't).
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import random
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[3]
|
||||
COMFY = ROOT / "vendor" / "comfyui"
|
||||
API = "http://127.0.0.1:8188"
|
||||
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--input", action="append", default=[])
|
||||
ap.add_argument("--outdir", required=True)
|
||||
ap.add_argument("--params", default="{}")
|
||||
a = ap.parse_args()
|
||||
p = json.loads(a.params)
|
||||
outdir = Path(a.outdir)
|
||||
|
||||
prompt = (p.get("prompt") or "").strip()
|
||||
if not prompt:
|
||||
print("ERROR: prompt is required")
|
||||
sys.exit(1)
|
||||
if not (COMFY / "main.py").exists():
|
||||
print(f"ERROR: ComfyUI not installed at {COMFY}. Run scripts/install_comfyui.sh")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def alive():
|
||||
try:
|
||||
urllib.request.urlopen(f"{API}/", timeout=3)
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def ensure_server():
|
||||
if alive():
|
||||
print("comfyui: already resident (models stay cached)", flush=True)
|
||||
return
|
||||
print("comfyui: starting resident server ...", flush=True)
|
||||
log = open("/tmp/comfyui.log", "ab")
|
||||
subprocess.Popen(
|
||||
[str(COMFY / ".venv" / "bin" / "python"), "main.py", "--port", "8188"],
|
||||
cwd=str(COMFY), stdout=log, stderr=log, start_new_session=True)
|
||||
for _ in range(90):
|
||||
if alive():
|
||||
print("comfyui: up", flush=True)
|
||||
return
|
||||
time.sleep(2)
|
||||
print("ERROR: ComfyUI did not come up — see /tmp/comfyui.log")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def build(seed):
|
||||
ckpt = p.get("checkpoint", "Hyper_Realism_1.2_fp16.safetensors")
|
||||
lora = (p.get("lora") or "").strip()
|
||||
g = {
|
||||
"1": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": ckpt}},
|
||||
"5": {"class_type": "EmptyLatentImage", "inputs": {
|
||||
"width": int(p.get("width", 512)), "height": int(p.get("height", 512)),
|
||||
"batch_size": int(p.get("batch", 1))}},
|
||||
"7": {"class_type": "VAEDecode", "inputs": {"samples": ["6", 0], "vae": ["1", 2]}},
|
||||
"8": {"class_type": "SaveImage", "inputs": {"filename_prefix": "mb_sd", "images": ["7", 0]}},
|
||||
}
|
||||
msrc, csrc = ["1", 0], ["1", 1]
|
||||
if lora:
|
||||
g["2"] = {"class_type": "LoraLoader", "inputs": {
|
||||
"lora_name": lora,
|
||||
"strength_model": float(p.get("lora_weight", 0.8)),
|
||||
"strength_clip": float(p.get("lora_weight", 0.8)),
|
||||
"model": msrc, "clip": csrc}}
|
||||
msrc, csrc = ["2", 0], ["2", 1]
|
||||
# clip_skip 2 == CLIPSetLastLayer -2 (what the local SD1.5 LoRAs were trained at)
|
||||
if int(p.get("clip_skip", 2)) == 2:
|
||||
g["9"] = {"class_type": "CLIPSetLastLayer", "inputs": {"clip": csrc, "stop_at_clip_layer": -2}}
|
||||
csrc = ["9", 0]
|
||||
g["3"] = {"class_type": "CLIPTextEncode", "inputs": {"text": prompt, "clip": csrc}}
|
||||
g["4"] = {"class_type": "CLIPTextEncode", "inputs": {"text": p.get("negative", ""), "clip": csrc}}
|
||||
g["6"] = {"class_type": "KSampler", "inputs": {
|
||||
"seed": seed, "steps": int(p.get("steps", 25)), "cfg": float(p.get("cfg", 7.0)),
|
||||
"sampler_name": p.get("sampler", "dpmpp_2m"), "scheduler": p.get("scheduler", "karras"),
|
||||
"denoise": 1.0, "model": msrc, "positive": ["3", 0], "negative": ["4", 0],
|
||||
"latent_image": ["5", 0]}}
|
||||
return g
|
||||
|
||||
|
||||
ensure_server()
|
||||
seed = int(p.get("seed", -1))
|
||||
if seed < 0:
|
||||
seed = random.randint(0, 2**31 - 1)
|
||||
CKPT = p.get("checkpoint", "Hyper_Realism_1.2_fp16.safetensors")
|
||||
LORA = (p.get("lora") or "").strip()
|
||||
print(f"seed={seed} ckpt={CKPT} "
|
||||
f"lora={LORA or 'none'}" + (f"@{p.get('lora_weight', 0.8)}" if LORA else ""), flush=True)
|
||||
if LORA and ("xl" in CKPT.lower() or "biglust" in CKPT.lower() or "v2-1" in CKPT.lower()):
|
||||
print(f"WARNING: {LORA} is SD1.5 but {CKPT} is not — the LoRA will silently do nothing. "
|
||||
f"Use Hyper_Realism_1.2_fp16.safetensors (see localmodels/README.md)", flush=True)
|
||||
|
||||
# Checkpoints are NOT identical across nodes (the M3 keeps only the LoRA-compatible
|
||||
# SD1.5 model; the M1 holds the full SDXL archive). Ask this node's ComfyUI what it
|
||||
# actually has and fail fast with a useful message rather than a cryptic 400.
|
||||
try:
|
||||
info = json.load(urllib.request.urlopen(f"{API}/object_info/CheckpointLoaderSimple"))
|
||||
have = info["CheckpointLoaderSimple"]["input"]["required"]["ckpt_name"][0]
|
||||
if CKPT not in have:
|
||||
print(f"ERROR: '{CKPT}' is not on this node.\n"
|
||||
f" available here: {', '.join(have) or '(none)'}\n"
|
||||
f" The full SDXL set lives on the m1 node; every node has "
|
||||
f"Hyper_Realism_1.2_fp16.safetensors (the only LoRA-compatible checkpoint).\n"
|
||||
f" Fix: use a checkpoint listed above, or rsync it from m1.", flush=True)
|
||||
sys.exit(1)
|
||||
if LORA:
|
||||
have_l = json.load(urllib.request.urlopen(f"{API}/object_info/LoraLoader"))
|
||||
have_l = have_l["LoraLoader"]["input"]["required"]["lora_name"][0]
|
||||
if LORA not in have_l:
|
||||
print(f"ERROR: LoRA '{LORA}' is not on this node. available: {', '.join(have_l) or '(none)'}", flush=True)
|
||||
sys.exit(1)
|
||||
except SystemExit:
|
||||
raise
|
||||
except Exception as e:
|
||||
print(f"(could not pre-check model availability: {e}) — continuing", flush=True)
|
||||
|
||||
body = json.dumps({"prompt": build(seed)}).encode()
|
||||
req = urllib.request.Request(f"{API}/prompt", data=body, headers={"Content-Type": "application/json"})
|
||||
try:
|
||||
pid = json.load(urllib.request.urlopen(req))["prompt_id"]
|
||||
except Exception as e:
|
||||
print(f"ERROR: ComfyUI rejected the workflow: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
t0 = time.time()
|
||||
images = []
|
||||
while time.time() - t0 < 900:
|
||||
h = json.load(urllib.request.urlopen(f"{API}/history/{pid}"))
|
||||
if pid in h:
|
||||
st = h[pid].get("status", {})
|
||||
if st.get("status_str") == "error":
|
||||
print("ERROR: generation failed — check /tmp/comfyui.log")
|
||||
print(json.dumps(st)[:400])
|
||||
sys.exit(1)
|
||||
if h[pid].get("outputs"):
|
||||
for node in h[pid]["outputs"].values():
|
||||
images += node.get("images", [])
|
||||
break
|
||||
time.sleep(2)
|
||||
|
||||
if not images:
|
||||
print("ERROR: no image produced (timeout)")
|
||||
sys.exit(1)
|
||||
|
||||
outputs = []
|
||||
for i, im in enumerate(images):
|
||||
q = urllib.parse.urlencode({"filename": im["filename"], "subfolder": im.get("subfolder", ""),
|
||||
"type": im.get("type", "output")})
|
||||
data = urllib.request.urlopen(f"{API}/view?{q}").read()
|
||||
name = f"sd_{seed}_{i}.png" if len(images) > 1 else f"sd_{seed}.png"
|
||||
dest = outdir / name
|
||||
dest.write_bytes(data)
|
||||
outputs.append({"path": str(dest), "name": name,
|
||||
"meta": {"tool": "comfyui_sd", "seed": seed,
|
||||
"checkpoint": p.get("checkpoint"), "lora": p.get("lora") or None}})
|
||||
|
||||
(outdir / "result.json").write_text(json.dumps({"outputs": outputs}))
|
||||
print(f"done: {len(outputs)} image(s) in {time.time() - t0:.1f}s -> {outputs[0]['path']}", flush=True)
|
||||
@ -2,7 +2,7 @@
|
||||
"id": "mflux_image_edit",
|
||||
"name": "Edit Image (local, Qwen)",
|
||||
"category": "image-prep",
|
||||
"description": "1-3 images + instruction \u2192 edited image (multi-image composes: 'put the shirt from image 2 on the person in image 1') entirely on this Mac via mflux Qwen-Image-Edit (ungated/Apache-2.0). Free local alternative to fal_image_edit / nano-banana edit \u2014 'remove the sticker', 'make it studio-lit on white', material/lighting changes. First run downloads weights.",
|
||||
"description": "1-3 images + instruction → edited image (multi-image composes: 'put the shirt from image 2 on the person in image 1') entirely on this Mac via mflux Qwen-Image-Edit (ungated/Apache-2.0). Free local alternative to fal_image_edit / nano-banana edit — 'remove the sticker', 'make it studio-lit on white', material/lighting changes. First run downloads weights.",
|
||||
"accepts": [
|
||||
"image"
|
||||
],
|
||||
@ -38,7 +38,12 @@
|
||||
"minimum": 0,
|
||||
"maximum": 10,
|
||||
"description": "Guidance scale"
|
||||
},
|
||||
"loras": {
|
||||
"type": "string",
|
||||
"default": "",
|
||||
"description": "Qwen-Edit LoRAs, comma-separated 'name' or 'name:scale' (e.g. 'qwen-studio-realism:0.8,gymnast'). Resolved from ~/Documents/localmodels/qwen-loras (mirror of ultra civit/_parked-qwen). Wrong-base LoRAs are silent no-ops."
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@ -1,30 +1,9 @@
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Qwen-Image-Edit LoRA search path (name → file). Drop .safetensors anywhere under these and they
|
||||
# become usable as {"loras": ["multiple-angles=1.0", ...]}. Same UX as sd_local's LoRA list.
|
||||
QWEN_LORA_DIRS = [Path(os.path.expanduser(d)) for d in os.environ.get(
|
||||
"QWEN_LORA_DIRS", "~/Documents/localmodels/QwenLora").split(":")]
|
||||
|
||||
|
||||
def find_lora(stem: str):
|
||||
"""Match by exact filename stem, else the first file whose stem contains it (case-insensitive)."""
|
||||
stem = stem.strip()
|
||||
cands = [f for d in QWEN_LORA_DIRS if d.is_dir() for f in d.rglob("*.safetensors")]
|
||||
for f in cands:
|
||||
if f.stem == stem:
|
||||
return f
|
||||
low = stem.lower()
|
||||
for f in cands:
|
||||
if low in f.stem.lower() or low in f.parent.name.lower():
|
||||
return f
|
||||
return None
|
||||
|
||||
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--input", action="append", default=[])
|
||||
ap.add_argument("--outdir", required=True)
|
||||
@ -55,19 +34,30 @@ cmd = [str(cli), "--image-paths", *[str(Path(x).resolve()) for x in a.input],
|
||||
"--guidance", str(p.get("guidance", 4.0)),
|
||||
"--output", str(out.resolve())]
|
||||
|
||||
# optional LoRAs: ["multiple-angles=1.0", "gymnastics-pose=0.8"] → --lora-paths/--lora-scales
|
||||
paths, scales = [], []
|
||||
for spec in (p.get("loras") or []):
|
||||
name, _, w = str(spec).partition("=")
|
||||
f = find_lora(name)
|
||||
if not f:
|
||||
print(f"LORA MISS '{name}' — searched {[str(d) for d in QWEN_LORA_DIRS]}", flush=True)
|
||||
continue
|
||||
paths.append(str(f))
|
||||
scales.append(str(float(w or 1.0)))
|
||||
if paths:
|
||||
cmd += ["--lora-paths", *paths, "--lora-scales", *scales]
|
||||
print(f"loras: {list(zip(paths, scales))}", flush=True)
|
||||
# Qwen-Edit LoRAs: "name" or "name:scale", comma-separated. Names resolve
|
||||
# (case-insensitive prefix match) against ~/Documents/localmodels/qwen-loras,
|
||||
# which mirrors ultra's civit/_parked-qwen stash. mflux natively takes
|
||||
# --lora-paths/--lora-scales; a LoRA for the wrong base is a silent no-op.
|
||||
LORA_DIR = Path.home() / "Documents" / "localmodels" / "qwen-loras"
|
||||
if p.get("loras"):
|
||||
paths, scales = [], []
|
||||
avail = {f.name.lower(): f for f in LORA_DIR.rglob("*.safetensors")}
|
||||
for item in str(p["loras"]).split(","):
|
||||
item = item.strip()
|
||||
if not item:
|
||||
continue
|
||||
name, _, scale = item.partition(":")
|
||||
name = name.strip().lower()
|
||||
hit = avail.get(name) or avail.get(name + ".safetensors") or next(
|
||||
(f for k, f in sorted(avail.items()) if k.startswith(name)), None)
|
||||
if not hit:
|
||||
print(f"ERROR: no LoRA matching '{name}' under {LORA_DIR}")
|
||||
print("available:", ", ".join(sorted(f.stem for f in avail.values())[:40]))
|
||||
sys.exit(1)
|
||||
paths.append(str(hit))
|
||||
scales.append(str(float(scale) if scale.strip() else 1.0))
|
||||
if paths:
|
||||
cmd += ["--lora-paths", *paths, "--lora-scales", *scales]
|
||||
|
||||
print("+", " ".join(cmd), flush=True)
|
||||
print("(first run downloads Qwen-Image-Edit weights)", flush=True)
|
||||
|
||||
Loading…
Reference in New Issue
Block a user