fal panel: 5 new operators, image outputs, grouped UI, no-input ops + review fixes

Operators (16 total):
- fal_hunyuan3d_v21: Hunyuan3D 2.1 single-image (live-verified dash variant,
  ~90s; v21 multi-view is broken on fal — v2 stays the multi-view path)
- fal_bg_remove (BiRefNet v2): subject cutout pre-pass — the biggest quality
  lever before image->3D
- fal_upscale (SeedVR faithful upscale), fal_image_edit (nano-banana prompt
  edits), fal_text_image (Ideogram v3 — first no-input operator)
- fal_common: collect='images' mode; recursive URL extractor now takes the
  wanted extension set

UI: operator dropdown grouped by category (optgroup); operators with
accepts: [] run without a selected asset ('No input asset needed' note);
run-button/launch logic updated accordingly.

Review fixes (Opus Phase 1 commit):
- runner._run_lane crashed (TypeError) when a queued job was deleted before
  the worker picked it up
- PUT /api/settings treated empty string as a masked placeholder, making
  secrets impossible to clear from the UI
- store.register_file mutated the caller's meta dict via pop

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
MODELBEAST 2026-07-12 21:51:16 +10:00
parent 605b1ae347
commit 7ea3c8b935
17 changed files with 228 additions and 17 deletions

View File

@ -11,7 +11,9 @@
- Frontend: Settings modal, operator gating (🔒 + disabled run), job actions, Compare grid (multi-select side-by-side viewers), SplatViewer (`@mkkellogg/gaussian-splats-3d`, format forced to Ply since asset URLs are extensionless), splat/colmap_dataset asset kinds. All browser-verified.
- Install scripts: `scripts/install_{colmap,brush,sf3d,trellis_mac}.sh`. Vendored repos in `vendor/` (gitignored), venvs in `venvs/` and `vendor/trellis-mac/.venv` (gitignored).
**Not yet built (next up):** object_capture (Swift CLI; Xcode present), freemocap, gvhmr_import, blender_retarget (multi-input UI wiring), workflow presets (§4.5), tripo/meshy character APIs (Phase 4), archive_to_m4pro, LLM copilot. See §58 below.
**fal panel (Fable session, later 2026-07-12):** 5 more operators — `fal_hunyuan3d_v21` (live-verified: dash-variant single-image works ~90s; **v21 multi-view is broken on fal**, keep v2 for that), `fal_bg_remove` (BiRefNet v2 — the image→3D quality pre-pass), `fal_upscale` (SeedVR), `fal_image_edit` (nano-banana), `fal_text_image` (Ideogram v3, first no-input operator). `fal_common` now collects image outputs (`collect="images"`). UI: operator dropdown grouped by category via optgroup; ops with `accepts: []` run without an asset. Fixed 3 review findings from the Opus commit: deleted-queued-job crash in `_run_lane`, secrets impossible to clear via PUT /api/settings, `register_file` mutating caller's meta. 16 operators total; smoke 12/12.
**Not yet built (next up):** object_capture (Swift CLI; Xcode present), freemocap, gvhmr_import, blender_retarget (multi-input UI wiring), workflow presets (§4.5) — first preset should be `image → bg_remove → trellis2/hunyuan_v21` — tripo/meshy character APIs (Phase 4), archive_to_m4pro, LLM copilot. See §58 below.
**Owner action to unblock local mesh-gen:** accept HF licenses + set HF token (see BENCHMARKS.md). fal operators need `FAL_KEY` in Settings.

View File

@ -42,6 +42,12 @@ Current operators:
| `brush_train` | gpu | colmap dataset → 3D gaussian splat (Brush, native Metal) |
| `sf3d` | gpu | image → GLB locally (Stable-Fast-3D, MPS) *[HF-gated]* |
| `trellis_mac` | gpu | image → GLB+PBR locally (TRELLIS.2 MPS port) *[HF-gated]* |
| `fal_trellis` / `fal_trellis2` / `fal_hunyuan3d` / `fal_rodin` | net | image → mesh via fal.ai API *[needs FAL_KEY]* |
| `fal_trellis` / `fal_trellis2` / `fal_hunyuan3d` / `fal_hunyuan3d_v21` / `fal_rodin` | net | image → mesh via fal.ai API *[needs FAL_KEY]* |
| `fal_bg_remove` | net | image → subject cutout (BiRefNet v2) — run before any image→3D for a big quality jump |
| `fal_upscale` | net | image → faithful upscale (SeedVR) |
| `fal_image_edit` | net | image + instruction → edited image (nano-banana) |
| `fal_text_image` | net | prompt → image (Ideogram v3, readable text) — no input asset needed |
Recommended fal chain for best image→3D: `fal_bg_remove` → (`fal_upscale` if thin) → `fal_trellis2` or `fal_hunyuan3d_v21`. Note: Hunyuan v21 **multi-view** is broken on fal (verified 2026-07) — v21 is single-image only; use v2 for multi-view.
Heavy operators get their own uv venv (`"python": "<abs venv path>"` in the manifest). Remaining roadmap (object_capture, freemocap, retargeting, tripo/meshy character APIs, workflow presets, LLM copilot) is in [HANDOFF.md](HANDOFF.md) §58.

View File

@ -45,8 +45,9 @@ def get_settings():
@app.put("/api/settings")
def put_settings(payload: dict):
# ignore masked secret placeholders so re-saving the form doesn't wipe a key
# (an empty string is a deliberate clear, not a placeholder)
clean = {k: v for k, v in payload.items()
if not (k in settings_mod.SECRET_KEYS and set(str(v)) <= {""})}
if not (k in settings_mod.SECRET_KEYS and v and set(str(v)) <= {""})}
settings_mod.set_many(app.state.con, clean)
return get_settings()

View File

@ -11,25 +11,26 @@ import urllib.request
from pathlib import Path
MODEL_EXTS = {".glb", ".gltf", ".obj", ".fbx", ".usdz", ".ply", ".stl", ".zip"}
IMAGE_EXTS = {".png", ".jpg", ".jpeg", ".webp"}
def _log(msg):
print(msg, flush=True)
def _find_file_urls(obj, acc):
def _find_file_urls(obj, acc, wanted_exts):
"""Recursively collect (url, ext) for any file-like url in the result JSON."""
if isinstance(obj, dict):
url = obj.get("url")
if isinstance(url, str):
ext = Path(url.split("?")[0]).suffix.lower()
if ext in MODEL_EXTS:
if ext in wanted_exts:
acc.append((url, ext, obj.get("file_name")))
for v in obj.values():
_find_file_urls(v, acc)
_find_file_urls(v, acc, wanted_exts)
elif isinstance(obj, list):
for v in obj:
_find_file_urls(v, acc)
_find_file_urls(v, acc, wanted_exts)
def parse_args():
@ -42,7 +43,9 @@ def parse_args():
def run(endpoint, arguments, outdir, image_path=None, image_arg="image_url",
image_as_list=False, extra_args=None, out_stem="model"):
image_as_list=False, extra_args=None, out_stem="model", collect="models"):
"""collect: 'models' (default) downloads 3D files from the result;
'images' downloads image files (for cutout/upscale/edit/gen endpoints)."""
import fal_client
if image_path:
@ -73,10 +76,11 @@ def run(endpoint, arguments, outdir, image_path=None, image_arg="image_url",
outdir = Path(outdir)
(outdir / "fal_result.json").write_text(json.dumps(result, indent=2))
wanted = IMAGE_EXTS if collect == "images" else MODEL_EXTS
urls = []
_find_file_urls(result, urls)
_find_file_urls(result, urls, wanted)
if not urls:
_log("ERROR: no 3D file url found in fal result. Raw result saved to fal_result.json:")
_log(f"ERROR: no {collect} file url found in fal result. Raw result saved to fal_result.json:")
_log(json.dumps(result, indent=2)[:2000])
sys.exit(1)

View File

@ -0,0 +1,18 @@
{
"id": "fal_bg_remove",
"name": "fal · Remove Background",
"category": "image-prep",
"description": "Image → clean subject cutout (transparent PNG) via fal-ai/birefnet/v2 (sub-cent). The single biggest quality lever before image→3D: run this first, then feed the cutout to TRELLIS/Hunyuan/SF3D so no geometry is wasted on background clutter. Needs FAL_KEY.",
"accepts": ["image"],
"produces": ["image"],
"resources": "net",
"requires_env": ["FAL_KEY"],
"entry": "run.py",
"params_schema": {
"type": "object",
"properties": {
"operating_resolution": {"type": "string", "enum": ["1024x1024", "2048x2048"], "default": "1024x1024", "description": "Processing resolution (2048 = finer edges, slower)"},
"extra_args": {"type": "string", "default": "", "description": "Raw JSON merged into fal arguments (e.g. model variant). Full control"}
}
}
}

View File

@ -0,0 +1,17 @@
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from _lib import fal_common # noqa: E402
a, p = fal_common.parse_args()
stem = Path(a.input[0]).stem if a.input else "cutout"
fal_common.run(
endpoint="fal-ai/birefnet/v2",
arguments={"operating_resolution": p.get("operating_resolution")},
outdir=a.outdir,
image_path=a.input[0] if a.input else None,
extra_args=p.get("extra_args"),
out_stem=f"{stem}_cutout",
collect="images",
)

View File

@ -0,0 +1,19 @@
{
"id": "fal_hunyuan3d_v21",
"name": "fal · Hunyuan3D 2.1 (PBR)",
"category": "mesh-gen",
"description": "Image → GLB with PBR textures via fal-ai/hunyuan3d-v21 (Hunyuan3D 2.1, ~90s, ~$0.05-0.10). Live-verified 2026-07: single-image works; the multi-view variant of v21 is broken on fal (use Hunyuan v2 for multi-view). Needs FAL_KEY.",
"accepts": ["image"],
"produces": ["model"],
"resources": "net",
"requires_env": ["FAL_KEY"],
"entry": "run.py",
"params_schema": {
"type": "object",
"properties": {
"seed": {"type": "integer", "description": "Random seed (leave blank for random)"},
"textured_mesh": {"type": "boolean", "default": true, "description": "Generate PBR textures (costs more)"},
"extra_args": {"type": "string", "default": "", "description": "Raw JSON merged into fal arguments — full control over any endpoint param"}
}
}
}

View File

@ -0,0 +1,16 @@
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from _lib import fal_common # noqa: E402
a, p = fal_common.parse_args()
fal_common.run(
endpoint="fal-ai/hunyuan3d-v21",
arguments={"seed": p.get("seed"), "textured_mesh": p.get("textured_mesh", True)},
outdir=a.outdir,
image_path=a.input[0] if a.input else None,
image_arg="input_image_url",
extra_args=p.get("extra_args"),
out_stem="hunyuan3d21",
)

View File

@ -0,0 +1,18 @@
{
"id": "fal_image_edit",
"name": "fal · Edit Image (prompt)",
"category": "image-prep",
"description": "Image + instruction → edited image via fal-ai/nano-banana/edit (remove objects, fix lighting, change materials — 'remove the price sticker', 'make it studio-lit on white'). Great for cleaning a photo before image→3D. Needs FAL_KEY.",
"accepts": ["image"],
"produces": ["image"],
"resources": "net",
"requires_env": ["FAL_KEY"],
"entry": "run.py",
"params_schema": {
"type": "object",
"properties": {
"prompt": {"type": "string", "default": "", "description": "Edit instruction (what to change)"},
"extra_args": {"type": "string", "default": "", "description": "Raw JSON merged into fal arguments — full control over any endpoint param"}
}
}
}

View File

@ -0,0 +1,22 @@
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from _lib import fal_common # noqa: E402
a, p = fal_common.parse_args()
if not p.get("prompt"):
print("ERROR: prompt is required — describe the edit you want")
sys.exit(1)
stem = Path(a.input[0]).stem if a.input else "image"
fal_common.run(
endpoint="fal-ai/nano-banana/edit",
arguments={"prompt": p["prompt"]},
outdir=a.outdir,
image_path=a.input[0] if a.input else None,
image_arg="image_urls",
image_as_list=True,
extra_args=p.get("extra_args"),
out_stem=f"{stem}_edited",
collect="images",
)

View File

@ -0,0 +1,19 @@
{
"id": "fal_text_image",
"name": "fal · Text → Image",
"category": "generate",
"description": "Prompt → image via fal-ai/ideogram/v3 (~$0.03; uniquely renders READABLE text — labels, posters, packaging). No input asset needed. Generate a concept image here, then feed it to a mesh generator. Needs FAL_KEY.",
"accepts": [],
"produces": ["image"],
"resources": "net",
"requires_env": ["FAL_KEY"],
"entry": "run.py",
"params_schema": {
"type": "object",
"properties": {
"prompt": {"type": "string", "default": "", "description": "What to generate"},
"image_size": {"type": "string", "enum": ["square_hd", "square", "portrait_4_3", "portrait_16_9", "landscape_4_3", "landscape_16_9"], "default": "square_hd", "description": "Output aspect/size"},
"extra_args": {"type": "string", "default": "", "description": "Raw JSON merged into fal arguments (style, seed, etc.). Full control"}
}
}
}

View File

@ -0,0 +1,18 @@
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from _lib import fal_common # noqa: E402
a, p = fal_common.parse_args()
if not p.get("prompt"):
print("ERROR: prompt is required")
sys.exit(1)
fal_common.run(
endpoint="fal-ai/ideogram/v3",
arguments={"prompt": p["prompt"], "image_size": p.get("image_size", "square_hd")},
outdir=a.outdir,
extra_args=p.get("extra_args"),
out_stem="generated",
collect="images",
)

View File

@ -0,0 +1,18 @@
{
"id": "fal_upscale",
"name": "fal · Upscale (SeedVR)",
"category": "image-prep",
"description": "Image → faithful upscale via fal-ai/seedvr/upscale/image (~$0.001/MP — sharpens without inventing fake detail). Use on thin/blurry inputs before mesh generation or texture work. Needs FAL_KEY.",
"accepts": ["image"],
"produces": ["image"],
"resources": "net",
"requires_env": ["FAL_KEY"],
"entry": "run.py",
"params_schema": {
"type": "object",
"properties": {
"upscale_factor": {"type": "number", "default": 2, "minimum": 1, "maximum": 4, "description": "Upscale multiplier"},
"extra_args": {"type": "string", "default": "", "description": "Raw JSON merged into fal arguments — full control over any endpoint param"}
}
}
}

View File

@ -0,0 +1,17 @@
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from _lib import fal_common # noqa: E402
a, p = fal_common.parse_args()
stem = Path(a.input[0]).stem if a.input else "image"
fal_common.run(
endpoint="fal-ai/seedvr/upscale/image",
arguments={"upscale_factor": p.get("upscale_factor", 2)},
outdir=a.outdir,
image_path=a.input[0] if a.input else None,
extra_args=p.get("extra_args"),
out_stem=f"{stem}_upscaled",
collect="images",
)

View File

@ -134,11 +134,15 @@ class Runner:
task.add_done_callback(self._tasks.discard)
async def _run_lane(self, con, job_id: str):
job = self.get_job(con, job_id)
if not job: # deleted while queued
self.cancelled.discard(job_id)
return
if job_id in self.cancelled:
self.cancelled.discard(job_id)
await self._update(con, job_id, status="cancelled", finished_at=db.now())
return
lane = self.lane_of(self.get_job(con, job_id)["operator"])
lane = self.lane_of(job["operator"])
async with self.lanes[lane]:
if job_id in self.cancelled:
self.cancelled.discard(job_id)

View File

@ -31,7 +31,7 @@ def register_file(con, src: Path, name: str | None = None, parent_job: str | Non
dest_dir = ASSETS_DIR / asset_id
dest_dir.mkdir(parents=True, exist_ok=True)
dest = dest_dir / name
meta = meta or {}
meta = dict(meta or {})
if src.is_dir():
if move:
shutil.move(str(src), dest)

View File

@ -139,10 +139,17 @@ export default function App() {
const selectedAsset = assets.find((a) => a.id === selected) || null;
const envSet = settings._env_set || [];
// no-input operators (accepts: []) are always available; others filter by asset kind
const usableOps = useMemo(() =>
operators.filter((op) => !selectedAsset || op.accepts.includes(selectedAsset.kind)),
operators.filter((op) => !op.accepts.length || !selectedAsset || op.accepts.includes(selectedAsset.kind)),
[operators, selectedAsset]);
const opGroups = useMemo(() => {
const groups = {};
for (const op of usableOps) (groups[op.category || "other"] ||= []).push(op);
return groups;
}, [usableOps]);
const activeOp = operators.find((o) => o.id === opId);
const needsInput = (activeOp?.accepts?.length ?? 0) > 0;
const gatedEnv = (activeOp?.requires_env || []).filter((e) => !envSet.includes(e));
const gated = gatedEnv.length > 0;
@ -159,8 +166,8 @@ export default function App() {
}, [usableOps]);
const launch = async () => {
if (!activeOp || !selectedAsset || gated) return;
await runJob(activeOp.id, selectedAsset.id, params);
if (!activeOp || gated || (needsInput && !selectedAsset)) return;
await runJob(activeOp.id, needsInput ? selectedAsset.id : null, params);
refreshJobs();
};
@ -219,12 +226,17 @@ export default function App() {
<aside className="workbench">
<h2>Run</h2>
<select value={opId || ""} onChange={(e) => setOpId(e.target.value)}>
{usableOps.map((op) => <option key={op.id} value={op.id}>{op.name}</option>)}
{Object.entries(opGroups).map(([cat, ops]) => (
<optgroup key={cat} label={cat}>
{ops.map((op) => <option key={op.id} value={op.id}>{op.name}</option>)}
</optgroup>
))}
</select>
{activeOp && <p className="dim">{activeOp.description}</p>}
{gated && <p className="warn">🔒 Needs {gatedEnv.join(", ")} add it in Settings.</p>}
{activeOp && !needsInput && <p className="dim">No input asset needed just set the parameters and run.</p>}
{activeOp && <ParamForm schema={activeOp.params_schema} values={params} onChange={setParams} />}
<button className="go" disabled={!selectedAsset || !activeOp || gated} onClick={launch}>
<button className="go" disabled={!activeOp || gated || (needsInput && !selectedAsset)} onClick={launch}>
Run {activeOp?.name || ""}
</button>