fal panel: 5 new operators, image outputs, grouped UI, no-input ops + review fixes
Operators (16 total):
- fal_hunyuan3d_v21: Hunyuan3D 2.1 single-image (live-verified dash variant,
~90s; v21 multi-view is broken on fal — v2 stays the multi-view path)
- fal_bg_remove (BiRefNet v2): subject cutout pre-pass — the biggest quality
lever before image->3D
- fal_upscale (SeedVR faithful upscale), fal_image_edit (nano-banana prompt
edits), fal_text_image (Ideogram v3 — first no-input operator)
- fal_common: collect='images' mode; recursive URL extractor now takes the
wanted extension set
UI: operator dropdown grouped by category (optgroup); operators with
accepts: [] run without a selected asset ('No input asset needed' note);
run-button/launch logic updated accordingly.
Review fixes (Opus Phase 1 commit):
- runner._run_lane crashed (TypeError) when a queued job was deleted before
the worker picked it up
- PUT /api/settings treated empty string as a masked placeholder, making
secrets impossible to clear from the UI
- store.register_file mutated the caller's meta dict via pop
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
605b1ae347
commit
7ea3c8b935
@ -11,7 +11,9 @@
|
||||
- Frontend: Settings modal, operator gating (🔒 + disabled run), job actions, Compare grid (multi-select side-by-side viewers), SplatViewer (`@mkkellogg/gaussian-splats-3d`, format forced to Ply since asset URLs are extensionless), splat/colmap_dataset asset kinds. All browser-verified.
|
||||
- Install scripts: `scripts/install_{colmap,brush,sf3d,trellis_mac}.sh`. Vendored repos in `vendor/` (gitignored), venvs in `venvs/` and `vendor/trellis-mac/.venv` (gitignored).
|
||||
|
||||
**Not yet built (next up):** object_capture (Swift CLI; Xcode present), freemocap, gvhmr_import, blender_retarget (multi-input UI wiring), workflow presets (§4.5), tripo/meshy character APIs (Phase 4), archive_to_m4pro, LLM copilot. See §5–8 below.
|
||||
**fal panel (Fable session, later 2026-07-12):** 5 more operators — `fal_hunyuan3d_v21` (live-verified: dash-variant single-image works ~90s; **v21 multi-view is broken on fal**, keep v2 for that), `fal_bg_remove` (BiRefNet v2 — the image→3D quality pre-pass), `fal_upscale` (SeedVR), `fal_image_edit` (nano-banana), `fal_text_image` (Ideogram v3, first no-input operator). `fal_common` now collects image outputs (`collect="images"`). UI: operator dropdown grouped by category via optgroup; ops with `accepts: []` run without an asset. Fixed 3 review findings from the Opus commit: deleted-queued-job crash in `_run_lane`, secrets impossible to clear via PUT /api/settings, `register_file` mutating caller's meta. 16 operators total; smoke 12/12.
|
||||
|
||||
**Not yet built (next up):** object_capture (Swift CLI; Xcode present), freemocap, gvhmr_import, blender_retarget (multi-input UI wiring), workflow presets (§4.5) — first preset should be `image → bg_remove → trellis2/hunyuan_v21` — tripo/meshy character APIs (Phase 4), archive_to_m4pro, LLM copilot. See §5–8 below.
|
||||
|
||||
**Owner action to unblock local mesh-gen:** accept HF licenses + set HF token (see BENCHMARKS.md). fal operators need `FAL_KEY` in Settings.
|
||||
|
||||
|
||||
@ -42,6 +42,12 @@ Current operators:
|
||||
| `brush_train` | gpu | colmap dataset → 3D gaussian splat (Brush, native Metal) |
|
||||
| `sf3d` | gpu | image → GLB locally (Stable-Fast-3D, MPS) *[HF-gated]* |
|
||||
| `trellis_mac` | gpu | image → GLB+PBR locally (TRELLIS.2 MPS port) *[HF-gated]* |
|
||||
| `fal_trellis` / `fal_trellis2` / `fal_hunyuan3d` / `fal_rodin` | net | image → mesh via fal.ai API *[needs FAL_KEY]* |
|
||||
| `fal_trellis` / `fal_trellis2` / `fal_hunyuan3d` / `fal_hunyuan3d_v21` / `fal_rodin` | net | image → mesh via fal.ai API *[needs FAL_KEY]* |
|
||||
| `fal_bg_remove` | net | image → subject cutout (BiRefNet v2) — run before any image→3D for a big quality jump |
|
||||
| `fal_upscale` | net | image → faithful upscale (SeedVR) |
|
||||
| `fal_image_edit` | net | image + instruction → edited image (nano-banana) |
|
||||
| `fal_text_image` | net | prompt → image (Ideogram v3, readable text) — no input asset needed |
|
||||
|
||||
Recommended fal chain for best image→3D: `fal_bg_remove` → (`fal_upscale` if thin) → `fal_trellis2` or `fal_hunyuan3d_v21`. Note: Hunyuan v21 **multi-view** is broken on fal (verified 2026-07) — v21 is single-image only; use v2 for multi-view.
|
||||
|
||||
Heavy operators get their own uv venv (`"python": "<abs venv path>"` in the manifest). Remaining roadmap (object_capture, freemocap, retargeting, tripo/meshy character APIs, workflow presets, LLM copilot) is in [HANDOFF.md](HANDOFF.md) §5–8.
|
||||
|
||||
@ -45,8 +45,9 @@ def get_settings():
|
||||
@app.put("/api/settings")
|
||||
def put_settings(payload: dict):
|
||||
# ignore masked secret placeholders so re-saving the form doesn't wipe a key
|
||||
# (an empty string is a deliberate clear, not a placeholder)
|
||||
clean = {k: v for k, v in payload.items()
|
||||
if not (k in settings_mod.SECRET_KEYS and set(str(v)) <= {"•"})}
|
||||
if not (k in settings_mod.SECRET_KEYS and v and set(str(v)) <= {"•"})}
|
||||
settings_mod.set_many(app.state.con, clean)
|
||||
return get_settings()
|
||||
|
||||
|
||||
@ -11,25 +11,26 @@ import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
MODEL_EXTS = {".glb", ".gltf", ".obj", ".fbx", ".usdz", ".ply", ".stl", ".zip"}
|
||||
IMAGE_EXTS = {".png", ".jpg", ".jpeg", ".webp"}
|
||||
|
||||
|
||||
def _log(msg):
|
||||
print(msg, flush=True)
|
||||
|
||||
|
||||
def _find_file_urls(obj, acc):
|
||||
def _find_file_urls(obj, acc, wanted_exts):
|
||||
"""Recursively collect (url, ext) for any file-like url in the result JSON."""
|
||||
if isinstance(obj, dict):
|
||||
url = obj.get("url")
|
||||
if isinstance(url, str):
|
||||
ext = Path(url.split("?")[0]).suffix.lower()
|
||||
if ext in MODEL_EXTS:
|
||||
if ext in wanted_exts:
|
||||
acc.append((url, ext, obj.get("file_name")))
|
||||
for v in obj.values():
|
||||
_find_file_urls(v, acc)
|
||||
_find_file_urls(v, acc, wanted_exts)
|
||||
elif isinstance(obj, list):
|
||||
for v in obj:
|
||||
_find_file_urls(v, acc)
|
||||
_find_file_urls(v, acc, wanted_exts)
|
||||
|
||||
|
||||
def parse_args():
|
||||
@ -42,7 +43,9 @@ def parse_args():
|
||||
|
||||
|
||||
def run(endpoint, arguments, outdir, image_path=None, image_arg="image_url",
|
||||
image_as_list=False, extra_args=None, out_stem="model"):
|
||||
image_as_list=False, extra_args=None, out_stem="model", collect="models"):
|
||||
"""collect: 'models' (default) downloads 3D files from the result;
|
||||
'images' downloads image files (for cutout/upscale/edit/gen endpoints)."""
|
||||
import fal_client
|
||||
|
||||
if image_path:
|
||||
@ -73,10 +76,11 @@ def run(endpoint, arguments, outdir, image_path=None, image_arg="image_url",
|
||||
outdir = Path(outdir)
|
||||
(outdir / "fal_result.json").write_text(json.dumps(result, indent=2))
|
||||
|
||||
wanted = IMAGE_EXTS if collect == "images" else MODEL_EXTS
|
||||
urls = []
|
||||
_find_file_urls(result, urls)
|
||||
_find_file_urls(result, urls, wanted)
|
||||
if not urls:
|
||||
_log("ERROR: no 3D file url found in fal result. Raw result saved to fal_result.json:")
|
||||
_log(f"ERROR: no {collect} file url found in fal result. Raw result saved to fal_result.json:")
|
||||
_log(json.dumps(result, indent=2)[:2000])
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
18
server/operators/fal_bg_remove/manifest.json
Normal file
18
server/operators/fal_bg_remove/manifest.json
Normal file
@ -0,0 +1,18 @@
|
||||
{
|
||||
"id": "fal_bg_remove",
|
||||
"name": "fal · Remove Background",
|
||||
"category": "image-prep",
|
||||
"description": "Image → clean subject cutout (transparent PNG) via fal-ai/birefnet/v2 (sub-cent). The single biggest quality lever before image→3D: run this first, then feed the cutout to TRELLIS/Hunyuan/SF3D so no geometry is wasted on background clutter. Needs FAL_KEY.",
|
||||
"accepts": ["image"],
|
||||
"produces": ["image"],
|
||||
"resources": "net",
|
||||
"requires_env": ["FAL_KEY"],
|
||||
"entry": "run.py",
|
||||
"params_schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"operating_resolution": {"type": "string", "enum": ["1024x1024", "2048x2048"], "default": "1024x1024", "description": "Processing resolution (2048 = finer edges, slower)"},
|
||||
"extra_args": {"type": "string", "default": "", "description": "Raw JSON merged into fal arguments (e.g. model variant). Full control"}
|
||||
}
|
||||
}
|
||||
}
|
||||
17
server/operators/fal_bg_remove/run.py
Normal file
17
server/operators/fal_bg_remove/run.py
Normal file
@ -0,0 +1,17 @@
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
from _lib import fal_common # noqa: E402
|
||||
|
||||
a, p = fal_common.parse_args()
|
||||
stem = Path(a.input[0]).stem if a.input else "cutout"
|
||||
fal_common.run(
|
||||
endpoint="fal-ai/birefnet/v2",
|
||||
arguments={"operating_resolution": p.get("operating_resolution")},
|
||||
outdir=a.outdir,
|
||||
image_path=a.input[0] if a.input else None,
|
||||
extra_args=p.get("extra_args"),
|
||||
out_stem=f"{stem}_cutout",
|
||||
collect="images",
|
||||
)
|
||||
19
server/operators/fal_hunyuan3d_v21/manifest.json
Normal file
19
server/operators/fal_hunyuan3d_v21/manifest.json
Normal file
@ -0,0 +1,19 @@
|
||||
{
|
||||
"id": "fal_hunyuan3d_v21",
|
||||
"name": "fal · Hunyuan3D 2.1 (PBR)",
|
||||
"category": "mesh-gen",
|
||||
"description": "Image → GLB with PBR textures via fal-ai/hunyuan3d-v21 (Hunyuan3D 2.1, ~90s, ~$0.05-0.10). Live-verified 2026-07: single-image works; the multi-view variant of v21 is broken on fal (use Hunyuan v2 for multi-view). Needs FAL_KEY.",
|
||||
"accepts": ["image"],
|
||||
"produces": ["model"],
|
||||
"resources": "net",
|
||||
"requires_env": ["FAL_KEY"],
|
||||
"entry": "run.py",
|
||||
"params_schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"seed": {"type": "integer", "description": "Random seed (leave blank for random)"},
|
||||
"textured_mesh": {"type": "boolean", "default": true, "description": "Generate PBR textures (costs more)"},
|
||||
"extra_args": {"type": "string", "default": "", "description": "Raw JSON merged into fal arguments — full control over any endpoint param"}
|
||||
}
|
||||
}
|
||||
}
|
||||
16
server/operators/fal_hunyuan3d_v21/run.py
Normal file
16
server/operators/fal_hunyuan3d_v21/run.py
Normal file
@ -0,0 +1,16 @@
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
from _lib import fal_common # noqa: E402
|
||||
|
||||
a, p = fal_common.parse_args()
|
||||
fal_common.run(
|
||||
endpoint="fal-ai/hunyuan3d-v21",
|
||||
arguments={"seed": p.get("seed"), "textured_mesh": p.get("textured_mesh", True)},
|
||||
outdir=a.outdir,
|
||||
image_path=a.input[0] if a.input else None,
|
||||
image_arg="input_image_url",
|
||||
extra_args=p.get("extra_args"),
|
||||
out_stem="hunyuan3d21",
|
||||
)
|
||||
18
server/operators/fal_image_edit/manifest.json
Normal file
18
server/operators/fal_image_edit/manifest.json
Normal file
@ -0,0 +1,18 @@
|
||||
{
|
||||
"id": "fal_image_edit",
|
||||
"name": "fal · Edit Image (prompt)",
|
||||
"category": "image-prep",
|
||||
"description": "Image + instruction → edited image via fal-ai/nano-banana/edit (remove objects, fix lighting, change materials — 'remove the price sticker', 'make it studio-lit on white'). Great for cleaning a photo before image→3D. Needs FAL_KEY.",
|
||||
"accepts": ["image"],
|
||||
"produces": ["image"],
|
||||
"resources": "net",
|
||||
"requires_env": ["FAL_KEY"],
|
||||
"entry": "run.py",
|
||||
"params_schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"prompt": {"type": "string", "default": "", "description": "Edit instruction (what to change)"},
|
||||
"extra_args": {"type": "string", "default": "", "description": "Raw JSON merged into fal arguments — full control over any endpoint param"}
|
||||
}
|
||||
}
|
||||
}
|
||||
22
server/operators/fal_image_edit/run.py
Normal file
22
server/operators/fal_image_edit/run.py
Normal file
@ -0,0 +1,22 @@
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
from _lib import fal_common # noqa: E402
|
||||
|
||||
a, p = fal_common.parse_args()
|
||||
if not p.get("prompt"):
|
||||
print("ERROR: prompt is required — describe the edit you want")
|
||||
sys.exit(1)
|
||||
stem = Path(a.input[0]).stem if a.input else "image"
|
||||
fal_common.run(
|
||||
endpoint="fal-ai/nano-banana/edit",
|
||||
arguments={"prompt": p["prompt"]},
|
||||
outdir=a.outdir,
|
||||
image_path=a.input[0] if a.input else None,
|
||||
image_arg="image_urls",
|
||||
image_as_list=True,
|
||||
extra_args=p.get("extra_args"),
|
||||
out_stem=f"{stem}_edited",
|
||||
collect="images",
|
||||
)
|
||||
19
server/operators/fal_text_image/manifest.json
Normal file
19
server/operators/fal_text_image/manifest.json
Normal file
@ -0,0 +1,19 @@
|
||||
{
|
||||
"id": "fal_text_image",
|
||||
"name": "fal · Text → Image",
|
||||
"category": "generate",
|
||||
"description": "Prompt → image via fal-ai/ideogram/v3 (~$0.03; uniquely renders READABLE text — labels, posters, packaging). No input asset needed. Generate a concept image here, then feed it to a mesh generator. Needs FAL_KEY.",
|
||||
"accepts": [],
|
||||
"produces": ["image"],
|
||||
"resources": "net",
|
||||
"requires_env": ["FAL_KEY"],
|
||||
"entry": "run.py",
|
||||
"params_schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"prompt": {"type": "string", "default": "", "description": "What to generate"},
|
||||
"image_size": {"type": "string", "enum": ["square_hd", "square", "portrait_4_3", "portrait_16_9", "landscape_4_3", "landscape_16_9"], "default": "square_hd", "description": "Output aspect/size"},
|
||||
"extra_args": {"type": "string", "default": "", "description": "Raw JSON merged into fal arguments (style, seed, etc.). Full control"}
|
||||
}
|
||||
}
|
||||
}
|
||||
18
server/operators/fal_text_image/run.py
Normal file
18
server/operators/fal_text_image/run.py
Normal file
@ -0,0 +1,18 @@
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
from _lib import fal_common # noqa: E402
|
||||
|
||||
a, p = fal_common.parse_args()
|
||||
if not p.get("prompt"):
|
||||
print("ERROR: prompt is required")
|
||||
sys.exit(1)
|
||||
fal_common.run(
|
||||
endpoint="fal-ai/ideogram/v3",
|
||||
arguments={"prompt": p["prompt"], "image_size": p.get("image_size", "square_hd")},
|
||||
outdir=a.outdir,
|
||||
extra_args=p.get("extra_args"),
|
||||
out_stem="generated",
|
||||
collect="images",
|
||||
)
|
||||
18
server/operators/fal_upscale/manifest.json
Normal file
18
server/operators/fal_upscale/manifest.json
Normal file
@ -0,0 +1,18 @@
|
||||
{
|
||||
"id": "fal_upscale",
|
||||
"name": "fal · Upscale (SeedVR)",
|
||||
"category": "image-prep",
|
||||
"description": "Image → faithful upscale via fal-ai/seedvr/upscale/image (~$0.001/MP — sharpens without inventing fake detail). Use on thin/blurry inputs before mesh generation or texture work. Needs FAL_KEY.",
|
||||
"accepts": ["image"],
|
||||
"produces": ["image"],
|
||||
"resources": "net",
|
||||
"requires_env": ["FAL_KEY"],
|
||||
"entry": "run.py",
|
||||
"params_schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"upscale_factor": {"type": "number", "default": 2, "minimum": 1, "maximum": 4, "description": "Upscale multiplier"},
|
||||
"extra_args": {"type": "string", "default": "", "description": "Raw JSON merged into fal arguments — full control over any endpoint param"}
|
||||
}
|
||||
}
|
||||
}
|
||||
17
server/operators/fal_upscale/run.py
Normal file
17
server/operators/fal_upscale/run.py
Normal file
@ -0,0 +1,17 @@
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
from _lib import fal_common # noqa: E402
|
||||
|
||||
a, p = fal_common.parse_args()
|
||||
stem = Path(a.input[0]).stem if a.input else "image"
|
||||
fal_common.run(
|
||||
endpoint="fal-ai/seedvr/upscale/image",
|
||||
arguments={"upscale_factor": p.get("upscale_factor", 2)},
|
||||
outdir=a.outdir,
|
||||
image_path=a.input[0] if a.input else None,
|
||||
extra_args=p.get("extra_args"),
|
||||
out_stem=f"{stem}_upscaled",
|
||||
collect="images",
|
||||
)
|
||||
@ -134,11 +134,15 @@ class Runner:
|
||||
task.add_done_callback(self._tasks.discard)
|
||||
|
||||
async def _run_lane(self, con, job_id: str):
|
||||
job = self.get_job(con, job_id)
|
||||
if not job: # deleted while queued
|
||||
self.cancelled.discard(job_id)
|
||||
return
|
||||
if job_id in self.cancelled:
|
||||
self.cancelled.discard(job_id)
|
||||
await self._update(con, job_id, status="cancelled", finished_at=db.now())
|
||||
return
|
||||
lane = self.lane_of(self.get_job(con, job_id)["operator"])
|
||||
lane = self.lane_of(job["operator"])
|
||||
async with self.lanes[lane]:
|
||||
if job_id in self.cancelled:
|
||||
self.cancelled.discard(job_id)
|
||||
|
||||
@ -31,7 +31,7 @@ def register_file(con, src: Path, name: str | None = None, parent_job: str | Non
|
||||
dest_dir = ASSETS_DIR / asset_id
|
||||
dest_dir.mkdir(parents=True, exist_ok=True)
|
||||
dest = dest_dir / name
|
||||
meta = meta or {}
|
||||
meta = dict(meta or {})
|
||||
if src.is_dir():
|
||||
if move:
|
||||
shutil.move(str(src), dest)
|
||||
|
||||
@ -139,10 +139,17 @@ export default function App() {
|
||||
|
||||
const selectedAsset = assets.find((a) => a.id === selected) || null;
|
||||
const envSet = settings._env_set || [];
|
||||
// no-input operators (accepts: []) are always available; others filter by asset kind
|
||||
const usableOps = useMemo(() =>
|
||||
operators.filter((op) => !selectedAsset || op.accepts.includes(selectedAsset.kind)),
|
||||
operators.filter((op) => !op.accepts.length || !selectedAsset || op.accepts.includes(selectedAsset.kind)),
|
||||
[operators, selectedAsset]);
|
||||
const opGroups = useMemo(() => {
|
||||
const groups = {};
|
||||
for (const op of usableOps) (groups[op.category || "other"] ||= []).push(op);
|
||||
return groups;
|
||||
}, [usableOps]);
|
||||
const activeOp = operators.find((o) => o.id === opId);
|
||||
const needsInput = (activeOp?.accepts?.length ?? 0) > 0;
|
||||
const gatedEnv = (activeOp?.requires_env || []).filter((e) => !envSet.includes(e));
|
||||
const gated = gatedEnv.length > 0;
|
||||
|
||||
@ -159,8 +166,8 @@ export default function App() {
|
||||
}, [usableOps]);
|
||||
|
||||
const launch = async () => {
|
||||
if (!activeOp || !selectedAsset || gated) return;
|
||||
await runJob(activeOp.id, selectedAsset.id, params);
|
||||
if (!activeOp || gated || (needsInput && !selectedAsset)) return;
|
||||
await runJob(activeOp.id, needsInput ? selectedAsset.id : null, params);
|
||||
refreshJobs();
|
||||
};
|
||||
|
||||
@ -219,12 +226,17 @@ export default function App() {
|
||||
<aside className="workbench">
|
||||
<h2>Run</h2>
|
||||
<select value={opId || ""} onChange={(e) => setOpId(e.target.value)}>
|
||||
{usableOps.map((op) => <option key={op.id} value={op.id}>{op.name}</option>)}
|
||||
{Object.entries(opGroups).map(([cat, ops]) => (
|
||||
<optgroup key={cat} label={cat}>
|
||||
{ops.map((op) => <option key={op.id} value={op.id}>{op.name}</option>)}
|
||||
</optgroup>
|
||||
))}
|
||||
</select>
|
||||
{activeOp && <p className="dim">{activeOp.description}</p>}
|
||||
{gated && <p className="warn">🔒 Needs {gatedEnv.join(", ")} — add it in Settings.</p>}
|
||||
{activeOp && !needsInput && <p className="dim">No input asset needed — just set the parameters and run.</p>}
|
||||
{activeOp && <ParamForm schema={activeOp.params_schema} values={params} onChange={setParams} />}
|
||||
<button className="go" disabled={!selectedAsset || !activeOp || gated} onClick={launch}>
|
||||
<button className="go" disabled={!activeOp || gated || (needsInput && !selectedAsset)} onClick={launch}>
|
||||
▶ Run {activeOp?.name || ""}
|
||||
</button>
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user