From 7911a59186e93cf254e47c1929d57f7d4ad5a285 Mon Sep 17 00:00:00 2001 From: type-two Date: Mon, 24 Aug 2026 14:17:50 +1000 Subject: [PATCH] VIDGOD: local AI video production toolkit Five CLIs around DaVinci Resolve music-video editing, all inference local on Apple Silicon (MPS/MLX): - vg-roto: SAM 2.1 + MatAnyone click-to-cutout -> ProRes 4444 alpha - vg-index / vg-find: PySceneDetect + mlx-whisper searchable clip library - vg-beats: librosa beat grid -> Resolve marker EDL - vg-transcode: legacy codecs -> ProRes LT, deinterlaced, resumable setup/setup_venvs.sh rebuilds venvs, tool clones, checkpoints and applies patches/matanyone-cv2-reader.patch (torchvision >= 0.23 removed read_video). Verified end-to-end on ultra 2026-08-24; setup/smoke_test.sh covers the lanes. Co-Authored-By: Claude Fable 5 --- .gitignore | 11 ++ CLAUDE.md | 25 ++++ README.md | 64 +++++++++ bin/vg-beats | 75 ++++++++++ bin/vg-find | 95 ++++++++++++ bin/vg-index | 181 +++++++++++++++++++++++ bin/vg-roto | 222 +++++++++++++++++++++++++++++ bin/vg-transcode | 67 +++++++++ patches/matanyone-cv2-reader.patch | 24 ++++ setup/setup_venvs.sh | 86 +++++++++++ setup/smoke_test.sh | 53 +++++++ 11 files changed, 903 insertions(+) create mode 100644 .gitignore create mode 100644 CLAUDE.md create mode 100644 README.md create mode 100755 bin/vg-beats create mode 100755 bin/vg-find create mode 100755 bin/vg-index create mode 100755 bin/vg-roto create mode 100755 bin/vg-transcode create mode 100644 patches/matanyone-cv2-reader.patch create mode 100755 setup/setup_venvs.sh create mode 100755 setup/smoke_test.sh diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..f8941ff --- /dev/null +++ b/.gitignore @@ -0,0 +1,11 @@ +# rebuilt by setup/setup_venvs.sh — never committed +venvs/ +tools/ +models/ +library/ +work/ +tests/ + +__pycache__/ +*.pyc +.DS_Store diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..07fa293 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,25 @@ +# VIDGOD — agent notes + +Read README.md first for what the tools do. This file is the stuff an agent needs beyond it. + +- **Canonical deploy: `~/Documents/VIDGOD` on ultra** (M1 Ultra 128GB, `johnking@100.91.239.7`). + Remote: `ssh://git@100.71.119.27:222/monster/vidgod.git` (Gitea, tailnet-only URL — never a + public one). The repo is scripts + patches only; venvs/tools/models/library are untracked + and rebuilt by `setup/setup_venvs.sh` (idempotent, resumable, heartbeats to + `~/.jobs/vidgod-setup.status`). +- Paths are **repo-relative everywhere**: bin scripts locate the root via their own location + (`#!/bin/sh` + exec-venv-python polyglot header; don't "fix" line 2 of the python CLIs — + it's what makes them run inside the right venv from any checkout location). +- Verified 2026-08-24 on ultra: all five lanes, including a real SAM2+MatAnyone cutout + (1917 trench demo clip, 2 clicks, matte held over 6s; ~3.4 fr/s at 1920×804 on MPS). +- The MatAnyone torchvision fix lives in `patches/matanyone-cv2-reader.patch`, applied by + setup step 6 (guard: grep CAP_PROP_FPS). Upstream may eventually fix this — if the patch + stops applying, check whether their `read_frame_from_videos` already avoids + `torchvision.io.read_video`, and drop the patch step if so. +- Test with `setup/smoke_test.sh` (fast) / `--roto` (full pipeline). Test media is synthesized + (ffmpeg lavfi + macOS `say`) — nothing copyrighted is committed. +- The clip library sqlite lives at `library/clips.sqlite` **per checkout** — the real library + on ultra is the one that matters; don't index into a scratch clone and expect vg-find on + ultra to see it. +- Fleet context (SSH map, farm, heartbeat convention): `~/.claude/skills/fleet` + + `~/.claude/skills/jobs` on JING5, GODVERSE_GUIDE.md §11 on ultra/JING5. diff --git a/README.md b/README.md new file mode 100644 index 0000000..1c19c85 --- /dev/null +++ b/README.md @@ -0,0 +1,64 @@ +# VIDGOD + +Open-source AI video production toolkit for music-video editing around DaVinci Resolve. +All inference is local on Apple Silicon (PyTorch MPS + MLX) — no cloud, no subscriptions. + +Built for the workflow: hoard old footage → normalize it → index every shot and every +spoken word → search it → arrange to a beat grid in Resolve → cut actors out with clicks +instead of rotoscoping splines. + +## The tools + +| Command | What it does | +|---|---| +| `bin/vg-roto CLIP --point x,y` | Click an actor → tracked cutout as **ProRes 4444 with alpha** (SAM 2.1 + MatAnyone). `--grab-frame` saves the prompt frame so you can find coords. `--mode mask` for hard binary masks via SAM2 video propagation. | +| `bin/vg-transcode DIR` | Batch-convert old AVI/WMV/MPEG/etc → ProRes LT, mirrored dir tree, auto-deinterlace, skips already-done files. `--h264` for small proxies. | +| `bin/vg-index PATHS` | Shot-detect (PySceneDetect) + transcribe (mlx-whisper large-v3-turbo) + thumbnail everything into `library/clips.sqlite` (FTS5). Resumable. | +| `bin/vg-find WORDS` | Full-text search the dialogue; shows the enclosing shot. `--cut DIR` exports each matching shot as a ProRes clip. `--shots VIDEO` lists a video's shots. | +| `bin/vg-beats SONG` | Beat-track a music file (librosa) → `.beats.csv` + a **Resolve marker EDL** (import: right-click timeline in media pool → Timelines → Import → Timeline Markers from EDL). | + +## Install + +Needs: Apple Silicon Mac, `brew install ffmpeg uv`, ~4GB disk for venvs + models. + +```bash +git clone ssh://git@100.71.119.27:222/monster/vidgod.git && cd vidgod +./setup/setup_venvs.sh # venvs, SAM2 + MatAnyone clones, checkpoints, patches (~10 min) +./setup/smoke_test.sh # fast lanes; add --roto for the full cutout pipeline +``` + +Everything heavy (venvs/, tools/, models/, library/, work/, tests/) lives untracked inside +the working copy; the repo is just the scripts. Long jobs heartbeat to `~/.jobs/vidgod-*.status`. + +## Typical session + +```bash +bin/vg-transcode ~/old-tv-rips # → ~/old-tv-rips-prores +bin/vg-index ~/old-tv-rips-prores # overnight for a big archive +bin/vg-find "radical" --cut ~/mv/pulls # grab every shot where someone says it +bin/vg-beats ~/mv/song.mp3 --fps 25 # markers for the Resolve timeline +bin/vg-roto ~/mv/pulls/003_*.mov --grab-frame # find your click coords +bin/vg-roto ~/mv/pulls/003_*.mov --point 970,220 # → ProRes 4444 alpha cutout +``` + +## Notes & gotchas + +- **MatAnyone patch**: torchvision ≥ 0.23 removed `torchvision.io.read_video`; setup applies + `patches/matanyone-cv2-reader.patch` to its clone automatically. If you re-clone + `tools/MatAnyone` by hand, re-run `setup/setup_venvs.sh`. +- MatAnyone's checkpoint comes from its GitHub release (the HF repo doesn't carry the + file at the root path); its inference script insists on `pretrained_models/` inside its + repo dir, so setup symlinks it there. +- MatAnyone propagates **forward from a first-frame mask** — `vg-roto --frame N` trims the + clip at N first. Prompt on a frame where the target is clearly visible. +- Scripts set `/opt/homebrew/bin` in PATH themselves (non-interactive SSH doesn't) and + `PYTORCH_ENABLE_MPS_FALLBACK=1` for stray unsupported MPS kernels. +- Rough speed on an M1 Ultra: MatAnyone ≈ 3.4 fr/s at 1920×804; whisper large-v3-turbo + transcribes far faster than realtime; ProRes transcodes are ffmpeg-bound. + +## Phase 2 (not built yet) + +ProPainter (object/logo removal), RIFE interpolation, Cutie interactive segmentation GUI, +MODELBEAST farm operators for roto/index jobs, model mirror to NAS modelzoo. Upscaling +old footage already exists fleet-side (`seedvr2_upscale` on the farm); Resolve's Optical +Flow retime covers slow-mo. diff --git a/bin/vg-beats b/bin/vg-beats new file mode 100755 index 0000000..7de0a77 --- /dev/null +++ b/bin/vg-beats @@ -0,0 +1,75 @@ +#!/bin/sh +"exec" "`dirname $0`/../venvs/index/bin/python" "$0" "$@" +"""vg-beats — extract a beat grid from a music track for cutting in Resolve. + +Usage: vg-beats SONG.mp3 [--fps 25] [--tc-start 01:00:00:00] [--bpm HINT] [--out PREFIX] + +Writes: + PREFIX.beats.csv one beat time (seconds) per line, with beat number + PREFIX.beats.edl Resolve timeline markers — import via + Timeline > (right-click) > Timelines > Import > Timeline Markers from EDL +Every 4th beat is a red marker (bar guess), others blue. +""" +import argparse, os, sys + +os.environ["PATH"] = "/opt/homebrew/bin:" + os.environ.get("PATH", "") + + +def tc(seconds, fps, start_frames): + total = int(round(seconds * fps)) + start_frames + f = total % int(fps) + s = (total // int(fps)) % 60 + m = (total // (int(fps) * 60)) % 60 + h = total // (int(fps) * 3600) + return f"{h:02d}:{m:02d}:{s:02d}:{f:02d}" + + +def parse_tc(t, fps): + h, m, s, f = (int(x) for x in t.split(":")) + return ((h * 3600 + m * 60 + s) * int(fps)) + f + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("audio") + ap.add_argument("--fps", type=float, default=25.0, help="timeline fps for the EDL (default 25)") + ap.add_argument("--tc-start", default="01:00:00:00", help="timeline start timecode (Resolve default 01:00:00:00)") + ap.add_argument("--bpm", type=float, default=None, help="starting tempo hint") + ap.add_argument("--out", default=None, help="output prefix (default: alongside the audio file)") + args = ap.parse_args() + + import librosa + import numpy as np + + print(f"loading {args.audio} ...") + y, sr = librosa.load(args.audio, sr=22050, mono=True) + kw = {"start_bpm": args.bpm} if args.bpm else {} + tempo, beats = librosa.beat.beat_track(y=y, sr=sr, units="time", **kw) + tempo = float(np.atleast_1d(tempo)[0]) + if len(beats) == 0: + sys.exit("no beats detected — is this a music track?") + print(f"tempo ~{tempo:.1f} BPM, {len(beats)} beats over {len(y)/sr:.1f}s") + + prefix = args.out or os.path.splitext(args.audio)[0] + start_frames = parse_tc(args.tc_start, args.fps) + + with open(prefix + ".beats.csv", "w") as f: + f.write("beat,seconds\n") + for i, t in enumerate(beats, 1): + f.write(f"{i},{t:.4f}\n") + + with open(prefix + ".beats.edl", "w") as f: + f.write(f"TITLE: {os.path.basename(prefix)} beats ({tempo:.1f}bpm)\nFCM: NON-DROP FRAME\n\n") + for i, t in enumerate(beats, 1): + a = tc(t, args.fps, start_frames) + b = tc(t + 1.0 / args.fps, args.fps, start_frames) + color = "ResolveColorRed" if (i - 1) % 4 == 0 else "ResolveColorBlue" + f.write(f"{i:03d} 001 V C {a} {b} {a} {b} \n") + f.write(f" |C:{color} |M:Beat {i} |D:1\n\n") + + print(f"wrote {prefix}.beats.csv and {prefix}.beats.edl") + print("Resolve: right-click the timeline in the media pool > Timelines > Import > Timeline Markers from EDL") + + +if __name__ == "__main__": + main() diff --git a/bin/vg-find b/bin/vg-find new file mode 100755 index 0000000..b3663bb --- /dev/null +++ b/bin/vg-find @@ -0,0 +1,95 @@ +#!/bin/sh +"exec" "`dirname $0`/../venvs/index/bin/python" "$0" "$@" +"""vg-find — search the clip library built by vg-index. + +Usage: + vg-find radical search dialogue for a word/phrase (FTS5 syntax ok) + vg-find "totally awesome" phrase search + vg-find radical --cut DIR also export each matching shot as a ProRes clip into DIR + vg-find --shots VIDEO list the detected shots of one video +""" +import argparse, os, sqlite3, subprocess, sys +from pathlib import Path + +os.environ["PATH"] = "/opt/homebrew/bin:" + os.environ.get("PATH", "") +DB = Path(__file__).resolve().parent.parent / "library/clips.sqlite" + + +def hms(t): + t = max(0, t) + return f"{int(t//3600):02d}:{int(t%3600//60):02d}:{t%60:06.3f}" + + +def cut(src, a, b, dest): + subprocess.run(["ffmpeg", "-hide_banner", "-loglevel", "error", "-y", + "-ss", f"{a:.3f}", "-to", f"{b:.3f}", "-i", src, + "-c:v", "prores_ks", "-profile:v", "1", "-c:a", "pcm_s16le", str(dest)], + check=False) + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("query", nargs="*") + ap.add_argument("--cut", metavar="DIR", help="export matching shots as ProRes clips") + ap.add_argument("--shots", metavar="VIDEO", help="list shots of one video (path substring ok)") + ap.add_argument("--limit", type=int, default=40) + args = ap.parse_args() + + if not DB.exists(): + sys.exit("no library yet — run vg-index first") + db = sqlite3.connect(DB) + + if args.shots: + rows = db.execute("""SELECT v.path, s.idx, s.start_s, s.end_s, s.thumb FROM shots s + JOIN videos v ON v.id=s.video_id WHERE v.path LIKE ? + ORDER BY v.path, s.idx""", (f"%{args.shots}%",)).fetchall() + if not rows: + sys.exit("no matching video in library") + last = None + for path, idx, a, b, thumb in rows: + if path != last: + print(f"\n{path}") + last = path + print(f" shot {idx:3d} {hms(a)} - {hms(b)} ({b-a:5.1f}s)") + return + + if not args.query: + v, s, g = db.execute("""SELECT (SELECT COUNT(*) FROM videos), + (SELECT COUNT(*) FROM shots), + (SELECT COUNT(*) FROM segs)""").fetchone() + print(f"library: {v} videos, {s} shots, {g} dialogue segments") + return + + q = " ".join(args.query) + fts = f'"{q}"' if " " in q and not any(c in q for c in '"*') else q + rows = db.execute(""" + SELECT v.path, g.start_s, g.end_s, g.text, + (SELECT s.start_s FROM shots s WHERE s.video_id=g.video_id + AND s.start_s <= (g.start_s+g.end_s)/2 ORDER BY s.start_s DESC LIMIT 1), + (SELECT s.end_s FROM shots s WHERE s.video_id=g.video_id + AND s.start_s <= (g.start_s+g.end_s)/2 ORDER BY s.start_s DESC LIMIT 1) + FROM segs_fts f JOIN segs g ON g.id=f.rowid JOIN videos v ON v.id=g.video_id + WHERE segs_fts MATCH ? ORDER BY v.path, g.start_s LIMIT ?""", (fts, args.limit)).fetchall() + if not rows: + print("no matches") + return + + outdir = None + if args.cut: + outdir = Path(args.cut).expanduser() + outdir.mkdir(parents=True, exist_ok=True) + + for i, (path, a, b, text, sa, sb) in enumerate(rows, 1): + sa = a if sa is None else sa + sb = b if sb is None else sb + print(f"{hms(a)} {Path(path).name}") + print(f" \"{text}\" (shot {hms(sa)} - {hms(sb)})") + if outdir: + name = f"{i:03d}_{Path(path).stem}_{int(sa)}s.mov" + cut(path, sa, sb, outdir / name) + print(f" -> {outdir/name}") + print(f"\n{len(rows)} matches" + (f", clips in {outdir}" if outdir else "")) + + +if __name__ == "__main__": + main() diff --git a/bin/vg-index b/bin/vg-index new file mode 100755 index 0000000..2ced750 --- /dev/null +++ b/bin/vg-index @@ -0,0 +1,181 @@ +#!/bin/sh +"exec" "`dirname $0`/../venvs/index/bin/python" "$0" "$@" +"""vg-index — build a searchable clip library: shot detection + transcript + thumbnails. + +Usage: vg-index PATH [PATH...] [--no-whisper] [--force] [--min-shot 0.5] + +For every video found under the given paths: + 1. PySceneDetect splits it into shots + 2. a thumbnail is saved for each shot -> library/thumbs/ + 3. Whisper (mlx) transcribes the audio -> timestamped segments + 4. everything lands in library/clips.sqlite (FTS5 full-text over dialogue) + +Search it with vg-find. Re-running skips unchanged files. Heartbeat: ~/.jobs/vidgod-index.status +""" +import argparse, hashlib, os, sqlite3, subprocess, sys, time +from datetime import datetime +from pathlib import Path + +os.environ["PATH"] = "/opt/homebrew/bin:" + os.environ.get("PATH", "") + +VG = Path(__file__).resolve().parent.parent +LIB = VG / "library" +DB = LIB / "clips.sqlite" +THUMBS = LIB / "thumbs" +WHISPER_MODEL = "mlx-community/whisper-large-v3-turbo" +VIDEO_EXTS = {".mov", ".mp4", ".mkv", ".avi", ".mpg", ".mpeg", ".wmv", ".webm", ".m4v", ".flv", ".mts", ".m2ts", ".vob"} + + +def hb(msg, note=""): + j = Path.home() / ".jobs" + j.mkdir(exist_ok=True) + (j / "vidgod-index.status").write_text(f"{datetime.now():%Y-%m-%d %H:%M:%S} | {msg} | {note}\n") + + +def open_db(): + LIB.mkdir(parents=True, exist_ok=True) + THUMBS.mkdir(parents=True, exist_ok=True) + db = sqlite3.connect(DB) + db.executescript(""" + CREATE TABLE IF NOT EXISTS videos( + id INTEGER PRIMARY KEY, path TEXT UNIQUE, mtime REAL, duration REAL, + fps REAL, width INT, height INT, has_transcript INT DEFAULT 0, indexed_at TEXT); + CREATE TABLE IF NOT EXISTS shots( + id INTEGER PRIMARY KEY, video_id INT, idx INT, start_s REAL, end_s REAL, thumb TEXT); + CREATE TABLE IF NOT EXISTS segs( + id INTEGER PRIMARY KEY, video_id INT, start_s REAL, end_s REAL, text TEXT); + CREATE VIRTUAL TABLE IF NOT EXISTS segs_fts USING fts5(text, content='segs', content_rowid='id'); + CREATE INDEX IF NOT EXISTS idx_shots_video ON shots(video_id); + CREATE INDEX IF NOT EXISTS idx_segs_video ON segs(video_id); + """) + return db + + +def probe(path): + out = subprocess.run( + ["ffprobe", "-v", "error", "-select_streams", "v:0", + "-show_entries", "stream=width,height,avg_frame_rate:format=duration", + "-of", "csv=p=0", str(path)], + capture_output=True, text=True).stdout.strip().splitlines() + w = h = 0; fps = 25.0; dur = 0.0 + for line in out: + parts = line.split(",") + if len(parts) >= 3: + w, h = int(parts[0]), int(parts[1]) + try: + num, den = parts[2].split("/") + fps = float(num) / float(den) if float(den) else 25.0 + except ValueError: + pass + elif len(parts) == 1 and parts[0]: + try: dur = float(parts[0]) + except ValueError: pass + return dur, fps, w, h + + +def detect_shots(path, min_shot): + from scenedetect import detect, ContentDetector + scenes = detect(str(path), ContentDetector(min_scene_len=int(min_shot * 25))) + if not scenes: + return None # single-shot video; caller uses full duration + return [(s.seconds, e.seconds) for s, e in scenes] + + +def save_thumb(path, t, dest): + subprocess.run(["ffmpeg", "-hide_banner", "-loglevel", "error", "-y", + "-ss", f"{t:.3f}", "-i", str(path), + "-frames:v", "1", "-vf", "scale=480:-2", str(dest)], + capture_output=True) + return dest.exists() + + +def transcribe(path): + import mlx_whisper + res = mlx_whisper.transcribe(str(path), path_or_hf_repo=WHISPER_MODEL) + return [(s["start"], s["end"], s["text"].strip()) for s in res.get("segments", []) if s["text"].strip()] + + +def index_file(db, path, args, pos, total): + st = path.stat() + row = db.execute("SELECT id, mtime FROM videos WHERE path=?", (str(path),)).fetchone() + if row and abs(row[1] - st.st_mtime) < 1 and not args.force: + return "skip" + if row: # stale entry — wipe and redo + vid = row[0] + db.execute("DELETE FROM segs_fts WHERE rowid IN (SELECT id FROM segs WHERE video_id=?)", (vid,)) + db.execute("DELETE FROM segs WHERE video_id=?", (vid,)) + db.execute("DELETE FROM shots WHERE video_id=?", (vid,)) + db.execute("DELETE FROM videos WHERE id=?", (vid,)) + + hb(f"{pos}/{total}", f"probing {path.name}") + dur, fps, w, h = probe(path) + if dur <= 0: + return "unreadable" + cur = db.execute( + "INSERT INTO videos(path, mtime, duration, fps, width, height, indexed_at) VALUES(?,?,?,?,?,?,?)", + (str(path), st.st_mtime, dur, fps, w, h, datetime.now().isoformat(timespec="seconds"))) + vid = cur.lastrowid + + hb(f"{pos}/{total}", f"shots: {path.name}") + shots = detect_shots(path, args.min_shot) or [(0.0, dur)] + tag = hashlib.md5(str(path).encode()).hexdigest()[:10] + for i, (a, b) in enumerate(shots): + thumb = THUMBS / f"{tag}_{i:04d}.jpg" + save_thumb(path, a + (b - a) / 2, thumb) + db.execute("INSERT INTO shots(video_id, idx, start_s, end_s, thumb) VALUES(?,?,?,?,?)", + (vid, i, a, b, str(thumb))) + + if not args.no_whisper: + hb(f"{pos}/{total}", f"whisper: {path.name}") + try: + for a, b, text in transcribe(path): + cur = db.execute("INSERT INTO segs(video_id, start_s, end_s, text) VALUES(?,?,?,?)", + (vid, a, b, text)) + db.execute("INSERT INTO segs_fts(rowid, text) VALUES(?,?)", (cur.lastrowid, text)) + db.execute("UPDATE videos SET has_transcript=1 WHERE id=?", (vid,)) + except Exception as e: # no audio track etc — keep the shots + print(f" whisper failed on {path.name}: {e}", file=sys.stderr) + + db.commit() + return f"{len(shots)} shots" + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("paths", nargs="+") + ap.add_argument("--no-whisper", action="store_true") + ap.add_argument("--force", action="store_true", help="re-index even if unchanged") + ap.add_argument("--min-shot", type=float, default=0.5, help="minimum shot length, seconds") + args = ap.parse_args() + + files = [] + for p in args.paths: + p = Path(p).expanduser().resolve() + if p.is_dir(): + files += sorted(x for x in p.rglob("*") if x.suffix.lower() in VIDEO_EXTS and x.is_file()) + elif p.is_file(): + files.append(p) + if not files: + sys.exit("no video files found") + + db = open_db() + t0 = time.time() + counts = {} + for i, f in enumerate(files, 1): + try: + r = index_file(db, f, args, i, len(files)) + except KeyboardInterrupt: + raise + except Exception as e: + r = "error" + print(f"ERROR indexing {f}: {e}", file=sys.stderr) + counts[r] = counts.get(r, 0) + 1 + print(f"[{i}/{len(files)}] {f.name}: {r}") + hb(f"DONE {len(files)}/{len(files)}", str(counts)) + n = db.execute("SELECT COUNT(*) FROM shots").fetchone()[0] + s = db.execute("SELECT COUNT(*) FROM segs").fetchone()[0] + print(f"\nlibrary now: {n} shots, {s} dialogue segments ({time.time()-t0:.0f}s). Search with vg-find.") + + +if __name__ == "__main__": + main() diff --git a/bin/vg-roto b/bin/vg-roto new file mode 100755 index 0000000..0874fd1 --- /dev/null +++ b/bin/vg-roto @@ -0,0 +1,222 @@ +#!/bin/sh +"exec" "`dirname $0`/../venvs/roto/bin/python" "$0" "$@" +"""vg-roto — one-click actor cutout: SAM 2.1 + MatAnyone on Apple Silicon. + +Usage: + vg-roto CLIP.mp4 --point 640,360 click on the actor (frame 0 coords) + vg-roto CLIP.mp4 --point 640,360 --point 700,200 multiple clicks refine the pick + vg-roto CLIP.mp4 --box 400,100,900,700 or a rough box around them + vg-roto CLIP.mp4 --point ... --neg 100,100 negative click excludes a region + vg-roto CLIP.mp4 --grab-frame just save frame 0 as a PNG so you + can find click coords, then exit + +Options: --out DIR (default CLIP_roto/), --frame N (prompt+start on frame N), + --mode matte|mask (matte = soft alpha via MatAnyone, default; + mask = hard binary via SAM2 video propagation), + --max-size N (downscale min side for inference, e.g. 720 for speed) + +Output in DIR: + CLIP_alpha.mov ProRes 4444 with real alpha -> drop straight into Resolve + CLIP_matte.mov the matte alone (grayscale) + CLIP_green.mp4 preview comp over green +""" +import argparse, os, shutil, subprocess, sys +from pathlib import Path + +os.environ["PATH"] = "/opt/homebrew/bin:" + os.environ.get("PATH", "") +os.environ.setdefault("PYTORCH_ENABLE_MPS_FALLBACK", "1") + +VG = Path(__file__).resolve().parent.parent +SAM2_CKPT = VG / "models/sam2.1_hiera_large.pt" +SAM2_CFG = "configs/sam2.1/sam2.1_hiera_l.yaml" +MATANYONE = VG / "tools/MatAnyone" +RPY = VG / "venvs/roto/bin/python" + + +def run(cmd, **kw): + r = subprocess.run([str(c) for c in cmd], **kw) + if r.returncode != 0: + sys.exit(f"command failed: {' '.join(str(c) for c in cmd)}") + return r + + +def probe_video(path): + out = subprocess.run( + ["ffprobe", "-v", "error", "-select_streams", "v:0", + "-show_entries", "stream=width,height,avg_frame_rate,nb_frames", + "-of", "csv=p=0", str(path)], capture_output=True, text=True).stdout.strip() + w, h, fr = out.split(",")[:3] + num, den = fr.split("/") + return int(w), int(h), (float(num) / float(den) if float(den) else 25.0) + + +def grab_frame(video, n, fps, dest): + run(["ffmpeg", "-hide_banner", "-loglevel", "error", "-y", + "-ss", f"{n / fps:.6f}", "-i", video, "-frames:v", "1", dest]) + + +def sam2_first_frame_mask(frame_png, points, neg_points, box, mask_png): + import numpy as np, torch + from PIL import Image + from sam2.build_sam import build_sam2 + from sam2.sam2_image_predictor import SAM2ImagePredictor + + device = "mps" if torch.backends.mps.is_available() else "cpu" + print(f"SAM2: loading {SAM2_CKPT.name} on {device} ...") + model = build_sam2(SAM2_CFG, str(SAM2_CKPT), device=device) + pred = SAM2ImagePredictor(model) + img = np.array(Image.open(frame_png).convert("RGB")) + pred.set_image(img) + + pc = pl = bx = None + if points or neg_points: + pc = np.array(points + neg_points, dtype=np.float32) + pl = np.array([1] * len(points) + [0] * len(neg_points), dtype=np.int32) + if box: + bx = np.array(box, dtype=np.float32) + masks, scores, _ = pred.predict(point_coords=pc, point_labels=pl, box=bx, multimask_output=True) + best = int(np.argmax(scores)) + m = (masks[best] > 0).astype(np.uint8) * 255 + Image.fromarray(m).save(mask_png) + cov = 100.0 * (m > 0).mean() + print(f"SAM2: mask score {scores[best]:.3f}, covers {cov:.1f}% of frame") + if cov < 0.05 or cov > 95: + print("WARNING: mask looks degenerate — check your click point (--grab-frame to inspect)") + return mask_png + + +def matanyone_matte(video, mask_png, outdir, max_size): + env = dict(os.environ) + cmd = [RPY, "inference_matanyone.py", "-i", str(video), "-m", str(mask_png), + "-o", str(outdir), "--max_size", str(max_size)] + print("MatAnyone: propagating matte (this is the slow part) ...") + r = subprocess.run([str(c) for c in cmd], cwd=MATANYONE, env=env) + if r.returncode != 0: + sys.exit("MatAnyone failed") + stem = Path(video).stem + pha = outdir / f"{stem}_pha.mp4" + if not pha.exists(): + cands = sorted(outdir.glob("*_pha.mp4")) + if not cands: + sys.exit(f"MatAnyone produced no *_pha.mp4 in {outdir}") + pha = cands[-1] + return pha + + +def sam2_video_masks(video, points, neg_points, box, frame_n, workdir, fps): + import numpy as np, torch + from PIL import Image + from sam2.build_sam import build_sam2_video_predictor + + frames = workdir / "frames" + frames.mkdir(parents=True, exist_ok=True) + if not list(frames.glob("*.jpg")): + run(["ffmpeg", "-hide_banner", "-loglevel", "error", "-y", "-i", video, + "-q:v", "2", "-start_number", "0", frames / "%05d.jpg"]) + device = "mps" if torch.backends.mps.is_available() else "cpu" + print(f"SAM2 video: loading on {device} ...") + pred = build_sam2_video_predictor(SAM2_CFG, str(SAM2_CKPT), device=device) + state = pred.init_state(video_path=str(frames), offload_video_to_cpu=True, + offload_state_to_cpu=True) + pc = pl = bx = None + if points or neg_points: + pc = np.array(points + neg_points, dtype=np.float32) + pl = np.array([1] * len(points) + [0] * len(neg_points), dtype=np.int32) + if box: + bx = np.array(box, dtype=np.float32) + pred.add_new_points_or_box(state, frame_idx=frame_n, obj_id=1, + points=pc, labels=pl, box=bx) + masks_dir = workdir / "masks" + masks_dir.mkdir(exist_ok=True) + n = 0 + for fidx, _, logits in pred.propagate_in_video(state): + m = (logits[0] > 0).cpu().numpy().squeeze().astype(np.uint8) * 255 + Image.fromarray(m).save(masks_dir / f"{fidx:05d}.png") + n += 1 + if n % 50 == 0: + print(f" propagated {n} frames") + print(f"SAM2 video: {n} frames masked") + matte = workdir / "matte_raw.mp4" + run(["ffmpeg", "-hide_banner", "-loglevel", "error", "-y", "-framerate", f"{fps}", + "-start_number", str(frame_n), "-i", masks_dir / "%05d.png", + "-c:v", "libx264", "-crf", "12", "-pix_fmt", "yuv420p", matte]) + return matte + + +def compose_outputs(video, pha, outdir, stem, fps, w, h): + alpha_mov = outdir / f"{stem}_alpha.mov" + matte_mov = outdir / f"{stem}_matte.mov" + green_mp4 = outdir / f"{stem}_green.mp4" + scale = f"scale={w}:{h}:flags=bicubic,format=gray" + print("composing ProRes 4444 + previews ...") + run(["ffmpeg", "-hide_banner", "-loglevel", "error", "-y", "-i", video, "-i", pha, + "-filter_complex", f"[1:v]{scale}[a];[0:v][a]alphamerge,format=yuva444p10le[out]", + "-map", "[out]", "-map", "0:a:0?", "-c:v", "prores_ks", "-profile:v", "4444", + "-c:a", "pcm_s16le", "-shortest", alpha_mov]) + run(["ffmpeg", "-hide_banner", "-loglevel", "error", "-y", "-i", pha, + "-vf", scale, "-c:v", "prores_ks", "-profile:v", "1", matte_mov]) + run(["ffmpeg", "-hide_banner", "-loglevel", "error", "-y", "-i", video, "-i", pha, + "-filter_complex", + f"color=0x00b140:size={w}x{h}:rate={fps}[bg];" + f"[1:v]{scale}[a];[0:v][a]alphamerge[fg];[bg][fg]overlay=shortest=1,format=yuv420p[out]", + "-map", "[out]", "-c:v", "libx264", "-crf", "18", green_mp4]) + return alpha_mov, matte_mov, green_mp4 + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("video") + ap.add_argument("--point", action="append", default=[], help="x,y positive click (repeatable)") + ap.add_argument("--neg", action="append", default=[], help="x,y negative click (repeatable)") + ap.add_argument("--box", help="x1,y1,x2,y2") + ap.add_argument("--frame", type=int, default=0, help="frame to prompt on / start from") + ap.add_argument("--mode", choices=["matte", "mask"], default="matte") + ap.add_argument("--max-size", type=int, default=-1, help="downscale min side for inference") + ap.add_argument("--out", default=None) + ap.add_argument("--grab-frame", action="store_true", help="save the prompt frame as PNG and exit") + args = ap.parse_args() + + video = Path(args.video).expanduser().resolve() + if not video.exists(): + sys.exit(f"no such file: {video}") + stem = video.stem + outdir = Path(args.out).expanduser() if args.out else video.parent / f"{stem}_roto" + outdir.mkdir(parents=True, exist_ok=True) + work = outdir / "work" + work.mkdir(exist_ok=True) + + w, h, fps = probe_video(video) + frame_png = work / f"frame{args.frame:05d}.png" + grab_frame(video, args.frame, fps, frame_png) + if args.grab_frame: + print(f"prompt frame saved: {frame_png} ({w}x{h}) — open it, note x,y of your click") + return + + points = [tuple(float(v) for v in p.split(",")) for p in args.point] + negs = [tuple(float(v) for v in p.split(",")) for p in args.neg] + box = [float(v) for v in args.box.split(",")] if args.box else None + if not points and not box: + sys.exit("give me --point x,y or --box x1,y1,x2,y2 (use --grab-frame to find coords)") + + src = video + if args.frame > 0 and args.mode == "matte": + src = work / f"{stem}_from{args.frame}.mp4" + print(f"trimming from frame {args.frame} (MatAnyone propagates forward from its mask)") + run(["ffmpeg", "-hide_banner", "-loglevel", "error", "-y", + "-ss", f"{args.frame / fps:.6f}", "-i", video, + "-c:v", "libx264", "-crf", "12", "-c:a", "aac", src]) + + mask_png = work / "first_mask.png" + sam2_first_frame_mask(frame_png, points, negs, box, mask_png) + + if args.mode == "matte": + pha = matanyone_matte(src, mask_png, work, args.max_size) + else: + pha = sam2_video_masks(src, points, negs, box, args.frame, work, fps) + + alpha, matte, green = compose_outputs(src, pha, outdir, stem, fps, w, h) + print(f"\ndone:\n {alpha} <- ProRes 4444 with alpha, drop into Resolve\n {matte}\n {green} <- preview") + + +if __name__ == "__main__": + main() diff --git a/bin/vg-transcode b/bin/vg-transcode new file mode 100755 index 0000000..ead1af9 --- /dev/null +++ b/bin/vg-transcode @@ -0,0 +1,67 @@ +#!/bin/zsh +# vg-transcode — batch-normalize old footage to ProRes LT for editing. +# Usage: vg-transcode SRC_DIR [DEST_DIR] [--h264] +# Recurses SRC_DIR, converts every video to ProRes LT .mov (or H.264 with --h264), +# mirrors the directory structure, skips files already done, deinterlaces +# interlaced sources (bwdif to progressive at double rate). +# Heartbeat: ~/.jobs/vidgod-transcode.status +export PATH=/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin +set -u + +SRC="${1:?usage: vg-transcode SRC_DIR [DEST_DIR] [--h264]}" +DEST="${2:-}" +CODEC=prores +[[ "${2:-}" == "--h264" ]] && { DEST=""; CODEC=h264; } +[[ "${3:-}" == "--h264" ]] && CODEC=h264 +SRC="${SRC:A}" +[[ -d "$SRC" ]] || { echo "not a dir: $SRC" >&2; exit 1; } +[[ -z "$DEST" ]] && DEST="${SRC}-prores" +mkdir -p "$DEST" + +hb(){ mkdir -p ~/.jobs; echo "$(date '+%F %T') | $1 | $2" > ~/.jobs/vidgod-transcode.status; } + +typeset -a files +files=() +while IFS= read -r -d '' f; do files+=("$f"); done < <( + find "$SRC" \( -iname '*.avi' -o -iname '*.wmv' -o -iname '*.mpg' -o -iname '*.mpeg' \ + -o -iname '*.asf' -o -iname '*.flv' -o -iname '*.rm' -o -iname '*.rmvb' -o -iname '*.vob' \ + -o -iname '*.m2v' -o -iname '*.divx' -o -iname '*.mov' -o -iname '*.mp4' -o -iname '*.mkv' \ + -o -iname '*.webm' -o -iname '*.m4v' -o -iname '*.3gp' -o -iname '*.mts' -o -iname '*.m2ts' \) \ + -not -path "$DEST/*" -print0 | sort -z) + +total=${#files[@]} +[[ $total -eq 0 ]] && { echo "no video files under $SRC"; exit 0; } +echo "transcoding $total files: $SRC -> $DEST ($CODEC)" + +n=0; ok=0; skip=0; fail=0 +for f in "${files[@]}"; do + n=$((n+1)) + rel="${f#$SRC/}" + out="$DEST/${rel%.*}.mov" + [[ "$CODEC" == "h264" ]] && out="$DEST/${rel%.*}.mp4" + mkdir -p "${out:h}" + hb "$n/$total" "${rel:t}" + if [[ -s "$out" && "$out" -nt "$f" ]]; then skip=$((skip+1)); continue; fi + + # detect interlacing + fo=$(ffprobe -v error -select_streams v:0 -show_entries stream=field_order -of csv=p=0 "$f" 2>/dev/null) + vf="format=yuv422p10le" + case "$fo" in + tt|bb|tb|bt) vf="bwdif=mode=1:parity=auto,format=yuv422p10le" ;; + esac + + if [[ "$CODEC" == "prores" ]]; then + ffmpeg -hide_banner -loglevel error -y -i "$f" -vf "$vf" \ + -c:v prores_ks -profile:v 1 -vendor apl0 \ + -c:a pcm_s16le -map 0:v:0 -map '0:a:0?' "$out" &2; fi +done + +hb "DONE $n/$total" "ok=$ok skip=$skip fail=$fail" +echo "done: $ok converted, $skip skipped (already done), $fail failed" +[[ $fail -eq 0 ]] diff --git a/patches/matanyone-cv2-reader.patch b/patches/matanyone-cv2-reader.patch new file mode 100644 index 0000000..f1876ba --- /dev/null +++ b/patches/matanyone-cv2-reader.patch @@ -0,0 +1,24 @@ +diff --git a/matanyone/utils/inference_utils.py b/matanyone/utils/inference_utils.py +index c5bb6a0..41c4c40 100644 +--- a/matanyone/utils/inference_utils.py ++++ b/matanyone/utils/inference_utils.py +@@ -12,8 +12,17 @@ VIDEO_EXTENSIONS = ('.mp4', '.mov', '.avi', '.MP4', '.MOV', '.AVI') + def read_frame_from_videos(frame_root): + if frame_root.endswith(VIDEO_EXTENSIONS): # Video file path + video_name = os.path.basename(frame_root)[:-4] +- frames, _, info = torchvision.io.read_video(filename=frame_root, pts_unit='sec', output_format='TCHW') # RGB +- fps = info['video_fps'] ++ # torchvision.io.read_video was removed in torchvision >= 0.23 — read via cv2 ++ cap = cv2.VideoCapture(frame_root) ++ fps = cap.get(cv2.CAP_PROP_FPS) or 24 ++ frames = [] ++ while True: ++ ok, frame = cap.read() ++ if not ok: ++ break ++ frames.append(frame[..., [2, 1, 0]]) # BGR -> RGB ++ cap.release() ++ frames = torch.from_numpy(np.array(frames)).permute(0, 3, 1, 2).contiguous() # TCHW + else: + video_name = os.path.basename(frame_root) + frames = [] diff --git a/setup/setup_venvs.sh b/setup/setup_venvs.sh new file mode 100755 index 0000000..8af6bc2 --- /dev/null +++ b/setup/setup_venvs.sh @@ -0,0 +1,86 @@ +#!/bin/zsh +# VIDGOD bootstrap — venvs, repos, checkpoints, patches. Idempotent: re-run to resume. +# Run from anywhere: paths derive from this script's location. +# Heartbeat: ~/.jobs/vidgod-setup.status +export PATH=/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin +VG="$(cd "$(dirname "$0")/.." && pwd)" +JOB=vidgod-setup + +hb(){ mkdir -p ~/.jobs; echo "$(date '+%F %T') | $1 | $2" > ~/.jobs/$JOB.status; } +die(){ hb "FAILED" "$1"; echo "FATAL: $1" >&2; exit 1; } + +command -v uv >/dev/null || die "uv not installed (brew install uv)" +command -v ffmpeg >/dev/null || die "ffmpeg not installed (brew install ffmpeg)" + +hb "1/10" "creating dirs" +mkdir -p "$VG"/{venvs,tools,models,library/thumbs,work,tests} || die "mkdir" + +# ---- roto venv (torch/MPS + SAM2 + MatAnyone) ---- +hb "2/10" "roto venv (py3.12)" +[ -x "$VG/venvs/roto/bin/python" ] || uv venv "$VG/venvs/roto" --python 3.12 || die "uv venv roto" +RPY="$VG/venvs/roto/bin/python" + +hb "3/10" "torch+torchvision into roto (big download)" +uv pip install --python "$RPY" torch torchvision numpy opencv-python pillow tqdm huggingface_hub imageio imageio-ffmpeg || die "torch install" + +hb "4/10" "clone + install SAM2" +if [ ! -d "$VG/tools/sam2" ]; then + git clone --depth 1 https://github.com/facebookresearch/sam2 "$VG/tools/sam2" || die "sam2 clone" +fi +SAM2_BUILD_CUDA=0 uv pip install --python "$RPY" --no-build-isolation -e "$VG/tools/sam2" || die "sam2 install" + +hb "5/10" "clone + install MatAnyone" +if [ ! -d "$VG/tools/MatAnyone" ]; then + git clone --depth 1 https://github.com/pq-yang/MatAnyone "$VG/tools/MatAnyone" || die "matanyone clone" +fi +if [ -f "$VG/tools/MatAnyone/requirements.txt" ]; then + uv pip install --python "$RPY" -r "$VG/tools/MatAnyone/requirements.txt" || echo "WARN: some MatAnyone reqs failed, continuing" +fi +if [ -f "$VG/tools/MatAnyone/pyproject.toml" ] || [ -f "$VG/tools/MatAnyone/setup.py" ]; then + uv pip install --python "$RPY" --no-build-isolation -e "$VG/tools/MatAnyone" || echo "WARN: matanyone -e install failed (will use sys.path)" +fi + +hb "6/10" "patch MatAnyone video reader (torchvision >= 0.23 removed read_video)" +if ! grep -q "CAP_PROP_FPS" "$VG/tools/MatAnyone/matanyone/utils/inference_utils.py"; then + git -C "$VG/tools/MatAnyone" apply "$VG/patches/matanyone-cv2-reader.patch" || die "matanyone patch failed to apply" + echo "patch applied" +else + echo "patch already applied" +fi + +# ---- index venv (scene detect + whisper + beats) ---- +hb "7/10" "index venv + packages" +[ -x "$VG/venvs/index/bin/python" ] || uv venv "$VG/venvs/index" --python 3.12 || die "uv venv index" +IPY="$VG/venvs/index/bin/python" +uv pip install --python "$IPY" mlx-whisper "scenedetect[opencv]" librosa soundfile numpy pillow tqdm || die "index pkgs" + +# ---- checkpoints ---- +hb "8/10" "SAM2.1 large checkpoint (~856MB)" +CKPT="$VG/models/sam2.1_hiera_large.pt" +if [ ! -s "$CKPT" ]; then + curl -fL --retry 3 -o "$CKPT.part" https://dl.fbaipublicfiles.com/segment_anything_2/092824/sam2.1_hiera_large.pt || die "sam2 ckpt download" + mv "$CKPT.part" "$CKPT" +fi + +hb "9/10" "MatAnyone checkpoint (~135MB, github release)" +MA="$VG/models/matanyone.pth" +if [ ! -s "$MA" ]; then + curl -fL --retry 3 -o "$MA.part" https://github.com/pq-yang/MatAnyone/releases/download/v1.0.0/matanyone.pth || die "matanyone ckpt download" + mv "$MA.part" "$MA" +fi +# its inference script hard-codes pretrained_models/ relative to the repo dir +mkdir -p "$VG/tools/MatAnyone/pretrained_models" +ln -sf "$MA" "$VG/tools/MatAnyone/pretrained_models/matanyone.pth" + +hb "10/10" "smoke: torch/MPS + imports" +"$RPY" - <<'PYEOF' || die "roto smoke" +import torch, sam2 +print("torch", torch.__version__, "mps:", torch.backends.mps.is_available()) +PYEOF +"$IPY" - <<'PYEOF' || die "index smoke" +import mlx_whisper, scenedetect, librosa +print("index venv ok") +PYEOF + +hb "DONE" "venvs+repos+ckpts+patches ready" +echo "SETUP DONE — try: $VG/setup/smoke_test.sh" diff --git a/setup/smoke_test.sh b/setup/smoke_test.sh new file mode 100755 index 0000000..fc44f65 --- /dev/null +++ b/setup/smoke_test.sh @@ -0,0 +1,53 @@ +#!/bin/zsh +# VIDGOD smoke test — fast lanes only (beats, transcode, index, find). ~1-2 min. +# Add --roto to also run the full SAM2+MatAnyone pipeline on a bundled demo clip (minutes). +export PATH=/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin +VG="$(cd "$(dirname "$0")/.." && pwd)" +T="$VG/tests/smoke" +mkdir -p "$T/oldstyle" +fails=0 +check(){ if [ $1 -eq 0 ]; then echo "PASS: $2"; else echo "FAIL: $2"; fails=$((fails+1)); fi } + +echo "== 1. vg-beats (synthetic 120bpm click) ==" +"$VG/venvs/index/bin/python" - </dev/null 2>&1 +[ -s "$T/click120.beats.edl" ] && grep -q "ResolveColor" "$T/click120.beats.edl" +check $? "vg-beats produced marker EDL" + +echo "== 2. vg-transcode (fake xvid avi + wmv) ==" +ffmpeg -hide_banner -loglevel error -y -f lavfi -i "testsrc=duration=4:size=640x480:rate=25" -c:v mpeg4 -vtag xvid -qscale:v 5 "$T/oldstyle/fake_xvid.avi" +ffmpeg -hide_banner -loglevel error -y -f lavfi -i "smptebars=duration=4:size=640x480:rate=25" -f lavfi -i "sine=frequency=440:duration=4" -c:v wmv2 -c:a wmav2 "$T/oldstyle/fake_wmv.wmv" +"$VG/bin/vg-transcode" "$T/oldstyle" >/dev/null 2>&1 +[ -s "$T/oldstyle-prores/fake_xvid.mov" ] && [ -s "$T/oldstyle-prores/fake_wmv.mov" ] +check $? "vg-transcode converted both to ProRes" + +echo "== 3. vg-index + vg-find (speech clip with a hard cut) ==" +if command -v say >/dev/null; then + say -o "$T/speech.aiff" "The radical robot destroyed the shopping mall. Totally awesome dude." + ffmpeg -hide_banner -loglevel error -y -f lavfi -i "testsrc2=duration=2.5:size=640x360:rate=25" -f lavfi -i "smptebars=duration=2.5:size=640x360:rate=25" -i "$T/speech.aiff" -filter_complex "[0:v][1:v]concat=n=2:v=1[v]" -map "[v]" -map 2:a -c:v libx264 -crf 20 -c:a aac -shortest "$T/dialogue_test.mp4" + "$VG/bin/vg-index" "$T/dialogue_test.mp4" --force >/dev/null 2>&1 + "$VG/bin/vg-find" radical | grep -qi radical + check $? "vg-index + vg-find round-trip (whisper heard 'radical')" +else + echo "SKIP: no 'say' binary for speech synthesis" +fi + +if [ "${1:-}" = "--roto" ]; then + echo "== 4. vg-roto (bundled demo clip, full SAM2+MatAnyone) ==" + DEMO="$VG/tools/MatAnyone/inputs/video/test-sample3.mp4" + ffmpeg -hide_banner -loglevel error -y -t 4 -i "$DEMO" -c:v libx264 -crf 12 -an "$T/roto_in.mp4" + "$VG/bin/vg-roto" "$T/roto_in.mp4" --point 970,220 --point 1150,500 --out "$T/roto_out" + [ -s "$T/roto_out/roto_in_alpha.mov" ] + check $? "vg-roto produced ProRes 4444 alpha" +fi + +echo +if [ $fails -eq 0 ]; then echo "SMOKE TEST: all passed"; else echo "SMOKE TEST: $fails FAILED"; fi +exit $fails