Lane E round 6 (E1): publish.py merges (not clobbers) + provenance-drift QA gate

- publish.py: _published.json now MERGE (read existing → union → sorted write), never
  replace — fixes the R5 --only clobber (27→1). Proven: a real no-op re-publish kept it
  at 27. Added --verify + post-publish drift-check vs the live depot (/api/list procity_*).
- validate_manifest.py: new provenance-drift gate — every manifest procity_* GLB must be
  in _published.json or QA fails. Proven: simulated clobber (→1) exits 1 w/ per-file errors.
- Record verified: 27-entry _published.json is byte-exact vs the live depot.
- qa.sh --strict GREEN (5 gates incl. F1 harness).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

E-progress.md

@@ -4,6 +4,24 @@
 Reviewed against `docs/LANES/LANE_E_ASSETS.md` + `docs/LANES/ROUND3_INSTRUCTIONS.md`. Nothing in
 other lanes' files was touched.*
 
+## Round 6 (Fable) — non-destructive pipeline + on-call
+
+- **E1 — publish.py non-destructive + provenance-drift gate: DONE + committed.**
+  - `publish.py` now **merges** into `_published.json` (read existing → union → sorted write),
+    never replaces — fixes the R5 `--only` clobber (record dropped 27→1). **Proven:** a real no-op
+    re-publish of `record_crate` kept the record at **27 (was 27)**, not 1.
+  - Post-publish + a new `--verify` mode **drift-check** the record against the live depot
+    (`/api/list` filtered to `procity_*`) and warn on either direction. `--verify` confirms 27 == 27.
+  - `validate_manifest.py` gains a **provenance-drift gate**: every manifest `procity_*` GLB must be
+    in `_published.json`, else QA fails. **Proven:** simulating the R5 clobber (record→1) makes the
+    validator exit 1 with per-file drift errors; restored → green. So the clobber class fails loudly.
+  - **Record verified**: `_published.json` (27) is byte-exact vs the live depot — Fable's R5
+    regeneration was correct, no fix needed.
+  - `qa.sh --strict` GREEN (5 gates incl. F1's new v2-flags harness).
+- **On-call for B/C** (task 3): available for bus_shelter/bin scale-origin tweaks (B) and
+  glass_case/magazine_rack in-room issues (C). Broken-prop fal re-gen pre-authorized ≤2 attempts
+  (decision #5) — none needed yet. No speculative generation.
+
 ## Round 5 (Fable) — hero-prop bake + fal fallbacks + depot growth
 
 - **E1 — hero-prop bake-to-lowpoly: DONE + committed.** New `pipeline/bake_lowpoly.py` (Blender

docs/LANES/LANE_E_NOTES.md

@@ -1,5 +1,20 @@
 # LANE E — cross-lane notes
 
+## Round 6
+
+**E1 done** — `publish.py` now merges into `_published.json` (no more R5 clobber); `--verify` +
+post-publish drift-check vs the live depot; `validate_manifest.py` fails if a manifest GLB isn't in
+`_published.json`. Record verified 27==depot. All 27 procity GLBs stay live + recorded.
+
+**→ Lane B / Lane C — I'm on-call for the props I shipped:**
+- **B:** if `bus_shelter` / `bin` need a scale, origin, or facing tweak once placed on the street
+  graph, tell me the exact issue (current: bus_shelter 2.33×1.41 m / 2.4 m, bin 0.57×0.68 m / 1.0 m,
+  both base-origin minY=0). I can re-normalize + re-publish same-name (idempotent, merge-safe now).
+- **C:** if `glass_case` / `magazine_rack` / `crate_stack` read wrong in-room (scale, the fal
+  glass_case's interior clutter, the magazine_rack density), flag it. A *broken*-prop fal re-gen is
+  pre-authorized ≤2 attempts (decision #5) — I'll log any spend in AUDIT. Footprints in the manifest;
+  all base-origin. Mapping guidance for the 3 orphans is in the Round-5 → Lane C section below.
+
 ## Round 5
 
 ### → Lane C: hero props BAKED to ≤8k tris — re-measure the record interior

pipeline/_published.json

@@ -26,4 +26,4 @@
   "procity_street_novelty_record_01.glb",
   "procity_street_park_bench_01.glb",
   "procity_street_streetlight_01.glb"
-]
+]

pipeline/publish.py

@@ -60,6 +60,12 @@ def post(path, body, ctype):
 
 
 def main():
+    if "--verify" in sys.argv:              # drift-check the record vs the live depot, no publish
+        recpath = os.path.join(ROOT, "pipeline", "_published.json")
+        record = sorted(set(json.load(open(recpath)))) if os.path.exists(recpath) else []
+        print(f"_published.json: {len(record)} entries")
+        drift_check(record)
+        return
     only = sys.argv[sys.argv.index("--only") + 1] if "--only" in sys.argv else ""
     # Passwordless tailnet path (round-3 decision #4): GOD3_DEPOT overridden to an allowlisted
     # tailnet endpoint (e.g. http://100.94.195.115:8788) trusts this box — no cookie needed.
@@ -103,8 +109,49 @@ def main():
         except Exception as e:
             print(f"  ERR {name:38} {e}")
     if not DRY:
-        json.dump(published, open(os.path.join(ROOT, "pipeline", "_published.json"), "w"), indent=2)
-        print(f"\npublished {len(published)}/{len(glbs)} → depot. verify: curl -s {DEPOT}/api/list")
+        # MERGE, never replace (decision #3): _published.json is the provenance record of everything
+        # ever pushed — a --only run must not clobber it to just this run's files (the R5 bug).
+        recpath = os.path.join(ROOT, "pipeline", "_published.json")
+        existing = set()
+        if os.path.exists(recpath):
+            try:
+                existing = set(json.load(open(recpath)))
+            except Exception:
+                pass
+        merged = sorted(existing | set(published))
+        json.dump(merged, open(recpath, "w"), indent=2)
+        print(f"\npublished {len(published)}/{len(glbs)} this run → "
+              f"_published.json now {len(merged)} (was {len(existing)}).")
+        drift_check(merged)
+
+
+def drift_check(record):
+    """After publishing, compare _published.json against the live depot procity_* list; warn on
+    drift (a published file missing from the record, or a recorded file absent from the depot)."""
+    try:
+        live = fetch_depot_procity()
+    except Exception as e:
+        print(f"  (drift check skipped — depot list unreachable: {e})")
+        return
+    rec = set(record)
+    missing_from_record = live - rec        # live on depot but not recorded → record is stale
+    missing_from_depot = rec - live          # recorded but not live → deleted/never landed
+    if missing_from_record:
+        print(f"  DRIFT: {len(missing_from_record)} on depot not in _published.json: "
+              f"{sorted(missing_from_record)[:5]}")
+    if missing_from_depot:
+        print(f"  DRIFT: {len(missing_from_depot)} in _published.json not live on depot: "
+              f"{sorted(missing_from_depot)[:5]}")
+    if not missing_from_record and not missing_from_depot:
+        print(f"  ✓ _published.json matches the live depot ({len(live)} procity assets).")
+
+
+def fetch_depot_procity():
+    import urllib.request as _u
+    with _u.urlopen(DEPOT + "/api/list", timeout=15) as r:
+        d = json.loads(r.read().decode("utf-8"))
+    assets = d.get("assets", d) if isinstance(d, dict) else d
+    return {a["file"] for a in assets if str(a.get("file", "")).startswith("procity_")}
 
 
 if __name__ == "__main__":

pipeline/validate_manifest.py

@@ -109,11 +109,25 @@ def main():
 
     # fittings + furniture GLBs
     n_glb = 0
+    manifest_files = set()
     for grp in ("fittings", "furniture"):
         for entry in m.get(grp, {}).values():
             check_glb(entry, depot)
+            manifest_files.add(entry["file"])
             n_glb += 1
 
+    # provenance-drift gate: every manifest depot GLB must be recorded in _published.json, so a
+    # clobbered/stale provenance record (the R5 bug) fails QA loudly instead of hiding.
+    recpath = os.path.join(ROOT, "pipeline", "_published.json")
+    try:
+        record = set(json.load(open(recpath)))
+    except Exception as e:
+        err(f"_published.json unreadable: {e}")
+        record = set()
+    for f in sorted(manifest_files):
+        if f.startswith("procity_") and f not in record:
+            err(f"manifest GLB not in _published.json (provenance drift): {f}")
+
     print(f"manifest v{m.get('version')} — facades {len(facade)}, "
           f"skins {sum(len(v) if isinstance(v, list) else (len(v) if isinstance(v, dict) else 0) for v in sk.values())} groups, "
           f"GLBs {n_glb}")