GODSIGH/SPEC2.md

# GODSIGH — Wave 2 Build Spec (upgrades & crossover)

**Audience:** Claude Opus 4.8, executing autonomously in this repo.
**Author:** Claude Fable 5, 2026-07-13, after reviewing the completed Wave 1 build.
**Prerequisite:** Wave 1 (SPEC.md phases 1–8) is COMPLETE at `7da9eea` — do not re-do it. Read SPEC.md §2 (locked decisions), §7 (verification protocol), §10 (pitfalls) before starting; every rule there still binds (relative URLs, no build step, entities-vs-primitives, `[DEMO]` labeling, verify-in-browser-then-commit-per-phase). Deployment remains SPEC.md §9, gated on John — unchanged by this document.

**Companion:** `~/Documents/godstrument/GODSIGH_CROSSOVER_BRIEF.md` — a sibling brief for the godstrument repo. Phase 3 here defines a **shared cache contract** with that repo; the contract text must stay in sync.

---

## 0. Mission

Six phases, in execution order: two new real-data layers (earthquakes, wildfires), a shared OpenSky cache (fixes a real quota collision with godstrument), historical record/replay (true time-travel for the timeline), a polish pack, and one investigate-first stretch. Each phase is independently shippable; commit + push after each.

## 1. Ground truth — verified 2026-07-13 by Fable (do not re-probe)

| Feed | URL | CORS | Quirk |
|---|---|---|---|
| USGS quakes | `https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_day.geojson` | `ACAO: *` — direct browser fetch OK, no proxy | ~166 KB/day; `all_hour.geojson` also available |
| NASA EONET wildfires | `https://eonet.gsfc.nasa.gov/api/v3/events?category=wildfires&status=open&limit=500` | `ACAO: *` — direct browser fetch OK, no proxy | **Content-Type lies: `application/rss+xml` but the body is JSON.** `res.json()` still works (fetch ignores content-type) — but do NOT gate on the header. |

Also ground truth from Wave 1: OpenSky anonymous quota is ~400 credits/day per IP and **godstrument's `workers/world_planes.py` polls the same API from the same machine** — the two projects currently compete for one quota. Phase 3 fixes this.

## 2. Phase 1 — Earthquakes layer (real data, time-anchored)

New file `js/layers/quakes.js`, registered in `main.js`'s `LAYER_MODULES` (before ships). Follow the layer contract exactly (see main.js header comment).

- **Fetch** `all_day.geojson` directly (no proxy). GeoJSON features: `properties.mag` (may be null — skip), `properties.time` (ms epoch), `properties.place`, `properties.url`, `geometry.coordinates = [lon, lat, depthKm]`. Refresh every 5 min (`setTimeout` self-scheduling like aircraft.js; keep polling regardless of scrub state — quakes are historical, not live-only).
- **Entities** in a `CustomDataSource('quakes')`. For each quake:
  - `availability`: if the quake time falls inside the clock window, start at its time (so scrubbing the slider back before a quake makes it vanish — the same trick as the events layer, but with REAL events); if it's older than `ctx.start`, available for the whole window.
  - `point`: `pixelSize = 4 + mag * 2.2`, color ramps amber→red with magnitude (e.g. lerp `#ffcf50` → `#ff3b30` over mag 2..6), `outlineColor` black, `disableDepthTestDistance: Infinity`. Quakes < 1 h old pulse (CallbackProperty like events.js).
  - `label` only for **mag ≥ 4.5** (declutter), `M{mag} {place}`, with the same `translucencyByDistance` pattern as infra.js.
  - `description`: place, magnitude, depth km, UTC time, and the USGS event link.
- Dedupe by feature `id` across refreshes (update, don't duplicate). Cap ~400 entities (drop smallest magnitudes first).
- HUD row `Earthquakes (USGS)`, status like `128 quakes · max M5.6 · 24h`, warn/err states on fetch failure. Return `{ id: 'quakes' }`.

## 3. Phase 2 — Wildfires layer (real data)

New file `js/layers/fires.js`, in `LAYER_MODULES` after quakes.

- **Fetch** the EONET URL from §1 directly. Each event: `title`, `id`, `link`, `geometry` (array of dated geometries — use the **latest** entry; it may be `Point` (`coordinates [lon,lat]`) or `Polygon` (use the first ring's centroid)). Refresh every 15 min.
- **Entities** in `CustomDataSource('fires')`: a small flame-colored glyph (shared canvas billboard — draw a simple upward-teardrop/triangle in `#ff7a1a`, dark outline, ~14 px), label the event title only when zoomed in (aggressive `translucencyByDistance` — fires cluster hard in California/Australia), `description` = title, category, last-geometry date, EONET link, and an "open event" note.
- HUD row `Wildfires (NASA EONET)`, status `N active fires`. Cap 500. Return `{ id: 'fires' }`.
- No time dynamics (EONET events are slow-moving; availability games would mislead more than inform).

## 4. Phase 3 — Shared OpenSky cache (the quota fix)

**The contract (mirrored verbatim in the godstrument brief — keep in sync):**

> **OpenSky shared cache v1.** Path: `~/.cache/godverse/opensky-states.json`, overridable via env `OPENSKY_CACHE_FILE`. Content: the raw, unmodified OpenSky `/states/all` JSON response body (global, no bbox). Writer: whoever successfully fetches from upstream writes the body **atomically** (write to `<path>.tmp` in the same directory, then `os.replace`). Freshness = file mtime. Readers treat the cache as fresh if `mtime` is within **120 s**; on upstream 429/failure, readers may serve/use a stale cache rather than nothing.

Implementation in `serve.py`'s proxy handler, for the `opensky` upstream only:
1. **Serve-from-cache:** if the cache file exists and is fresh (<120 s) — and the request has no bbox query (global request) — return the cached body with `X-Godsigh-Cache: hit` instead of hitting upstream. (Bbox requests bypass the cache; the app default is global.)
2. **Write-through:** on a successful upstream fetch of the global feed, write the body to the cache atomically (`pathlib.Path(...).expanduser()`, `mkdir(parents=True, exist_ok=True)` for the dir).
3. **Stale-on-error:** on upstream 429 or exception, if a cache file exists, serve it with 200 + `X-Godsigh-Cache: stale` (the client keeps working through quota exhaustion); otherwise behave as today.
4. Keep forwarding `X-Rate-Limit-*` headers when they exist (they won't on cache hits — that's fine, the client already guards nulls).

Result: when GODSIGH and godstrument run at once, only one of them actually spends quota per 120 s window. Test: two rapid `curl 'http://127.0.0.1:8137/proxy/opensky'` calls — the second must return `X-Godsigh-Cache: hit` and be near-instant.

## 5. Phase 4 — Historical record & replay (true time-travel)

Today, scrubbing off-live hides aircraft (orbits alone are computable in the past). This phase records live snapshots so the timeline replays actual traffic. Pattern borrowed from godstrument's `recorder.py`/`timewarp.py` (read them: `~/Documents/godstrument/recorder.py`, `timewarp.py` — change-logged SQLite, decimation, compressed replay).

### 5a. `record.py` (new, repo root — dev-side daemon, NOT shipped to prod)
- Stdlib-only (like serve.py). Loop: every **180 s**, fetch `http://127.0.0.1:8137/proxy/opensky` (going through the proxy means the shared cache dedupes quota with the live app; if the dev server is down, sleep and retry — do not fetch upstream directly).
- Reduce each snapshot to essentials: `{t: epoch_s, planes: [[icao24, callsign, lon, lat, baro_alt, track, vel], ...]}` (airborne only), then `zlib`-compress and INSERT into SQLite `data/history.db` (gitignored — add `data/` to `.gitignore`), table `snapshots(ts INTEGER PRIMARY KEY, kind TEXT, body BLOB)`, WAL mode.
- **Retention:** on startup and hourly, `DELETE` snapshots older than **72 h**, then `PRAGMA wal_checkpoint`. (~70 MB steady-state.)
- **Heartbeat:** this is a long-running script — invoke the `/jobs` skill and wire it into John's heartbeat convention so "is the recorder alive?" is answerable.

### 5b. Replay endpoint in `serve.py`
- `GET history/aircraft?t=<epoch_s>` → nearest snapshot within ±10 min of `t`, decompressed, as JSON `{t, planes}`; 404 with a JSON error body if none. (Relative path — same-origin like everything else. Read the db with a short-lived sqlite3 connection per request; WAL makes concurrent read safe.)

### 5c. Frontend: aircraft.js replay mode
- When `onClockTick` reports **not live**: instead of just hiding, fetch `history/aircraft?t=` for the clock time (debounce: only refetch when the requested time moves > 60 s from the last fetched snapshot; cache the last response). On hit: rebuild billboards from the snapshot (reuse the existing build loop — factor it out to take a plane array) and **show** them; status `replay · {snapshot time} · {n} aircraft` with state `warn` (amber = replayed, not live). On 404: hide + status `no history for this time` (this is the pre-recorder behavior, and prod-without-recorder behavior — must degrade gracefully).
- When live again: discard replay state, resume the live poll loop exactly as today.
- The LIVE/SCRUBBED chip stays as-is (SCRUBBED remains true); the amber aircraft status is what signals replay.
- **Prod note:** `record.py` + `history/` are local-dev capabilities. The prod nginx has no recorder; the graceful 404 path covers it. A VPS recorder service is possible later but is OUT OF SCOPE and would be gated on John (see SPEC.md §9 doctrine).

## 6. Phase 5 — Polish pack

Four small, independent items; verify each in the browser.

1. **Military callsign highlighting** (`aircraft.js` + `config.js`): add `CONFIG.aircraft.militaryPrefixes` (a commented list — start with `RCH, REACH, LAGR, DUKE, POLO, CNV, RRR, ASCOT, GAF, BAF, IAM, PLF, HKY, NATO, CFC, MC, SAM, EVAC`) and tint matching-callsign billboards `#ff5964` regardless of altitude band. Add a second HUD row `Military filter` (default off): when on, only matching aircraft render. Status shows `N military of M`.
2. **Satellite ground track on selection** (`satellites.js`): listen to `viewer.selectedEntityChanged`; when the selected entity is one of ours, add a single reusable entity whose `polyline` is the **sub-satellite ground track** (same SampledPositionProperty samples but at height 0, over ±half period around now — precompute per sat during the build loop and stash on the entity via `entity.properties`, or recompute on demand from the satrec, whichever is cleaner), dashed, in the sat's color, width 1.5. Deselect → hide it. One track at a time.
3. **Shareable URL state** (`main.js`): serialize to `location.hash` (via `history.replaceState`, debounced 1 s): camera (`lon,lat,height,heading,pitch` at 3 decimals), mode (`d`/`p`), layer-toggle states, and clock offset-from-now in seconds (0 = live). On boot, parse the hash and apply before/instead of the default camera + mode. Keep it human-readable: `#c=53.0,25.5,2800000,0,-90&m=d&L=sat,paths,infra,events,ships,air&t=0`. Ignore malformed hashes silently.
4. **README screenshot:** add to `serve.py` a tiny localhost-only dev endpoint `POST snap` that accepts a `data:image/png;base64,...` body and writes `docs/<name>.png` (name from a `?name=` param, sanitized to `[a-z0-9-]`, always `.png`, always inside `docs/`). From the browser console (or a small dev button you remove after): render, `canvas.toDataURL('image/png')`, POST it. **Pitfall:** Cesium's context has `preserveDrawingBuffer: false` by default — call `viewer.render()` synchronously immediately before `toDataURL` in the same task, exactly as Wave 1 verification did, or you'll capture transparent black. Capture one Data-Mode hero shot (`docs/screenshot-data.png`, oblique Gulf view with everything on) and one Photo-Mode shot, wire them into README.md, commit the PNGs. serve.py is dev-only (never deployed), so the endpoint is not an exposure — but still restrict the write path as described.

## 7. Phase 6 — STRETCH (investigate-first): GPS-jamming layer

Time-boxed investigation, then build only if the data cooperates:
1. Probe gpsjam.org's daily data (John Wiseman publishes H3-hex CSVs; look for `https://gpsjam.org/data/` patterns — check robots, CORS with an Origin header, and file format). Also check `https://github.com/wiseman/gpsjam` for documented data paths.
2. If accessible: add `gpsjam` as a proxied upstream if CORS blocks; render H3 hexes via `h3-js` (CDN global, pin a version) as red-tinted translucent polygons (Cesium `PolygonHierarchy` per hex, alpha scaled by interference %). One HUD row, dated status (`GPS interference · YYYY-MM-DD`), and a description noting the data is daily-aggregated from ADS-B, per gpsjam methodology.
3. If not accessible or ambiguous licensing: **write your findings in the README roadmap and stop.** Do not scrape around an unwilling origin.

## 8. Verification & workflow

- Per phase: the SPEC.md §7 browser protocol (console clean on a single fresh load, network 200s, screenshots, HUD statuses, toggle checks) plus the phase's own checks above. `node --check` every touched JS file.
- The subpath audit must stay clean: `grep -rnE "fetch\(['\"]/|src=['\"]/" js/` → empty. New external fetches (USGS/EONET) are full `https://` URLs in `config.js` — that's fine (they're cross-origin CDN-style feeds, not same-origin paths).
- Commit per phase (`wave2 phase N: …`), push after each. End commits with `Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>`.
- Update the README per phase: layers table (quakes/fires rows), the real-vs-DEMO section (quakes/fires are REAL — say so), replay section, URL-share mention, screenshots.

## 9. Pitfalls (Wave 2 specific)

- EONET's Content-Type header lies (§1). Don't trust headers; trust the body.
- USGS `mag` can be null and `place` can be missing — guard both.
- Quake `availability` intervals: quakes happen in the past — an event newer than `ctx.stop` can't happen, but clock-window edges can; clamp intervals to `[ctx.start, ctx.stop]`.
- Atomic cache writes: `os.replace` on the same filesystem only — keep the `.tmp` beside the target, not in `/tmp`.
- sqlite from serve.py: open per-request, read-only (`file:...?mode=ro` URI) to avoid locking the recorder.
- Replay fetch loop: never let a scrub gesture fire dozens of history fetches — debounce on requested-time distance, not wall time alone.
- `selectedEntityChanged` fires with `undefined` on deselect — handle it.
- Do not touch SPEC.md §9 deployment or `deploy.sh` in this wave. If John says "deploy" mid-wave, finish the current phase, then follow SPEC.md §9 (including the resolver gotcha) — the new layers deploy as plain static files with zero extra prod config (USGS/EONET are direct; replay 404s gracefully).

## 10. Out of scope for Wave 2

Live AIS by default, oil-futures panel, Cesium ion terrain, agentic event ingestion, VPS-side recorder, accounts/auth. Roadmap only.