GODSIGH/SPEC2.md

GODSIGH — Wave 2 Build Spec (upgrades & crossover)

Audience: Claude Opus 4.8, executing autonomously in this repo. Author: Claude Fable 5, 2026-07-13, after reviewing the completed Wave 1 build. Prerequisite: Wave 1 (SPEC.md phases 1–8) is COMPLETE at 7da9eea — do not re-do it. Read SPEC.md §2 (locked decisions), §7 (verification protocol), §10 (pitfalls) before starting; every rule there still binds (relative URLs, no build step, entities-vs-primitives, [DEMO] labeling, verify-in-browser-then-commit-per-phase). Deployment remains SPEC.md §9, gated on John — unchanged by this document.

Companion: ~/Documents/godstrument/GODSIGH_CROSSOVER_BRIEF.md — a sibling brief for the godstrument repo. Phase 3 here defines a shared cache contract with that repo; the contract text must stay in sync.

---

0. Mission

Six phases, in execution order: two new real-data layers (earthquakes, wildfires), a shared OpenSky cache (fixes a real quota collision with godstrument), historical record/replay (true time-travel for the timeline), a polish pack, and one investigate-first stretch. Each phase is independently shippable; commit + push after each.

1. Ground truth — verified 2026-07-13 by Fable (do not re-probe)

Feed	URL	CORS	Quirk
USGS quakes	https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_day.geojson	ACAO: * — direct browser fetch OK, no proxy	~166 KB/day; all_hour.geojson also available
NASA EONET wildfires	https://eonet.gsfc.nasa.gov/api/v3/events?category=wildfires&status=open&limit=500	ACAO: * — direct browser fetch OK, no proxy	Content-Type lies: application/rss+xml but the body is JSON. res.json() still works (fetch ignores content-type) — but do NOT gate on the header.

Also ground truth from Wave 1: OpenSky anonymous quota is ~400 credits/day per IP and godstrument's workers/world_planes.py polls the same API from the same machine — the two projects currently compete for one quota. Phase 3 fixes this.

2. Phase 1 — Earthquakes layer (real data, time-anchored)

New file js/layers/quakes.js, registered in main.js's LAYER_MODULES (before ships). Follow the layer contract exactly (see main.js header comment).

• Fetch all_day.geojson directly (no proxy). GeoJSON features: properties.mag (may be null — skip), properties.time (ms epoch), properties.place, properties.url, geometry.coordinates = [lon, lat, depthKm]. Refresh every 5 min (setTimeout self-scheduling like aircraft.js; keep polling regardless of scrub state — quakes are historical, not live-only).
• Entities in a CustomDataSource('quakes'). For each quake:
  • availability: if the quake time falls inside the clock window, start at its time (so scrubbing the slider back before a quake makes it vanish — the same trick as the events layer, but with REAL events); if it's older than ctx.start, available for the whole window.
  • point: pixelSize = 4 + mag * 2.2, color ramps amber→red with magnitude (e.g. lerp #ffcf50 → #ff3b30 over mag 2..6), outlineColor black, disableDepthTestDistance: Infinity. Quakes < 1 h old pulse (CallbackProperty like events.js).
  • label only for mag ≥ 4.5 (declutter), M{mag} {place}, with the same translucencyByDistance pattern as infra.js.
  • description: place, magnitude, depth km, UTC time, and the USGS event link.
• Dedupe by feature id across refreshes (update, don't duplicate). Cap ~400 entities (drop smallest magnitudes first).
• HUD row Earthquakes (USGS), status like 128 quakes · max M5.6 · 24h, warn/err states on fetch failure. Return { id: 'quakes' }.

3. Phase 2 — Wildfires layer (real data)

New file js/layers/fires.js, in LAYER_MODULES after quakes.

• Fetch the EONET URL from §1 directly. Each event: title, id, link, geometry (array of dated geometries — use the latest entry; it may be Point (coordinates [lon,lat]) or Polygon (use the first ring's centroid)). Refresh every 15 min.
• Entities in CustomDataSource('fires'): a small flame-colored glyph (shared canvas billboard — draw a simple upward-teardrop/triangle in #ff7a1a, dark outline, ~14 px), label the event title only when zoomed in (aggressive translucencyByDistance — fires cluster hard in California/Australia), description = title, category, last-geometry date, EONET link, and an "open event" note.
• HUD row Wildfires (NASA EONET), status N active fires. Cap 500. Return { id: 'fires' }.
• No time dynamics (EONET events are slow-moving; availability games would mislead more than inform).

4. Phase 3 — Shared OpenSky cache (the quota fix)

The contract (mirrored verbatim in the godstrument brief — keep in sync):

OpenSky shared cache v1. Path: ~/.cache/godverse/opensky-states.json, overridable via env OPENSKY_CACHE_FILE. Content: the raw, unmodified OpenSky /states/all JSON response body (global, no bbox). Writer: whoever successfully fetches from upstream writes the body atomically (write to <path>.tmp in the same directory, then os.replace). Freshness = file mtime. Readers treat the cache as fresh if mtime is within 120 s; on upstream 429/failure, readers may serve/use a stale cache rather than nothing.

Implementation in serve.py's proxy handler, for the opensky upstream only:

1. Serve-from-cache: if the cache file exists and is fresh (<120 s) — and the request has no bbox query (global request) — return the cached body with X-Godsigh-Cache: hit instead of hitting upstream. (Bbox requests bypass the cache; the app default is global.)
2. Write-through: on a successful upstream fetch of the global feed, write the body to the cache atomically (pathlib.Path(...).expanduser(), mkdir(parents=True, exist_ok=True) for the dir).
3. Stale-on-error: on upstream 429 or exception, if a cache file exists, serve it with 200 + X-Godsigh-Cache: stale (the client keeps working through quota exhaustion); otherwise behave as today.
4. Keep forwarding X-Rate-Limit-* headers when they exist (they won't on cache hits — that's fine, the client already guards nulls).

Result: when GODSIGH and godstrument run at once, only one of them actually spends quota per 120 s window. Test: two rapid curl 'http://127.0.0.1:8137/proxy/opensky' calls — the second must return X-Godsigh-Cache: hit and be near-instant.

5. Phase 4 — Historical record & replay (true time-travel)

Today, scrubbing off-live hides aircraft (orbits alone are computable in the past). This phase records live snapshots so the timeline replays actual traffic. Pattern borrowed from godstrument's recorder.py/timewarp.py (read them: ~/Documents/godstrument/recorder.py, timewarp.py — change-logged SQLite, decimation, compressed replay).

5a. record.py (new, repo root — dev-side daemon, NOT shipped to prod)

• Stdlib-only (like serve.py). Loop: every 180 s, fetch http://127.0.0.1:8137/proxy/opensky (going through the proxy means the shared cache dedupes quota with the live app; if the dev server is down, sleep and retry — do not fetch upstream directly).
• Reduce each snapshot to essentials: {t: epoch_s, planes: [[icao24, callsign, lon, lat, baro_alt, track, vel], ...]} (airborne only), then zlib-compress and INSERT into SQLite data/history.db (gitignored — add data/ to .gitignore), table snapshots(ts INTEGER PRIMARY KEY, kind TEXT, body BLOB), WAL mode.
• Retention: on startup and hourly, DELETE snapshots older than 72 h, then PRAGMA wal_checkpoint. (~70 MB steady-state.)
• Heartbeat: this is a long-running script — invoke the /jobs skill and wire it into John's heartbeat convention so "is the recorder alive?" is answerable.

5b. Replay endpoint in serve.py

• GET history/aircraft?t=<epoch_s> → nearest snapshot within ±10 min of t, decompressed, as JSON {t, planes}; 404 with a JSON error body if none. (Relative path — same-origin like everything else. Read the db with a short-lived sqlite3 connection per request; WAL makes concurrent read safe.)

5c. Frontend: aircraft.js replay mode

• When onClockTick reports not live: instead of just hiding, fetch history/aircraft?t= for the clock time (debounce: only refetch when the requested time moves > 60 s from the last fetched snapshot; cache the last response). On hit: rebuild billboards from the snapshot (reuse the existing build loop — factor it out to take a plane array) and show them; status replay · {snapshot time} · {n} aircraft with state warn (amber = replayed, not live). On 404: hide + status no history for this time (this is the pre-recorder behavior, and prod-without-recorder behavior — must degrade gracefully).
• When live again: discard replay state, resume the live poll loop exactly as today.
• The LIVE/SCRUBBED chip stays as-is (SCRUBBED remains true); the amber aircraft status is what signals replay.
• Prod note: record.py + history/ are local-dev capabilities. The prod nginx has no recorder; the graceful 404 path covers it. A VPS recorder service is possible later but is OUT OF SCOPE and would be gated on John (see SPEC.md §9 doctrine).

6. Phase 5 — Polish pack

Four small, independent items; verify each in the browser.

1. Military callsign highlighting (aircraft.js + config.js): add CONFIG.aircraft.militaryPrefixes (a commented list — start with RCH, REACH, LAGR, DUKE, POLO, CNV, RRR, ASCOT, GAF, BAF, IAM, PLF, HKY, NATO, CFC, MC, SAM, EVAC) and tint matching-callsign billboards #ff5964 regardless of altitude band. Add a second HUD row Military filter (default off): when on, only matching aircraft render. Status shows N military of M.
2. Satellite ground track on selection (satellites.js): listen to viewer.selectedEntityChanged; when the selected entity is one of ours, add a single reusable entity whose polyline is the sub-satellite ground track (same SampledPositionProperty samples but at height 0, over ±half period around now — precompute per sat during the build loop and stash on the entity via entity.properties, or recompute on demand from the satrec, whichever is cleaner), dashed, in the sat's color, width 1.5. Deselect → hide it. One track at a time.
3. Shareable URL state (main.js): serialize to location.hash (via history.replaceState, debounced 1 s): camera (lon,lat,height,heading,pitch at 3 decimals), mode (d/p), layer-toggle states, and clock offset-from-now in seconds (0 = live). On boot, parse the hash and apply before/instead of the default camera + mode. Keep it human-readable: #c=53.0,25.5,2800000,0,-90&m=d&L=sat,paths,infra,events,ships,air&t=0. Ignore malformed hashes silently.
4. README screenshot: add to serve.py a tiny localhost-only dev endpoint POST snap that accepts a data:image/png;base64,... body and writes docs/<name>.png (name from a ?name= param, sanitized to [a-z0-9-], always .png, always inside docs/). From the browser console (or a small dev button you remove after): render, canvas.toDataURL('image/png'), POST it. Pitfall: Cesium's context has preserveDrawingBuffer: false by default — call viewer.render() synchronously immediately before toDataURL in the same task, exactly as Wave 1 verification did, or you'll capture transparent black. Capture one Data-Mode hero shot (docs/screenshot-data.png, oblique Gulf view with everything on) and one Photo-Mode shot, wire them into README.md, commit the PNGs. serve.py is dev-only (never deployed), so the endpoint is not an exposure — but still restrict the write path as described.

7. Phase 6 — STRETCH (investigate-first): GPS-jamming layer

Time-boxed investigation, then build only if the data cooperates:

1. Probe gpsjam.org's daily data (John Wiseman publishes H3-hex CSVs; look for https://gpsjam.org/data/ patterns — check robots, CORS with an Origin header, and file format). Also check https://github.com/wiseman/gpsjam for documented data paths.
2. If accessible: add gpsjam as a proxied upstream if CORS blocks; render H3 hexes via h3-js (CDN global, pin a version) as red-tinted translucent polygons (Cesium PolygonHierarchy per hex, alpha scaled by interference %). One HUD row, dated status (GPS interference · YYYY-MM-DD), and a description noting the data is daily-aggregated from ADS-B, per gpsjam methodology.
3. If not accessible or ambiguous licensing: write your findings in the README roadmap and stop. Do not scrape around an unwilling origin.

8. Verification & workflow

• Per phase: the SPEC.md §7 browser protocol (console clean on a single fresh load, network 200s, screenshots, HUD statuses, toggle checks) plus the phase's own checks above. node --check every touched JS file.
• The subpath audit must stay clean: grep -rnE "fetch\(['\"]/|src=['\"]/" js/ → empty. New external fetches (USGS/EONET) are full https:// URLs in config.js — that's fine (they're cross-origin CDN-style feeds, not same-origin paths).
• Commit per phase (wave2 phase N: …), push after each. End commits with Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>.
• Update the README per phase: layers table (quakes/fires rows), the real-vs-DEMO section (quakes/fires are REAL — say so), replay section, URL-share mention, screenshots.

9. Pitfalls (Wave 2 specific)

• EONET's Content-Type header lies (§1). Don't trust headers; trust the body.
• USGS mag can be null and place can be missing — guard both.
• Quake availability intervals: quakes happen in the past — an event newer than ctx.stop can't happen, but clock-window edges can; clamp intervals to [ctx.start, ctx.stop].
• Atomic cache writes: os.replace on the same filesystem only — keep the .tmp beside the target, not in /tmp.
• sqlite from serve.py: open per-request, read-only (file:...?mode=ro URI) to avoid locking the recorder.
• Replay fetch loop: never let a scrub gesture fire dozens of history fetches — debounce on requested-time distance, not wall time alone.
• selectedEntityChanged fires with undefined on deselect — handle it.
• Do not touch SPEC.md §9 deployment or deploy.sh in this wave. If John says "deploy" mid-wave, finish the current phase, then follow SPEC.md §9 (including the resolver gotcha) — the new layers deploy as plain static files with zero extra prod config (USGS/EONET are direct; replay 404s gracefully).

10. Out of scope for Wave 2

Live AIS by default, oil-futures panel, Cesium ion terrain, agentic event ingestion, VPS-side recorder, accounts/auth. Roadmap only.