GODSIGH/SPEC3.md
jing bf4d1d16de Wave 3 spec for Opus: the layer registry + generic GeoJSON/imagery factories
Refactor quakes/fires onto a data-driven registry, then seed ~6 verified new
real feeds (adsb.lol military [needs proxy], NWS alerts, GDACS disasters,
Launch Library, NOAA aurora, NSW RFS bushfires) under a categorized collapsible
HUD, plus NASA GIBS time-dimensioned imagery. GDELT news is investigate-first.
CORS/shape ground truth for all 8 candidates probed and tabled in §1.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 18:22:56 +10:00

17 KiB
Raw Blame History

GODSIGH — Wave 3 Build Spec (the layer registry)

Audience: Claude Opus 4.8, executing autonomously in this repo. Author: Claude Fable 5, 2026-07-13, after reviewing the completed Wave 1 + Wave 2 builds. Prerequisite: Waves 12 are COMPLETE at 403884e. Read SPEC.md §2/§7/§10 and SPEC2.md §1/§8/§9 first; every rule there still binds (relative URLs, no build step, entities-vs-primitives, [DEMO] labeling, per-phase verify-then-commit). Deployment (SPEC.md §9) is unchanged and still gated on John.


0. Mission & the core idea

Wave 2 proved that most new layers are the same shape: fetch GeoJSON-ish data on an interval, drop styled entities on the globe, report status. quakes.js and fires.js are 90% identical boilerplate. That doesn't scale to the 15+ feeds we want.

Wave 3 makes adding a layer a data-entry task, not a coding task. Build a data-source registry + a generic GeoJSON layer factory, refactor the two Wave 2 real-data layers onto it to prove parity, then seed the registry with a batch of verified new feeds and a collapsible categorized HUD. The bespoke layers (satellites, aircraft, ships, infra, events) stay exactly as they are — they have real per-layer logic that a generic factory shouldn't swallow.

Done looks like: a js/registry.js manifest where one object literal = one working layer; quakes/fires rebuilt as registry entries with no behavior change; ~6 new real layers live (military aircraft, severe-weather alerts, global disasters, rocket launches, aurora, NSW bushfires) under a categorized, collapsible HUD; and a NASA GIBS time-dimensioned imagery option. Two feeds (GDELT news, adsb.lol proxying) are investigate-first.

1. Ground truth — verified 2026-07-13 by Fable (do not re-probe)

Probed with a browser-like Origin header from this machine:

Feed URL CORS Geometry / shape Verdict
adsb.lol military https://api.adsb.lol/v2/mil NO ACAO — needs proxy {ac:[{hex,flight,lat,lon,alt_baro,gs,track,...}]}, ~251 ac proxy it
NWS alerts (US) https://api.weather.gov/alerts/active?severity=Severe,Extreme ACAO: * GeoJSON FeatureCollection; features may have null geometry (skip those) direct
GDACS disasters https://www.gdacs.org/gdacsapi/api/events/geteventlist/MAP ACAO: * GeoJSON FeatureCollection, Point geometry direct
Launch Library 2 https://ll.thespacedevs.com/2.2.0/launch/upcoming/?limit=30&mode=list ACAO: * JSON {results:[{name,net,pad:{latitude,longitude,location}}]} — NOT GeoJSON direct, custom adapter
NOAA aurora https://services.swpc.noaa.gov/json/ovation_aurora_latest.json ACAO: * {coordinates:[[lon,lat,aurora%],...]} ~920 KB grid — NOT points direct, special layer
NSW RFS bushfires https://www.rfs.nsw.gov.au/feeds/majorIncidents.json ACAO: * GeoJSON; features carry GeometryCollection (Point + Polygon) direct
NASA GIBS https://gibs.earthdata.nasa.gov/wmts/epsg3857/best/{Layer}/default/{Time}/{TileMatrixSet}/{z}/{y}/{x}.jpg ACAO: * WMTS raster tiles, time-dimensioned direct, imagery
GDELT geo https://api.gdeltproject.org/api/v2/geo/geo?query=…&format=GeoJSON 404 on my query formats GeoJSON when the query is right, but params are finicky investigate-first

adsb.lol has no rate limit or key (community ADS-B). GIBS/NWS/GDACS/Launch Library/NOAA are all free, keyless, public.

2. Locked design decisions

  1. Registry, not inheritance. js/registry.js default-exports an array of plain-object layer specs. main.js iterates it and instantiates each via the generic factory. Bespoke layers stay as their own modules in LAYER_MODULES — the registry is additive, both lists load.
  2. Two generic factories, not one. js/layers/geojson-layer.js (entity layers from point/polygon GeoJSON) and js/layers/imagery-layer.js (WMTS/tile overlays like GIBS). A feed that's neither (aurora grid, Launch Library's non-GeoJSON JSON) either gets a small adapt() function in its registry entry that returns a normalized feature array, or stays a bespoke module — decide per feed (§5).
  3. Everything relative or full-https://. Same rule as always: same-origin paths relative (proxy/…), cross-origin feeds are full https:// URLs living only in the registry. The subpath audit (grep -rnE "fetch\(['\"]/|src=['\"]/" js/) must stay empty.
  4. No behavior regressions. After refactoring quakes/fires onto the factory, the app must look and behave identically — same counts, colors, time-anchoring, labels, statuses. Verify side-by-side against the current build.
  5. Categories + collapse. At 12+ layers a flat HUD is unusable. Group rows by category, collapsible, most new layers default OFF to keep first paint clean and quota/clutter down.
  6. [DEMO] rule still holds — but note almost everything in this wave is REAL. Only label demo data.

3. Phase 1 — the GeoJSON layer factory

New file js/layers/geojson-layer.js. Export createGeoJsonLayer(ctx, spec) returning the standard { id, onClockTick?, ... } contract object. It must express everything quakes/fires need, driven by spec:

// A registry entry consumed by createGeoJsonLayer.
{
  id: 'quakes',
  name: 'Earthquakes (USGS)',
  category: 'Earth',
  defaultOn: true,
  url: 'https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_day.geojson',
  refreshMs: 300000,
  // Turn a raw response into an array of normalized features. Default assumes
  // GeoJSON FeatureCollection; override for odd shapes.
  adapt: (json) => json.features,
  // Pull geometry → {lon, lat} from one feature (handles Point, Polygon-centroid,
  // GeometryCollection — factor the fires.js/quakes.js logic here as the default).
  // Return null to skip a feature.
  locate: (f) => ({ lon, lat }),
  // Stable id for dedupe across refreshes; return null to rebuild-wholesale each time.
  featureId: (f) => f.id,
  // Optional: availability start for time-anchoring (return a Date/ms, or null for always-on).
  timeAt: (f) => f.properties.time,
  // Visual spec per feature.
  style: (f) => ({
    color: '#ff3b30', pixelSize: 8, // OR glyph: sharedCanvas / billboard opts
    label: 'M5.2 …' | null,          // null = no label (declutter)
    labelFade: [2.0e6, 1.2e7],       // NearFarScalar near/far, optional
    pulseIf: (f) => boolean,          // recent-event pulse, optional
  }),
  description: (f) => '<table>…</table>', // InfoBox HTML
  cap: 400,                              // max entities; drop by a sort key
  sortKey: (f) => f.properties.mag,      // for the cap (keep highest)
  status: ({shown, extra}) => `${shown} quakes${extra}`, // HUD status text
}

Requirements the factory must honor (all already solved in quakes.js/fires.js — lift the logic):

  • Promise-safe polling loop (self-scheduling setTimeout, inFlight guard, keeps polling regardless of scrub since these are historical/live-record feeds).
  • Dedupe-by-id with revision detection (rebuild an entity when its content signature changes — quakes.js already does this; make it the default when featureId is provided).
  • Time-anchoring via timeAt (clamp intervals to [ctx.start, ctx.stop], exactly quakes.js's availabilityFor).
  • Cap by sortKey (drop smallest/least-important first).
  • Never trust Content-Type (EONET lies — §SPEC2). Guard null geometry, null coords, missing props everywhere.
  • Full error→status reporting (ok/warn/err), no silent failures.
  • Register its HUD row + category via the new UI API (§6).

Do not build any real feed yet — just the factory + a trivial inline test spec, verified to render.

4. Phase 2 — refactor quakes & fires onto the factory (parity proof)

Rebuild quakes and fires as registry entries (move them into registry.js), delete js/layers/quakes.js and js/layers/fires.js, and remove them from LAYER_MODULES. This is the acid test that the factory is expressive enough.

  • Quakes entry: timeAt = origin time (keeps the appears-as-slider-crosses behavior), amber→red magColor ramp in style, labels for M≥4.5, revision-aware dedupe, cap 400 by magnitude, the pulse-if-<1h.
  • Fires entry: no timeAt (wholesale rebuild), flame glyph billboard, aggressive label fade, cap 500.
  • Verify parity in the browser: counts, colors, labels, time-anchoring (scrub back → quakes vanish), statuses must match the pre-refactor build exactly. Screenshot both before/after if unsure. Commit only when identical.

5. Phase 3 — seed the registry with new real layers

Add these as registry entries (or bespoke where noted). Each gets a category, defaultOn: false unless noted, verify each renders, commit per feed or in small batches.

Category "Air":

  • Military aircraft (adsb.lol/v2/mil) — needs the proxy: add mil to serve.py's UPSTREAMS (https://api.adsb.lol/v2/mil) and an nginx block in SPEC.md §9's list (same resolver caveat — document it). Fetch proxy/mil. This is a BillboardCollection primitive like civil aircraft, NOT entities (could be hundreds) — so it may be a bespoke module reusing aircraft.js's rendering, or a factory "primitive mode." Judge which is cleaner; if bespoke, factor the shared billboard build out of aircraft.js. Tint military-red #ff5964, glyph rotated by track, click overlay with flight/hex/alt/speed. Poll ~60 s (no quota limit). This supersedes Wave 2's callsign-guess military filter — keep that filter for civil OpenSky, but real mil data is separate and better.

Category "Human":

  • Severe weather alerts (api.weather.gov/alerts/active?severity=Severe,Extreme) — GeoJSON factory. Many features have polygon geometry AND some have null geometry (skip null). Style: amber translucent polygons (PolygonHierarchy from the ring) + a point+label at the centroid; timeAt = properties.onset or sent (time-anchored). US-only — note that in the description. defaultOn: false.
  • Global disasters (gdacs.org/…/geteventlist/MAP) — GeoJSON factory, Point geometry. Icon/color by properties event type (EQ/TC/FL/VO/DR/WF) and alert level (green/orange/red). Label the event name. Global. timeAt if a timestamp is present.

Category "Space":

  • Rocket launches (Launch Library 2) — NOT GeoJSON, so adapt: (j) => j.results + locate: (r) => ({lon: r.pad.latitude, lon…}) reading pad.latitude/longitude. Time-anchor to net (the launch window) so upcoming launches appear as the slider reaches them — but note most net values are in the FUTURE beyond the +6h window, so also render them statically at their pad with a "T-…" countdown in the label/description. A rising rocket glyph. defaultOn: false.
  • Aurora oval (NOAA OVATION) — NOT points, it's a ~920 KB [[lon,lat,prob],...] grid. This is a bespoke module, not a factory layer: sample the grid (every Nth point, prob>10), render as a translucent green point cloud or a coarse polygon band near both poles, alpha ∝ probability. Refresh ~5 min. defaultOn: false. Gorgeous in Photo mode.

Category "Regional — Australia" (collapsed by default):

  • NSW RFS bushfires (rfs.nsw.gov.au/feeds/majorIncidents.json) — GeoJSON factory. Features carry GeometryCollection (Point + Polygon) — locate uses the Point; optionally draw the polygon fire-ground. Color by properties alert level (advice/watch-act/emergency). Label the incident name. timeAt from the pubDate if present. This is John's home-turf detail pack; a stub for adding VIC/QLD equivalents later.

6. Phase 4 — categorized collapsible HUD

Extend js/ui.js:

  • addLayer(id, name, defaultOn, onToggle, category) — new optional category (default 'Core'). Rows render grouped under a collapsible category header (▸/▾), with a per-category count and a category-level show/hide-all toggle. Preserve the existing flat API for the bespoke layers (they land in 'Core').
  • Categories in display order: Core, Space, Air, Sea, Earth, Human, Regional — Australia. Bespoke Wave 1 layers map to Core/Space/Air/Sea; quakes→Earth, fires→Earth (or a "Land" category — your call, keep it sensible).
  • getLayerIds/getLayerChecked/setLayerChecked must keep working (URL-state depends on them). Collapsed state is UI-only — do NOT serialize it into the hash (keep the hash stable).
  • Keep the HUD scrollable and within its existing max-height; categories collapsed-by-default for Regional and any all-off category, expanded for Core.
  • CSS: match the existing dark HUD idiom (the category header styled like a quiet section divider). Don't fight Cesium widget CSS.

7. Phase 5 — NASA GIBS imagery option

New file js/layers/imagery-layer.js + registry support for kind: 'imagery' entries. GIBS is time-dimensioned raster — a third basemap-ish skin:

  • Add a GIBS UrlTemplateImageryProvider whose {Time} is driven by the Cesium clock (the current day, YYYY-MM-DD; MODIS true-color is daily). Start with MODIS_Terra_CorrectedReflectance_TrueColor (GoogleMapsCompatible_Level9, .jpg), and VIIRS_SNPP_Thermal_Anomalies_375m_All as a fire overlay option.
  • This is NOT a HUD entity layer — it's an imagery layer with an opacity slider, sitting above the Photo basemap. A small "Imagery" section in the HUD (or a mode addition): off by default; when on, add the layer to viewer.imageryLayers with alpha ~0.85.
  • When the clock is scrubbed to a past day, refetch tiles for that date (GIBS has years of history) — a genuine "satellite imagery time machine." Debounce on day-change, not every tick.
  • Attribution: "NASA EOSDIS GIBS" — add to the credits.
  • If wiring the clock→time dimension cleanly proves fiddly, ship it fixed to "yesterday" (today's mosaic is often incomplete) with a note, rather than half-working scrubbing.

8. Phase 6 — investigate-first: GDELT geocoded news

Time-boxed, same discipline as the Wave 2 GPS-jam stretch:

  1. GDELT's GEO 2.0 API returned 404 on my ?query=protest&format=GeoJSON attempts — the query grammar is particular. Investigate the correct params (it wants a specific query= DSL and format=GeoJSON; check https://blog.gdeltproject.org GEO 2.0 docs). Confirm a working URL that returns GeoJSON points with ACAO.
  2. If it works: a GeoJSON-factory layer, category "Human", defaultOn: false, points at article locations, label = the location name, description links the source articles, timeAt from the article time if present. Note GDELT's terms/attribution.
  3. If the query grammar stays ambiguous or CORS blocks it: document findings in the README roadmap and stop. Don't scrape around it.

9. Verification & workflow

  • Per phase: SPEC.md §7 protocol (clean console on fresh load, network 200s, screenshots, HUD statuses, toggle checks) + the phase's own checks. node --check every touched JS file. Subpath audit stays empty.
  • Parity is the headline check for Phase 2 — the refactor must be behavior-invisible.
  • The preview tab backgrounds itself (document.hidden), pausing live polls and rAF — Waves 12 documented the workarounds (drive onClockTick directly; viewer.render() before toDataURL; override document.hidden to force a poll). Reuse them.
  • Commit per phase (wave3 phase N: …), push after each. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>.
  • Update README per phase: the layers table grows a lot — consider grouping it by category to match the HUD. Mark which feeds are REAL (nearly all) vs DEMO (ships/events only). Add attributions for every new source (adsb.lol, NWS, GDACS, Launch Library/The Space Devs, NOAA SWPC, NSW RFS, NASA GIBS).
  • Run an adversarial-review workflow over the new code before the final commit — the Waves 12 recipe (reviewers by dimension → verify each finding → apply confirmed). Focus dimensions: factory correctness/expressiveness, parity of the refactored layers, CORS/subpath safety, HUD state + URL-state interaction, quota/perf (poll cadences, the 920 KB aurora grid, hundreds of mil billboards).

10. Deployment note (for whenever John runs SPEC.md §9)

  • New same-origin proxy upstream: milhttps://api.adsb.lol/v2/mil. Add the nginx location block alongside opensky/celestrak, with the resolver directive (the §9 gotcha — a variable in proxy_pass forces request-time DNS; no resolver ⇒ 502 while nginx -t passes). All other Wave 3 feeds are direct https:// and need zero prod config.
  • Everything else deploys as plain static files. The registry is just JS.

11. Out of scope for Wave 3

GPS-jam (still gated on licensing per Wave 2), oil-futures panel, Cesium ion terrain, VPS-side recorder, accounts/auth, live AIS by default, VIC/QLD/other regional packs beyond the NSW stub. Roadmap only.