PROCITY/docs/LANES/LANE_D_NOTES.md
m3ultra a5e4b64a9d Lane D round-4: in-shell verify (D1) + decimated-ped validation (D2) + impostor tone-map fix (D3)
D1: peds+keepers upgrade placeholder->rig in the shell; 12 enter/exit shops leak-free
(renderer.info.memory back to exact baseline); determinism holds across reload; ?noassets
fully placeholder with zero ped fetches.

D2 (gate-3 critical path): validated Lane E's decimated fleet — all 19 peds <=3k tris
(1564-2805, avg 2450), rigs bind+animate (skinning/skeleton preserved), 0 broken meshes under
eviction churn, silhouettes+identity intact, atlas bakes clean. Near fleet ~59.6k tris at the
24-cap (was ~1.2M) — F clear to re-measure gate 3.

D3: fixed a real bug found only in-shell — the impostor self-tone-mapped (toneMapped:false +
in-shader ACES), correct on the direct-canvas test page but DOUBLE-tone-mapped under the shell's
EffectComposer/OutputPass. Reworked to a standard toneMapped:true material outputting linear via
three's own tonemapping/colorspace chunks, so it matches the near rigs on both render paths.
setExposure is now a no-op (exposure is global). Verified noon+night in-shell.

Only web/js/citizens/impostor.js changed (+ NOTES/progress/R4 shots).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 18:56:28 +10:00

18 KiB
Raw Blame History

LANE D — NOTES (measured budgets + clip wishlist)

Written by PROCITY-D, 2026-07-14. Standalone verification via web/citizens_test.html. Reference stack ported from 90sDJsim/web/world/index.html ~405520 (loadRig/spawnRig/_canon/ _rotOnly/head-bone normalize/upgradeStreetPeople). Measurements on the M3 Ultra dev box.


ROUND 4 — in-shell verify + decimation validation (→ Fable, → Lane F)

All three R4 tasks done. tools/qa.sh --strict GREEN. Sign-off for Lane F below.

D1 — in-shell rig/keeper verification (web/index.html)

Verified in the real game (not the test page), via window.PROCITY:

  • Placeholder→rig upgrade fires for both peds (citizens.mode'rig', 19 fleet, pool active) and keepers (keeper spawns at the counter keeperStand pose as a rig).
  • Leak-free: baseline renderer.info.memory = {geometries 213, textures 198}; after 12 enter/exit shop cycles it returned to exactly 213/198 (Δ geom 0, Δ tex 0). Interiors peak ~303 geoms inside, drop back every exit. Confirms the R-review dispose fix (free the keeper's Skeleton only, never the fleet-shared geo/mats) holds in-shell alongside Lane C's interior dispose.
  • Determinism in-shell: seed 20261990 → byte-identical identity signature across a full reload.
  • ?noassets=1: fleet===null, mode stays placeholder, peds + keeper are placeholder actors, zero ped GLB fetches — town fully playable asset-free.

D2 — decimated ped validation (critical path — E1 landed 518678d) SIGN-OFF FOR F

Validated Lane E's decimated fleet (web/models/peds/, ref lane-e/round4-peds):

  • Per-ped tris: 15642805, avg 2450 — all ≤3k (matches E's _peds_decim.json).
  • Rigs bind + animate: skinning preserved, the shared walk.glb/idle.glb retarget unchanged (skeleton/65 joints intact), near actors animate, mixer 0.10.3 ms.
  • No skinning explosions: 24-teleport pool-eviction churn → 0 broken meshes; silhouettes read at street distance; identity variety intact (verified visually — kid/luchador/hi-vis/elder/ business all distinct, no melted hands/faces).
  • Impostor atlas bakes clean from the decimated rigs; determinism + leak checks re-pass.
  • Gate-3 number for Lane F: at the 24-rig near cap the fleet now contributes ≈59.6k tris (was ~1.2M). Test-page whole-view total 88.9k tris (was ~1.5M). In-shell sample near rig 2729 tris, keeper 2073 tris (was 41k). F is clear to re-measure gate 3 — the fleet leaves ~140k for the town, comfortably ≤200k. (The R1 "peds too heavy" finding is now resolved by E's decimation.)

D3 — exposure sync at dusk/night — FOUND + FIXED A REAL BUG

Verifying the shell exposed a genuine rendering bug (only correct on my direct-canvas test page): the impostor material was toneMapped:false and applied ACES in-shader, but the shell renders through an EffectComposer (RenderPass → bloom → OutputPass). three only tone-maps materials when rendering to the canvas (three.module.js:69216931, currentRenderTarget === null); into a composer target, materials output linear and OutputPass applies ACES once. So my self-tone-mapping impostors were double-tone-mapped in the actual game (too dark), matching the rigs only on the test page. Fix (impostor.js): the impostor is now a normal toneMapped:true material — it outputs linear and uses three's own <tonemapping_fragment> + <colorspace_fragment> call-sites (the *_pars_* are auto-injected — including them ourselves double-defines RRTAndODTFit and fails to compile), so it tone-maps identically to the near rigs on both paths: canvas → ACES per-material; composer → linear → OutputPass ACES. Verified: near rig + mid impostor match brightness at noon AND the scene reads correctly at night 22:00 in the shell (docs/shots/laneD/r4_shell_*.jpg), and the test-page canvas path still matches.

  • Note for Lane F: citizens.setExposure() is now a no-op (exposure is global via renderer.toneMappingExposure, which both three and OutputPass read). The call at web/index.html:245 is harmless — keep or drop it, your call.

(Round-1/3 notes below remain valid; the tri-budget finding #1 is superseded by D2's decimation.)

What shipped

file role
web/js/citizens/rigs.js ported rig stack: fleet loader, _canon/canonRig/_rotOnly, buildFigure (head-bone height-normalise + feet plant), spawnRig (single clip), makeActor (walk↔idle crossfade), pickRig
web/js/citizens/placeholder.js seeded low-poly box humanoid (POLY lock) — walks/idles, planted at y=0, hot-swaps to a rig
web/js/citizens/impostor.js 4-yaw sprite-atlas baker + instanced billboard layer (mid tier, 1 draw call)
web/js/citizens/sim.js deterministic roster, footpath lanes, near/mid/far LOD, rig pool, staggered mixer budget, time-of-day density
web/js/citizens/keepers.js one keeper per shop at the counter slot, idle + greet head/body-turn
web/citizens_test.html standalone harness: fixture 3×3 street graph, sliders, tier debug, determinism check
web/models/peds/* 19 rigged GLBs + walk.glb + idle.glb, byte-identical copies from 90sDJsim (checksummed)

The fleet (copied, never edited)

19 rigged peds (17 normal + 2 comical) + 2 clip-only GLBs, ~37 MB total, copied from johnking@100.91.239.7:~/Documents/90sDJsim/web/world/models/peds/ (also present locally at ~/Documents/90sDJsim/…). SHA-verified byte-identical (house law: canonical source stays upstream).

  • Skeletons canonicalise cleanly: mixamorig1…mixamorig…, so one shared walk clip drives all 19 and one shared idle clip too. Verified in-scene — a single walk.glb animates every ped.
  • walk.glb/idle.glb position tracks + Hips.quaternion are stripped (_rotOnly) — without this the different-scale root track inflates peds to giants. Confirmed: no giants/ants across all 19.

Measured budgets (M3 Ultra, citizens_test.html)

Preview note: the in-app browser reports visibilityState:'hidden', which throttles requestAnimationFrame, so the on-screen fps counter reads low (4784). True cost was measured by timing 140 manual update()+render() frames.

200 citizens, midday (the acceptance fixture):

metric measured CITY_SPEC / Lane-D budget verdict
near (rigged) actives 24 (hard cap) ≤ 24
mid (impostor) ~122
far (culled) ~54
mixer update / frame 0.4 ms max, ~0.2 ms avg < 2 ms comfortably
sim logic / frame 0.24 ms avg 4 ms build budget
full frame CPU (update+render) ~2.6 ms → ~380 fps headroom 60 fps (16.6 ms) huge margin
mid tier draw calls 1 (69 instances, 138 tris) 1 per atlas
impostor atlas texture 2048×640 (16×5 cells, 4 yaws × 19 peds) ≤ 2048, < 512 MB total (~5 MB)
determinism live roster ≡ seed recompute, stable while walking same seed → same crowd

Time-of-day density (200 base): 00:00 → 12 active, 06:00 → 70, 12:00 → 200, 15:00 → 150, 21:00 → 36. Curve in sim.js DAY_CURVE.

⚠️ Findings for Lane E / Lane F (need a decision)

  1. The inherited peds are ~49.7k tris each — too heavy for the CITY_SPEC 200k-tri "typical view". 24 near rigs ≈ 1.2M tris; a realistic 16-rig street view already measured 1.02M tris. Framerate holds on M-series (GPU eats it), but this leaves nothing for Lane B buildings. Recommendation: decimate the ped fleet to ~58k tris in the asset pipeline (Lane E) — at 6k/ped the near cap costs ~144k tris, back within budget. The rig stack is indifferent to poly count; this is purely an asset job. Until then, NEAR_MAX (sim.js) is the throttle.

  2. Draw calls scale with near rigs (~6 calls/ped — each GLB has several sub-meshes/materials). 16 rigs ≈ 144 calls; 24 rigs ≈ ~190. Under the 300-call street budget today, but tight once Lane B adds shells. Recommendation: merge each ped's sub-meshes by material on load (BufferGeometryUtils.mergeGeometries) → ~12 calls/ped. Cheap win, deferred (not blocking).

  3. Peds are metallic-PBR (metalness≈0.5) and render black without a scene.environment. The test scene builds a neutral PMREM; the impostor bake is fed the same env. Lane B's shell must set scene.environment (a sky PMREM) or every citizen — near and mid — goes dark. This is wired: CitizenSim reads scene.environment and passes it to the impostor baker.

  4. Impostor material is toneMapped:false and does ACES + sRGB itself (matching ACESFilmicToneMapping, exposure /0.6). The shell DOES animate renderer.toneMappingExposure (Lane B lighting.js sets it per day segment), so mid billboards drift brighter/darker than the near rigs across the day unless matched. Round-3 fix: CitizenSim.setExposure(e) passthrough added — see the Lane F hook below. (For a non-ACES curve the in-shader ACES in impostor.js IMP_FRAG would also need to match; today both are ACESFilmic so exposure is the only variable.)

Hardened by a 4-dimension adversarial code review (see D-progress.md): 6 confirmed defects fixed — non-deterministic fleet order, shared-geometry disposal, impostor under-exposure, unrestored viewport, missing modelMatrix, frozen exposure.

Integration hooks for Lane F

  • new CitizenSim({ renderer, scene, camera, citySeed, graph, fleet })graph = { nodes:[{id,x,z}], edges:[{id,a,b,width,kind}] }. Swap the fixture graph for Lane A's CityPlan.streets. Footpath lanes are derived from edge width + a 0.9 m margin; pedestrians keep to the right of travel.
  • sim.setPopulation(n) (slider / density) · sim.setTimeOfDay(t01) (drive from the shell's day segment) · sim.setPaused(bool) (wire to visibilitychange) · sim.update(dt) each frame.
  • [Round-3] sim.setExposure(e) — one line, needed for dusk/night. The shell already calls citizens.setTimeOfDay(...) in the street loop (index.html:225); add right after it:
    citizens.setExposure(renderer.toneMappingExposure);   // [Lane F] keep mid billboards matched to day/night exposure
    
    Cheap (sets one uniform), safe to call every frame, and survives the placeholder→rig atlas re-bake (stored internally). Verified in citizens_test.html: uniform tracks 0.55↔2.3 and persists across the fleet upgrade. Without it, mid impostors read too bright at night / too dark at noon vs the near rigs. (This is a Lane-F seam edit in index.html; flagged here per the cross-lane note rule.)
  • [Round-3] Keeper fleet-upgrade (enables F §3.4). Keepers today are placeholder-only because F builds new KeeperManager({ camera, citySeed }) with no fleet. To upgrade them to shared GLB rigs, pass the loaded fleet: new KeeperManager({ camera, citySeed, fleet }). keepers.js already picks a rig when fleet.ready and falls back to a placeholder otherwise — no other change. Dispose is leak-free: verified 15 enter/exit cycles with a rig keeper → renderer.info.memory geo/tex return exactly to baseline (_disposeInner frees the clone's skeleton bone-texture only; shared fleet geo/mats are preserved for siblings). Safe to spawn/remove() per interior enter/exit.
  • Chunk streaming (v1.5): roster is currently whole-graph — fine for v1, but big towns look sparse (see the design note "Chunk-streamed roster" below for the why + the full plan).
  • Keepers: keepers.spawn(roomGroup, { x, z, ry, shopId, type }) — feed x,z,ry from Lane C's interior counter places; call on interior build, keepers.remove(handle) on dispose. keepers.update(dt, playerPos) each interior frame.
  • ?noassets=1 verified: full placeholder town, mixers=0, zero crashes.

Chunk-streamed roster — v1.5 design note (design only; no implementation)

Round-3 task 3. This is the durable fix for "big-town streets look empty." Written as a spec for a future Lane D/F pass — nothing here is wired yet; v1 keeps the whole-graph roster.

The problem (measured). setPopulation(N) builds one global roster and spreads it uniformly over all edges (identityOf picks edge = rng()*edgeCount across the whole graph). On the 12-edge test fixture that fills the view. On a real town — "Boolarra Heads" has hundreds of edges — those same N citizens scatter town-wide, so almost all sit beyond the 70 m cull from any one camera (I measured a 27-node generated town: pop 200 → 4 near / 6 mid / 190 far). Raising N to fill the near streets wastes roster + advance cost on citizens nobody can see, and still can't guarantee local density.

The fix. Generate + tick citizens per chunk, keyed so identity is independent of town size and visit order, and stream them in lockstep with Lane B's chunk window (the chunks already building/ disposing around the player). Density becomes constant per unit street, and total roster is bounded by the live-chunk window, not the town.

Design:

  1. Chunk-local identity. Replace the global running id with a per-chunk key: identityOf(citySeed, chunkKey, i) for i in [0, perChunkCount), and choose the citizen's home edge from that chunk's edges (chunkIndex(plan).chunks[chunkKey].edges), not the global list. Then "same seed + same chunk → same people" holds regardless of what else exists or the order chunks were visited — the determinism property that whole-graph keying quietly loses at scale. perChunkCount is seeded per chunk and scaled by district (main-street chunks busier than residential) × densityAt(tod).
  2. Windowing — poll-driven (works today, no Lane B change). Each frame derive the active chunk set from the camera: chunkKey(⌊camX/chunkSize⌋, ⌊camZ/chunkSize⌋) + neighbours within a radius R (R≥1 so a walker never steps into an unloaded chunk before its owner unloads). Diff vs the live set → build sub-rosters for newly-active chunks, dispose for newly-inactive. This mirrors B's own window and needs nothing from B. Alternative — hook-driven: chunks.js exposes an optional ctx.onChunkBuilt (see chunks.js:40 — "inert unless a consumer sets it"); if B also adds onChunkDisposed, drive spawn/despawn off those instead of polling. Prefer this once B commits the pair; poll until then.
  3. Storage. Swap the single this.roster array for this.chunkRosters = Map<chunkKey, Citizen[]>. The per-frame update() iterates the union of live-chunk citizens. Everything downstream is unchanged — LOD tiers, 24 near-cap, mixer stagger, rig pool, impostor layer already operate per-citizen and are chunk-agnostic; the near-cap + mixer budget stay global across live chunks (nearest-first selection already does the right thing on the merged set).
  4. Ownership + hand-off. A citizen is owned by its spawn chunk. It walks freely (the graph is continuous); because R≥1 keeps neighbours loaded, crossing a chunk boundary is seamless. It despawns only when its owner chunk unloads — even if it's momentarily standing in a still-loaded neighbour (acceptable: that's behind the player, past the cull). No re-keying on cross, so no identity churn.
  5. Budget. Live window ≈ (2R+1)² chunks × perChunkCount. Tune perChunkCount (~1530) so the near streets reach the 24-cap while total roster stays ~150270 — similar cost to today's flat 200, but concentrated where the camera is instead of smeared across the map.

Migration. Opt-in and back-compatible: new CitizenSim({ …, chunkStream: { chunkIndex, chunkSize, radius } }) switches on chunk-keyed identity + windowing; omit it and the current whole-graph roster is unchanged. So v1 ships as-is and v1.5 flips a constructor option — no rewrite of the hot path.

Explicitly NOT in scope here: cross-chunk social groups, per-district behaviour trees, persistent citizens (a citizen you saw yesterday) — all v2 (docs/V2_IDEAS.md).

Clip wishlist (for the mixamo-fetch run — CHECK THE 34 EXISTING CLIPS FIRST)

Only walk + idle are wired today (they ship with the ped fleet). CITY_SPEC says ~/Documents/mixamo-fetch/out/ already holds 34 clips (sit, lean, look-around, phone, …) — these bind directly through the canonical mixamorig skeleton exactly like walk/idle, so wiring them is a data task, not new rig work. Verify each exists before requesting; never re-download.

Wanted for richer loiter/keeper behaviour, in priority order:

  1. sit — bench-sit at benches/verandah steps (sim loiter already has the hook; needs the clip).
  2. lean — loiter against a shopfront (window-shopping stops).
  3. look-around / idle-look — window-shopping variety + a better keeper greet than the body-turn.
  4. talk / gesture — pairs chatting on the footpath (spawn in seeded 2-groups).
  5. carry / laden walk — shoppers with bags after a purchase (content-phase tie-in).
  6. browse / reach — keeper stocking shelves; customer reaching a shelf inside interiors.

Bespoke (won't be on Mixamo, hand-author later): crate-riffle at market stalls, till-operation.

Known limitations (honest)

  • Near↔mid swap is a hard LOD switch at ~25 m (hysteresis 24/27 m). At that range a 128px impostor and the full rig subtend near-identical screen size, so it reads as seamless — but it is a switch, not a cross-fade (rig materials are shared across SkeletonUtils clones, so per-instance opacity fade isn't free). If a pop is ever visible after buildings land, the fix is a short dither/fade band.
  • In ?noassets mode, mid impostors use 8 generic placeholder variants while near placeholders are per-citizen coloured — so a citizen's colour can shift slightly crossing 25 m. Fallback-only; with the real fleet, near and mid are the same baked ped.
  • Loiter timing is dt-driven (cosmetic), so exact positions at time T aren't reproducible across runs; identity (who, which ped, height, speed, spawn beat) is fully seeded and asserted.