PROCITY/docs/LANES/LANE_D_NOTES.md
m3ultra a5e4b64a9d Lane D round-4: in-shell verify (D1) + decimated-ped validation (D2) + impostor tone-map fix (D3)
D1: peds+keepers upgrade placeholder->rig in the shell; 12 enter/exit shops leak-free
(renderer.info.memory back to exact baseline); determinism holds across reload; ?noassets
fully placeholder with zero ped fetches.

D2 (gate-3 critical path): validated Lane E's decimated fleet — all 19 peds <=3k tris
(1564-2805, avg 2450), rigs bind+animate (skinning/skeleton preserved), 0 broken meshes under
eviction churn, silhouettes+identity intact, atlas bakes clean. Near fleet ~59.6k tris at the
24-cap (was ~1.2M) — F clear to re-measure gate 3.

D3: fixed a real bug found only in-shell — the impostor self-tone-mapped (toneMapped:false +
in-shader ACES), correct on the direct-canvas test page but DOUBLE-tone-mapped under the shell's
EffectComposer/OutputPass. Reworked to a standard toneMapped:true material outputting linear via
three's own tonemapping/colorspace chunks, so it matches the near rigs on both render paths.
setExposure is now a no-op (exposure is global). Verified noon+night in-shell.

Only web/js/citizens/impostor.js changed (+ NOTES/progress/R4 shots).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 18:56:28 +10:00

245 lines
18 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# LANE D — NOTES (measured budgets + clip wishlist)
*Written by PROCITY-D, 2026-07-14. Standalone verification via `web/citizens_test.html`.
Reference stack ported from `90sDJsim/web/world/index.html` ~405520 (loadRig/spawnRig/_canon/
_rotOnly/head-bone normalize/upgradeStreetPeople). Measurements on the M3 Ultra dev box.*
---
## ROUND 4 — in-shell verify + decimation validation (→ Fable, → Lane F)
**All three R4 tasks done. `tools/qa.sh --strict` GREEN. Sign-off for Lane F below.**
### D1 — in-shell rig/keeper verification (`web/index.html`) ✅
Verified in the real game (not the test page), via `window.PROCITY`:
- **Placeholder→rig upgrade** fires for both peds (`citizens.mode` → `'rig'`, 19 fleet, pool active)
and keepers (keeper spawns at the counter `keeperStand` pose as a **rig**).
- **Leak-free**: baseline `renderer.info.memory` = {geometries 213, textures 198}; after **12
enter/exit shop cycles** it returned to **exactly** 213/198 (Δ geom 0, Δ tex 0). Interiors peak
~303 geoms inside, drop back every exit. Confirms the R-review dispose fix (free the keeper's
Skeleton only, never the fleet-shared geo/mats) holds in-shell alongside Lane C's interior dispose.
- **Determinism in-shell**: seed 20261990 → byte-identical identity signature across a full reload.
- **`?noassets=1`**: `fleet===null`, mode stays `placeholder`, peds + keeper are placeholder actors,
**zero ped GLB fetches** — town fully playable asset-free.
### D2 — decimated ped validation (critical path — E1 landed `518678d`) ✅ SIGN-OFF FOR F
Validated Lane E's decimated fleet (`web/models/peds/`, ref `lane-e/round4-peds`):
- **Per-ped tris: 15642805, avg 2450 — all ≤3k** (matches E's `_peds_decim.json`).
- **Rigs bind + animate**: skinning preserved, the shared `walk.glb`/`idle.glb` retarget unchanged
(skeleton/65 joints intact), near actors animate, mixer 0.10.3 ms.
- **No skinning explosions**: 24-teleport pool-eviction churn → **0 broken meshes**; silhouettes
read at street distance; identity variety intact (verified visually — kid/luchador/hi-vis/elder/
business all distinct, no melted hands/faces).
- **Impostor atlas bakes clean** from the decimated rigs; determinism + leak checks re-pass.
- **Gate-3 number for Lane F**: at the 24-rig near cap the fleet now contributes **≈59.6k tris**
(was ~1.2M). Test-page whole-view total **88.9k tris** (was ~1.5M). In-shell sample near rig 2729
tris, keeper 2073 tris (was 41k). **F is clear to re-measure gate 3** — the fleet leaves ~140k for
the town, comfortably ≤200k. (The R1 "peds too heavy" finding is now resolved by E's decimation.)
### D3 — exposure sync at dusk/night — FOUND + FIXED A REAL BUG ✅
Verifying the shell exposed a genuine rendering bug (only correct on my direct-canvas test page):
**the impostor material was `toneMapped:false` and applied ACES in-shader**, but the shell renders
through an **EffectComposer (RenderPass → bloom → OutputPass)**. three only tone-maps materials when
rendering to the **canvas** (`three.module.js:69216931`, `currentRenderTarget === null`); into a
composer target, materials output **linear** and OutputPass applies ACES once. So my self-tone-mapping
impostors were **double-tone-mapped** in the actual game (too dark), matching the rigs only on the
test page. **Fix (impostor.js):** the impostor is now a normal `toneMapped:true` material — it outputs
**linear** and uses three's own `<tonemapping_fragment>` + `<colorspace_fragment>` call-sites (the
`*_pars_*` are auto-injected — including them ourselves double-defines `RRTAndODTFit` and fails to
compile), so it tone-maps **identically to the near rigs on both paths**: canvas → ACES per-material;
composer → linear → OutputPass ACES. Verified: near rig + mid impostor match brightness at noon AND
the scene reads correctly at night 22:00 in the shell (`docs/shots/laneD/r4_shell_*.jpg`), and the
test-page canvas path still matches.
- **Note for Lane F**: `citizens.setExposure()` is now a **no-op** (exposure is global via
`renderer.toneMappingExposure`, which both three and OutputPass read). The call at
`web/index.html:245` is harmless — keep or drop it, your call.
*(Round-1/3 notes below remain valid; the tri-budget finding #1 is superseded by D2's decimation.)*
## What shipped
| file | role |
|---|---|
| `web/js/citizens/rigs.js` | ported rig stack: fleet loader, `_canon`/`canonRig`/`_rotOnly`, `buildFigure` (head-bone height-normalise + feet plant), `spawnRig` (single clip), `makeActor` (walk↔idle crossfade), `pickRig` |
| `web/js/citizens/placeholder.js` | seeded low-poly box humanoid (POLY lock) — walks/idles, planted at y=0, hot-swaps to a rig |
| `web/js/citizens/impostor.js` | 4-yaw sprite-atlas baker + instanced billboard layer (mid tier, 1 draw call) |
| `web/js/citizens/sim.js` | deterministic roster, footpath lanes, near/mid/far LOD, rig pool, staggered mixer budget, time-of-day density |
| `web/js/citizens/keepers.js` | one keeper per shop at the counter slot, idle + greet head/body-turn |
| `web/citizens_test.html` | standalone harness: fixture 3×3 street graph, sliders, tier debug, determinism check |
| `web/models/peds/*` | 19 rigged GLBs + `walk.glb` + `idle.glb`, **byte-identical** copies from 90sDJsim (checksummed) |
## The fleet (copied, never edited)
19 rigged peds (17 normal + 2 comical) + 2 clip-only GLBs, ~37 MB total, copied from
`johnking@100.91.239.7:~/Documents/90sDJsim/web/world/models/peds/` (also present locally at
`~/Documents/90sDJsim/…`). SHA-verified byte-identical (house law: canonical source stays upstream).
- Skeletons canonicalise cleanly: `mixamorig1…`→`mixamorig…`, so **one shared walk clip drives all 19**
and one shared idle clip too. Verified in-scene — a single `walk.glb` animates every ped.
- `walk.glb`/`idle.glb` position tracks + `Hips.quaternion` are stripped (`_rotOnly`) — without this
the different-scale root track inflates peds to giants. Confirmed: no giants/ants across all 19.
## Measured budgets (M3 Ultra, `citizens_test.html`)
Preview note: the in-app browser reports `visibilityState:'hidden'`, which throttles `requestAnimationFrame`,
so the on-screen fps counter reads low (4784). True cost was measured by timing 140 manual
`update()+render()` frames.
**200 citizens, midday (the acceptance fixture):**
| metric | measured | CITY_SPEC / Lane-D budget | verdict |
|---|---|---|---|
| near (rigged) actives | 24 (hard cap) | ≤ 24 | ✅ |
| mid (impostor) | ~122 | — | — |
| far (culled) | ~54 | — | — |
| **mixer update / frame** | **0.4 ms max, ~0.2 ms avg** | < 2 ms | comfortably |
| sim logic / frame | 0.24 ms avg | 4 ms build budget | |
| full frame CPU (update+render) | ~2.6 ms ~380 fps headroom | 60 fps (16.6 ms) | huge margin |
| mid tier draw calls | **1** (69 instances, 138 tris) | 1 per atlas | |
| impostor atlas texture | 2048×640 (16×5 cells, 4 yaws × 19 peds) | 2048, < 512 MB total | (~5 MB) |
| determinism | live roster seed recompute, stable while walking | same seed same crowd | |
**Time-of-day density** (200 base): 00:00 12 active, 06:00 70, 12:00 200, 15:00 150,
21:00 36. Curve in `sim.js DAY_CURVE`.
## ⚠️ Findings for Lane E / Lane F (need a decision)
1. **The inherited peds are ~49.7k tris each — too heavy for the CITY_SPEC 200k-tri "typical view".**
24 near rigs **1.2M tris**; a realistic 16-rig street view already measured **1.02M tris**.
Framerate holds on M-series (GPU eats it), but this leaves *nothing* for Lane B buildings.
**Recommendation:** decimate the ped fleet to ~58k tris in the asset pipeline (Lane E) at 6k/ped
the near cap costs ~144k tris, back within budget. The rig stack is indifferent to poly count;
this is purely an asset job. Until then, `NEAR_MAX` (sim.js) is the throttle.
2. **Draw calls scale with near rigs (~6 calls/ped — each GLB has several sub-meshes/materials).**
16 rigs 144 calls; 24 rigs ~190. Under the 300-call street budget *today*, but tight once
Lane B adds shells. **Recommendation:** merge each ped's sub-meshes by material on load
(`BufferGeometryUtils.mergeGeometries`) ~12 calls/ped. Cheap win, deferred (not blocking).
3. **Peds are metallic-PBR (`metalness≈0.5`) and render black without a `scene.environment`.**
The test scene builds a neutral PMREM; the impostor bake is fed the same env. **Lane B's shell must
set `scene.environment`** (a sky PMREM) or every citizen near and mid goes dark. This is wired:
`CitizenSim` reads `scene.environment` and passes it to the impostor baker.
4. **Impostor material is `toneMapped:false` and does ACES + sRGB itself** (matching
`ACESFilmicToneMapping`, exposure `/0.6`). **The shell DOES animate `renderer.toneMappingExposure`**
(Lane B `lighting.js` sets it per day segment), so mid billboards drift brighter/darker than the near
rigs across the day unless matched. **Round-3 fix:** `CitizenSim.setExposure(e)` passthrough added
see the Lane F hook below. (For a non-ACES curve the in-shader ACES in `impostor.js IMP_FRAG` would
also need to match; today both are ACESFilmic so exposure is the only variable.)
*Hardened by a 4-dimension adversarial code review (see D-progress.md): 6 confirmed defects fixed
non-deterministic fleet order, shared-geometry disposal, impostor under-exposure, unrestored
viewport, missing modelMatrix, frozen exposure.*
## Integration hooks for Lane F
- `new CitizenSim({ renderer, scene, camera, citySeed, graph, fleet })` `graph = { nodes:[{id,x,z}],
edges:[{id,a,b,width,kind}] }`. Swap the fixture graph for Lane A's `CityPlan.streets`. Footpath
lanes are derived from edge `width` + a 0.9 m margin; pedestrians keep to the right of travel.
- `sim.setPopulation(n)` (slider / density) · `sim.setTimeOfDay(t01)` (drive from the shell's day
segment) · `sim.setPaused(bool)` (wire to `visibilitychange`) · `sim.update(dt)` each frame.
- **[Round-3] `sim.setExposure(e)` one line, needed for dusk/night.** The shell already calls
`citizens.setTimeOfDay(...)` in the street loop (`index.html:225`); add right after it:
```js
citizens.setExposure(renderer.toneMappingExposure); // [Lane F] keep mid billboards matched to day/night exposure
```
Cheap (sets one uniform), safe to call every frame, and survives the placeholderrig atlas re-bake
(stored internally). **Verified** in `citizens_test.html`: uniform tracks 0.552.3 and persists
across the fleet upgrade. Without it, mid impostors read too bright at night / too dark at noon vs
the near rigs. *(This is a Lane-F seam edit in `index.html`; flagged here per the cross-lane note rule.)*
- **[Round-3] Keeper fleet-upgrade (enables F §3.4).** Keepers today are placeholder-only because F
builds `new KeeperManager({ camera, citySeed })` with no fleet. To upgrade them to shared GLB rigs,
pass the loaded fleet: `new KeeperManager({ camera, citySeed, fleet })`. `keepers.js` already picks a
rig when `fleet.ready` and falls back to a placeholder otherwise no other change. **Dispose is
leak-free**: verified 15 enter/exit cycles with a rig keeper `renderer.info.memory` geo/tex return
exactly to baseline (`_disposeInner` frees the clone's skeleton bone-texture only; shared fleet
geo/mats are preserved for siblings). Safe to spawn/`remove()` per interior enter/exit.
- **Chunk streaming (v1.5):** roster is currently whole-graph fine for v1, but big towns look sparse
(see the design note **"Chunk-streamed roster"** below for the why + the full plan).
- **Keepers:** `keepers.spawn(roomGroup, { x, z, ry, shopId, type })` feed `x,z,ry` from Lane C's
interior counter `places`; call on interior build, `keepers.remove(handle)` on dispose.
`keepers.update(dt, playerPos)` each interior frame.
- `?noassets=1` verified: full placeholder town, mixers=0, zero crashes.
## Chunk-streamed roster — v1.5 design note (design only; no implementation)
*Round-3 task 3. This is the durable fix for "big-town streets look empty." Written as a spec for a
future Lane D/F pass nothing here is wired yet; v1 keeps the whole-graph roster.*
**The problem (measured).** `setPopulation(N)` builds one global roster and spreads it uniformly over
*all* edges (`identityOf` picks `edge = rng()*edgeCount` across the whole graph). On the 12-edge test
fixture that fills the view. On a real town "Boolarra Heads" has hundreds of edges those same N
citizens scatter town-wide, so almost all sit beyond the 70 m cull from any one camera (I measured a
27-node generated town: pop 200 **4 near / 6 mid / 190 far**). Raising N to fill the near streets
wastes roster + advance cost on citizens nobody can see, and still can't guarantee local density.
**The fix.** Generate + tick citizens **per chunk**, keyed so identity is independent of town size and
visit order, and stream them in lockstep with Lane B's chunk window (the chunks already building/
disposing around the player). Density becomes *constant per unit street*, and total roster is bounded
by the live-chunk window, not the town.
**Design:**
1. **Chunk-local identity.** Replace the global running `id` with a per-chunk key:
`identityOf(citySeed, chunkKey, i)` for `i in [0, perChunkCount)`, and choose the citizen's home
edge from **that chunk's** edges (`chunkIndex(plan).chunks[chunkKey].edges`), not the global list.
Then "same seed + same chunk same people" holds regardless of what else exists or the order chunks
were visited the determinism property that whole-graph keying quietly loses at scale. `perChunkCount`
is seeded per chunk and scaled by district (main-street chunks busier than residential) × `densityAt(tod)`.
2. **Windowing — poll-driven (works today, no Lane B change).** Each frame derive the active chunk set
from the camera: `chunkKey(⌊camX/chunkSize⌋, ⌊camZ/chunkSize⌋)` + neighbours within a radius R (R1
so a walker never steps into an unloaded chunk before its owner unloads). Diff vs the live set
build sub-rosters for newly-active chunks, dispose for newly-inactive. This mirrors B's own window and
needs nothing from B. *Alternative — hook-driven:* `chunks.js` exposes an optional `ctx.onChunkBuilt`
(see `chunks.js:40` "inert unless a consumer sets it"); if B also adds `onChunkDisposed`, drive
spawn/despawn off those instead of polling. Prefer this once B commits the pair; poll until then.
3. **Storage.** Swap the single `this.roster` array for `this.chunkRosters = Map<chunkKey, Citizen[]>`.
The per-frame `update()` iterates the union of live-chunk citizens. **Everything downstream is
unchanged** LOD tiers, 24 near-cap, mixer stagger, rig pool, impostor layer already operate
per-citizen and are chunk-agnostic; the near-cap + mixer budget stay **global** across live chunks
(nearest-first selection already does the right thing on the merged set).
4. **Ownership + hand-off.** A citizen is owned by its spawn chunk. It walks freely (the graph is
continuous); because R1 keeps neighbours loaded, crossing a chunk boundary is seamless. It despawns
only when **its owner chunk** unloads even if it's momentarily standing in a still-loaded neighbour
(acceptable: that's behind the player, past the cull). No re-keying on cross, so no identity churn.
5. **Budget.** Live window (2R+1 chunks × `perChunkCount`. Tune `perChunkCount` (~1530) so the
near streets reach the 24-cap while total roster stays ~150270 similar cost to today's flat 200,
but concentrated where the camera is instead of smeared across the map.
**Migration.** Opt-in and back-compatible: `new CitizenSim({ …, chunkStream: { chunkIndex, chunkSize,
radius } })` switches on chunk-keyed identity + windowing; omit it and the current whole-graph roster is
unchanged. So v1 ships as-is and v1.5 flips a constructor option no rewrite of the hot path.
**Explicitly NOT in scope here:** cross-chunk social groups, per-district behaviour trees, persistent
citizens (a citizen you saw yesterday) all v2 (`docs/V2_IDEAS.md`).
## Clip wishlist (for the mixamo-fetch run — CHECK THE 34 EXISTING CLIPS FIRST)
Only `walk` + `idle` are wired today (they ship with the ped fleet). CITY_SPEC says
`~/Documents/mixamo-fetch/out/` already holds **34 clips (sit, lean, look-around, phone, …)** these
bind directly through the canonical `mixamorig` skeleton exactly like walk/idle, so wiring them is a
data task, not new rig work. **Verify each exists before requesting; never re-download.**
Wanted for richer loiter/keeper behaviour, in priority order:
1. `sit` bench-sit at benches/verandah steps (sim loiter already has the hook; needs the clip).
2. `lean` loiter against a shopfront (window-shopping stops).
3. `look-around` / `idle-look` window-shopping variety + a better keeper greet than the body-turn.
4. `talk` / `gesture` pairs chatting on the footpath (spawn in seeded 2-groups).
5. `carry` / laden walk shoppers with bags after a purchase (content-phase tie-in).
6. `browse` / `reach` keeper stocking shelves; customer reaching a shelf inside interiors.
Bespoke (won't be on Mixamo, hand-author later): crate-riffle at market stalls, till-operation.
## Known limitations (honest)
- Nearmid swap is a hard LOD switch at ~25 m (hysteresis 24/27 m). At that range a 128px impostor
and the full rig subtend near-identical screen size, so it reads as seamless but it is a switch,
not a cross-fade (rig materials are shared across SkeletonUtils clones, so per-instance opacity fade
isn't free). If a pop is ever visible after buildings land, the fix is a short dither/fade band.
- In `?noassets` mode, mid impostors use 8 generic placeholder variants while near placeholders are
per-citizen coloured so a citizen's colour can shift slightly crossing 25 m. Fallback-only; with
the real fleet, near and mid are the same baked ped.
- Loiter *timing* is dt-driven (cosmetic), so exact positions at time T aren't reproducible across
runs; **identity** (who, which ped, height, speed, spawn beat) is fully seeded and asserted.