README final pass: current numbers everywhere, rtk calibration, tradeoffs

- hero matches the measured receipts (87k->6.7k mixed log)
- rtk's third-party LogDx-CI numbers cited as the lossy end of the curve
- hook documented as fail-open; Drain3 comparison matches two-tier design
- stale 300x claim corrected (~230x), assert count 48, LogDx bug credited
- Known tradeoffs gains templated-readability note, moves above License

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

README.md

@@ -1,14 +1,14 @@
 # lessismore 📉
 
-**Turn 50,000 tokens of log spam into 4,500 tokens of pure signal — before it
+**Turn 87,000 tokens of log spam into 6,700 tokens of pure signal — before it
 ever hits your LLM.**
 
 lessismore is a deterministic compression filter for the bulk text that
 actually fills context windows: logs, CI output, tracebacks, JSON dumps,
 captured tool output. Pure stdlib Python. Zero dependencies. Every claim below
-was measured against the real o200k tokenizer — and every idea that failed the
+was measured against the real o200k tokenizer, on three public benchmarks as
-measurement is documented at the bottom, so you know exactly what you're
+well as synthetic ones — and every idea that failed the measurement is
-getting.
+documented in the graveyard section, so you know exactly what you're getting.
 
 ```bash
 tail -5000 app.log | python3 lessismore.py | llm "why did this crash?"
@@ -178,7 +178,12 @@ of the 7 misses, most are layout artifacts where templating separates a value
 from its line). And the answer-quality check: on 12 cases, claude-haiku-4.5
 diagnosed **12/12 from raw and 12/12 from compressed** — parity at ~2.4x
 fewer tokens. (Ground-truth caveat inherited from the benchmark: its labels
-are AI-drafted with single-author verification, n=35.)
+are AI-drafted with single-author verification, n=35.) For calibration
+against the field: rtk's most aggressive mode measures 810 tokens/case on
+this same corpus at a 0.249 diagnosis score
+([rtk#2012](https://github.com/rtk-ai/rtk/issues/2012)) — a far lossier
+point on the curve. lessismore's stance is the opposite end: keep the
+signals, then `--budget` down to whatever your cap is.
 
 **The equal-budget receipt** — the question every harness actually faces is
 not "compress or not" but "we cap tool output anyway; is compressing before
@@ -274,7 +279,8 @@ outputs under 2,000 chars pass through untouched. In
 
 Because the passes are deterministic, hook-compressed transcripts stay
 byte-stable across re-runs — the prompt-cache argument above is strongest
-exactly here.
+exactly here. The hook fails open: on any input it doesn't recognize it
+emits nothing, and Claude Code keeps the original output untouched.
 
 ## What we refused to build (measured so it stays dead)
 
@@ -318,8 +324,9 @@ non-UTF-8 stdin, distinct SHA-256s falsely merging as "repeated",
 regression tests, four documented below, one wontfix (adversarial in-band
 marker collision). A second review round found the big one: similar-line
 collapse silently eating distinct status codes — fixed with value summaries
-and now guarded by the downstream eval. `python3 test_lessismore.py` —
+and now guarded by the downstream eval. A third bug (camelCase test names
-42 asserts, no framework.
+eaten as base64 blobs) was caught by the LogDx-CI benchmark rather than a
+reviewer. `python3 test_lessismore.py` — 48 asserts, no framework.
 
 ## Where it does nothing (on purpose)
 
@@ -328,7 +335,7 @@ and machine noise; it cannot compress information, and doesn't pretend to.
 
 | Content | Expect | Verdict |
 |---|---|---|
-| Repetitive machine output | 5–16x, up to 300x on pathological repeats | the reason this exists |
+| Repetitive machine output | 5–16x, up to ~230x on pathological repeats | the reason this exists |
 | Real-world CI / system logs | 2–4x typical, up to 10x on template-heavy logs (measured on LogHub, LogChunks, LogDx-CI) | the honest production number |
 | Structured data (JSON, tracebacks) | 30–64% | worthwhile, lossless where it fires |
 | Varied prose | ~2% (level 3) / 38% lossy (level 4) | gist mode only |
@@ -354,9 +361,11 @@ Honest placement, so you can pick the right tool:
   lessismore doesn't have.
 - **[Drain3](https://github.com/logpai/Drain3)** — industrial log template
   mining. `alias_templates` is a deterministic, single-pass, zero-dep
-  Drain-lite (digit-token masking instead of similarity clustering); if you
+  Drain-lite (two-tier hex-run/whole-token masking with exact payoff
-  need streaming template state across files or letter-varying parameters
+  accounting, instead of similarity clustering); if you need streaming
-  (usernames, hostnames), use Drain3.
+  template state across files, or one template spanning letter-varying
+  parameters (we measured that merge and it didn't pay — see the graveyard),
+  use Drain3.
 - **[LLMLingua-2](https://github.com/microsoft/LLMLingua)** — ML token
   pruning; wrapped here as the optional `--ml` last mile, *after* the
   deterministic passes, so you never pay a classifier to delete duplicate
@@ -376,10 +385,6 @@ the cache-stability argument as a design constraint rather than a patch.
 ugliest log, pick a level, watch the token count drop. Stdlib only, binds
 localhost only.
 
-## License
-
-MIT — see [LICENSE](LICENSE).
-
 ## Known tradeoffs
 
 - Levels 2+ assume line *order* matters but wall-clock timing doesn't. When
@@ -389,10 +394,11 @@ MIT — see [LICENSE](LICENSE).
 - At levels 3–4, dedupe counts describe the post-stripped text — five
   differently-phrased "retry the job" lines can legitimately merge.
 - In-band markers can collide with input that already contains them;
-  `alias_repeats` bails out if its own `@N` markers already appear as lines.
+  `alias_repeats` and `alias_templates` bail out if their own `@N`/`@tN`
+  markers already appear as lines.
 - Blob squashing eats base64url runs (real JWTs) only when they look like
-  blobs (digits + mixed case); a 64-char kebab-case or snake_case identifier
+  blobs (digits + mixed case, no 16+ letter run); a kebab-case slug or a
-  is treated as content and survives.
+  camelCase test name is treated as content and survives.
 - Similar-line collapse summarizes varying digits per column
   (`values 404/429/503`, or a min–max range past 4 distinct); if the digit
   *runs* per line differ in count, it falls back to a plain count marker.
@@ -401,3 +407,11 @@ MIT — see [LICENSE](LICENSE).
   original spacing/alignment is not preserved). Parameters that vary in
   *letters* (usernames, hostnames) split into separate templates rather than
   merging — less compression, never confusion.
+- Reading a templated log requires joining refs back to the legend. Models
+  handle this fine (the evals above are all on templated output, and answers
+  routinely cite `@N` legend entries), but a human skimming the compressed
+  file will prefer the raw one.
+
+## License
+
+MIT — see [LICENSE](LICENSE).