Commit Graph

1 Commits

Author SHA1 Message Date
jing
160d0cc484 bench_real.py: three public benchmarks with deterministic metrics; 0.3.0
- LogHub samples (real system logs): 3.0x over 1.15M tokens
- LogChunks (797 Travis failures, human-labeled failure chunks): 1.9x over
  66M tokens, 78.5% of labeled failure-explaining lines survive verbatim
- LogDx-CI (35 real GitHub Actions failures, arXiv 2605.28876): 2.2x over
  15M tokens, 95.2% critical diagnostic signals retained; with --budget
  8000, 6,334 tok/case at 77.4% retention; haiku diagnosed 12/12 from
  compressed vs 12/12 raw on the 12-case subset
- README: Real logs, real benchmarks section; updated synthetic receipts
  (templates lift ANSI CI to 15.9x, synthetic eval total to 6.0x, 8/8
  re-verified); honest 2-4x expectation row for production logs

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 17:20:35 +10:00