Chronicle records model and tool boundaries as immutable envelopes, then replays selected boundaries while running the rest against changed code. This cut-point replay turns a nondeterministic agent incident into a CI regression test. On six recorded failures, full replay made zero model calls and was bit-stable across 20 runs; cut-point tests distinguished faulty, guarded and benign changes in every case. The implementation and benchmark are public.
Chronicle turns failed agent runs into deterministic regression tests
2026-09-18Frontier watchTRIAL
Chronicle records model and tool boundaries as immutable envelopes, then replays selected boundaries while running the rest against changed code. This cut-point replay turns a nondeterministic agent incident into a CI regression test. On six recorded failures, full replay made zero model calls and was bit-stable across 20 runs; cut-point tests distinguished faulty, guarded and benign changes in every case. The implementation and benchmark are public.
Availability: public code and benchmark
Imported from the existing research archive. Migration preserves the recorded judgment and is not a fresh verification.
What this means for you
For teams operating tool-using agents, traces can become executable tests instead of passive observability records. The six-case benchmark is small, so the right next step is a focused POC on one costly production incident rather than broad adoption.
FRONTIER WATCH
Public artifacts and validated results are different.
The recorded fact is the public work and its artifacts. Author-reported outcomes do not establish independent reproduction. Next checks include usable code, task conditions and third-party validation.
Evidence & related material
Technical details & scores
| Method | immutable boundary envelopes plus cut-point replay |
|---|---|
| Benchmark | 6 recorded failures with simulated model boundaries |
| Full Replay | zero model calls; bit-stable over 20 repetitions |
| Recording Overhead | 23 microseconds per boundary crossing |
| Availability | public code and benchmark |
Technical impact 4
Engineering value 5
Adoption signal 1
Maturity 2
Verification cost 2
Risk 2