RRSI adds regularization to automated agent-harness evolution so the system favors reusable changes over benchmark-specific edits. It limits how many edits a proposal can bundle, explores less-used evolution paths, screens benchmark leakage and prunes small, costly or stale changes. Across eight coding, workspace-agent and engineering-design benchmarks, it reports gains of up to 14.1 points in distribution and 4.7 points on five held-out benchmarks, while using 30% fewer policy tokens than unregularized evolution.
RRSI keeps self-improving agent harnesses from memorizing their benchmarks
2026-09-22Frontier watchWATCH
RRSI adds regularization to automated agent-harness evolution so the system favors reusable changes over benchmark-specific edits. It limits how many edits a proposal can bundle, explores less-used evolution paths, screens benchmark leakage and prunes small, costly or stale changes. Across eight coding, workspace-agent and engineering-design benchmarks, it reports gains of up to 14.1 points in distribution and 4.7 points on five held-out benchmarks, while using 30% fewer policy tokens than unregularized evolution.
Imported from the existing research archive. Migration preserves the recorded judgment and is not a fresh verification.
What this means for you
For teams letting agents rewrite their own prompts, tools or control flow, tuning-set gains can hide harness overfitting. Use held-out tasks, proposal-size budgets and automatic pruning before accepting a self-generated harness change. RRSI is public and reproducible enough to study, but the results remain a research signal rather than proof that recursive harness evolution is safe in production.
FRONTIER WATCH
Public artifacts and validated results are different.
The recorded fact is the public work and its artifacts. Author-reported outcomes do not establish independent reproduction. Next checks include usable code, task conditions and third-party validation.
Evidence & related material
Technical details & scores
| Scope | 8 coding, agentic-workspace and engineering-design benchmarks |
|---|---|
| Regularization | annealed proposal-size budget, evolution-history exploration, benchmark-specific critic and cost/staleness pruner |
| In Distribution Gain | up to +14.1 points |
| Out Of Distribution Gain | up to +4.7 points across five held-out benchmarks |
| Policy Tokens | 30% fewer than unregularized evolution |
| Artifacts | public code and project page |
Technical impact 4
Engineering value 4
Adoption signal 2
Maturity 2
Verification cost 3
Risk 3