ExecCritic (arXiv 2609.09133, Wed 9 Sep digest) addresses the false-confidence failure of coding agents whose tests and patches are written by the same trajectory — the errors agree, so execution feedback looks green while behavior is wrong. Its test-verify-revise scaffold separates the roles: a Test agent independently generates repository-native tests, a fail-closed harness qualifies and freezes them, and a Repair agent revises source code against their execution feedback without touching the tests. Both roles use Qwen-3.5-35B-A3B as backbone and are trained separately with role-specific RL; on Learn-to-Test and execution-feedback benchmarks the scaffold outperforms single-trajectory agents.
ExecCritic separates test-writing from repair and trains both with RL
2026-09-10Frontier watchTRIAL
ExecCritic (arXiv 2609.09133, Wed 9 Sep digest) addresses the false-confidence failure of coding agents whose tests and patches are written by the same trajectory — the errors agree, so execution feedback looks green while behavior is wrong. Its test-verify-revise scaffold separates the roles: a Test agent independently generates repository-native tests, a fail-closed harness qualifies and freezes them, and a Repair agent revises source code against their execution feedback without touching the tests. Both roles use Qwen-3.5-35B-A3B as backbone and are trained separately with role-specific RL; on Learn-to-Test and execution-feedback benchmarks the scaffold outperforms single-trajectory agents.
Imported from the existing research archive. Migration preserves the recorded judgment and is not a fresh verification.
What this means for you
The same-trajectory test-and-patch failure mode is well known to anyone running agentic CI. Teams can adopt the scaffold pattern without the RL: routing test generation and repair through different contexts already removes the correlated-error failure, and the fail-closed freeze is a one-line policy.
FRONTIER WATCH
Public artifacts and validated results are different.
The recorded fact is the public work and its artifacts. Author-reported outcomes do not establish independent reproduction. Next checks include usable code, task conditions and third-party validation.
Evidence & related material
Technical details & scores
| Arxiv | 2609.09133, Wed 9 Sep 2026 digest |
|---|---|
| Scaffold | Test agent -> fail-closed qualification/freeze -> Repair agent (cannot touch tests) |
| Training | both roles Qwen-3.5-35B-A3B, role-specific RL, trained separately |
Technical impact 3
Engineering value 4
Adoption signal 2
Maturity 2
Verification cost 2
Risk 1