EarlyEval cuts agent evaluation cost by stopping doomed runs early

EarlyEval (arXiv 2609.02783, Thu 3 Sep digest; SMU) reduces the cost of agent evaluation by predicting outcomes early: LightGBM success/failure classifiers over behavioral, textual, and reference-solution features stop an agent run once confidence crosses a calibrated threshold. On SWE-bench Verified, TerminalBench, and Toolathlon it eliminates 13-26% of agent steps and up to 44.1% of input tokens and 29.4% of output tokens at 89-97% accuracy, perturbing per-agent solve rates by only 1-2 points. Code and data are public.

Agent evaluation is a real line item: frontier-agent runs cost dollars to hundreds of dollars each, and most of that spend is on runs that were doomed by step three. This is a drop-in saver for any team running agent evals at scale — the 1-2-point solve-rate perturbation is the honest cost of a 30-40% token cut.