Argo-Bench releases tasks, a harness and a large synthetic warehouse with outcome-based grading.
Argo-Bench grades data agents by the business actions they produce
2026-10-02Frontier watchTRIAL
Argo-Bench releases 210 enterprise-style data tasks over a synthetic warehouse with 235 tables and 7.5 billion rows. It scores the filings and operational actions produced by an agent rather than the plausibility of its prose. Across 14 tested models, the strongest system reached at least 95% on only 34.8% of tasks and averaged 59.5%, according to the authors.
What this means for you
For teams deploying data agents, this benchmark tests whether a system can reconstruct business state and take the right downstream action, not merely write a convincing SQL explanation. The public harness and warehouse make a bounded internal-model comparison possible.
FRONTIER WATCH
Public artifacts and validated results are different.
The recorded fact is the public work and its artifacts. Author-reported outcomes do not establish independent reproduction. Next checks include usable code, task conditions and third-party validation.
Evidence & related material
Technical details & scores
| Tasks | 210 |
|---|---|
| Warehouse | 235 tables and 7.5 billion synthetic rows; 71 GiB public dataset |
| Evaluation | outcome-based grading of filings and operational actions |
| Author Result | best of 14 models averaged 59.5%; at least 95% on 34.8% of tasks |
| Artifacts | Apache-2.0 harness, CC-BY warehouse and 20 reference solutions; answer keys remain server-side |
Technical impact 3
Engineering value 4
Adoption signal 2
Maturity 3
Verification cost 4
Risk 2