The paper and public data artifacts confirm corpus scale, benchmark design and author-reported results.
OpenFrontiers tests whether agents rediscover or retrieve mathematics
2026-10-28Frontier watchWATCH
OpenFrontiers publishes a 12,151-problem corpus and a 200-problem benchmark for separating retrieval of known mathematical solutions from closed-book rediscovery. The authors also provide a higher-quality 50-problem subset and Lean 4 statements. On 40 resolved HQ-50 tasks, the best closed-book agent rediscovered at most seven, while live-web success mostly retrieved published solutions.
Availability: Three public Hugging Face artifacts
What this means for you
For autonomous-research evaluations, answer accuracy alone can mistake retrieval for discovery. The dataset makes that distinction explicit, but it remains an author-designed benchmark rather than evidence of general scientific autonomy.
FRONTIER WATCH
Public artifacts and validated results are different.
The recorded fact is the public work and its artifacts. Author-reported outcomes do not establish independent reproduction. Next checks include usable code, task conditions and third-party validation.
Evidence & related material
Technical details & scores
| Corpus | 12,151 problems |
|---|---|
| Benchmark | 200 problems plus an HQ-50 subset and Lean 4 statements |
| Reported Result | At most 7 rediscoveries among 40 resolved HQ-50 tasks for the best closed-book agent |
| Availability | Three public Hugging Face artifacts |
| Time Basis | arXiv API publication 2026-10-27T16:49:33Z; discovered from the 2026-10-31 announcement feed |
Technical impact 3
Engineering value 2
Adoption signal 1
Maturity 2
Verification cost 3
Risk 2