ScienceClaw-Eval publishes paper and code and accepts updates only after replay and independent-task checks.
ScienceClaw tests whether scientific agents retain verified improvements
2026-10-07Frontier watchTRIAL
ScienceClaw-Eval measures whether a fixed model can turn verified task repairs into persistent Skill and Operator changes across sequential scientific work. It spans 23 disciplines and evaluates correctness, improvement, retention, transfer and cost with reset tests. Public code keeps an update only after replay reproduces the repair and independent tasks improve.
Availability: public paper and code
What this means for you
For teams building self-improving agents, this replaces “the agent wrote a skill” with replay and held-out acceptance gates. Results remain author-reported, and the broad research stack should be isolated before evaluation.
FRONTIER WATCH
Public artifacts and validated results are different.
The recorded fact is the public work and its artifacts. Author-reported outcomes do not establish independent reproduction. Next checks include usable code, task conditions and third-party validation.
Evidence & related material
Technical details & scores
| Scope | 23 natural and social science disciplines |
|---|---|
| Metrics | scientific correctness, evolutionary gain, retention, cross-dataset transfer and evolution cost |
| Acceptance Gate | source-task replay plus improvement on independent scientific tasks |
| Availability | public paper and code |
Technical impact 3
Engineering value 4
Adoption signal 2
Maturity 2
Verification cost 3
Risk 4