ParaRecover moves agent evaluation from final success to the process of diagnosing and recovering from failures across parallel tool branches. Its 10,626 instances cover 14 error types spanning dependency planning, tool selection and argument matching. The SDE rubric scores structural integrity, diagnostic reasoning and replanning strategy. Tests on more than ten models find persistent weakness in multi-turn error propagation, implicit failures and precise replanning; MIT-licensed code and data are public.
ParaRecover evaluates recovery across parallel tool-call failures
2026-09-14Frontier watchTRIAL
ParaRecover moves agent evaluation from final success to the process of diagnosing and recovering from failures across parallel tool branches. Its 10,626 instances cover 14 error types spanning dependency planning, tool selection and argument matching. The SDE rubric scores structural integrity, diagnostic reasoning and replanning strategy. Tests on more than ten models find persistent weakness in multi-turn error propagation, implicit failures and precise replanning; MIT-licensed code and data are public.
Availability: MIT-licensed GitHub repository with data and evaluation code
Imported from the existing research archive. Migration preserves the recorded judgment and is not a fresh verification.
What this means for you
For teams operating parallel or multi-agent workflows, a final pass rate hides whether recovery is reliable or accidental. ParaRecover provides a regression suite for localization and replanning before these paths reach production. Because most scoring still depends on a research rubric and the repository has no visible adoption, it should complement environment-specific failure injection rather than replace it.
FRONTIER WATCH
Public artifacts and validated results are different.
The recorded fact is the public work and its artifacts. Author-reported outcomes do not establish independent reproduction. Next checks include usable code, task conditions and third-party validation.
Evidence & related material
Technical details & scores
| Dataset | 10,626 instances at two difficulty levels |
|---|---|
| Taxonomy | 14 error types across planning dependencies, tool selection and argument matching |
| Rubric | SDE: structural integrity, diagnostic reasoning and evolutionary/replanning strategy |
| Model Coverage | more than ten mainstream LLMs |
| Availability | MIT-licensed GitHub repository with data and evaluation code |
| Adoption | 0 GitHub stars at scan cutoff |
Technical impact 3
Engineering value 4
Adoption signal 1
Maturity 2
Verification cost 2
Risk 2