Summary

ParaRecover moves agent evaluation from final success to the process of diagnosing and recovering from failures across parallel tool branches. Its 10,626 instances cover 14 error types spanning dependency planning, tool selection and argument matching. The SDE rubric scores structural integrity, diagnostic reasoning and replanning strategy. Tests on more than ten models find persistent weakness in multi-turn error propagation, implicit failures and precise replanning; MIT-licensed code and data are public.

Why it matters
For teams operating parallel or multi-agent workflows, a final pass rate hides whether recovery is reliable or accidental. ParaRecover provides a regression suite for localization and replanning before these paths reach production. Because most scoring still depends on a research rubric and the repository has no visible adoption, it should complement environment-specific failure injection rather than replace it.
Technical details
Dataset 10,626 instances at two difficulty levels
Taxonomy 14 error types across planning dependencies, tool selection and argument matching
Rubric SDE: structural integrity, diagnostic reasoning and evolutionary/replanning strategy
Model Coverage more than ten mainstream LLMs
Availability MIT-licensed GitHub repository with data and evaluation code
Adoption 0 GitHub stars at scan cutoff
Tags
agent-evaluationparallel-tool-useerror-recoveryreplanningbenchmark