EVOHARNESSBENCH measures what agents lose as their harness evolves

EVOHARNESSBENCH (arXiv 2609.04280, Mon 7 Sep digest) is a benchmark for evaluating agents under controlled harness evolution — placing non-stationarity not in the task stream (as continual-learning agent benchmarks do) but in the externally supplied harness of tools, skills and agents itself. It contains 17 deterministic multi-stage harness streams built from verifier-based benchmarks: 802 tasks, 520 tools, 42 skills, 62 agents. Two complementary settings isolate the central challenges: deployment evaluation (does previously accessible competence survive as the harness expands?) and self-evolving adaptation evaluation (does accumulated experience transfer to unseen harness configurations?).

Every team shipping agent platforms adds tools weekly; this is the first controlled instrument for the resulting regression class — competence quietly lost when a harness grows — plus the other direction: whether experience gained on one harness config transfers after the next platform update. The deterministic stream construction (verifier-based) means it can run in CI as a harness-change canary, which is exactly how AutoSaddler/HarnessLens-style findings should be operationalized.