Tool-calling study finds action-class calibration, not execution, is the bottleneck

'Calibration is the Bottleneck' (arXiv 2609.00949, Wed 2 Sep digest) decomposes multi-turn tool-calling failures into two orthogonal modes over a four-class action space (TOOL_CALL / ASK / REFUSE / CONFIRM): action-class miscalibration and action-execution failure. It introduces a self-revealing upper bound, Acc <= GAR (Gold Action Recall), where bound violations expose state-grader masking of miscalibration. The context: on public tool-calling benchmarks, open-weight models now approach or surpass closed-source frontier models in aggregate accuracy, but aggregate numbers average over very different multi-turn situations.

For anyone building tool-using agents, this gives a decomposition you can run on your own traces: separate 'chose the wrong kind of action' (call vs ask vs refuse vs confirm) from 'executed the right action badly'. Aggregate accuracy hides the first mode entirely, and the paper's bound gives a cheap audit — if your grader only checks state, miscalibrated models look better than they are. That distinction maps directly onto where to spend effort: prompting/selection policies for calibration, tool ergonomics for execution.