Study finds shell access beats typed tools on enterprise agent tasks

Microsoft and Carnegie Mellon compare five agent tool interfaces on TheAgentCompany and APEX-Agents using Opus 4.8 and GPT-5.5. Bash alone beats typed tools by 21.8–24.5 points on TheAgentCompany and 4.8–7.4 points on APEX-Agents while consuming 19–72% fewer tokens. Adding typed tools or persistent synthesized tools to Bash produces no detectable pooled gain. Programmatic tool calling narrows the action surface and uses fewer tokens than direct typed calls, but usually trails Bash on quality and cost efficiency.

For teams choosing an action interface for enterprise agents, this is evidence that a broad shell can be both more capable and cheaper when it runs inside a strong sandbox. Where policy requires a fixed catalog, programmatic tool calling remains the safer compromise. The study does not justify unrestricted host access: isolation and consequence controls are part of the architecture decision.