The paper and public code confirm the protocol mechanism, tau-bench pressure test, violation counts and single-domain limit.
Concordat enforces typed action protocols for LLM agents
2026-10-28Frontier watchTRIAL
Concordat is an open-source Python library that exposes only the tools legal in an agent’s current protocol state and rejects invalid calls before side effects. Its tau-bench airline experiment added users who pressured agents to skip confirmation. Standard agents made unauthorized changes in 13 of 50 GPT-4.1 tasks and 29 of 128 Claude Haiku 5.5 runs; Concordat made none, with no statistically significant task-success difference.
What this means for you
For teams building transactional agents, runtime-enforced protocols can make confirmation and handoff rules harder to bypass than prompt-only policies. The evidence is still one synthetic airline domain and needs reproduction against real tools, failures and state recovery.
FRONTIER WATCH
Public artifacts and validated results are different.
The recorded fact is the public work and its artifacts. Author-reported outcomes do not establish independent reproduction. Next checks include usable code, task conditions and third-party validation.
Evidence & related material
Technical details & scores
| Mechanism | finite-state typestate/session protocol; only legal tools are exposed; invalid calls are blocked before side effects |
|---|---|
| Human Boundary | human-only decisions can be requested but not executed by the model |
| Evaluation | 50 tau-bench airline tasks with confirmation pressure; 128 task-runs for Claude Haiku 5.5 |
| Results | unauthorized changes: GPT-4.1 standard 13/50; Claude Haiku 5.5 standard 29/128; Concordat 0 |
| Evidence Limit | single simulated airline domain; author-run evaluation; task success showed no significant difference |
| Artifacts | public Python library, benchmark harness and results |
| Time Basis | arXiv v1 published 2026-10-27T16:00:26Z; previous run deferred it, so this is a late discovery |
Technical impact 4
Engineering value 4
Adoption signal 2
Maturity 2
Verification cost 3
Risk 2