The paper reports that truthful tool output missing a critical fact can induce unauthorized agent action.
EpiSEM-Bench exposes unsafe agent actions when tools omit critical evidence
2026-10-23Frontier watchWATCH
EpiSEM-Bench uses 300 matched safe and unsafe task pairs where every tool statement is true but one fact needed to judge permission is omitted. Across ten models, unsafe execution, active evidence recovery and correct post-recovery action vary substantially. Providing a recovery path reduces unsafe action for every model but does not ensure agents use it. No public implementation was found.
What this means for you
For tool-using agent teams, risk comes not only from false data or malicious prompts: truthful but incomplete tool output can still cause unauthorized action. Permission facts should be mandatory runtime inputs with deterministic gates, not optional follow-up questions.
FRONTIER WATCH
Public artifacts and validated results are different.
The recorded fact is the public work and its artifacts. Author-reported outcomes do not establish independent reproduction. Next checks include usable code, task conditions and third-party validation.
Evidence & related material
Technical details & scores
| Benchmark | 300 matched unsafe/safe task pairs across six domains |
|---|---|
| Models | 10 |
| Failure Mode | critical safety fact omitted while returned statements remain true |
| Audit | independent tool checks evidence seeking and appropriate action |
| Result Boundary | recovery access reduced unsafe execution for all tested models, but active recovery varied |
| Artifact | paper only during this run; no public code found |
| Time Basis | arXiv v1 submitted 2026-10-22T17:57:04Z |
Technical impact 4
Engineering value 4
Adoption signal 1
Maturity 1
Verification cost 3
Risk 3