The paper releases FIBS, inference code and probe weights and reports 98.8% AUC on SHADE-Arena.
Probes detect hidden agent deception before text monitors do
2026-10-09Frontier watchTRIAL
Caught in the Act scales white-box activation probes to agent-deception monitoring. The authors report 98.8% AUC on SHADE-Arena and detect hidden goals that cannot be inferred from text alone. The FIBS training data, inference code and probe weights are public.
What this means for you
For teams deploying privileged agents, internal model signals may cover blind spots in text monitors. This is still author-reported research; test cross-model transfer, false positives and activation-access cost before connecting it to a blocking path.
FRONTIER WATCH
Public artifacts and validated results are different.
The recorded fact is the public work and its artifacts. Author-reported outcomes do not establish independent reproduction. Next checks include usable code, task conditions and third-party validation.
Evidence & related material
Technical details & scores
| Reported Sha De Arena Auc | 0.988 |
|---|---|
| Hidden Goal Auc | up to 0.997 |
| Artifacts | MIT inference code, FIBS data and published probe weights |
| Method | multi-layer, multi-token white-box activation probes |
Technical impact 4
Engineering value 4
Adoption signal 1
Maturity 2
Verification cost 3
Risk 3