A study of 66,192 ClawHub skill versions separates artifact maliciousness from whether an action is allowed on a specific machine. It finds 705 unanimously clean skills from 135 publishers that still document policy-relevant actions, mostly fetching and executing remote code; a 100-skill audit estimates 92% detector precision. In live runs, 34.7% of agent commands introduced a consequence class absent from the skill's code blocks. A deterministic runtime resolver stopped all 23 forbidden actions attempted across 53 command-producing clean skills, with 67.6 ms median end-to-end hook latency.
Runtime gate blocks policy-breaking actions from clean agent skills
2026-09-14Frontier watchTRIAL
A study of 66,192 ClawHub skill versions separates artifact maliciousness from whether an action is allowed on a specific machine. It finds 705 unanimously clean skills from 135 publishers that still document policy-relevant actions, mostly fetching and executing remote code; a 100-skill audit estimates 92% detector precision. In live runs, 34.7% of agent commands introduced a consequence class absent from the skill's code blocks. A deterministic runtime resolver stopped all 23 forbidden actions attempted across 53 command-producing clean skills, with 67.6 ms median end-to-end hook latency.
Availability: Apache-2.0 code and reproducible corpus queries released
Imported from the existing research archive. Migration preserves the recorded judgment and is not a fresh verification.
What this means for you
For teams distributing or installing agent skills, registry scanning cannot express local operator policy and cannot fully predict commands an agent will compose at runtime. A deterministic action gate and per-resource trust ledger provide a practical second layer. The live study uses one model and a small sample, so teams should reproduce both false-positive rates and latency under their own workloads.
FRONTIER WATCH
Public artifacts and validated results are different.
The recorded fact is the public work and its artifacts. Author-reported outcomes do not establish independent reproduction. Next checks include usable code, task conditions and third-party validation.
Evidence & related material
Technical details & scores
| Corpus | 66,192 public ClawHub skill versions |
|---|---|
| Clean But Policy Relevant | 705 skills from 135 publishers; 506 from one publisher; 92% precision in a 100-skill hand audit |
| Runtime Drift | 50 of 144 commands (34.7%) had a consequence class absent from all documentation code blocks |
| Live Gate | 23 of 23 attempted never-graduating actions held or blocked across 53 command-producing skills |
| Latency | 67.6 ms median end-to-end through the current CLI hook, dominated by process spawn |
| Availability | Apache-2.0 code and reproducible corpus queries released |
Technical impact 4
Engineering value 5
Adoption signal 2
Maturity 2
Verification cost 2
Risk 2