Summary
A study of 66,192 ClawHub skill versions separates artifact maliciousness from whether an action is allowed on a specific machine. It finds 705 unanimously clean skills from 135 publishers that still document policy-relevant actions, mostly fetching and executing remote code; a 100-skill audit estimates 92% detector precision. In live runs, 34.7% of agent commands introduced a consequence class absent from the skill's code blocks. A deterministic runtime resolver stopped all 23 forbidden actions attempted across 53 command-producing clean skills, with 67.6 ms median end-to-end hook latency.
Why it matters
For teams distributing or installing agent skills, registry scanning cannot express local operator policy and cannot fully predict commands an agent will compose at runtime. A deterministic action gate and per-resource trust ledger provide a practical second layer. The live study uses one model and a small sample, so teams should reproduce both false-positive rates and latency under their own workloads.
Technical details
| Corpus | 66,192 public ClawHub skill versions |
|---|---|
| Clean But Policy Relevant | 705 skills from 135 publishers; 506 from one publisher; 92% precision in a 100-skill hand audit |
| Runtime Drift | 50 of 144 commands (34.7%) had a consequence class absent from all documentation code blocks |
| Live Gate | 23 of 23 attempted never-graduating actions held or blocked across 53 command-producing skills |
| Latency | 67.6 ms median end-to-end through the current CLI hook, dominated by process spawn |
| Availability | Apache-2.0 code and reproducible corpus queries released |
Tags
agent-skillssupply-chainruntime-policysandboxauthorizationsecurity