'Scanning the Harness' audits 3,171 public GitHub repositories (2,600 Claude Code / Cursor / Copilot / Codex configurations and 511 published skill sets), with every finding independently re-derived and dual-adjudicated: 9.8% install unpinned MCP servers; 3.1% pre-approve arbitrary execution behind seemingly scoped grants; 3.8% of skills carry pre-approved shells; 16.0% of configurations carry at least one security defect (25.5% raw scan rate). No confirmed credential exposure was found. Tools, corpus manifest, prompts and adjudications are released.
A supply-chain audit finds security defects in 16% of public coding-agent configs
2026-09-10Frontier watchTRIAL
'Scanning the Harness' audits 3,171 public GitHub repositories (2,600 Claude Code / Cursor / Copilot / Codex configurations and 511 published skill sets), with every finding independently re-derived and dual-adjudicated: 9.8% install unpinned MCP servers; 3.1% pre-approve arbitrary execution behind seemingly scoped grants; 3.8% of skills carry pre-approved shells; 16.0% of configurations carry at least one security defect (25.5% raw scan rate). No confirmed credential exposure was found. Tools, corpus manifest, prompts and adjudications are released.
Imported from the existing research archive. Migration preserves the recorded judgment and is not a fresh verification.
What this means for you
This is the first quantified security audit of the emerging MCP/skills config supply chain — the numbers say one in six public agent configs ships a defect class for which no lockfiles, install-time checks or capability vocabulary exist yet. Teams curating shared MCP configs or skill libraries should pin versions and add install-time review; platform vendors now have a measured baseline to design against.
FRONTIER WATCH
Public artifacts and validated results are different.
The recorded fact is the public work and its artifacts. Author-reported outcomes do not establish independent reproduction. Next checks include usable code, task conditions and third-party validation.
Evidence & related material
Technical details & scores
| Arxiv | 2609.07360 (Wed 9 Sep digest, announced 2026-09-10T00:00Z) |
|---|---|
| Corpus | 3,171 public repos: 2,600 agent configs (Claude Code / Cursor / Copilot / Codex) + 511 published skill sets |
| Findings | 9.8% unpinned MCP servers; 3.1% pre-approved arbitrary execution behind scoped-looking grants; 3.8% skills with pre-approved shells; 16.0% configs with >=1 defect (25.5% raw); no confirmed credential exposure |
| Method | every finding independently re-derived, dual-adjudicated |
| Artifacts | tools + corpus manifest + prompts + adjudications public |
Technical impact 3
Engineering value 5
Adoption signal 3
Maturity 3
Verification cost 2
Risk 3