Summary

'Scanning the Harness' audits 3,171 public GitHub repositories (2,600 Claude Code / Cursor / Copilot / Codex configurations and 511 published skill sets), with every finding independently re-derived and dual-adjudicated: 9.8% install unpinned MCP servers; 3.1% pre-approve arbitrary execution behind seemingly scoped grants; 3.8% of skills carry pre-approved shells; 16.0% of configurations carry at least one security defect (25.5% raw scan rate). No confirmed credential exposure was found. Tools, corpus manifest, prompts and adjudications are released.

Why it matters
This is the first quantified security audit of the emerging MCP/skills config supply chain — the numbers say one in six public agent configs ships a defect class for which no lockfiles, install-time checks or capability vocabulary exist yet. Teams curating shared MCP configs or skill libraries should pin versions and add install-time review; platform vendors now have a measured baseline to design against.
Technical details
Arxiv 2609.07360 (Wed 9 Sep digest, announced 2026-09-10T00:00Z)
Corpus 3,171 public repos: 2,600 agent configs (Claude Code / Cursor / Copilot / Codex) + 511 published skill sets
Findings 9.8% unpinned MCP servers; 3.1% pre-approved arbitrary execution behind scoped-looking grants; 3.8% skills with pre-approved shells; 16.0% configs with >=1 defect (25.5% raw); no confirmed credential exposure
Method every finding independently re-derived, dual-adjudicated
Artifacts tools + corpus manifest + prompts + adjudications public
Tags
agent-securitymcpsupply-chainauditskills