VLoc Bench finds agents struggle to locate vulnerable code

VLoc Bench pairs vulnerable and patched snapshots for 500 real vulnerabilities across 290 repositories. Agents receive a weakness description and read-only terminal access, then identify affected files or say the patch removed the flaw. Across 27 models and four static analyzers, the best File F1 reaches only 0.229; in 38.4% of tasks no tested model finds a correct location. Even stronger locators can report unsupported files after a fix.

For teams deploying security agents, file localization and abstention on patched code need separate acceptance tests. A high-level vulnerability verdict alone does not establish reliable repository navigation.