Verifier audit: RLVR reward errors concentrate in whitespace and punctuation

'Where the Verifier Fails' (arXiv 2609.01354, Wed 2 Sep digest, sole author Esther Xin) audits the automatic verifiers that RLVR and benchmark evaluation depend on, using metamorphic testing: rewrites that preserve mathematical meaning by construction, so any rejection is a provable false negative. Across four widely used verifiers and 307,420 verdicts: self-validation rates range from 53.8% to 95.2% on identical inputs (a 41.3-point spread); two configurations of the same library disagree on 49.9% of pairs; whitespace and punctuation account for 93.0% of in-contract failures under the default LaTeX configuration (trailing periods and newlines dominate); and one numeric cascade accepts off-by-one wrong answers as a step function of magnitude — 0% below 10^4, 100% at or above — due to scale-invariant relative tolerance. Code, the transform suite, the contract matrix, and per-sample verdict records are public.

Every RLVR pipeline and every benchmark leaderboard sits on top of one of these verifiers, and the failure modes are exactly the boring ones: trailing periods, newlines, tolerance regimes. For teams running RLVR, the audit is cheap to replicate on your own reward stack before trusting reward curves — a wrong verifier silently distorts both training signal and eval rankings, and two configs of the same library disagreeing on half of pairs means verifier config is itself an experiment variable.