Prefix caching quietly breaks reproducibility, and quantization multiplies the damage

Same Request, Different Answer (arXiv 2609.04748, Mon 7 Sep digest) measures what prefix caching — enabled by default in the major open-source serving stacks and treated as a transparent optimization — costs in reproducibility. Holding model, decoding parameters, seed and request order fixed and issuing every request serially at batch size one, the author ran an 80-episode multi-turn agentic tool-use workload with caching on and off across two engines and four weight formats. Enabling the cache changed the agent's trajectory on 36.2% of episodes at 16-bit precision and on 75.0% at 4-bit; with caching disabled, repeated execution was bit-identical in every configuration (0 of 800 episodes), bounding all other sources of nondeterminism at 0.5%. Three follow-up experiments locate the cause: a single server-level prompt-cache setting moves run-to-run outcomes, and the divergence compounds across a multi-turn session.

Anyone running quantized models with default prefix caching is silently running a different agent on every cache hit — this turns 'the same prompt gave a different answer' from a mystery into a measured, reproducible effect with a 2x jump between FP16 and INT4. For teams doing agent evals, A/B tests or regression tracking on quantized deployments, the actionable move is either pinning cache state in the harness or reporting cache configuration alongside results; at 4-bit, three quarters of episodes diverge, which is enough to swamp most effect sizes.