Same Request, Different Answer (arXiv 2609.04748, Mon 7 Sep digest) measures what prefix caching — enabled by default in the major open-source serving stacks and treated as a transparent optimization — costs in reproducibility. Holding model, decoding parameters, seed and request order fixed and issuing every request serially at batch size one, the author ran an 80-episode multi-turn agentic tool-use workload with caching on and off across two engines and four weight formats. Enabling the cache changed the agent's trajectory on 36.2% of episodes at 16-bit precision and on 75.0% at 4-bit; with caching disabled, repeated execution was bit-identical in every configuration (0 of 800 episodes), bounding all other sources of nondeterminism at 0.5%. Three follow-up experiments locate the cause: a single server-level prompt-cache setting moves run-to-run outcomes, and the divergence compounds across a multi-turn session.
Prefix caching quietly breaks reproducibility, and quantization multiplies the damage
Same Request, Different Answer (arXiv 2609.04748, Mon 7 Sep digest) measures what prefix caching — enabled by default in the major open-source serving stacks and treated as a transparent optimization — costs in reproducibility. Holding model, decoding parameters, seed and request order fixed and issuing every request serially at batch size one, the author ran an 80-episode multi-turn agentic tool-use workload with caching on and off across two engines and four weight formats. Enabling the cache changed the agent's trajectory on 36.2% of episodes at 16-bit precision and on 75.0% at 4-bit; with caching disabled, repeated execution was bit-identical in every configuration (0 of 800 episodes), bounding all other sources of nondeterminism at 0.5%. Three follow-up experiments locate the cause: a single server-level prompt-cache setting moves run-to-run outcomes, and the divergence compounds across a multi-turn session.
Imported from the existing research archive. Migration preserves the recorded judgment and is not a fresh verification.
What this means for you
Anyone running quantized models with default prefix caching is silently running a different agent on every cache hit — this turns 'the same prompt gave a different answer' from a mystery into a measured, reproducible effect with a 2x jump between FP16 and INT4. For teams doing agent evals, A/B tests or regression tracking on quantized deployments, the actionable move is either pinning cache state in the harness or reporting cache configuration alongside results; at 4-bit, three quarters of episodes diverge, which is enough to swamp most effect sizes.
Public artifacts and validated results are different.
The recorded fact is the public work and its artifacts. Author-reported outcomes do not establish independent reproduction. Next checks include usable code, task conditions and third-party validation.
Evidence & related material
Technical details & scores
| Experiment | 80-episode multi-turn agentic tool-use workload; 2 engines x 4 weight formats; model/seed/order fixed; serial, batch size 1 |
|---|---|
| Divergence Rates | cache ON changed trajectory: 36.2% of episodes at 16-bit; 75.0% at 4-bit; cache OFF bit-identical (0/800), bounding other nondeterminism at 0.5% |
| Cause Localization | a single server-level prompt-cache setting moves run-to-run outcomes; divergence compounds across multi-turn sessions |