Compress and Forget: 4-bit quantization worsens recall of overwritten values

The paper asks whether post-training quantization, the default deployment path for open-weight models, worsens proactive interference (PI): the more times a value is overwritten, the harder it becomes for the model to recall its latest version, much like human working memory. The authors tested three precisions (FP16, INT8, INT4/NF4 via bitsandbytes) on three instruction-tuned models (Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.3, Phi-3.5-mini-instruct) with a fixed retrieval task. INT4 significantly cut accuracy under high interference in every model; Qwen2.5-7B fell from 81.0% to 68.3%. INT8, often assumed safe, showed a smaller but real penalty in two of the three models. Ablations trace the effect to the quantized transformer backbone rather than the output projection, and it appears only with semantically similar distractors; same-key intrusion errors rose from 21.5% to 24.6% (p = 4.8e-7), with paired McNemar's tests at p <= 2.6e-6. The takeaway: even when aggregate benchmark accuracy looks flat, 4-bit quantization can quietly tax applications that rely on long, updatable, semantically dense contexts. Code and data are released.

Aggregate benchmarks stay largely flat, but one behavior agent workloads depend on is quietly degrading: recalling the latest version of a frequently-updated value in a long, semantically dense context. For teams serving quantized open-weight models behind agent workspaces (memory systems, editable documents, tool-state tracking), PI-style probes should become part of quantization acceptance testing rather than an optional extra. The method is cheap, reproducible, and fully released.