What Attention Recalls and Recurrence Controls in Hybrid Language Models (arXiv 2609.04434, Mon 7 Sep digest) introduces two cache-level interventions on hybrid attention+recurrence models — split-prefill (keep only the KV cache or only the recurrent state from a prefilled context, then generate) and state-swap (pair the KV cache from one context with the recurrent state from another in a single forward pass). On Qwen3.5 and Falcon-H1 the two channels split sharply by function: exact retrieval survives only through attention (64-98% of full accuracy) and collapses to zero through recurrence, while output language and persona reverse the pattern (surviving recurrence at 70-80% and 3-5x, with KV-only dropping to ~1% language accuracy). State-swap confirms it causally: the answer takes its value from the KV side and its language from the recurrent side. The authors' summary: attention provides a lookup over what was said; the recurrent state shapes how the model says it next.
Causal experiments split hybrid models: attention recalls, recurrence controls
What Attention Recalls and Recurrence Controls in Hybrid Language Models (arXiv 2609.04434, Mon 7 Sep digest) introduces two cache-level interventions on hybrid attention+recurrence models — split-prefill (keep only the KV cache or only the recurrent state from a prefilled context, then generate) and state-swap (pair the KV cache from one context with the recurrent state from another in a single forward pass). On Qwen3.5 and Falcon-H1 the two channels split sharply by function: exact retrieval survives only through attention (64-98% of full accuracy) and collapses to zero through recurrence, while output language and persona reverse the pattern (surviving recurrence at 70-80% and 3-5x, with KV-only dropping to ~1% language accuracy). State-swap confirms it causally: the answer takes its value from the KV side and its language from the recurrent side. The authors' summary: attention provides a lookup over what was said; the recurrent state shapes how the model says it next.
Imported from the existing research archive. Migration preserves the recorded judgment and is not a fresh verification.
What this means for you
This is the first clean causal division of labor between the two channels of the hybrid architectures now shipping in open weights (GLM-5.3-Flash, Qwen3.8-Flash-Next — trend #1), and it lands exactly where serving engineering is being designed: prefix caching and state checkpointing must treat the KV side and recurrent state differently, because only attention carries exact recall — recurrent-state-only caches silently lose lookup while preserving style and persona. Teams building hybrid-model caching or state migration should test both channels separately; this paper supplies the intervention protocol.
Public artifacts and validated results are different.
The recorded fact is the public work and its artifacts. Author-reported outcomes do not establish independent reproduction. Next checks include usable code, task conditions and third-party validation.
Evidence & related material
Technical details & scores
| Interventions | split-prefill: keep only KV cache or only recurrent state from prefilled context, then generate; state-swap: KV cache from one context + recurrent state from another in a single forward pass |
|---|---|
| Models | Qwen3.5, Falcon-H1 |
| Findings | exact retrieval survives attention only (64-98% of full accuracy), collapses to 0 through recurrence; language/persona survive recurrence (70-80%, 3-5x) while KV-only ~1% language accuracy; state-swap: answer value from KV side, language from recurrent side |
| One Liner | attention = lookup over what was said; recurrent state = how the model says it next |