Hybrid attention mechanisms do not fully fix attention sinks at 1M tokens

Do New Attention Mechanisms Actually Fix Attention Sinks at Million-Token Context? (arXiv 2609.08574, Wed 9 Sep digest) builds SinkProbe, a diagnostic suite measuring sink mass, massive activation, position-resolved recall and the recency gap, and applies it to four small models that differ only in how they mix tokens — including gated attention (which cut first-token attention from 46.7% to 4.8%) and Kimi K3's gated attention + Kimi Delta Attention + Attention Residuals behind a one-million-token window, eight times past the range where these diagnostics had been reported. The paper maps where each mechanism holds and where sinks and positional-recall pathologies reappear at extreme range.

The open-weight wave is shipping 262k-1M-token hybrid-attention models, and serving teams design prefix caching and eviction around assumptions about what attention does at those lengths. SinkProbe turns those assumptions into measurable quantities: check whether recall is actually uniform across the advertised window before committing cache and memory budgets.