Tail-Replay brings unconstrained prefix caching to hybrid attention LLMs

Tail-Replay (arXiv 2608.30310, Tue 1 Sep digest) enables token-level prefix reuse for hybrid LLMs that interleave full-attention and linear-attention layers. Full-attention KV caches are token-addressable, but linear-attention recurrent states cannot roll back to arbitrary boundaries, so existing hybrid prefix caching stores recurrent-state checkpoints and only reuses prefixes at checkpoint-aligned positions. Tail-Replay replays the tail of the sequence from the nearest state checkpoint, allowing prefix reuse at any token boundary, removing the discrete-boundary constraint. The same digest carries DASC (2608.30386), which compresses hybrid recurrent-state checkpoints by exploiting per-head/per-channel 'retention horizons' in Gated DeltaNet and Kimi Delta Attention.

The open-weight frontier just moved to hybrid architectures (GLM-5.3-Flash, Qwen3.8-Flash-Next), and prefix caching is the single biggest cost lever in agentic serving — yet naive hybrid serving throws most of it away because recurrent states cannot be rolled back. This pair of papers is the serving stack catching up: any team self-hosting hybrid-attention models should track these techniques for their vLLM/SGLang roadmaps before assuming cache economics from full-attention models carry over.