Tail-Replay (arXiv 2608.30310, Tue 1 Sep digest) enables token-level prefix reuse for hybrid LLMs that interleave full-attention and linear-attention layers. Full-attention KV caches are token-addressable, but linear-attention recurrent states cannot roll back to arbitrary boundaries, so existing hybrid prefix caching stores recurrent-state checkpoints and only reuses prefixes at checkpoint-aligned positions. Tail-Replay replays the tail of the sequence from the nearest state checkpoint, allowing prefix reuse at any token boundary, removing the discrete-boundary constraint. The same digest carries DASC (2608.30386), which compresses hybrid recurrent-state checkpoints by exploiting per-head/per-channel 'retention horizons' in Gated DeltaNet and Kimi Delta Attention.
Tail-Replay brings unconstrained prefix caching to hybrid attention LLMs
Tail-Replay (arXiv 2608.30310, Tue 1 Sep digest) enables token-level prefix reuse for hybrid LLMs that interleave full-attention and linear-attention layers. Full-attention KV caches are token-addressable, but linear-attention recurrent states cannot roll back to arbitrary boundaries, so existing hybrid prefix caching stores recurrent-state checkpoints and only reuses prefixes at checkpoint-aligned positions. Tail-Replay replays the tail of the sequence from the nearest state checkpoint, allowing prefix reuse at any token boundary, removing the discrete-boundary constraint. The same digest carries DASC (2608.30386), which compresses hybrid recurrent-state checkpoints by exploiting per-head/per-channel 'retention horizons' in Gated DeltaNet and Kimi Delta Attention.
Imported from the existing research archive. Migration preserves the recorded judgment and is not a fresh verification.
What this means for you
The open-weight frontier just moved to hybrid architectures (GLM-5.3-Flash, Qwen3.8-Flash-Next), and prefix caching is the single biggest cost lever in agentic serving — yet naive hybrid serving throws most of it away because recurrent states cannot be rolled back. This pair of papers is the serving stack catching up: any team self-hosting hybrid-attention models should track these techniques for their vLLM/SGLang roadmaps before assuming cache economics from full-attention models carry over.
Public artifacts and validated results are different.
The recorded fact is the public work and its artifacts. Author-reported outcomes do not establish independent reproduction. Next checks include usable code, task conditions and third-party validation.
Evidence & related material
Technical details & scores
| Arxiv | 2608.30310 (Tail-Replay), announced in the Tue 1 Sep 2026 digest |
|---|---|
| Problem | hybrid LLM prefix caching: full-attention KV is token-addressable; linear-attention recurrent states are not roll-back-able, so existing methods store state checkpoints and constrain reuse to checkpoint-aligned boundaries |
| Mechanism | replay the sequence tail from the nearest stored recurrent state to reach arbitrary token-level prefix boundaries |
| Cluster | same-window companion work: DASC (2608.30386) — decay-aware compression of hybrid recurrent-state checkpoints via per-head/per-channel retention horizons (Gated DeltaNet, Kimi Delta Attention); DAMP (2608.27513) — decay-aware mixed-precision recurrent-state quantization |
| Relevance | targets the serving cost of the hybrid-attention open-weight wave (GLM-5.3-Flash, Qwen3.8-Flash-Next) |