Content-Based Addressing for Long Context (arXiv 2609.07314, Wed 9 Sep digest) replaces RoPE's growing positional counter: the token stream is divided into units, ordinary RoPE positions are retained within each unit, and every completed unit gets an address computed from its content. Adding units applies the same learned map to new content rather than extending a positional range or identifier table. The authors prove the construction preserves local RoPE exactly and leaves attention over non-local content unchanged in form.
Content-based addressing replaces positional stretching for long context
2026-09-10Frontier watchWATCH
Content-Based Addressing for Long Context (arXiv 2609.07314, Wed 9 Sep digest) replaces RoPE's growing positional counter: the token stream is divided into units, ordinary RoPE positions are retained within each unit, and every completed unit gets an address computed from its content. Adding units applies the same learned map to new content rather than extending a positional range or identifier table. The authors prove the construction preserves local RoPE exactly and leaves attention over non-local content unchanged in form.
Imported from the existing research archive. Migration preserves the recorded judgment and is not a fresh verification.
What this means for you
Every rescaling/interpolation scheme attacks the positional train-test mismatch by stretching a counter; this removes the counter for macro structure and lets content identity do the addressing. Early and small-scale, but a genuinely different primitive for the 1M-context problem — DeepSeek V4.1-Flash's CSA2 takes the sparse-indexing route to the same end (ev-20260910-01).
FRONTIER WATCH
Public artifacts and validated results are different.
The recorded fact is the public work and its artifacts. Author-reported outcomes do not establish independent reproduction. Next checks include usable code, task conditions and third-party validation.
Evidence & related material
Technical details & scores
| Arxiv | 2609.07314, Wed 9 Sep 2026 digest |
|---|---|
| Construction | units with internal RoPE; content-computed unit addresses; no positional-range extension |
| Guarantees | local RoPE preserved exactly; non-local attention unchanged in form |
Technical impact 4
Engineering value 2
Adoption signal 2
Maturity 1
Verification cost 3
Risk 2