An end-to-end system for online draft-model co-training during RL post-training: rank-local branch attention merged into compact zigzag-ring attention (context parallelism), and a separate TapChannel path moves intermediate target features across pipeline stages. The system achieves significant rollout and end-to-end acceleration on 122B-scale models with strong context-parallel scaling at 256K tokens. Code is available (link in the paper).
Speculative decoding brought into RL rollout at 122B scale
2026-09-10Frontier watchTRIAL
An end-to-end system for online draft-model co-training during RL post-training: rank-local branch attention merged into compact zigzag-ring attention (context parallelism), and a separate TapChannel path moves intermediate target features across pipeline stages. The system achieves significant rollout and end-to-end acceleration on 122B-scale models with strong context-parallel scaling at 256K tokens. Code is available (link in the paper).
Imported from the existing research archive. Migration preserves the recorded judgment and is not a fresh verification.
What this means for you
RL rollout generation is the dominant cost of RL post-training, and long-context rollouts have been the hardest to accelerate. This shows the speculative-decoding machinery surviving contact with a real distributed training stack at a scale and context length most teams cannot run.
FRONTIER WATCH
Public artifacts and validated results are different.
The recorded fact is the public work and its artifacts. Author-reported outcomes do not establish independent reproduction. Next checks include usable code, task conditions and third-party validation.
Evidence & related material
Technical details & scores
| Arxiv | 2609.07108 (Wed 9 Sep digest, announced 2026-09-10T00:00Z) |
|---|---|
| Mechanism | online draft co-training during RL; rank-local branch attention -> compact zigzag-ring attention (context parallel); TapChannel crosses pipeline stages with intermediate target features |
| Results | significant rollout + end-to-end acceleration at 122B scale; strong CP scaling at 256K tokens |
| Code | available (link in paper) |
Technical impact 4
Engineering value 3
Adoption signal 2
Maturity 2
Verification cost 4
Risk 2