MATRIX · SGLang

Version 0.5.22 expands serving paths and fixes recurrent-state correctness

Mon, 19 Oct 2026 16:04:44 GMT

Version 0.5.22 expands serving paths and fixes recurrent-state correctness

Original story: SGLang 0.5.22 speeds model loading and fixes recurrent state corruption

Why read: A feature combination could silently resume the wrong state, requiring a correctness regression before production upgrade.

Recommendation: TRIAL

Sources: SGLang 0.5.22 release notes

Version 0.5.21 adds runtime role switching, the default Rust cache core and new decision APIs

Fri, 02 Oct 2026 16:00:56 GMT

Version 0.5.21 adds runtime role switching, the default Rust cache core and new decision APIs.

Original story: SGLang 0.5.22 speeds model loading and fixes recurrent state corruption

Why read: SGLang 0.5.21 switches prefill and decode roles at runtime.

Recommendation: TRIAL

Sources: SGLang 0.5.21 release

SGLang 0.5.22 speeds model loading and fixes recurrent state corruption

Fri, 18 Sep 2026 23:21:35 GMT

SGLang 0.5.22 adds an InstantTensor loader that the project reports as 10× faster, broader model and hardware support, and new paths for speculative decoding and prefill/decode disaggregation. It also fixes a correctness bug where hybrid linear models could resume from the wrong Mamba state when speculative decoding and structured output were combined. Python 3.11 is now required, and all performance gains remain project-reported.

Recommendation: TRIAL

Sources: sgl-project/sglang · releases / tag / v0.5.20

Training-free lossy speculative decoding lifts SGLang throughput up to 56%

Sat, 05 Sep 2026 00:00:21 GMT

'Margins, Not Windows' (arXiv 2609.02897, Fri 4 Sep digest) presents AdaptiveSpec, a training-free approach to lossy speculative decoding that co-adapts two knobs at decode time: a per-step margin verification rule that promotes mismatched draft tokens when the ratio of target to draft top-1 probability exceeds a threshold, and a draft-tree shape (depth/width/node count) adjusted from the draft model's top-1 confidence plus rolling accept history. Both signals are already available during decoding, so nothing needs training. It is implemented in the SGLang production engine and reaches up to 56% higher throughput than EAGLE-3 on SGLang across three target models (DeepSeek-R1-Distill-Llama-8B, Llama-3.1-8B-Instruct, Qwen3-8B), recovering 93-100% of lossless task accuracy on GSM8K, MATH-500, and HumanEval.

Recommendation: TRIAL

Sources: arXiv 2609.02897