SLOWeave sizes prefill chunks to decode deadlines with a performance guarantee

Deadline-Aware Adaptive Prefill Chunking (arXiv 2609.07883, Wed 9 Sep digest) presents SLOWeave, an online scheduler that selects the largest prefill chunk predicted to finish before the earliest active decode deadline — replacing the fixed chunk size that trades launch overhead against latency spikes. The decision needs no workload-specific tuning and runs as a logarithmic-time search over a monotone iteration-cost model. The authors prove that whenever a decode-only iteration is feasible and the cost predictor is accurate, SLOWeave maximizes immediate prefill progress among deadline-safe choices.

Chunked prefill is standard in vLLM/SGLang-class engines, but the chunk size is a hand-tuned constant per deployment. This replaces it with a deadline-driven online decision that provably does not hurt decode SLOs — an afternoon-scale A/B upgrade for serving teams.