Training-Free Halving of Activated Experts in Fine-Grained Mixture-of-Experts Models (arXiv 2609.04575, Mon 7 Sep digest) exploits a subtle property of fine-grained MoE routing: renormalizing router probabilities implicitly calibrates expert output gain to the training top-k, so naively reducing k at inference changes both which experts fire and their strength. The fix is one integer: activate the top k1 experts while normalizing by the probability mass of the top k2 experts — no parameters, no training, no measurable overhead. On Qwen3.6-35B-A3B, going from 8 to 4 experts costs 4.65 MMLU points under standard renormalization but only 0.35 points with k2=16, while halving routed-expert compute; the result replicates on the 11x larger Qwen3.5-397B-A17B (10 to 5 experts, 0.55-point loss with an appropriate reference set). Removing renormalization entirely is catastrophic — preserving a suitable reference mass is what matters. Perplexity and downstream accuracy also favor different k2 values, so pick by task metric.
One integer halves activated MoE experts with near-zero quality loss
Training-Free Halving of Activated Experts in Fine-Grained Mixture-of-Experts Models (arXiv 2609.04575, Mon 7 Sep digest) exploits a subtle property of fine-grained MoE routing: renormalizing router probabilities implicitly calibrates expert output gain to the training top-k, so naively reducing k at inference changes both which experts fire and their strength. The fix is one integer: activate the top k1 experts while normalizing by the probability mass of the top k2 experts — no parameters, no training, no measurable overhead. On Qwen3.6-35B-A3B, going from 8 to 4 experts costs 4.65 MMLU points under standard renormalization but only 0.35 points with k2=16, while halving routed-expert compute; the result replicates on the 11x larger Qwen3.5-397B-A17B (10 to 5 experts, 0.55-point loss with an appropriate reference set). Removing renormalization entirely is catastrophic — preserving a suitable reference mass is what matters. Perplexity and downstream accuracy also favor different k2 values, so pick by task metric.
Imported from the existing research archive. Migration preserves the recorded judgment and is not a fresh verification.
What this means for you
For anyone serving fine-grained MoE models, this is a free ~2x cut in expert-compute with a one-line change, validated across an 11x model-size range — the kind of result you test the afternoon you read it. The caveat matters: perplexity is not the right selection metric, so validate on your downstream task, and note the effect is about inference-time expert skipping on already-trained models, not training-time sparsity.
Public artifacts and validated results are different.
The recorded fact is the public work and its artifacts. Author-reported outcomes do not establish independent reproduction. Next checks include usable code, task conditions and third-party validation.
Evidence & related material
Technical details & scores
| Method | activate top k1 experts, normalize by probability mass of top k2 experts (k2 > k1); one integer, no parameters/training/overhead |
|---|---|
| Results | Qwen3.6-35B-A3B 8->4 experts: 4.65 MMLU drop (standard renorm) vs 0.35 (k2=16), halving routed-expert compute; replicates on Qwen3.5-397B-A17B (10->5, 0.55 loss); removing renormalization entirely is catastrophic |
| Caveats | perplexity and downstream accuracy favor different k2 — select by task metric |