One integer halves activated MoE experts with near-zero quality loss

Training-Free Halving of Activated Experts in Fine-Grained Mixture-of-Experts Models (arXiv 2609.04575, Mon 7 Sep digest) exploits a subtle property of fine-grained MoE routing: renormalizing router probabilities implicitly calibrates expert output gain to the training top-k, so naively reducing k at inference changes both which experts fire and their strength. The fix is one integer: activate the top k1 experts while normalizing by the probability mass of the top k2 experts — no parameters, no training, no measurable overhead. On Qwen3.6-35B-A3B, going from 8 to 4 experts costs 4.65 MMLU points under standard renormalization but only 0.35 points with k2=16, while halving routed-expert compute; the result replicates on the 11x larger Qwen3.5-397B-A17B (10 to 5 experts, 0.55-point loss with an appropriate reference set). Removing renormalization entirely is catastrophic — preserving a suitable reference mass is what matters. Perplexity and downstream accuracy also favor different k2 values, so pick by task metric.

For anyone serving fine-grained MoE models, this is a free ~2x cut in expert-compute with a one-line change, validated across an 11x model-size range — the kind of result you test the afternoon you read it. The caveat matters: perplexity is not the right selection metric, so validate on your downstream task, and note the effect is about inference-time expert skipping on already-trained models, not training-time sparsity.