AMD published the Instella-MoE technical report (arXiv 2609.00791, Wed 2 Sep digest): a fully open 16B-total / 2.8B-active MoE trained from scratch on AMD Instinct MI300X and MI325X GPUs. It combines Gated Multi-head Latent Attention and FarSkip-Collective connectivity, and was built through a multi-stage pipeline: pre-training, mid-training, long-context extension, SFT with feedback-driven data curation, DPO, and RL with Multi-Teacher On-Policy Distillation.
AMD open-sources Instella-MoE, a 16B MoE trained entirely on Instinct GPUs
2026-09-02Frontier watchWATCH
AMD published the Instella-MoE technical report (arXiv 2609.00791, Wed 2 Sep digest): a fully open 16B-total / 2.8B-active MoE trained from scratch on AMD Instinct MI300X and MI325X GPUs. It combines Gated Multi-head Latent Attention and FarSkip-Collective connectivity, and was built through a multi-stage pipeline: pre-training, mid-training, long-context extension, SFT with feedback-driven data curation, DPO, and RL with Multi-Teacher On-Policy Distillation.
Imported from the existing research archive. Migration preserves the recorded judgment and is not a fresh verification.
What this means for you
The open-weight frontier is currently a Chinese-lab story with a US response forming via Nvidia/Poolside; Instella-MoE adds a third vector — open weights proven out on non-NVIDIA silicon at scale. At 16B it is not frontier-tier, but it is a credible reference for anyone evaluating MI300X/MI325X training stacks, and MLA-class architecture choices show AMD targeting the same serving-efficiency conventions as DeepSeek-family models.
FRONTIER WATCH
Public artifacts and validated results are different.
The recorded fact is the public work and its artifacts. Author-reported outcomes do not establish independent reproduction. Next checks include usable code, task conditions and third-party validation.
Evidence & related material
Technical details & scores
| Arxiv | 2609.00791, announced in the Wed 2 Sep 2026 digest |
|---|---|
| Scale | 16B total / 2.8B active MoE |
| Hardware | trained entirely on AMD Instinct MI300X and MI325X |
| Architecture | Gated Multi-head Latent Attention (Gated MLA); FarSkip-Collective connectivity |
| Pipeline | pre-training -> mid-training -> long-context extension -> SFT (feedback-driven data curation) -> DPO -> RL with Multi-Teacher On-Policy Distillation |
| Openness | fully open (technical report + model family on Hugging Face under AMD org) |
Technical impact 3
Engineering value 3
Adoption signal 2
Maturity 3
Verification cost 2
Risk 2