MoEMB scales universal multimodal embedders with MoE encoders while keeping single-vector, non-autoregressive encoding — the first systematic study of adaptive computation for MoE embedders (training and inference). With only 3B active parameters it sets a new SOTA on MMEB-V2 and MRMR among models trained on public MMEB-family data, surpassing Think-Then-Embed approaches with >4x the active parameters.
MoEMB: multimodal embeddings scaled through the expert axis
2026-09-10Frontier watchWATCH
MoEMB scales universal multimodal embedders with MoE encoders while keeping single-vector, non-autoregressive encoding — the first systematic study of adaptive computation for MoE embedders (training and inference). With only 3B active parameters it sets a new SOTA on MMEB-V2 and MRMR among models trained on public MMEB-family data, surpassing Think-Then-Embed approaches with >4x the active parameters.
Imported from the existing research archive. Migration preserves the recorded judgment and is not a fresh verification.
What this means for you
For latency-bound retrieval services, this is the efficiency path for multimodal embedding: capacity scales through experts without autoregressive reasoning costs — the same trade GLM-5.3-Flash/Qwen Flash-Next made for LLMs, applied to the embedder.
FRONTIER WATCH
Public artifacts and validated results are different.
The recorded fact is the public work and its artifacts. Author-reported outcomes do not establish independent reproduction. Next checks include usable code, task conditions and third-party validation.
Evidence & related material
Technical details & scores
| Arxiv | 2609.08663 (Wed 9 Sep digest, announced 2026-09-10T00:00Z) |
|---|---|
| Design | MoE encoder; single-vector, non-autoregressive; first systematic adaptive-computation study for MoE embedders |
| Results | 3B active params; SOTA on MMEB-V2 + MRMR among public-MMEB-trained models; beats Think-Then-Embed with >4x active params |
| Code | not stated in abstract |
Technical impact 3
Engineering value 3
Adoption signal 2
Maturity 2
Verification cost 3
Risk 2