MoEMB: multimodal embeddings scaled through the expert axis

MoEMB scales universal multimodal embedders with MoE encoders while keeping single-vector, non-autoregressive encoding — the first systematic study of adaptive computation for MoE embedders (training and inference). With only 3B active parameters it sets a new SOTA on MMEB-V2 and MRMR among models trained on public MMEB-family data, surpassing Think-Then-Embed approaches with >4x the active parameters.

For latency-bound retrieval services, this is the efficiency path for multimodal embedding: capacity scales through experts without autoregressive reasoning costs — the same trade GLM-5.3-Flash/Qwen Flash-Next made for LLMs, applied to the embedder.