The Omni Interaction Agent technical report presents Gander, an end-to-end model unifying omni perception, realtime interaction and agentic capabilities: continuous streaming inputs across video, speech and text enable full-duplex interaction in everyday conversation and workflow-oriented agent scenarios, with user interruption, proactive intermediate feedback and follow-up questions. Two architectural designs: a Cerebellum-Brain split (Cerebellum handles realtime omni conversation; Brain handles complex reasoning and agentic tasks; they interact via tool calling and the agent orchestration runtime) and a streaming Thinker-Talker Cerebellum that flattens user inputs and model outputs into one ordered chunk-level token stream. Internal human evals: SOTA open-source spoken-dialogue naturalness with competitive omni-interaction performance, robust under background noise, multi-party interaction and backchannel communication. Released with models, code and data.
Gander: an open full-duplex omni agent released with models, code and data
The Omni Interaction Agent technical report presents Gander, an end-to-end model unifying omni perception, realtime interaction and agentic capabilities: continuous streaming inputs across video, speech and text enable full-duplex interaction in everyday conversation and workflow-oriented agent scenarios, with user interruption, proactive intermediate feedback and follow-up questions. Two architectural designs: a Cerebellum-Brain split (Cerebellum handles realtime omni conversation; Brain handles complex reasoning and agentic tasks; they interact via tool calling and the agent orchestration runtime) and a streaming Thinker-Talker Cerebellum that flattens user inputs and model outputs into one ordered chunk-level token stream. Internal human evals: SOTA open-source spoken-dialogue naturalness with competitive omni-interaction performance, robust under background noise, multi-party interaction and backchannel communication. Released with models, code and data.
Imported from the existing research archive. Migration preserves the recorded judgment and is not a fresh verification.
What this means for you
Full-duplex voice agents are shipping as commercial APIs (GPT-Live 1 GA the same week); Gander is the open-weights counterpart plus the orchestration runtime — the first open release bundling realtime omni interaction with agentic tool use, and a reference architecture (fast/slow two-tier + unified token stream) for anyone building voice agents they can inspect.
Public artifacts and validated results are different.
The recorded fact is the public work and its artifacts. Author-reported outcomes do not establish independent reproduction. Next checks include usable code, task conditions and third-party validation.
Evidence & related material
Technical details & scores
| Arxiv | 2609.08977 (Thu 10 Sep digest, announced 2026-09-11T00:00Z) |
|---|---|
| Architecture | Cerebellum-Brain split (realtime omni conversation vs complex reasoning + agentic tasks; interact via tool calling + orchestration runtime); streaming Thinker-Talker Cerebellum; chunk-level ordered token stream |
| Capabilities | full-duplex, user interruption, proactive intermediate feedback, follow-up questions; video+speech+text streaming |
| Evaluation | internal human evals: SOTA open-source spoken-dialogue naturalness; competitive omni interaction; robust to noise / multi-party / backchannel |
| Release | models + code + data released |