Qwen3-Omni · 30B-A3B Instruct
Ask questions about recordings alongside text, images or video; the speech-output path adds Talker and Code2Wav stages.
UnderstandingOpen weights30B total / 3B active
- Start here
- Use the author's reference example with a short clip you know. Ask which sound sources are present, then compare with CLAP candidate descriptions. Record unsupported details, latency and peak memory before testing shared serving.
- Training data
- The card describes multimodal training and audio evaluations, without a recording-level corpus inventory. MoE routes work among trained expert blocks; it does not combine arbitrary standalone models.
- Training rights
- The checkpoint's published license does not identify the permissions for every underlying training recording.
- License & output terms
- Apache-2.0 on the linked Instruct checkpoint; retain its license and notices.
- Limitations
- Audio understanding and speech generation do not establish editable MIDI output or a controllable musical instrument. Active parameter count is not total weight memory. No Maru integration is listed.
- Technique
- Multimodal Thinker–Talker mixture of experts
- Runtime
- Author examples use Transformers and vLLM; vLLM-Omni supports staged serving. Check exact checkpoint, software versions and accelerator memory. One endpoint can span multiple workers and model stages.
- Parameters
- 30B total / 3B active
Instruct checkpoint card (opens in a new tab)Architecture and Transformers examples (opens in a new tab)Staged model serving (opens in a new tab)
Sources reviewed . Training and rights notes summarize provider disclosures. A model license does not clear every source recording or generated output. Parameter counts refer to the named variant or component, not total runtime memory.