Overview
On August 10, the Qwen team (QwenLM) released Qwen-MM-Plugins on GitHub under Apache-2.0. It is not a model, not a UI, and not a harness; it is a protocol plugin layer that makes any agent harness multimodal-native. The core promise is *Make any agent harness multimodal-native*. A same-day commit added Qwen-Omni audio/video understanding as a new capability plugin (feat(omni-av): add Qwen-Omni audio/video understanding capability).
The release fits into the August protocol-stack arms race, complementing the August 8 Agent Plugins 1.0.0 (led by Vercel). Where Agent Plugins 1.0.0 standardizes the protocol layer, Qwen-MM-Plugins standardizes the multimodal capability layer; together they complete the picture of "any harness can call any AI capability out of the box."
Architecture: Skill + MCP Server
Each capability plugin = a Skill (so the model knows the tool exists) + an MCP Server (the tool implementation). The benefit is harness decoupling: Claude Code, Codex, Qoder, OpenClaw, Qwen Code, Gemini CLI, opencode, pi, and QwenPaw need no code changes — just an MCP configuration.
Seven Capability Plugins
- core (default-installed, base vision) — dynamic-resolution image reading (4K screenshots, documents, 3D models, PDFs auto-split into VL-model patches), OCR, grounding (bounding boxes), segmentation, ASR, vision chat, web search (Serper API key required), image search, media-info metadata.
- video-memory (long-video memory) — hierarchical graph memory supporting >2 hour video QA. Memory is a hierarchical graph, not linear KV cache, so query efficiency stays stable as video length grows.
- omni-av (Qwen-Omni audio/video, new August 10) — 7 tools:
omni_asr(plain text transcription),omni_asr_timestamped(SRT sentence/word timestamps),omni_multi_speaker_asr(multi-speaker separation),omni_av_caption(time-segment descriptions),omni_av_grounding(text query to event timestamp),omni_av_counting(event/action/object counts),omni_music_caption(genre/mood/instrument/key/vocal tags). - video-edit — editing workflows plus image/video/audio generation integrations.
- blender (3D modeling) — 22 tools, thin-client mode (local Blender required, driven via remote Python).
- freecad (parametric CAD) — 14 tools, thin-client mode, STEP/STL import-export and FEM analysis.
- edu-agent (instructional videos) — converts math/science problems into Chinese explainer videos. Pure Skill mode, no MCP — capability is delivered as agent prompt templates.
- Past 12 months — multimodal features were model-native (Qwen-VL, GPT-4V, Gemini 1.5 Pro, Claude 3.5 Sonnet). Switching models meant rewriting tool-calling code.
- Past 6 months — Anthropic's MCP protocol (November) began standardizing tool calls, but multimodal capability remained model-bound.
- Qwen-MM-Plugins — splits Qwen's multimodal capabilities into 7 independent plugins, each exposed via MCP. Harnesses add one MCP config to call Qwen's multimodal tools without being locked to the Qwen model.
- "AI coding tool reads images" is now standardized —
vision_chatworks in any harness via MCP, replacing Claude Code's Read tool, Codex's vision API, and opencode's file-include mechanism. - "AI coding tool edits Blender" is now solvable — 22 thin Blender wrappers replace custom Python servers.
- "AI coding tool watches long videos" now has a memory model — hierarchical graph memory replaces "dump all frames" or "summarize" heuristics, with log-scaling rather than linear growth.
- Multimodal capability portability is now protocol-guaranteed — switch from Claude Code to Codex without code changes; only the MCP config changes.
- Apache-2.0 main, MIT exception — Blender and FreeCAD plugins include MIT-licensed third-party code (for direct Blender/FreeCAD Python API calls); all others are Apache-2.0.
- One-line install —
curl -fsSL https://raw.githubusercontent.com/QwenLM/Qwen-MM-Plugins/main/install.sh | bashfor the core plugin; each harness adds its own MCP config. - WSL2 is mandatory on Windows — the docs explicitly require WSL2 (Ubuntu recommended); clone inside the WSL home directory, not on
/mnt/cmount points. No cross-platform out-of-the-box support. - Cookbooks largely TBD — only the
corecookbook is complete; others are marked "TBD," so the August 10 release mainly demos core capabilities. - System dependencies —
ffmpeg,libreoffice,blender,texlive,chromium. A full install requires ~2–3 GB and ~30 minutes on macOS. - DashScope API key is required — Qwen-MM-Plugins is not an independent capability layer; it is a capability wrapper around the Qwen model ecosystem. Serper API key is optional (for web search).
- 8-08 Agent Plugins 1.0.0 — Vercel-led protocol-layer standardization.
- 8-09 Microsoft SkillOpt — experience-layer portability.
- 8-10 OpenChamber — harness / runtime / UI three-layer decoupling.
- 8-10 Qwen-MM-Plugins — multimodal capability-layer protocolization.
- Repository: https://github.com/QwenLM/Qwen-MM-Plugins
- Chinese README: https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/README.zh.md
- Alibaba_Qwen announcement: https://x.com/Alibaba_Qwen/status/2086664887560970531
- Installation docs: https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/docs/en/installation.md
- Cookbooks index: https://github.com/QwenLM/Qwen-MM-Plugins/tree/main/cookbooks
- One-line installer: https://raw.githubusercontent.com/QwenLM/Qwen-MM-Plugins/main/install.sh
- DashScope API key: https://dashscope.console.aliyun.com/apiKey
- Serper API key: https://serper.dev/
Why This Matters for AI Coding Workflows
The trajectory of multimodal capability delivery:
Concrete wins for AI coding:
Practical Notes and Constraints
August AI Coding Tool-Chain Timeline
August 10 marks the "full-stack decoupling" moment for AI coding tooling: harness (opencode), runtime (OpenChamber), capability layer (Qwen-MM-Plugins), experience layer (SkillOpt), protocol layer (Agent Plugins 1.0.0), model layer. Each layer now has an independent open-source replacement, turning "AI coding tools" from single-vendor products into assemblies of swappable parts.