{"id":"ccc8c713-3685-427a-b199-f4a2042b1bdf","slug":"qwen-omni-mcp","kind":"mcp","name":"qwen-omni-mcp","summary":"An MCP server that gives Claude Code and other AI agents video, image, audio, and audio-video understanding via Bailian using the multimodal Qwen3.7-Plus and Qwen3.5-Omni models.","owner":"sommio","repoUrl":"https://github.com/sommio/qwen-omni-mcp","license":"MIT","licenseTier":"permissive","trust":"community","stars":2,"match":null,"fewStars":true,"format":"mcp","createdAt":"2026-07-28T07:29:24.000Z","addedAt":"2026-10-21T17:50:00.789Z","logoUrl":"https://avatars.githubusercontent.com/u/64189793?u=b9968f19cd86d8942bdc15ced6b576014eee023c&v=4","linkOnly":false,"collection":null,"skillPath":null,"categories":["design-media","developer-tools","cloud-infra"],"tags":[],"runtime":"node","local":true,"cloudCapable":false,"cloudCapability":"unknown","cloudReadyOnPlatform":false,"needsApiKey":false,"reviewed":false,"syncedAt":"2026-10-25T22:39:14.380Z","sample":false,"url":"https://conduits.dev/tools/qwen-omni-mcp","language":"Python","transports":["stdio"],"install":{"npm":"qwen-omni-mcp"},"requirements":{"env":[],"upstreamMayCharge":false},"tools":null,"readme":"# qwen-omni-mcp\n\nAn [MCP](https://modelcontextprotocol.io) server that gives Claude Code and other AI agents **video, image, audio, and audio-video understanding** via [Bailian (DashScope)](https://bailian.console.aliyun.com/) using the multimodal **Qwen3.8-Max** and **Qwen3.5-Omni** models.\n\nQwen3.8-Max reads video natively — **no client-side frame extraction**. Qwen3.5-Omni adds native **audio** understanding (and audio-track awareness for video). Pass a public media URL **or a local file path**; the model does the rest. The server also ships MCP **instructions** that teach text-only agents to reach for these tools when they need to view/read media — while telling natively multimodal agents to prefer their own vision.\n\n## Highlights\n\n- **Native video understanding** — send a video URL or local file, get grounded analysis\n- **Image understanding** — describe, Q&A, OCR; doubles as the \"eyes\" for text-only agents whose file reader can't display images\n- **Audio understanding** — transcribe, summarize, analyze speech/sound (mp3/wav/flac/ogg/m4a/aac)\n- **Audio-video understanding** — analyze a video's visuals **and** its sound track together\n- **Thinking control** — optional per-call `thinking_budget` on every media tool; omitted = provider default\n- **Local file support** — pass a local path; files are sent inline as base64 (25MB guardrail)\n- **npx-launchable** — one line in your MCP client config\n\n## Install\n\nNo global install needed. Run directly with npx:\n\n```bash\nnpx -y qwen-omni-mcp\n```\n\nFor local development:\n\n```bash\ngit clone <this-repo>\ncd qwen-omni-mcp\nnpm install            # also installs husky git hooks\ncp .env.example .env   # fill in DASHSCOPE_API_KEY\nnpm run dev            # run from source via tsx\n```\n\n## Configuration\n\nAll config is via environment variables (loaded from `.env` by `dotenv`):\n\n| Variable               | Required | Default                                             | Description                         |\n| ---------------------- | -------- | --------------------------------------------------- | ----------------------------------- |\n| `DASHSCOPE_API_KEY`    | yes      | —                                                   | Bailian API key                     |\n| `QWEN_MODEL`           | no       | `qwen3.8-max`                                       | Model id for video/image analysis   |\n| `QWEN_OMNI_MODEL`      | no       | `qwen3.5-omni-plus`                                 | Omni model id for audio/audio-video |\n| `DASHSCOPE_BASE_URL`   | no       | `https://dashscope.aliyuncs.com/compatible-mode/v1` | OpenAI-compatible endpoint          |\n| `QWEN_REQUEST_TIMEOUT` | no       | `300`                                               | Per-request timeout in seconds      |\n\nGet a key at <https://bailian.console.aliyun.com/cn-beijing?tab=model#/api-key>.\n\n> The Anthropic-compatible `/apps/anthropic` endpoint does **not** support video input, so this server uses the OpenAI-compatible endpoint.\n\n## Use with Claude Code\n\nAdd to your MCP client config:\n\n```json\n{\n  \"mcpServers\": {\n    \"qwen-omni-mcp\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"qwen-omni-mcp\"],\n      \"env\": {\n        \"DASHSCOPE_API_KEY\": \"your-key\"\n      }\n    }\n  }\n}\n```\n\nFor local development without publishing:\n\n```json\n{\n  \"mcpServers\": {\n    \"qwen-omni-mcp\": {\n      \"command\": \"npx\",\n      \"args\": [\"tsx\", \"src/index.ts\"],\n      \"env\": { \"DASHSCOPE_API_KEY\": \"your-key\" }\n    }\n  }\n}\n```\n\n## Tools\n\n| Tool                    | Description                                                           |\n| ----------------------- | --------------------------------------------------------------------- |\n| `analyze_video`         | Analyze a video (URL or local file) with a custom prompt              |\n| `analyze_image`         | Analyze an image (URL or local file) with a custom prompt             |\n| `analyze_audio`         | Analyze an audio file (URL or local) with a custom prompt (Omni)      |\n| `analyze_audio_video`   | Analyze a video's visuals + sound (URL or local) with a prompt (Omni) |\n| `check_endpoint_status` | Show configured endpoint/model (key redacted)                         |\n\nEach media tool accepts a public `http`/`https` URL **or a local file path**. Local files are read and sent inline as base64, with a 25MB guardrail (verified up to a 14MB video / ~18MB body on Qwen3.7-Plus, and an 8.8MB video / ~11.7MB base64 body on Qwen3.5-Omni, both HTTP 200). Files larger than 25MB must be hosted at a public URL instead. Local input is validated by extension + magic-byte signature before encoding, so non-media files are rejected.\n\nEach media tool also accepts an optional `thinking_budget` (positive integer): the maximum tokens the model may spend thinking before answering. Omit it to use the provider default (thinking on at full budget for Qwen3.8 hybrid-thinking models). Thinking tokens are billed but do **not** count against `max_tokens`, which limits the answer itself.\n\n`analyze_audio` / `analyze_audio_video` use the omni model (`QWEN_OMNI_MODEL`, default `qwen3.5-omni-plus`) and force text-only output. Audio is sent as an `input_audio` block in the `data:;base64,<b64>` form with a `format` field (mp3/wav/flac/ogg/m4a/aac).\n\n## Development\n\n```bash\nnpm run typecheck     # strict tsc\nnpm run lint          # eslint, --max-warnings 0\nnpm run format:check  # prettier\nnpm test              # unit + mocked e2e (no API cost)\nnpm run build         # emit dist/\nLIVE=1 npm run test:live   # real API calls (costs tokens)\n```\n\nCI (`.github/workflows/ci.yml`) runs the same gates on Node 20/22. `secrets-scan.yml` runs gitleaks. `smoke-live.yml` (manual / weekly) runs one real image call.\n\nSee [AGENTS.md](AGENTS.md) for the full set of agent rules (never bypass hooks, never commit secrets, etc.).\n\n## License\n\nMIT\n"}