{"id":"97d37aa8-c2e2-4886-9fdc-e5bfb9320687","slug":"ubuntu-mcp","kind":"mcp","name":"Ubuntu-MCP","summary":"在 Ubuntu 上让 AI 直接操控桌面","owner":"ShieldDog-ACE","repoUrl":"https://github.com/ShieldDog-ACE/Ubuntu_MCP","license":"MIT","licenseTier":"permissive","trust":"community","stars":0,"match":null,"fewStars":true,"format":"mcp","createdAt":"2026-10-25T12:52:16.000Z","addedAt":"2026-10-28T09:37:34.058Z","logoUrl":"https://avatars.githubusercontent.com/u/300552701?u=b4a6c0558dc1f77835249c0a921afecaa64ff31b&v=4","linkOnly":false,"collection":null,"skillPath":null,"categories":["cloud-infra"],"tags":["ai","at-spi2","claude","desktop-automation","linux","ubuntu"],"runtime":"python","local":true,"cloudCapable":false,"cloudCapability":"unknown","cloudReadyOnPlatform":false,"needsApiKey":false,"reviewed":false,"syncedAt":"2026-10-28T10:30:06.492Z","sample":false,"url":"https://conduits.dev/tools/ubuntu-mcp","language":"Python","transports":["stdio"],"install":{"pypi":"linux-desktop-mcp"},"requirements":{"env":[],"upstreamMayCharge":false},"tools":null,"readme":"# Linux Desktop MCP Server\n\n[![PyPI version](https://badge.fury.io/py/linux-desktop-mcp.svg)](https://pypi.org/project/linux-desktop-mcp/)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)\n\n> **Built with [Claude Code](https://claude.ai/claude-code)** - This entire MCP server was developed using Claude Code, Anthropic's AI-powered coding assistant. We're proud to showcase what's possible with AI-assisted development!\n\nAn MCP server that provides Chrome-extension-level semantic element targeting for native Linux desktop applications using AT-SPI2 (Assistive Technology Service Provider Interface).\n\n> ### ⚠️ 本仓库是衍生版本（Derivative Work）\n>\n> 本仓库并非原创项目，而是以下游开源项目的**适配版本**：\n>\n> - **上游项目**：[`BeckhamLabsLLC/linux-desktop-mcp`](https://github.com/BeckhamLabsLLC/linux-desktop-mcp)\n> - **上游版本**：v0.2.0\n> - **上游版权**：Copyright © 2025 Austin Beck，以 MIT License 发布\n> - **本仓库维护者**：[ShieldDog-ACE](https://github.com/ShieldDog-ACE)\n>\n> 本仓库在上游代码基础上只做了**本地化适配**，未改动上游核心的 AT-SPI2 语义定位与输入后端逻辑。具体差异见下方 [本地适配改动](#本地适配改动)。\n>\n> 上游代码版权归原作者所有。本仓库沿用上游的 **MIT License**（见 [LICENSE](LICENSE)），上游版权声明完整保留。\n\n## 本地适配改动\n\n相对上游 v0.2.0，本仓库的改动如下：\n\n| 类型 | 文件 | 说明 |\n|---|---|---|\n| 新增 | `src/linux_desktop_mcp/system_tools.py` | 4 个系统级桌面工具的实现，参考 [Windows-MCP](https://github.com/CursorTouch/Windows-MCP) 的工具集设计移植到 Linux |\n| 修改 | `src/linux_desktop_mcp/tool_definitions.py` | 注册上述 4 个工具的定义 |\n| 修改 | `src/linux_desktop_mcp/handlers.py` | 将 4 个工具接入 `HANDLER_MAP` 派发表 |\n| 修改 | `pyproject.toml` | 依赖收敛为 `mcp>=1.0.0,<2`，避免 MCP SDK 2.x 的破坏性变更 |\n\n目标很直接：**在 Ubuntu 上让 AI 直接操控桌面** —— 既能按语义引用点控件，也能执行命令、读写剪贴板、启动应用、发通知。\n\n## Features\n\n- **Semantic Element References**: Just like Chrome extension's `ref_1`, `ref_2` system\n- **Role Detection**: Identifies buttons, text fields, links, menus, etc.\n- **State Detection**: Tracks focused, enabled, checked, editable states\n- **Natural Language Search**: Find elements by description (\"save button\", \"search field\")\n- **Cross-Platform Input**: Works on X11, Wayland, and XWayland\n- **GTK/Qt/Electron Support**: Works with any application that exposes accessibility\n\n## Installation\n\n### System Dependencies\n\n```bash\n# Ubuntu/Debian\nsudo apt install python3-pyatspi gir1.2-atspi-2.0 at-spi2-core\n\n# For X11 input simulation\nsudo apt install xdotool\n\n# For Wayland input simulation (recommended)\n# Install ydotool from source or your package manager\n# Then start the daemon:\nsudo ydotoold &\n```\n\n### Python Package\n\n```bash\n# From PyPI\npip install linux-desktop-mcp\n\n# Or from source (this adapted repository)\ngit clone https://github.com/ShieldDog-ACE/Ubuntu_MCP.git\ncd Ubuntu_MCP\npip install -e .\n```\n\n### Enable Accessibility\n\nEnsure accessibility is enabled in your desktop environment:\n\n- **GNOME**: Settings → Accessibility → Enable accessibility features\n- **KDE**: System Settings → Accessibility\n- Most modern desktops have this enabled by default\n\n## Configuration\n\nAdd to `~/.claude/settings.json`:\n\n```json\n{\n  \"mcpServers\": {\n    \"linux-desktop\": {\n      \"command\": \"linux-desktop-mcp\"\n    }\n  }\n}\n```\n\nOr if installed from source:\n\n```json\n{\n  \"mcpServers\": {\n    \"linux-desktop\": {\n      \"command\": \"python\",\n      \"args\": [\"-m\", \"linux_desktop_mcp\"]\n    }\n  }\n}\n```\n\n## Available Tools\n\n### `desktop_snapshot`\n\nCapture the accessibility tree with semantic element references.\n\n```\nParameters:\n  app_name: str (optional) - Filter to specific application\n  max_depth: int (default: 15) - Tree traversal depth\n\nReturns:\n  Tree of elements with ref_ids:\n  - ref_1: [application] Firefox\n    - ref_2: [frame] \"GitHub - Mozilla Firefox\"\n      - ref_3: [button] \"Back\" (clickable)\n      - ref_4: [entry] \"Search or enter address\" (editable, focused)\n```\n\n### `desktop_find`\n\nFind elements by natural language query.\n\n```\nParameters:\n  query: str - \"save button\", \"search field\", \"menu containing File\"\n  app_name: str (optional)\n\nReturns:\n  Matching elements with refs, states, and actions\n```\n\n### `desktop_click`\n\nClick an element by reference or coordinates.\n\n```\nParameters:\n  ref: str - Element reference (e.g., \"ref_5\")\n  element: str - Human description for logging\n  coordinate: [x, y] - Fallback if no ref\n  button: left|right|middle\n  click_type: single|double\n  modifiers: [ctrl, shift, alt, super]\n```\n\n### `desktop_type`\n\nType text into an element.\n\n```\nParameters:\n  text: str - Text to type\n  ref: str - Element to focus first (optional)\n  element: str - Human description\n  clear_first: bool - Ctrl+A, Delete before typing\n  submit: bool - Press Enter after\n```\n\n### `desktop_key`\n\nPress keyboard keys/shortcuts.\n\n```\nParameters:\n  key: str - Key name (Return, Tab, Escape, a, etc.)\n  modifiers: [ctrl, shift, alt, super]\n```\n\n### `desktop_capabilities`\n\nCheck available automation capabilities.\n\n### 系统级工具（本地适配新增）\n\n以下 4 个工具由本仓库新增，设计参考 Windows-MCP 的同名能力，移植到 Linux 桌面。\n\n#### `desktop_shell`\n\n在系统上执行 bash 命令（带危险命令拦截）。\n\n```\nParameters:\n  command: str - 要执行的 bash 命令（最长 5000 字符）\n  timeout: int (default: 30, max: 300) - 超时秒数\n\nReturns:\n  exit code + 合并后的 stdout/stderr（截断至 10000 字符）\n```\n\n内置危险命令拦截（正则匹配即拒绝）：`shutdown`/`reboot`/`poweroff`、`mkfs`、\n`dd if=`、`rm -rf /`、`rm -rf ~`、fork bomb、`chmod -R 777 /`、覆盖块设备等。\n\n#### `desktop_clipboard`\n\n读取或写入系统剪贴板文本（Wayland 用 `wl-clipboard`，X11 用 `xclip`）。\n\n```\nParameters:\n  action: get|set - 读取或写入\n  text: str - action=set 时要写入的文本\n\nReturns:\n  action=get 时返回当前剪贴板文本；action=set 时返回确认信息\n```\n\n#### `desktop_launch`\n\n按名称启动已安装的桌面应用（解析 `.desktop` 启动项）。\n\n```\nParameters:\n  app: str - 应用名或 .desktop 文件名（如 \"firefox\"、\"code\"）\n\nReturns:\n  启动结果与进程信息\n```\n\n#### `desktop_notify`\n\n发送桌面通知（基于 `notify-send`）。\n\n```\nParameters:\n  title: str - 通知标题\n  message: str - 通知正文\n  urgency: low|normal|critical (optional) - 紧急程度\n\nReturns:\n  发送结果\n```\n\n## Example Usage\n\n### Example 1: Navigating to a Website in Firefox\n\n```\nUser: \"Open GitHub in Firefox\"\n\nClaude uses:\n1. desktop_snapshot(app_name=\"Firefox\")\n   → Returns UI tree with elements like:\n     - ref_5: [entry] \"Search or enter address\" (editable, focused)\n     - ref_12: [button] \"Go\" (clickable)\n\n2. desktop_click(ref=\"ref_5\", element=\"URL bar\")\n   → Clicks to focus the address bar\n\n3. desktop_type(text=\"https://github.com\", ref=\"ref_5\", clear_first=True, submit=True)\n   → Types the URL and presses Enter\n\nResult: Firefox navigates to GitHub\n```\n\n### Example 2: Saving a File in LibreOffice\n\n```\nUser: \"Save this document as 'report.odt'\"\n\nClaude uses:\n1. desktop_key(key=\"s\", modifiers=[\"ctrl\"])\n   → Opens the Save dialog\n\n2. desktop_snapshot(app_name=\"LibreOffice\")\n   → Returns dialog elements including:\n     - ref_8: [entry] \"File name:\" (editable)\n     - ref_15: [button] \"Save\" (clickable)\n\n3. desktop_type(text=\"report.odt\", ref=\"ref_8\", clear_first=True)\n   → Types the filename\n\n4. desktop_click(ref=\"ref_15\", element=\"Save button\")\n   → Clicks Save\n\nResult: Document saved as report.odt\n```\n\n### Example 3: Searching in a Code Editor\n\n```\nUser: \"Search for 'TODO' comments in VS Code\"\n\nClaude uses:\n1. desktop_find(query=\"search\", app_name=\"Code\")\n   → Finds search-related elements\n\n2. desktop_key(key=\"f\", modifiers=[\"ctrl\", \"shift\"])\n   → Opens global search panel\n\n3. desktop_snapshot(app_name=\"Code\")\n   → Returns search panel elements:\n     - ref_22: [entry] \"Search\" (editable, focused)\n     - ref_25: [checkbox] \"Match Case\"\n\n4. desktop_type(text=\"TODO\", ref=\"ref_22\", submit=True)\n   → Types search query and executes search\n\nResult: VS Code shows all TODO comments across the project\n```\n\n### Example 4: Window Targeting for Multi-Window Automation\n\n```\nUser: \"Help me copy data from the spreadsheet to the email\"\n\nClaude uses:\n1. desktop_context(list_available=True)\n   → Lists all available windows\n\n2. desktop_target_window(app_name=\"LibreOffice Calc\", color=\"green\")\n   → Targets spreadsheet with green border\n\n3. desktop_target_window(app_name=\"Thunderbird\", color=\"blue\")\n   → Targets email client with blue border\n\n4. desktop_snapshot()\n   → Only shows elements from targeted windows (reduced context)\n\n5. [Proceeds with copy/paste operations between windows]\n\nResult: Claude can efficiently work across multiple applications\n```\n\n## Platform Support\n\n| Feature | X11 | Wayland | XWayland |\n|---------|-----|---------|----------|\n| AT-SPI discovery | Full | Full | Full |\n| Click by ref | Full | Full | Full |\n| Type text | Full | Full | Full |\n| ydotool input | Full | Full | Full |\n| xdotool input | Full | No | Yes |\n\n## Troubleshooting\n\n### \"AT-SPI2 not available\"\n\n```bash\nsudo apt install python3-pyatspi gir1.2-atspi-2.0 at-spi2-core\n```\n\n### \"AT-SPI2 registry not running\"\n\nEnsure accessibility is enabled in your desktop settings. You may need to log out and back in.\n\n### \"No input backend available\" (Wayland)\n\n```bash\n# Install and start ydotool daemon\nsudo ydotoold &\n```\n\n### Elements not showing up\n\nSome applications may not expose accessibility information. Modern GTK3/4, Qt5/6, and Electron apps generally work well.\n\n## Architecture\n\n```\n┌─────────────────────────────────────────────────────────────┐\n│                     MCP Protocol Layer                       │\n│              (JSON-RPC over stdio, tool defs)                │\n└─────────────────────────────────────────────────────────────┘\n                              │\n┌─────────────────────────────────────────────────────────────┐\n│                  Reference Manager                           │\n│          (ref_1, ref_2 mapping, lifecycle, GC)              │\n└─────────────────────────────────────────────────────────────┘\n                              │\n          ┌───────────────────┴───────────────────┐\n          │                                       │\n┌─────────────────────┐             ┌─────────────────────────┐\n│   AT-SPI2 Backend   │             │   Input Backends        │\n│     (pyatspi)       │             │ (ydotool/xdotool/wtype) │\n└─────────────────────┘             └─────────────────────────┘\n```\n\n## Contributing\n\nThis project was created with Claude Code and we warmly welcome contributions! Whether you want to:\n\n- Report bugs or request features\n- Submit pull requests\n- Fork and build your own version\n- Improve documentation\n\nWe're very open to help and collaboration. See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.\n\n## Privacy Policy\n\nSee [PRIVACY_POLICY.md](PRIVACY_POLICY.md) for our full privacy policy. In short: Linux Desktop MCP runs entirely on your local machine, collects no data, and requires no network connectivity.\n\n## License\n\n**MIT** - See [LICENSE](LICENSE) for details.\n\n本仓库是 [`BeckhamLabsLLC/linux-desktop-mcp`](https://github.com/BeckhamLabsLLC/linux-desktop-mcp) 的衍生版本，\n因此 `LICENSE` 中保留上游原始版权声明 `Copyright (c) 2025 Austin Beck`。MIT 协议允许修改、再分发与商用，\n但要求保留该版权声明与本许可声明——本仓库遵守这一要求。本仓库新增部分的适配工作由 ShieldDog-ACE 完成。\n"}