{"id":"5cd329eb-9b9b-4605-86db-b7019838eec4","slug":"computer-harness","kind":"skill","name":"computer-harness","summary":"Native macOS computer use for agents — non-disturbing background control via CGEvent + CGWindowList + Accessibility. Sibling to browser-harness.","owner":"huytieu","repoUrl":"https://github.com/huytieu/computer-harness","license":"MIT","licenseTier":"permissive","trust":"community","stars":3,"match":null,"fewStars":true,"format":"skill","createdAt":"2026-07-23T11:09:06.000Z","addedAt":"2026-10-21T23:32:05.145Z","logoUrl":"https://avatars.githubusercontent.com/u/10217502?u=719adafc9f0e13d2caa56769d4d9e721bcebf183&v=4","linkOnly":false,"collection":null,"skillPath":null,"categories":["browser-web"],"tags":[],"runtime":"python","local":true,"cloudCapable":false,"cloudCapability":"unknown","cloudReadyOnPlatform":false,"needsApiKey":false,"reviewed":false,"syncedAt":"2026-10-26T04:27:08.693Z","sample":false,"url":"https://conduits.dev/tools/computer-harness","language":"Python","transports":["stdio"],"install":{"pypi":"computer-harness"},"requirements":{"env":[],"upstreamMayCharge":false},"tools":null,"readme":"# computer-harness\n\nThe simplest, thinnest harness to control **native macOS apps** with your agent —\nand, by default, **without disturbing you**. A desktop sibling to browser-harness:\nsame daemon + heredoc shape, but it drives any native app, not just Chrome.\n\n```bash\ncomputer-harness <<'PY'\nst = get_app_state(\"Notes\")       # background screenshot + numbered a11y tree\nclick(\"Notes\", element_index=4)   # AXPress — runs in the background\nset_value(\"Notes\", 7, \"hello\")    # set a text field directly, no keystrokes\nPY\n```\n\n## Why\n\nmacOS's ChatGPT/Codex desktop app ships computer use whose best property is that\n**it doesn't disturb you** — the agent works in one app while you keep working in\nanother. Under the hood that's OpenAI's `@oai/sky`: a signed helper that captures\na specific window and acts through the accessibility API over a local socket.\n\ncomputer-harness rebuilds that architecture on **public Apple APIs** — no private\nentitlements, no dependency on OpenAI's binary:\n\n- **Input** — CoreGraphics `CGEvent` (global foreground) and `CGEventPostToPid`\n  + `AXUIElementPerformAction` / set-value (background, non-disturbing).\n- **Pixels** — `CGWindowListCreateImage` captures a window even when it's\n  occluded or not frontmost; `screencapture` for full-screen and video.\n- **Structure** — the `AXUIElement` accessibility tree, surfaced as an indexed,\n  clickable element list (sky's interaction model).\n\n## Two tiers\n\n- **Non-disturbing (default):** operate on a target app's window in the\n  background — no cursor move, no focus steal, so you keep working in the\n  foreground. `get_app_state`, `click`, `set_value`, `perform_action`,\n  `capture_window`, `list_windows`, plus passive `capture_screenshot` and screen\n  recording. Works reliably on standard AppKit apps.\n- **Foreground (explicit, interrupts you):** global cursor/keyboard for canvas\n  apps and games. `click_at_xy`, `type_text`, `press_key`, `drag`, `scroll`,\n  `activate_app`. Opt-in only.\n\n## Install\n\nmacOS only (CoreGraphics, CGWindowList, Accessibility API).\n\n```bash\ngit clone https://github.com/huytieu/computer-harness && cd computer-harness\npython3 -m venv .venv && .venv/bin/pip install -e .\nln -sf \"$PWD/.venv/bin/computer-harness\" ~/.local/bin/computer-harness\ncomputer-harness --doctor\n```\n\nGrant the printed Python binary **Accessibility** and **Screen Recording** in\nSystem Settings → Privacy & Security (once). `--doctor` tells you what's missing.\nSee [install.md](install.md) for details, [SKILL.md](SKILL.md) for the full\nworkflow.\n\n## Security model — read before use\n\ncomputer-harness is a \"run code as me\" tool, like a shell. Specifically:\n\n- **It executes arbitrary Python from stdin as your user** (`run.py`) and can\n  synthesize input and read your screen. Only pipe it code you trust.\n- **The daemon is a standing proxy to your Accessibility + Screen Recording\n  grants.** Its control socket is `0700` (owner-only), so **another user cannot\n  reach it** — but *any process running as you* can connect and drive it, whether\n  or not that process was itself granted those TCC permissions. That is by design\n  for a single-user machine; on a shared account, treat the daemon as ambient\n  authority and stop it (`--restart` kills it) when unused.\n- **Screen captures may contain secrets.** Screenshots and recordings land\n  unencrypted in `~/.computer-harness/` (dirs `0700`, files `0600` — owner-only).\n  Nothing prunes them automatically; delete the directory to purge. Uninstalling\n  removes it.\n- **No network egress.** The tool makes no outbound connections and sends no\n  telemetry; the only socket is the local `AF_UNIX` control channel.\n\n## Status\n\nv0.1.0. macOS only. Verified end-to-end: background window capture, AX-action\nclicks, direct text set-value, full-screen foreground input, and desktop video\nrecording — all through the CLI/daemon. Not affiliated with OpenAI.\n\n## License\n\nMIT — see [LICENSE](LICENSE).\n"}