uiacli

AI agents can browse the web — but they can't click buttons in Notepad, fill forms in legacy enterprise apps, or automate anything without a browser. UIA CLI fixes that. It gives any AI agent full control of any Windows application through simple JSON commands.

Quick Start

Install and try in 30 seconds — no .NET SDK or Go toolchain required:

# Install
irm https://raw.githubusercontent.com/amitse/uiacli/master/install.ps1 | iex

# Try it
uia windows                              # See every open window
uia tree Calculator --depth 2            # Inspect Calculator's UI
uia click --window Calculator --name Five # Click the "5" button
uia batch actions.json --verbose          # Run a batch with visual overlay

Or download the zip and extract anywhere.

🤖 Agent Skill — works with 50+ AI agents

Give your AI coding agent the ability to control Windows apps:

npx skills add amitse/uiacli

Works with Claude Code, GitHub Copilot, Cursor, Codex, Windsurf, Roo Code, and 50+ more agents. Once installed, just ask your agent to interact with any Windows app.

Why UIA CLI vs Alternatives

Features

• 20+ commands — windows, tree, click, type, key, batch, screenshot, clipboard, overlay, and more
• Action batching — 7 actions in one HTTP call, no per-command overhead
• Visual overlay — ghost cursor, highlights, and annotations so humans can follow what the agent does
• Hybrid input — UIA patterns first, SendInput fallback for apps with no accessibility
• Structured JSON — every response has ok, error.code, error.message, error.hint
• Auto-start — server launches on first CLI command, shuts down after 30min idle

Action Batching

Send multiple actions in one call — the killer feature for agent speed:

{
  "window": "Calculator",
  "actions": [
    {"type": "click", "element": {"name": "Two"}},
    {"type": "click", "element": {"name": "Five"}},
    {"type": "click", "element": {"name": "Multiply by"}},
    {"type": "click", "element": {"name": "Two"}},
    {"type": "click", "element": {"name": "Five"}},
    {"type": "click", "element": {"name": "Equals"}},
    {"type": "read", "element": {"automationId": "CalculatorResults"}}
  ],
  "verbose": true
}

With --verbose, the overlay shows a ghost cursor, highlights, and annotations for each action as it executes.

Commands

Run uia <command> --help for details.

HTTP API

The server exposes a REST API on http://localhost:9721 — use it from any language:

Full API reference →

Architecture

┌─────────────┐         HTTP/JSON          ┌──────────────────┐
│   uia.exe   │    localhost:9721          │   UIA Server     │
│   (Go CLI)  │ ◄────────────────────────► │   (.NET 8)       │
└─────────────┘                            │  ┌────────────┐  │
                                           │  │ Uia.Core   │  │
     Any agent or script                   │  │ Uia.Overlay│  │
     can call uia.exe                      │  └────────────┘  │
     or hit the HTTP API                   └──────────────────┘

The Go CLI auto-starts the .NET server on first use. The server holds the UIA context, executes actions, and drives the visual overlay. Why client-server? →

📚 Hard-Won Patterns

Automating real Windows apps comes with quirks — SetCursorPos doesn't generate mouse events, UWP apps return temporary PIDs, some modern controls don't appear in the UIA tree. We documented 14 patterns from working with Calculator, Paint, Notepad, and Edge:

• Why SendInput works but SetCursorPos doesn't for drawing
• The screenshot + tree correlation trick for custom-rendered UIs
• How to calibrate canvas coordinates when bounds don't line up
• Freehand drawing: 601-point Archimedean spiral in 1.77 seconds

Read LEARNINGS.md →

📰 News

• [2026-05-17] 🚀 v0.1.0 released — 20+ commands, action batching, visual overlay, HTTP API
• [2026-05-17] 🤖 Agent skill published — npx skills add amitse/uiacli for 50+ agents
• [2026-05-17] 📦 One-liner install — irm .../install.ps1 | iex, no SDK needed

Build from Source

# Build the .NET server
dotnet build UiaCli.sln

# Build the Go CLI
cd cli && go build -o ../uia.exe . && cd ..

# Run
uia windows

Known Limitations

• Windows only — requires Windows UI Automation APIs
• Single monitor DPI — mixed-DPI multi-monitor setups may have coordinate offset issues
• WinUI3/UWP apps — some modern apps expose minimal UIA trees; coordinate fallback works
• No UAC access — can't automate elevated windows from a non-elevated process

Contributing

Contributions welcome! Fork, build (dotnet build && cd cli && go build), test against a real app, and open a PR. Check open issues for ideas.

License

MIT