The design decision that matters is refusing the easy fallback: actions route through accessibility or process-targeted events rather than global mouse and keyboard, so an agent driving an app does not grab your cursor or pull the window to the front.
An open-source computer-use service wrapped as an MCP server, so any agent or MCP client can drive a desktop app. It works through the accessibility layer, which is what lets it target elements directly instead of moving a physical cursor around.
- `list_apps` and `get_app_state` — what is running, and the state of one app including its accessibility tree and a screenshot
- `click` with a repeat count, targeted at an element through accessibility rather than a global pointer event
- `set_value` for writing into a field, with a settable-attribute check first, so a non-settable target returns an error instead of silently falling back to keyboard or clipboard tricks
- `type_text` and `press_key` for keyboard input, posted to the target process so focus is not stolen
- `scroll` with fractional page support, and `drag`
- `perform_secondary_action` for menu items and secondary actions
- A CLI alongside the server: `open-computer-use call <tool>` runs a single tool and prints the MCP-style JSON result, and `--calls` runs a sequence in one process so element index state can be reused
- Installers that write the config for you — `install-codex-mcp`, `install-claude-mcp`, `install-gemini-mcp`, `install-opencode-mcp` and `install-codex-plugin`
- `open-computer-use doctor` to check permissions, opening onboarding only when something is missing
`npm i -g open-computer-use`, which also exposes the short alias `ocu`. The macOS runtime requires macOS 14.0 or later, and on macOS you run it once to grant Accessibility and Screen Recording; Windows and Linux need neither step. Register it with your agent through one of the installer subcommands, or add the MCP config by hand with `open-computer-use` as the command and `mcp` as the argument. Global physical pointer events are off by default and only enabled behind an explicit opt-in environment variable, which is why the agent working in the background does not fight you for the mouse.
One command — npm i -g open-computer-use