LabscoMCP early access ↗
iFuryStMCP SERVER · OFFICIAL PROJECT

Open Codex Computer Use

Computer use for macOS, Linux and Windows built on Accessibility rather than screenshots — the agent reads the real UI tree and acts on elements, without stealing your mouse.

by iFurySt ↗Desktop & OS Automation✓ Official

LABSCO SUMMARY

Non-intrusive computer use — the agent works while you keep using the machine.

The design decision that matters is refusing the easy fallback: actions route through accessibility or process-targeted events rather than global mouse and keyboard, so an agent driving an app does not grab your cursor or pull the window to the front.

What it is

An open-source computer-use service wrapped as an MCP server, so any agent or MCP client can drive a desktop app. It works through the accessibility layer, which is what lets it target elements directly instead of moving a physical cursor around.

What you get

  • list_apps and get_app_state — what is running, and the state of one app including its accessibility tree and a screenshot
  • click with a repeat count, targeted at an element through accessibility rather than a global pointer event
  • set_value for writing into a field, with a settable-attribute check first, so a non-settable target returns an error instead of silently falling back to keyboard or clipboard tricks
  • type_text and press_key for keyboard input, posted to the target process so focus is not stolen
  • scroll with fractional page support, and drag
  • perform_secondary_action for menu items and secondary actions
  • A CLI alongside the server: open-computer-use call <tool> runs a single tool and prints the MCP-style JSON result, and --calls runs a sequence in one process so element index state can be reused
  • Installers that write the config for you — install-codex-mcp, install-claude-mcp, install-gemini-mcp, install-opencode-mcp and install-codex-plugin
  • open-computer-use doctor to check permissions, opening onboarding only when something is missing

Requirements

npm i -g open-computer-use, which also exposes the short alias ocu. The macOS runtime requires macOS 14.0 or later, and on macOS you run it once to grant Accessibility and Screen Recording; Windows and Linux need neither step. Register it with your agent through one of the installer subcommands, or add the MCP config by hand with open-computer-use as the command and mcp as the argument. Global physical pointer events are off by default and only enabled behind an explicit opt-in environment variable, which is why the agent working in the background does not fight you for the mouse.

Setup effort

One command — npm i -g open-computer-use

See installation options