Summary
Raw desktop input and screen reads, with no application-specific layer in between.
Because it works in coordinates and pixels rather than accessibility trees, it drives any application at all — and it needs the agent to know where things are, usually from a screenshot first. Treat it like giving something else your mouse: it can click anything you can click.
What it is
A server that exposes PyAutoGUI's desktop control to an MCP client — pointer, keystrokes, and screen reads on Windows, macOS and Linux.
What you get
- Mouse control — move to coordinates, click at the current or a given position, drag and drop, read the current position
- Keyboard control — type text, press individual keys, send hotkey combinations
- Screen operations — take a screenshot, read the screen size, locate an image on screen, sample a pixel colour
- The same set of operations on Windows, macOS and Linux
Requirements
Python 3.12+ and the published package: pip install mcp-pyautogui-server, or run it with uvx. Remaining dependencies install with it. Nothing is sandboxed — the server drives the real desktop of whatever machine it runs on.
Setup effort
One command — uvx mcp-pyautogui-server