Move mouse to specific coordinatesLABSCO SUMMARY
Raw desktop input and screen reads, with no application-specific layer in between.
Because it works in coordinates and pixels rather than accessibility trees, it drives any application at all — and it needs the agent to know where things are, usually from a screenshot first. Treat it like giving something else your mouse: it can click anything you can click.
What it is
A server that exposes PyAutoGUI's desktop control to an MCP client — pointer, keystrokes, and screen reads on Windows, macOS and Linux.
What you get
- Mouse control — move to coordinates, click at the current or a given position, drag and drop, read the current position
- Keyboard control — type text, press individual keys, send hotkey combinations
- Screen operations — take a screenshot, read the screen size, locate an image on screen, sample a pixel colour
- The same set of operations on Windows, macOS and Linux
Requirements
Python 3.12+ and the published package: pip install mcp-pyautogui-server, or run it with uvx. Remaining dependencies install with it. Nothing is sandboxed — the server drives the real desktop of whatever machine it runs on.
Setup effort
One command — uvx mcp-pyautogui-server
USE CASES
01
Automating an application that has no API
Screenshot the window, locate the control by image, then click and type the way a person would.
02
Driving a GUI test end to end
Chain moves, clicks and keystrokes, checking the screen between steps to confirm the app got where you expected.
03
Filling a form the agent can see but not reach
Type into the focused field and send Tab or Enter as real key events, so the application behaves as it would for a human.
04
Checking what is actually on screen right now
Take a screenshot or sample a pixel colour to confirm a state change before the next step runs.
11 tools
Listed in project docs. Your client may show a different list.
Click at current or specified positionDrag and drop operationsGet current mouse positionType textPress individual keysHotkey combinationsTake screenshotsGet screen sizeFind image locations on screenGet pixel colorsBEFORE YOU INSTALL
What to know first
Tools you get
11 tools named in the project's documentation.
Requires
- uvx on PATH
- Python 3.12+
Where it runs
The documented client runs the server on your machine.
INSTALLATION
Connect your client
Server setup
TERMINAL
uvx mcp-pyautogui-serverMCP Clients
Claude Desktop
CLAUDE DESKTOP · JSON
{
"mcpServers": {
"mcp-pyautogui-server": {
"command": "uvx",
"args": [
"mcp-pyautogui-server"
]
}
}
}
