Everything else on this list is table stakes for desktop automation; the Windows Graphics Capture path is not. Traditional capture returns a black rectangle for anything GPU-accelerated — which is games, media players, Electron apps and half of modern software — and this server lets you switch capture method per call or, better, name the window titles once in an environment variable and forget about it. OCR returning coordinates alongside text is the other practical detail: the model can see a button and click where it is.
A desktop control server built on PyAutoGUI, RapidOCR and ONNXRuntime, giving an assistant the mouse, the keyboard, window management and text extraction from the screen. Everything runs locally with no external service.
- Mouse control including press-and-hold as separate calls, so drag gestures and modifier-held clicks are expressible — `click_screen`, `move_mouse`, `drag_mouse`, `mouse_down`, `mouse_up`
- Keyboard input covering single keys, sequences and combinations, plus holding a key down across other actions — `type_text`, `press_keys`, `key_down`, `key_up`
- Screenshots of the whole screen or a window matched by title pattern, optionally regex, optionally saved to disk — `take_screenshot`
- Text extracted from the screen with its coordinates, which is what makes clicking on something the model can see possible — `take_screenshot_with_ocr`
- Window listing and activation by title pattern, plus screen resolution — `list_windows`, `activate_window`, `get_screen_size`
- An explicit wait, for the moments where the UI needs time — `wait_milliseconds`
- A Windows Graphics Capture path for GPU-accelerated windows, where the usual GDI and PrintWindow methods return black frames — games, media players, Electron apps, GPU-accelerated browsers, streaming software and CAD tools; either a flag on the screenshot call or an environment variable listing window-title patterns to apply it to automatically
Python with `uvx`: `uvx computer-control-mcp@latest`, or `pip install computer-control-mcp` for a global install. The first run downloads around 70MB of Python dependencies, and the README suggests doing that in a terminal first rather than waiting inside an MCP client. `COMPUTER_CONTROL_MCP_SCREENSHOT_DIR` moves saved screenshots off the OS downloads directory, and `COMPUTER_CONTROL_MCP_WGC_PATTERNS` takes comma-separated title patterns — such as `obs, discord, game, steam` — that automatically use Windows Graphics Capture. The enhanced GPU-window capture is Windows-only. Licensed MIT.
One command — uvx computer-control-mcp@latest