LabscoMCP early access ↗
AB498MCP SERVER

Computer Control MCP

Mouse, keyboard, window control and on-screen OCR from an MCP client — with a Windows Graphics Capture path so screenshots of GPU-accelerated windows are not just black rectangles.

by AB498 ↗Desktop & OS AutomationCommunity

LABSCO SUMMARY

It solved the black-screenshot problem.

Everything else on this list is table stakes for desktop automation; the Windows Graphics Capture path is not. Traditional capture returns a black rectangle for anything GPU-accelerated — which is games, media players, Electron apps and half of modern software — and this server lets you switch capture method per call or, better, name the window titles once in an environment variable and forget about it. OCR returning coordinates alongside text is the other practical detail: the model can see a button and click where it is.

What it is

A desktop control server built on PyAutoGUI, RapidOCR and ONNXRuntime, giving an assistant the mouse, the keyboard, window management and text extraction from the screen. Everything runs locally with no external service.

What you get

  • Mouse control including press-and-hold as separate calls, so drag gestures and modifier-held clicks are expressible — click_screen, move_mouse, drag_mouse, mouse_down, mouse_up
  • Keyboard input covering single keys, sequences and combinations, plus holding a key down across other actions — type_text, press_keys, key_down, key_up
  • Screenshots of the whole screen or a window matched by title pattern, optionally regex, optionally saved to disk — take_screenshot
  • Text extracted from the screen with its coordinates, which is what makes clicking on something the model can see possible — take_screenshot_with_ocr
  • Window listing and activation by title pattern, plus screen resolution — list_windows, activate_window, get_screen_size
  • An explicit wait, for the moments where the UI needs time — wait_milliseconds
  • A Windows Graphics Capture path for GPU-accelerated windows, where the usual GDI and PrintWindow methods return black frames — games, media players, Electron apps, GPU-accelerated browsers, streaming software and CAD tools; either a flag on the screenshot call or an environment variable listing window-title patterns to apply it to automatically

Requirements

Python with uvx: uvx computer-control-mcp@latest, or pip install computer-control-mcp for a global install. The first run downloads around 70MB of Python dependencies, and the README suggests doing that in a terminal first rather than waiting inside an MCP client. COMPUTER_CONTROL_MCP_SCREENSHOT_DIR moves saved screenshots off the OS downloads directory, and COMPUTER_CONTROL_MCP_WGC_PATTERNS takes comma-separated title patterns — such as obs, discord, game, steam — that automatically use Windows Graphics Capture. The enhanced GPU-window capture is Windows-only. Licensed MIT.

Setup effort

One command — uvx computer-control-mcp@latest

See installation options