get_screen_size followed by mouse_move gives an agent coordinates it can act on, and keyboard_type reaches applications that expose no API at all. Responses are capped at 1MB and a high-resolution screen_capture goes over that cap; the author names a modest display resolution as the one that works reliably, so lower the display or capture a smaller screen before a long session.
A desktop control server built on RobotJS. It reports the screen size, captures the screen, moves and clicks the mouse, and types text or presses key combinations at the current cursor.
- The screen dimensions, so coordinates can be worked out before anything moves
- A capture of the current screen, also readable afterwards through screenshot:// resources
- Mouse movement to an x, y coordinate
- Left, right or middle clicks, single or double
- Text typed at the current cursor position
- Single keys or combinations, holding control, shift, alt or command
Node.js 14.x or newer, and operating-system permission to capture the screen and to control the mouse and keyboard — granted in your security settings the first time the client launches the server. Installs with npx mcp-desktop-automation.
One command — npx -y mcp-desktop-automation