project_initLABSCO SUMMARY
The blast radius is the virtual machine.
Everything runs over SSH into a guest you built, so a wrong click or a bad shell command hits the VM rather than your host. What makes it workable is the targeting: text search, magnify-and-map, and numbered marks each remove a place where a model would otherwise guess at coordinates. One rule to know up front — a screenshot requires an active project.
What it is
A Python server that drives a Linux VM over SSH: X11 mouse and keyboard control, screenshots, OCR-based clicking, and shell commands, all confined to the guest.
What you get
- Input control —
move_mouse,click,click_in_window,drag,scroll,type_text,press_keys,key_downandkey_up,set_clipboard,paste - Clicking by what's on screen rather than by pixel —
find_textOCRs the screen and returns every match with coordinates,click_textclicks the match's exact centre zoommagnifies a region and keeps the crop mapping, soclick_zoomedturns a point in the zoomed image back into the right full-screen pixelmark_screennumbers every detected text element andclick_markpicks one by number — a discrete choice instead of an estimaterun_actionsexecutes a whole sequence of UI actions in a single call- Shell and file transfer —
ssh_execute,ssh_upload,ssh_download,ssh_connection_info; screenshots viatake_screenshot, stored in the project and exposed as resources - Session memory —
project_init,project_log,project_save_resultandproject_save_advice, so one run leaves notes for the next
Requirements
Host: Python 3.12+, uv or pip, and QEMU/KVM with libvirt. Guest: Linux with an X11 desktop, an SSH server, and openssh, xdotool, scrot, xrandr and xinput installed. Connection settings come from VM_HOST, VM_USER, VM_PORT, VM_DISPLAY and optionally VM_IDENTITY. The OCR and zoom tools additionally need tesseract on the host plus pillow and pytesseract — the rest of the server runs without them.
Setup effort
Build from source — git clone https://github.com/Neanderthal/mcp-qemu-vm.git
USE CASES
01
Letting an agent use a desktop app safely
Give it the VM's screen and keyboard; the host filesystem is never in reach.
02
Clicking a button whose position you don't know
Search the screen for its label and click the match, instead of describing where you think it is.
03
Hitting a small or low-contrast control
Magnify the region, pick the point in the zoomed image, and the server maps it back to the real pixel.
04
Running a multi-step UI sequence reliably
Send the keystrokes, waits and typing as one batched action list rather than a round trip per step.
36 tools
Listed in project docs. Your client may show a different list.
project_loadproject_listproject_infoproject_logproject_read_logsproject_save_resultproject_save_adviceproject_read_advicemove_mouseclickclick_in_windowget_active_window_infoscrolldragtype_textpress_keyskey_downkey_upset_clipboardpasteactivate_windowwaitrun_actionsfind_textclick_textzoomclick_zoomedmark_screenclick_markssh_executessh_uploadssh_downloadssh_connection_infotake_screenshotdisplay_calibration_infoBEFORE YOU INSTALL
What to know first
Tools you get
36 tools named in the project's documentation.
Requires
Python 3.12+
Where it runs
The documented client runs the server on your machine.
INSTALLATION
Connect your client
This one is built from source rather than installed with a single command.
Server setup
TERMINAL
git clone https://github.com/Neanderthal/mcp-qemu-vm.git
cd mcp-qemu-vm
uv venv && source .venv/bin/activate
uv pip install -r requirements.txt
python server.pyMCP Clients
Claude Desktop
CLAUDE DESKTOP · JSON
{
"mcpServers": {
"qemu-vm-control": {
"command": "python3",
"args": [
"<YOUR_PATH_TO_MCP_QEMU_VM>/server.py"
],
"env": {
"VM_HOST": "192.168.122.79",
"VM_USER": "vmrobot",
"VM_PORT": "22",
"VM_DISPLAY": ":0"
}
}
}
}
