There is no element model here — nothing queries the screen for a button — so every action is a coordinate the caller worked out from the previous vnc_screenshot, and the loop is screenshot-driven by construction. What that buys is reach: an installer, a native desktop app or a BIOS-era dialog is as addressable as a web page, which no browser-based tool can say. vnc_type_multiline and the delay on vnc_screenshot are the two details that keep the loop from being one call per line.
A VNC input server, six tools that send mouse and keyboard events to a remote screen and read it back as a screenshot.
- vnc_click at x and y with a button choice and a double-click flag, and vnc_move_mouse to position the pointer without clicking
- vnc_type_text for a single string, with an option to press Enter after it, and vnc_type_multiline to send several newline-separated lines in one call
- vnc_key_press for a key or a key combination, which is what covers shortcuts and dialogs that have no clickable target
- vnc_screenshot with a delay parameter, so the capture can be taken after the screen has settled rather than mid-redraw
A reachable VNC server, and screen coordinates to act on: the tools address the screen by x and y rather than by element. MIT licensed.
One command plus a key — npm install -g @hrrrsn/mcp-vnc, then supply credentials