LabscoMCP early access ↗
Touchpoint-LabsMCP SERVER

Touchpoint

Let an agent read and operate any desktop app through the accessibility tree — no vision model needed.

by Touchpoint-Labs ↗Desktop & OS AutomationCommunity

LABSCO SUMMARY

Structured elements instead of pixels, which is what makes a local model able to drive a desktop.

Because the tree carries names, roles and states, no-vision mode works with models that cannot see at all — the agent reads a compact text snapshot and acts on element IDs. The auto-verify flags on each action are the quiet win: the agent learns that focus moved or nothing changed without spending a screenshot to find out.

What it is

A cross-platform desktop control server: it reads native accessibility APIs on Linux, macOS and Windows, plus browser and Electron content over CDP, and returns structured elements with names, roles, states and positions.

What you get

  • snapshot and screenshot orient the agent; apps and windows list what is open; diff_snapshot shows what changed
  • find locates elements by text and role, get_element returns one by ID, read_text pulls the text out
  • click, set_value, set_numeric_value, select_text, focus and action operate controls; type_text and press_key send keyboard input
  • activate_window, minimize_window, fullscreen_window, close_window, move_window and resize_window manage windows; mouse_move and scroll handle the pointer
  • wait_for, wait_for_app and wait_for_window hold until the UI catches up; diagnostics reports health
  • A no-vision mode where the agent works from the structured tree alone, and action results carry auto-verify flags such as (new window: ...), (focus moved) and (no change detected)
  • Electron apps like Slack and VS Code addressable as both native and web content

Requirements

Python 3.10+ and pip install touchpoint-py, which brings the platform backend, CDP support, the MCP server and screenshots together. macOS needs Accessibility permission granted in System Settings. Linux needs xdotool for input and wmctrl for window management. Windows uses built-in COM APIs and needs nothing extra. The client launches touchpoint-mcp; TOUCHPOINT_MODE switches between vision and no-vision. The project is marked alpha.

Setup effort

One command — pip install touchpoint-py

See installation options