screenshotLABSCO SUMMARY
A button press is an exact element target by role and name, not a coordinate guess.
That is what survives a layout shift, and it is why a click does not need a screenshot or an LLM call in the loop. Browser work goes through the DevTools Protocol in the background, so Chrome does not need focus — which also means you can keep using your machine while it works.
What it is
A desktop automation MCP server for macOS and Windows. It drives native applications through the accessibility API, and Chrome through the DevTools Protocol in the background.
What you get
- 111 tools across desktop control, browser automation and fallback execution
- Native UI: ui_tree, ui_find, ui_press, ui_set_value, menu_click, click, type_text, key, drag, scroll
- Seeing the screen when you need to: screenshot, screenshot_file, ocr, click_text
- App handling: apps, windows, focus, launch, and applescript on macOS
- Browser over CDP without stealing focus: browser_tabs, browser_open, browser_navigate, browser_js, browser_dom, browser_click, browser_type, browser_wait, browser_page_info
- Human-like input and anti-detection: browser_fill_form, browser_human_click, browser_stealth
- Automatic fallback from accessibility API to CDP to OCR to coordinates: execution_plan, click_with_fallback, type_with_fallback
- Prebuilt app references, playbooks and UI maps that load when the matching app or site is detected
Requirements
macOS or Windows. Add it with claude mcp add screenhand -- npx -y screenhand, or run npx -y screenhand from any MCP client. On macOS, enable your terminal under System Settings > Privacy & Security > Accessibility. Browser tools need Chrome started with --remote-debugging-port=9222. Version 0.5.3.
Setup effort
One command — npx -y screenhand
USE CASES
01
Driving an app that has no API
Find the control by its title, press it, and type into the field that opens.
02
Filling a web form the way a person would
Type with realistic delays and click with a full mouse event sequence.
03
Working in a browser tab while you use another window
Navigate, click and read page content over CDP without the window coming forward.
04
Reaching something the accessibility tree does not expose
Fall back to OCR: read the text on screen and click it where it sits.
111 tools
Listed in project docs. Your client may show a different list.
screenshot_fileocrappswindowsfocuslaunchui_treeui_findui_pressui_set_valuemenu_clickclickclick_texttype_textkeydragscrollapplescriptbrowser_tabsbrowser_openbrowser_navigatebrowser_jsbrowser_dombrowser_clickbrowser_typebrowser_waitbrowser_page_infobrowser_stealthbrowser_fill_formbrowser_human_clickexecution_planclick_with_fallbacktype_with_fallbackread_with_fallbacklocate_with_fallbackselect_with_fallbackscroll_with_fallbackwait_for_statememory_snapshotmemory_recallmemory_savememory_record_errormemory_record_learningmemory_query_patternsmemory_errorsmemory_statsmemory_clearlearning_statuslearning_resetrecovery_statusrecovery_configurecoverage_reportplatform_guideplaybook_preflightplaybook_recordexport_playbookplatform_exploreplatform_learnscan_menu_baringest_documentationingest_tutorialjob_createjob_create_chainjob_statusjob_listjob_transitionjob_step_donejob_step_failjob_resumejob_dequeuejob_removejob_runjob_run_allworker_startworker_stopworker_statussession_claimsession_heartbeatsession_releasesupervisor_statussupervisor_startsupervisor_stopsupervisor_pausesupervisor_resumesupervisor_installsupervisor_uninstallrecovery_queue_addrecovery_queue_listperception_startperception_stopperception_statusworld_stateworld_state_diffplan_goalplan_executeplan_stepplan_step_resolveplan_statusplan_listplan_cancelorchestrator_startorchestrator_submitorchestrator_statusorchestrator_stopobserver_startobserver_statusobserver_stopobserver_ocr_roicommunity_publishcommunity_fetchBEFORE YOU INSTALL
What to know first
Tools you get
111 tools named in the project's documentation. Older docs describing 70+ desktop tools such as app_list, app_launch and session_start refer to an earlier build; those names do not exist in the current set.
Requires
npx on PATH
INSTALLATION
Connect your client
Server setup
TERMINAL
npx -y screenhandMCP Clients
Claude Code
CLAUDE CODE · TERMINAL
claude mcp add screenhand -- npx -y screenhandClaude Desktop, Cursor
JSON
{
"mcpServers": {
"screenhand": {
"command": "npx",
"args": [
"-y",
"screenhand"
]
}
}
}Codex
CODEX · TOML
[mcp_servers.screenhand]
command = "npx"
args = ["-y", "screenhand"]Claude Code — Plugin
CLAUDE CODE — PLUGIN · TERMINAL
git clone https://github.com/manushi4/screenhand.git
cd screenhand && npm install && npm run build:native
./install-plugin.shRun the installer from the root of the clone: it copies the plugin into ~/.claude/plugins/ and points it at the mcp-desktop.ts in that folder.
