interact_with_screenLABSCO SUMMARY
Screen analysis first, then the tap - which is what makes the automation survive a layout change.
Coordinate-driven phone automation breaks the moment a dialog moves. Here the agent reads a structured description of the screen, finds an element by its text or description, waits for one to appear, and only then acts. Launching by package and activity is the reliable path when launching by app name does not work on a given device.
What it is
A bridge from an MCP client to a USB-connected Android device through ADB. Alongside the raw gestures it exposes a structured read of the current screen, so an agent can decide what to touch instead of guessing at coordinates.
What you get
- A structured analysis of the current screen and its UI elements, with an optional screenshot
- One interaction entry point covering tap, swipe, key press, text input, find, wait and scroll - by coordinates, by element text, or by content description
- Calls and SMS: place and end a call, send a message, read received and sent messages
- Contacts, including creating a new one by driving the contacts UI
- Apps: launch by name, by package, or by package and activity, list what is installed, and close an app
- Screenshots, screen recording, media control, app shortcuts and opening a URL in the device browser
Requirements
Python 3.7+, ADB installed and configured, USB debugging enabled on the device, and the phone connected by USB. Run it with uvx phone-mcp. Nothing else to sign up for - the credential is physical access to an unlocked phone.
Setup effort
One command — uvx phone-mcp
USE CASES
01
Automating a flow through an app you cannot script
Read the screen, find the control by its label, tap it, then wait for the next screen to appear before continuing.
02
Reaching a screen that the app launcher will not open
Launch the exact activity by package and activity name - the path that works across devices when launching by app name does not.
03
Handling a call and a follow-up message
Place the call from contacts, and if nobody answers, send the SMS instead - both from the same session.
04
Adding a contact without opening the app yourself
Hand over a name and number; the tool opens the contacts app, fills the fields and saves.
5 tools
Listed in project docs. Your client may show a different list.
analyze_screencreate_contactlaunch_app_activitylaunch_intentBEFORE YOU INSTALL
What to know first
Tools you get
5 tools named in the project's documentation.
Requires
- uvx on PATH
- Python 3.7+
Where it runs
All 4 documented clients run the server on your machine.
INSTALLATION
Connect your client
Server setup
TERMINAL
uvx phone-mcpMCP Clients
Cursor — uvx, Trae — uvx, Claude — uvx
JSON
{
"mcpServers": {
"phone-mcp": {
"command": "uvx",
"args": [
"phone-mcp"
]
}
}
}Any MCP client — pip
ANY MCP CLIENT — PIP · JSON
{
"mcpServers": {
"phone-mcp": {
"command": "/usr/local/bin/python",
"args": [
"-m",
"phone_mcp"
]
}
}
}
