ScreenHand
Let an assistant click real buttons through the OS accessibility API — no screenshot, no model call per action.Read full description
manushi4LABSCO SUMMARY
Per-pipe data permissions are enforced outside the prompt, which is the part that makes this deployable.
Handing an assistant a searchable record of everything on your screen is only acceptable if you can bound it. Here the bounds are YAML frontmatter per pipe — apps, windows, content types, hours, endpoints — enforced at three layers, so a denied endpoint is never even described to the model. Data stays in a local SQLite database by default; cloud sync is an opt-in tier.
What it is
A continuous local recorder with an MCP server on top. Screen and audio are captured to a local SQLite database on your machine; the MCP endpoint lets Claude Desktop, Cursor, VS Code extensions and other clients query that history instead of you scrolling back through your day.
What you get
- Your screen history searched from the assistant — "what did I see in the last 5 mins", "summarize today conversations"
- Recent context and meeting transcriptions available to the assistant as tool results
- Capture across macOS, Windows and Linux, all monitors, with local Whisper transcription for audio
- Pipes: scheduled AI agents written as markdown files in
~/.screenpipe/pipes/, with YAML frontmatter that restricts what data each one may touch — allowed and denied apps and windows, content types, time ranges and days, and endpoint gating - A full REST API on localhost, default port 3030, with raw SQL access to the underlying SQLite database and a JavaScript/TypeScript SDK
Requirements
The desktop app, or the CLI via npx screenpipe record. Wiring it to an assistant is one command — claude mcp add screenpipe -- npx -y screenpipe-mcp@latest — or npx screenpipe setup to configure every supported agent it finds. No account is needed for the core application. The source is available for personal, non-commercial use; the signed desktop app is a subscription: Standard $25/month, Pro $50/seat/month adding cloud sync and integrations, Enterprise $150/seat/month with central config, per-pipe permissions, SSO/SAML and MDM support.
Setup effort
Follow the published steps — npx screenpipe record
USE CASES
01
Recovering something you saw and cannot find again
Ask for it in natural language over your recent history rather than retracing tabs, files and chats.
02
Writing up a meeting you did not take notes in
The audio was transcribed locally as it happened; ask the assistant to summarise the conversation and pull out what was agreed.
03
Automating something that should happen when you do certain work
Write a pipe as a markdown file — for example updating a tracker whenever you work on a given task — and it runs on schedule against your captured history.
04
Rolling this out to a team without seeing their screens
Push capture settings and per-pipe permissions centrally: employees can add stricter filters, admins cannot see the data, which stays on each device.
BEFORE YOU INSTALL
What to know first
Requires
- The screenpipe recorder running — the desktop app or npx screenpipe record — since the MCP server only reads what it captures
- npx on PATH
Where it runs
The documented client runs the server on your machine.
INSTALLATION
Connect your client
Server setup
Install the screenpipe desktop app from https://screenpipe.com/how-to-install, or start the recorder from the CLI:
TERMINAL
npx screenpipe recordKeep it recording, then register the MCP server with Claude Code (or run npx screenpipe setup to configure every supported agent it finds on your machine):
TERMINAL
claude mcp add screenpipe -- npx -y screenpipe-mcp@latestMCP Clients
Claude Code
CLAUDE CODE · TERMINAL
claude mcp add screenpipe --transport stdio --scope user -- npx -y screenpipe-mcp
