Foreground actions, shared desktop input and ambient browser auto-connect are rejected before anything runs, so an MCP client cannot quietly take over your screen — establishing a browser connection is a separate, explicit foreground step. `verify_state` is the other design decision worth copying: it polls real accessibility state and returns `unknown` rather than guessing when the evidence is ambiguous. The `capture` tool writes video and frames, so use tool filters before exposing this to a client you do not control.
A macOS CLI and menu-bar app for screen capture, accessibility inspection and native UI automation, with `peekaboo mcp` running the same toolset as an MCP server over stdio. The core loop is observe, pick an element from the result, act on it — the map comes back with opaque element ids rather than raw coordinates.
- A structured UI map of a running app or the whole screen, with element ids to act on — `see`, backed by `screen list` and `window list`
- Native interaction: clicking that tries accessibility actions before synthetic input, plus typing, key presses, scrolling and dragging — `click`, and the CLI's `type`, `press`, `scroll`, `drag`
- Action-first additions that skip synthetic input entirely: setting an accessibility value directly, and invoking a named accessibility action on an element — `set_value`, `action`
- Waiting that replaces fixed sleeps: 1 to 8 predicates about window existence, bounds, or element existence, value, enabled and selected state, polled every 100 ms for at most 10 seconds, reported as satisfied, unsatisfied or a conservative unknown — `verify_state`
- Bounded recording of a screen, window or region that writes retained frames, `contact.png`, `metadata.json` and optional MP4 — `capture`
- A natural-language agent run over the same tools — `agent` from the CLI, and a nested background-only agent tool over MCP
- Window-pinned automation: list Safari's windows, take the `window_id`, and keep every subsequent call bound to that exact window
macOS 15 or later. The CLI installs with `brew install steipete/tap/peekaboo`; the npm package `@steipete/peekaboo` 4.2.3 needs Node.js 22 or later and includes the CLI plus the MCP launcher. Screen Recording permission is required for capture and Accessibility permission for inspection and control — check both with `peekaboo permissions status`. The MCP server runs over stdio; HTTP and SSE transports are reserved and currently fail with an explicit error. Agent runs need a model provider configured under `~/.peekaboo`. MIT licensed.
One command — brew install steipete/tap/peekaboo