LabscoMCP early access ↗
arizawanMCP SERVER · OFFICIAL PROJECT

vidlizer

Turn a video, image or PDF into a structured scene-by-scene account — what was on screen, what changed, what text was visible, and what was said.

by arizawan ↗Speech Synthesis & Transcription✓ Official

LABSCO SUMMARY

The analysis is stored, and the model reads only what it asks for.

The design decision that makes this practical is that analyze_video returns an identifier rather than the content: a long video registers cheaply and the agent then pulls the summary, one phase, or the steps that match a search. Locking provider and model to environment variables is the other good call — an agent cannot quietly switch to an expensive cloud model mid-session.

What it is

A video-understanding server that runs on local models by default. It extracts frames with ffmpeg, sends them to a vision model, and returns a flow array with one entry per scene carrying the visible scene, the subjects, the action, on-screen text, persistent context and observations. When the video has audio it transcribes with Apple MLX Whisper and merges the speech into each step. Inputs can be a local file, an image, a PDF, or a URL from YouTube, Loom, Vimeo or Twitter.

What you get

  • analyze_video — run the analysis and get back an analysis_id plus metadata, with the full result stored on disk rather than dumped into the conversation
  • Pull only the part you need: get_summary at brief, medium or full depth, get_step for one step, get_steps for a range, get_phase for every step in a named phase, and get_full_analysis when you do want everything
  • search_analysis — return only the steps matching a query, so a long video costs you the hits rather than the whole transcript
  • get_transcript — a slice of the speech transcript between two timestamps
  • Housekeeping: list_analyses, delete_analysis, get_usage_stats for a per-model token and cost breakdown, and clear_usage_stats
  • Analyses also readable as resources at vidlizer://analyses, vidlizer://analyses/{id} and vidlizer://analyses/{id}/summary

Requirements

Nothing when running locally against Ollama — no API key and no data leaving the machine. macOS with Python 3.10 or higher; the package is vidlizer (version 0.2.19) and the MCP entry point is vidlizer-mcp, run with uvx "vidlizer[mcp]" vidlizer-mcp or installed through pipx with the [mcp] extra — a plain pipx install creates the entry point without the mcp package, and needs pipx inject vidlizer mcp. Provider is chosen with PROVIDER: Ollama with a vision model pulled, any OpenAI-compatible server through OPENAI_BASE_URL and OPENAI_MODEL, or OpenRouter with OPENROUTER_API_KEY. ffmpeg is installed via Homebrew on first run if missing. The model and provider are fixed by environment variables and cannot be changed by the calling agent.

Setup effort

One command plus a key — uvx "vidlizer[mcp]" vidlizer-mcp, then supply credentials

See installation options