Comet Opik

Ask Ollie, the Opik in-product AI assistant, a question.

Use this for investigative questions ("why did X fail?"), cross-entity synthesis, or when domain expertise is required. For straightforward "show me X" / "what is Y" reads, prefer read / list which are cheaper. Ollie has direct read access to the workspace; the surrounding chat does not.

Returns the assistant's final text plus a thread_id. Reuse that thread_id on subsequent calls by default — Ollie has no memory across threads, so dropping it loses all prior context. Start a fresh thread (omit thread_id) only when the user explicitly changes topics.

Writes Ollie performs mid-stream (scores, comments, test-suite items, prompts) execute without a per-action confirmation step; auto-approvals are recorded on the opik_mcp.audit logger.

List Opik entities with optional name filter and pagination.

Output is a pipe-delimited table with id, name, and a few entity-specific columns, plus a pagination footer when more pages exist. Use read() to get full details on any specific item.

Project-scoped types require their parent:

• trace: project_id or project_name
• thread: project_id or project_name
• test_suite_item: test_suite_id
• prompt_version: prompt_id

Workspace-wide types (project, experiment, prompt, test_suite) accept an optional name substring filter.

Read any Opik entity by ID, name, or opik:// URI, with adaptive compression.

Prefer a UUID for id — it's faster (single API call) and unambiguous. Name lookup is available for: project, experiment, prompt, test_suite — name lookup is slower (two API calls) and may return multiple matches, in which case the tool lists the candidates so you can retry with the correct ID.

Special shapes:

• trace: returns {trace, spans, spansTruncated} with up to 200 spans inlined.
• prompt: returns {prompt, versions, versionsTruncated} with up to 100 versions.
• thread: returns {thread, messages, messagesTruncated} — each message is one turn's trace input/output + a trace_id to read('trace', id). Needs project scope: pass a thread link/URI, or project_id/project_name.
• All others: the flat record from /v1/private/{entity}/{id}.

Output is a one-line [read: …] header (entity_type, id, compression tier, returned tokens, full tokens) followed by compact JSON.

Submit an experiment on a test-suite-backed dataset. opik-backend runs each prompt variant against every item in the suite asynchronously and applies the suite's scoring assertions. Returns the created experiment_ids immediately — the run itself typically takes 10-30+ minutes server-side.

This tool is fire-and-return: it does NOT wait for completion. To check progress, the caller uses read("experiment", <id>); the experiment record carries status and trace_count. The result also includes a summary_url deep-linking the Opik UI compare view.

Use for: rerunning an evaluation, trying a prompt on a known test suite, comparing models against the same test suite.

Does NOT support ad-hoc (non-test-suite) datasets — those require client-side LLM execution which this tool intentionally does not do.

Return the JSON Schema, OAuth scope, and one validated example for a write operation's data payload. Pure lookup — no backend call.

Create, update, or annotate Opik entities. The operation field selects which entity/verb pair to invoke; data carries the payload for that operation (single object, or array up to 1000 for batch).

Operations:

• trace.create: Log a single trace (or a batch). Sets up the parent for spans/scores/comments. (batch ok)
• trace.update: Finalize or amend an existing trace by id. (batch ok)
• span.create: Log a single span on an existing trace (or a batch).; required: trace_id (batch ok)