Eval logs have irregular shapes — fields appear as prompts change — so a model writing SQL against them from memory will reference columns that do not exist in this project. Discovering the schema first turns that from a guess into a lookup. The pattern tools are the unusual part: an agent can not only find that a class of trace fails, but record that finding with evidence and attach it to every matching trace, so the next investigation starts where this one ended.
Braintrust's endpoint over their evaluation platform, with a separate EU address for EU organisations. Ten tools: documentation, object navigation, SQL, and a pattern log for recording what you found.
- sql_query — SELECT over experiments, datasets and logs, with an overflow path when results exceed 1MB
- infer_schema — discover fields, types and common values before writing the query
- resolve_object and list_recent_objects — turn names into IDs, parse Braintrust URLs, list what you have access to
- summarize_experiment — aggregated metrics, optionally against a baseline
- search_patterns, new_pattern and update_pattern — record a finding with evidence and attach it to matching traces
- search_docs and generate_permalink
OAuth in most clients, or a bearer API key. EU organisations connect to api-eu.braintrust.dev instead.
Paste a URL, then authorize — add the endpoint to your client, then approve the OAuth consent screen