Galileo Evals in the IDE

Integrate Galileo observability into LangChain applications.

Supported Languages: python Parameters:

• language (optional): Programming language for integration guide. Defaults to "python".

Provides:

• Complete integration code examples and setup instructions
• Automatic logging of prompts, responses, model parameters, and token usage
• Environment configuration templates
• Language-specific best practices

Returns working code snippets ready to copy and paste for robust LLM observability.

Integrate Galileo observability into OpenAI API applications.

Supported Languages: python, typescript

Parameters:

• language (optional): Programming language for integration guide. Defaults to "python".

Provides:

• Complete integration code examples and setup instructions
• Automatic logging of prompts, responses, model parameters, and token usage
• Environment configuration templates
• Language-specific best practices

Returns working code snippets ready to copy and paste for robust LLM observability.

Get the insights (also known as signals) for a log stream in a specific project. This tool analyzes log streams and provides insights/signals that identify issues, patterns, and suggest improvements.

When to use this tool:

• User asks for "insights" or "signals" about a log stream
• User mentions a project name and wants to analyze a log stream
• User asks questions like: "What's wrong with my application?", "Get insights about my log stream", "Show me signals for my logs"

Parameters:

• project (required): The exact name of the Galileo project.

Get the signals (also known as insights) for a log stream in a specific project. This tool analyzes log streams and provides signals/insights that identify issues, patterns, and suggest improvements.

When to use this tool:

• User asks for "signals" or "insights" about a log stream
• User mentions a project name and wants to analyze a log stream
• User asks questions like: "What's wrong with my application?", "Get signals about my log stream", "Show me insights for my logs"

Parameters:

• project (required): The exact name of the Galileo project.

Check the status of a synthetic dataset generation job and view dataset contents. This tool allows the user to check the progress of a synthetic dataset generation job that was previously started using the create_galileo_dataset tool. It always shows the first 10 rows of the dataset for preview, regardless of generation status. Users can start by asking a question such as: Check the status of my dataset generation job.

Parameters:

• dataset_id (required): The id of the dataset to check the generation status for.

Create a new Galileo dataset with multiple data source options.

Prerequisites:

• Requires a valid Galileo API key configured in your MCP settings
• You must have an active Galileo account with dataset creation permissions

Creation Options: Ask the user which option they prefer:

1. Synthetic data generation: Automatically generate data based on description
2. CSV file: Provide CSV content to create dataset (columns: input, output, optional metadata)
3. JSON data: Provide structured JSON data

Create a new global prompt template that can be used across all projects.

Prerequisites:

• Requires a valid Galileo API key configured in your MCP settings
• You must have an active Galileo account with prompt template creation permissions

Required User Inputs: Please ask the user to provide the following information:

• name: A descriptive name for the prompt template.
• template: The actual prompt template content. Can be plain text or JSON format for chat templates.

Optional User Inputs:

• raw: Boolean flag indicating if this is a raw template (default: false).

Setup and create a Galileo experiment using the Galileo SDK.

Supported Languages: python Parameters:

• language (optional): Programming language for experiment setup guide. Defaults to "python".

Provides:

• Complete experiment setup code examples and instructions
• Step-by-step guide for creating and running experiments
• Dataset preparation and metrics configuration
• Integration with existing application code
• Advanced usage patterns for agentic/RAG workflows

What this tool helps you do:

• Set up Galileo experiments with datasets (remote or inline)

Search across the Galileo documentation to find relevant information, code examples, API references, and guides. Use this tool when you need to answer questions about Galileo, find specific documentation, understand how features work, or locate implementation details. The search returns contextual content with titles and direct links to the documentation pages.

Parameters:

• query (required): The search query to find relevant documentation.

Returns:

• Relevant documentation content and links based on the search query.

Example: search_docs(query="how to set up data logging")