langchain-ai / langsmith-skills
★ 134A skill package that teaches your agent 3 capabilities — every one documented and browsable below, no GitHub required · by langchain-ai.
Each skill below is one capability this package teaches your agent. Install the whole package, or open a skill to install just that one.
Create, manage, and upload evaluation datasets to LangSmith for testing and validation. Supports four dataset types: final_response (full conversations), single_step (individual node behavior), trajectory (tool call sequences), and RAG (question/chunks/answer/citations) CLI commands for dataset lifecycle management: create, list, get, delete, export, and upload from local JSON files SDK-based dataset creation in Python and JavaScript with programmatic example addition Example management...
1 file — installable on its own
Build evaluation pipelines for LangSmith with LLM-as-Judge and custom code evaluators. Three core components: creating evaluators (LLM-as-Judge or custom code), defining run functions to capture agent outputs and trajectories, and running evaluations locally or auto-running via uploaded evaluators Supports both offline evaluators (comparing run outputs to dataset examples) and online evaluators (real-time quality checks on production runs) Requires LangSmith API key and project...
1 file — installable on its own
Add tracing to LangChain/LangGraph apps and query trace data via CLI for debugging and dataset generation. Automatic tracing for LangChain/LangGraph apps with environment variables; manual tracing via @traceable decorator and wrap_openai() for other frameworks in Python and TypeScript Query traces (complete execution trees) or runs (individual nodes) using langsmith CLI with filters for time, latency, errors, tags, and custom metadata Export traces to JSONL files preserving hierarchy;...
1 file — installable on its own
LangSmith Skills
⚠️ — This project is in early development. APIs and skill content may change.
Agent skills for observing and evaluating LLM applications with LangSmith. Query traces, build evaluation datasets, and create custom evaluators — all from your coding agent.
Looking for skills to build and improve agents with LangChain, LangGraph, or Deep Agents? See langchain-skills.
Supported Coding Agents
These skills can be installed for any agent supported by skills.sh, including Claude Code, Deep Agents CLI, Cursor, Windsurf, Goose, and many more.
Prerequisites
Installation
Quick Install
Using npx skills:
Local (current project):
npx skills add langchain-ai/langsmith-skills --skill '*' --yesGlobal (all projects):
npx skills add langchain-ai/langsmith-skills --skill '*' --yes --globalTo link skills to a specific agent (e.g. Claude Code):
npx skills add langchain-ai/langsmith-skills --agent claude-code --skill '*' --yes --globalClaude Code Plugin
Install directly as a Claude Code plugin:
/plugin marketplace add langchain-ai/langsmith-skills
/plugin install langsmith-skills@langsmith-skillsInstall Script (Claude Code & Deep Agents CLI only)
Alternatively, clone the repo and use the install script:
# Install for Claude Code in current directory (default)
./install.sh
# Install for Claude Code in a specific project directory
./install.sh ~/my-project
# Install for Claude Code globally
./install.sh --global
# Install for DeepAgents CLI in a specific project directory
./install.sh --deepagents ~/my-project
# Install for DeepAgents CLI globally (includes agent persona)
./install.sh --deepagents --global| Flag / Argument | Description |
|---|---|
DIRECTORY | Target project directory (default: current directory, ignored with --global) |
--claude | Install for Claude Code (default) |
--deepagents | Install for DeepAgents CLI |
--global, -g | Install globally instead of current directory |
--force, -f | Overwrite skills with same names as this package |
--yes, -y | Skip confirmation prompts |
Usage
After installation, set your API keys:
export LANGSMITH_API_KEY=<your-key>
export OPENAI_API_KEY=<your-key> # For OpenAI models
export ANTHROPIC_API_KEY=<your-key> # For Anthropic modelsThen run your coding agent from the directory where you installed (for local installs) or from anywhere (for global installs).
Available Skills (3)
LangSmith
- langsmith-trace - Query and export traces (includes helper scripts)
- langsmith-dataset - Generate evaluation datasets from traces (includes helper scripts)
- langsmith-evaluator - Create custom evaluators (includes helper scripts)
Note: All skills include Python and TypeScript helper scripts for common operations.
Development
Agent configuration lives in config/. To update an existing installation:
./install.sh --forceInstall the whole package (3 skills):
npx skills add https://github.com/langchain-ai/langsmith-skillsOr install a single skill:
npx skills add https://github.com/langchain-ai/langsmith-skills --skill <name>Pick the skill name from the Skills tab — each entry there installs independently.