Takes the spreadsheet of test results Copilot Studio gives you and explains why each test failed. It then suggests the edits that would fix them, and you decide which ones to apply.
WHEN TO USE
SCOPE
USE FOR
Sort a long export down to the handful of questions the bot actually got wrong.
See what to change when an answer came back half-finished or without a source.
Tell a genuine mistake in your bot apart from a one-off glitch worth just retrying.
This is a saved copy of the skill's SKILL.md instructions. It may differ from the upstream file or the version you install.
Analyze Copilot Studio Evaluation Results
Analyze evaluation results exported from the Copilot Studio UI as CSV.
Phase 1: Get Results
Ask the user for the CSV file path if not already provided. The file is typically exported from Copilot Studio's Evaluate tab and named Evaluate <agent name> <date>.csv in their Downloads folder.
Read the CSV file. The in-product evaluation CSV has these columns:
Column
Meaning
question
The test utterance
expectedResponse
Expected response (may be empty)
actualResponse
What the agent responded
testMethodType_1
Eval method (e.g., GeneralQuality)
result_1
Pass or Fail
passingScore_1
Score threshold (may be empty)
explanation_1
Why it passed/failed (e.g., "Seems relevant; Seems incomplete; Knowledge sources not cited")
The _1 suffix indicates the first eval method. There may be additional methods (_2, _3, etc.) with the same column pattern.
Phase 2: Analyze Results
Focus on failed evaluations (result_1 = Fail, or any result_N = Fail).
For each failure, use the explanation column to understand the issue:
"Question not answered" — The agent couldn't handle the question. Check if there's a matching topic or knowledge source.
"Knowledge sources not cited" — The agent responded but didn't cite sources. Check knowledge source configuration and SearchAndSummarizeContent nodes.
"Seems incomplete" — The response was partial. Check topic flow for early exits, missing branches, or incomplete SendActivity messages.
Error messages in actualResponse (e.g., GenAIToolPlannerRateLimitReached) — These are runtime errors, not authoring issues. Flag them to the user as transient failures to retry.
Phase 3: Propose Fixes
For each failure, identify the relevant YAML file(s):
Auto-discover the agent: Glob: **/agent.mcs.yml
Find the relevant topic by matching the test utterance against trigger phrases and model descriptions
Read the topic file to understand the current flow
Propose specific YAML changes to fix each failure. Present them to the user as a summary:
Which test(s) failed and why
Which file(s) need changes
What the proposed change is (show the diff)
Wait for user decision. The user can:
Accept all — apply all proposed changes
Accept partially — apply only some changes (ask which ones)
Reject — discard proposed changes and discuss alternative approaches
Apply accepted changes using the Edit tool. After applying, remind the user to push and publish again before re-running evaluations.
Account or credentials required
1 FOR YOU
01
Export the evaluation results as CSV from the Evaluate tab
In Copilot Studio's Evaluate tab, export the test run results to CSV (named like "Evaluate <agent name> <date>.csv") and save it locally so this skill can read it.
🔌 This skill ships inside the Copilot Studio YAML Toolkit plugin. Listed package contents include 4 agents and hooks. Installed components and skill versions may vary by AI tool.
Install just this skill
Run this outside your AI tool, using Node.js 22.20.0 or newer. Skills CLI 1.7.2 skips skill selection; prompts cancel installation without an interactive terminal. If choosing Project when asked about installation scope, open your project's folder before running this.
Install checklist
Check selected tools if a picker appears; Claude Code may be unchecked under Additional agents. ↑/↓ moves, Space toggles selections, Enter confirms.
Without a tool picker, check Installing to: for your intended tool. Stop if it names a different tool.
If asked, choose Symlink (Recommended) for shared files or Copy to all agents for separate copies; review the summary before confirming installation.
Install the find-skills skill? defaults to Yes, adding another skill for all projects. Choose No if you don't want that extra installation.
Skills CLI 1.7.2 sends install data to skills.sh by default, including the source, skill names, target tools and file details; GitHub sources are reported only when GitHub confirms the repository is public, while other remote sources can also be reported. Confirmed-public GitHub sources also trigger a security-audit request. Before installing, set the environment variable DO_NOT_TRACK=1 or DISABLE_TELEMETRY=1 to disable both.
After installing for Claude Code, open it in the installation project if you chose Project; Global makes the skill available across projects on this computer. Once Claude Code loads the skill, its description helps Claude decide when to use it.
If installation created a top-level skills folder that did not exist when your Claude Code session started, run /reload-skills in that session. Run it again after later changes in that folder.
Install the whole plugin
This route installs the Copilot Studio YAML Toolkit plugin package selected for your AI tool.
Add Microsoft's marketplace, then install the plugin
Type both lines inside a Claude Code session, not at your shell prompt. They add Microsoft's marketplace, which is this repository, and the copilot-studio plugin, with its Copilot Studio skills and four sub-agents.
Install checklist
Microsoft lists Node.js 18 or later, and VS Code with the Copilot Studio extension for cloning, pushing and pulling agents.
Claude Code installs it once you pick an install option.
Good to know
A hook that runs whenever a session starts adds Microsoft's Copilot Studio instructions; one tells Claude to lean towards reading your requests as Copilot Studio requests unless there are clear signs otherwise.
The option that installs it for you in every project brings the hook's instructions into every project; to keep them to one Copilot Studio repository, pick the option for this repo only.
On first run, and after an update that changes its dependencies, the session-start hook runs npm install in the plugin's data folder; every session it also writes ~/.copilot-studio-cli/plugin-paths.json.
Add Microsoft's marketplace, then install the plugin
Type both lines in an interactive GitHub Copilot CLI session, not at your shell prompt. The first adds Microsoft's marketplace, which is this repository; the second installs the copilot-studio plugin.
Install checklist
Microsoft lists Node.js 18 or later, and VS Code with the Copilot Studio extension for cloning, pushing and pulling agents.
The plugin also carries a hooks/hooks.json file with SessionStart hooks, a path GitHub's plugin reference lists as a hooks file for plugins in this format; read the file in the repository before you install.
Guide users through adding a new connector action to a Copilot Studio agent. Connector actions require UI-based connection setup, so this skill walks users through the Copilot Studio portal steps, then delegates to edit-action for YAML modifications.Read full description
Generate and insert an Adaptive Card into a Copilot Studio topic using AdaptiveCardPrompt. Use when the user asks to add an adaptive card, rich card, form card, info card, confirmation card, or interactive card to a topic.Read full description
Add generative answer nodes (SearchAndSummarizeContent or AnswerQuestionWithAI) to a Copilot Studio topic. Use this instead of /add-node when the user asks to add grounded answers, knowledge search, generative answers, or AI-powered responses — these nodes require specific patterns (ConditionGroup follow-up, knowledge source references, autoSend, responseCaptureType) that /add-node does not cover.Read full description
Add a global variable to a Copilot Studio agent. Use when the user needs a variable that persists across topics in the same conversation and can optionally be visible to the AI orchestrator.Read full description
Add a knowledge source (public website or SharePoint) to a Copilot Studio agent. Use when the user asks to add a knowledge source, documentation URL, website, or SharePoint site for the agent to search.Read full description
Add or modify a node in an existing Copilot Studio topic. Use when the user asks to add a question, message, condition, variable, or other node to a topic. Do NOT use this for generative answers or knowledge search — use /add-generative-answers instead.Read full description
