Labsco
google-gemini logo

behavioral-evals

✓ Official★ 105,700

by google-gemini · part of google-gemini/gemini-cli

Guidance for creating, running, fixing, and promoting behavioral evaluations. Use when verifying agent decision logic, debugging failures, debugging prompt…

🔥🔥🔥✓ VerifiedFreeQuick setup
#automation#git
🧰 Not standalone. This skill ships with google-gemini/gemini-cli and only works together with that tool — install the tool first, then add this skill.

Guidance for creating, running, fixing, and promoting behavioral evaluations. Use when verifying agent decision logic, debugging failures, debugging prompt…

Inspect the full instructions your agent will receiveExpand

This is the exact playbook injected into your agent when the skill activates — shown here so you can audit it before installing. You don't need to read it to use the skill.

by google-gemini

Guidance for creating, running, fixing, and promoting behavioral evaluations. Use when verifying agent decision logic, debugging failures, debugging prompt… npx skills add https://github.com/google-gemini/gemini-cli --skill behavioral-evals Download ZIPGitHub105.7k

Behavioral Evals

Overview

Behavioral evaluations (evals) are tests that validate the agent's decision-making (e.g., tool choice) rather than pure functionality. They are critical for verifying prompt changes, debugging steerability, and preventing regressions.

[!NOTE] Single Source of Truth: For core concepts, policies, running tests, and general best practices, always refer to ** evals/README.md **.

🔄 Workflow Decision Tree

  • Does a prompt/tool change need validation?

  • No -> Normal integration tests.

  • Yes -> Continue below.

  • Is it UI/Interaction heavy?

  • Yes -> Use appEvalTest (AppRig). See creating.md.

  • No -> Use evalTest (TestRig). See creating.md.

  • Is it a new test?

  • Yes -> Set policy to USUALLY_PASSES.

  • No -> ALWAYS_PASSES (locks in regression).

  • Are you fixing a failure or promoting a test?

  • Fixing -> See fixing.md.

  • Promoting -> See promoting.md.

📋 Quick Checklist

1. Setup Workspace

Seed the workspace with necessary files using the files object to simulate a realistic scenario (e.g., NodeJS project with package.json).

2. Write Assertions

Audit agent decisions using rig.setBreakpoint() (AppRig only) or index verification on rig.readToolLogs().

3. Verify

Run single tests locally with Vitest. Confirm stability locally before relying on CI workflows.

  • See ** evals/README.md ** for running commands.

📦 Bundled Resources

Detailed procedural guides:

  • creating.md: Assertion strategies, Rig selection, Mock MCPs.

  • fixing.md: Step-by-step automated investigation, architecture diagnosis guidelines.

  • promoting.md: Candidate identification criteria and threshold guidelines.