google-agents-cli-eval
The google-agents-cli-eval skill provides a command-line interface for evaluating AI agents, including dataset synthesis, trace generation, grading, failure analysis, and prompt optimization.
Is google-agents-cli-eval safe to install?
Review the source first: our audit of google-agents-cli-eval's source files found 10 shell commands, 0 external URLs, file reads and writes (high risk). Every command and URL listed appears verbatim in the skill's source. The skill executes shell commands via the agents-cli binary and performs file system operations to read datasets and write evaluation traces and results.
How we audit skills: our security review methodology.
Who is this skill for?
Developers building and iterating on AI agents using the Google Agent Platform.
What can you do with it?
- Synthesize evaluation datasets for multi-turn conversations
- Generate agent traces over evaluation datasets
- Grade agent traces using built-in or custom metrics
- Analyze agent failure modes through LLM-based clustering
- Optimize agent prompts using ADK GEPA
- Compare evaluation results to track performance improvements
How good is this skill?
Quality score: 5/10. The documentation is comprehensive, providing clear workflows, command examples, and guidance on the evaluation lifecycle.
What does the skill file contain?
# Agent Evaluation Guide > **Requires:** `agents-cli` (`uv tool install google-agents-cli`) — [install uv](https://docs.astral.sh/uv/getting-started/installation/index.md) first if needed. > **Scaffolded project?** If you used `/google-agents-cli-scaffold`, you already have `agents-cli eval run` (chains `generate` + `grade`), `tests/eval/datasets/`, and `tests/eval/eval_config.yaml`. Start with executing `eval run` and iterate from there. ## Reference Files | File | Contents | |------|----------| | `references/dataset_schema.md` | Canonical EvaluationDataset schema — all field types, JSON ...
Frequently asked questions
How do I start evaluating an agent?
If using a scaffolded project, execute agents-cli eval run to chain generation and grading.
What should I do if my agent fails an evaluation?
Inspect the generated JSON or HTML results for metric scores and judge rationales, then adjust prompts, tool descriptions, or instructions based on the failure type.
Can I use custom metrics?
Yes, you can define custom LLMMetric or CodeExecutionMetric types in your evaluation configuration.
When should I use eval optimize?
Use it only for prompt-only failures after manual iteration, as it is a long-running and expensive process.
Related skills
find-skills
2.3MUsers seeking to extend agent capabilities with specialized tools, workflows, or knowledge packages
The find-skills skill enables agents to search for, discover, and install modular packages from the open agent skills ecosystem using the Skills CLI.
agent-browser
506.7KAI agents and developers requiring programmatic web interaction, exploratory testing, or automation of Electron desktop applications
The agent-browser CLI provides browser automation for AI agents using Chrome or Chromium via CDP. It supports page navigation, form interaction, data extraction, and testing. The tool utilizes accessibility-tree snapshots and element references for interaction.
microsoft-foundry
429.2KDevelopers building, deploying, and managing AI agents on Azure AI Foundry
The Microsoft Foundry skill provides a framework for the end-to-end lifecycle of AI agents, including scaffolding, deployment, evaluation, fine-tuning, and troubleshooting within Azure AI Foundry.
image-to-video
355.0KDevelopers and users who need to automate video generation from images using the RunComfy platform
The image-to-video skill routes user requests to specific RunComfy animation models based on intent. It selects HappyHorse 1.0 I2V for general animation, Wan 2.7 for custom-voiceover lip-sync, or Seedance 2.0 Pro for multi-modal composition. The skill executes the RunComfy CLI to process inputs and download generated video files.