Pular para o conteúdo principal

Evaluating Your AI Agent

Avançado
What you'll learn
  • Understand why agent evals differ from prompt evals — trajectory matters, not just the final answer
  • Build a golden set of 20–100 real cases with clear pass criteria
  • Score four layers: tool-call correctness, trajectory quality, task success, production drift
  • Use LLM-as-judge safely: rubric first, calibrate against humans, spot-check verdicts
  • Ship an eval that runs in CI and fails a bad change before it reaches users

An agent eval answers a harder question than "did the prompt return the right words?" It asks: did a model running in a loop pick the right tools, in the right order, with the right arguments, arrive at the right outcome — and stay within budget and safety bounds?

Skip this step and you'll ship a "helpful" agent that quietly regresses every time you tweak the system prompt.

Why agents need their own evals​

A single-prompt eval scores one input → one output. An agent produces a trajectory: a chain of reasoning, tool calls, intermediate observations, and revisions across many turns. Two failure modes make this hard:

  • Right answer, wrong path. The agent stumbles onto the correct output after wasteful loops, unsafe actions, or lucky guesses. Final-answer-only evals mark this a pass; production won't.
  • Wrong answer, plausible path. Every step looks reasonable in isolation, but the agent misused a tool, ignored a constraint, or hallucinated an intermediate fact. You need to look at the trace, not just the reply.

The four eval layers​

Layer them cheapest-first so a bad change fails fast without waiting for expensive graders.

Guided walkthrough1 of 4
  1. For each expected step, check tool name matches, required parameters are present, and types validate. Pure code, milliseconds, no model needed. Catches 'called search when it should have called write_file' before anything else runs.

Metrics that predict value​

Not every metric belongs on the dashboard. These five drive shipping decisions in 2026:

MetricWhat it measuresWhy it matters
Task success rate% of golden-set cases the agent finishes correctlyThe headline. Everything else is diagnostic.
Cost per successful task$ / passing case (tokens in + out, tool costs)Success at 10× the cost is a regression.
Latency (p50 / p95)Wall-clock per task, tail includedp95 is what real users feel — averages lie.
Tool-call accuracy% of expected tool calls with correct name + argsPredicts trajectory quality; cheap to compute.
Intervention rate% of tasks needing human takeover in prodThe autonomy number. Rising = trust falling.

Track them together — one moving without the others is usually a leading signal, not noise.

Build the golden set​

Guided walkthrough1 of 5
  1. Pull 20–100 tasks from actual usage (logs, support tickets, user requests). Cover the frequent easy path, the tricky middle, and the edge cases that already bit you.

LLM-as-judge — cheap, fast, but calibrate it​

Grading fuzzy outputs by hand doesn't scale. A capable model reading against an explicit rubric does — Anthropic's own eval methodology guide recommends this pattern for tone, faithfulness, helpfulness, and safety.

Judges have well-documented biases: they prefer longer answers, the first option shown, and outputs that echo their own phrasing. Three habits keep them honest:

  • Rubric, not vibes. "Rate helpfulness 1–5" is useless. Anchor every point on the scale to observable behavior.
  • Calibrate on a human-labeled sample. Have humans grade 30–50 cases; measure judge-vs-human agreement (aim for Cohen's κ ≥ 0.6). If it disagrees, tighten the rubric.
  • Use a different model as judge. Grading with the same model that produced the output leaks bias in both directions.
  • Spot-check verdicts weekly. Read 10 random judge scores and their reasoning. It's the cheapest way to catch drift.

LLM-as-judge rubric template

You are grading an AI assistant's response against a rubric. Be strict. Cite exact evidence from the response.

<task>{task}</task>
<response>{response}</response>

Rubric (rate 1–5 per dimension):
- Task completion: 1 = ignored task; 3 = partial; 5 = fully done, no gaps.
- Faithfulness: 1 = contains false claims; 3 = mostly grounded, one soft claim; 5 = every claim traceable to input/tools.
- Efficiency: 1 = wandered/looped; 3 = extra steps; 5 = minimum viable path.

Output JSON only:
{"task_completion": N, "faithfulness": N, "efficiency": N, "evidence": "<quote>", "verdict": "pass"|"fail"}

Trajectory review prompt (Layer 2)

You are auditing an AI agent's tool-call trajectory. The goal was: {goal}
Expected minimum steps: {n_min}

<trajectory>
{list of tool_name(args) -> result, in order}
</trajectory>

Answer in JSON:
{"steps_taken": N, "wasted_steps": N, "wrong_tool_calls": [<indices>], "unsafe_actions": [<indices>], "verdict": "pass"|"fail", "reason": "<one sentence>"}

Adversarial case generator (grow the set)

Generate 5 new eval cases that are likely to break an agent whose current failures cluster around: {failure_pattern}.

For each case give: input, expected output OR pass criterion, ideal tool sequence, and why this case is hard.

Return YAML.

CI gate: fail the bad change before it ships​

The eval only pays off when it blocks regressions automatically. Wire it into CI as a check on every prompt / model / tool change:

# tests/eval_gate.py — runs on every PR
import json, sys
from anthropic import Anthropic
from my_agent import run_agent

client = Anthropic()
golden = json.load(open("evals/golden.v3.json"))

results = []
for case in golden:
    trace = run_agent(case["input"])
    layer1 = tool_calls_match(trace, case["expected_tools"])   # deterministic
    layer3 = judge(client, case, trace.final_output)           # LLM rubric
    results.append({"id": case["id"], "layer1": layer1, "layer3": layer3["verdict"]})

pass_rate = sum(r["layer3"] == "pass" for r in results) / len(results)
tool_acc  = sum(r["layer1"] for r in results) / len(results)

# Gates — tighten over time
assert pass_rate >= 0.85, f"Task success dropped to {pass_rate:.0%}"
assert tool_acc  >= 0.90, f"Tool-call accuracy dropped to {tool_acc:.0%}"
print(f"PASS: task={pass_rate:.0%} tools={tool_acc:.0%}")

Store per-run scores so you can chart the trend. A drop of 3+ points between merges is a real regression, not noise.

Anti-patterns that make evals under-deliver
  • Judging the final answer only — misses every trajectory bug. Score Layers 1 and 2 too.
  • Static golden set — if it doesn't grow with every prod failure it stops predicting prod. Budget time monthly.
  • Same model as agent and judge — bias in both directions. Rotate to a different model for grading.
  • No cost or latency in the gate — a prompt tweak that adds 8 tool calls can 'pass' the eval while 10×-ing the bill.
  • Vibes-only scoring — 'feels better' is not a metric. If you can't diff two numbers, you can't ship confidently.
Key takeaways
  • Agents produce trajectories, not answers — evaluate the path, not only the outcome
  • Layer cheapest-first: tool-call correctness → trajectory quality → task success → production drift
  • The five metrics that ship decisions: task success rate, cost per success, p50/p95 latency, tool-call accuracy, intervention rate
  • LLM-as-judge scales, but only with an explicit rubric, a different model, and calibration against human labels
  • A golden set that doesn't grow from prod failures stops predicting prod — grow it monthly
  • Wire the eval into CI as a hard gate — the check that catches a regression before users do

Check yourself​

Check yourself

0/4
  1. Why do agents need trajectory evals, not just final-answer evals?
  2. You're layering your evals. Which order is cheapest-to-most-expensive and correct?
  3. Which pair of habits actually keeps LLM-as-judge trustworthy over time?
  4. Your CI gate passes task success rate but latency and cost per task doubled. What's the right call?
Pressione Enter ou Espaço para virar o cartão. Use as setas esquerda e direita para navegar entre os cartões.Termo exibido.
1 / 7

Next​