إنتقل إلى المحتوى الرئيسي

Evaluating Your AI Agent

متقدّم
What you'll learn
  • Understand why agent evals differ from prompt evals — trajectory matters, not just the final answer
  • Build a golden set of 20–100 real cases with clear pass criteria
  • Score four layers: tool-call correctness, trajectory quality, task success, production drift
  • Use LLM-as-judge safely: rubric first, calibrate against humans, spot-check verdicts
  • Ship an eval that runs in CI and fails a bad change before it reaches users

An agent eval answers a harder question than "did the prompt return the right words?" It asks: did a model running in a loop pick the right tools, in the right order, with the right arguments, arrive at the right outcome — and stay within budget and safety bounds?

Skip this step and you'll ship a "helpful" agent that quietly regresses every time you tweak the system prompt.

Why agents need their own evals​

A single-prompt eval scores one input → one output. An agent produces a trajectory: a chain of reasoning, tool calls, intermediate observations, and revisions across many turns. Two failure modes make this hard:

  • Right answer, wrong path. The agent stumbles onto the correct output after wasteful loops, unsafe actions, or lucky guesses. Final-answer-only evals mark this a pass; production won't.
  • Wrong answer, plausible path. Every step looks reasonable in isolation, but the agent misused a tool, ignored a constraint, or hallucinated an intermediate fact. You need to look at the trace, not just the reply.

The four eval layers​

Layer them cheapest-first so a bad change fails fast without waiting for expensive graders.

Guided walkthrough1 of 4
  1. For each expected step, check tool name matches, required parameters are present, and types validate. Pure code, milliseconds, no model needed. Catches 'called search when it should have called write_file' before anything else runs.

Metrics that predict value​

Not every metric belongs on the dashboard. These five drive shipping decisions in 2026:

MetricWhat it measuresWhy it matters
Task success rate% of golden-set cases the agent finishes correctlyThe headline. Everything else is diagnostic.
Cost per successful task$ / passing case (tokens in + out, tool costs)Success at 10× the cost is a regression.
Latency (p50 / p95)Wall-clock per task, tail includedp95 is what real users feel — averages lie.
Tool-call accuracy% of expected tool calls with correct name + argsPredicts trajectory quality; cheap to compute.
Intervention rate% of tasks needing human takeover in prodThe autonomy number. Rising = trust falling.

Track them together — one moving without the others is usually a leading signal, not noise.

Build the golden set​

Guided walkthrough1 of 5
  1. Pull 20–100 tasks from actual usage (logs, support tickets, user requests). Cover the frequent easy path, the tricky middle, and the edge cases that already bit you.

LLM-as-judge — cheap, fast, but calibrate it​

Grading fuzzy outputs by hand doesn't scale. A capable model reading against an explicit rubric does — Anthropic's own eval methodology guide recommends this pattern for tone, faithfulness, helpfulness, and safety.

Judges have well-documented biases: they prefer longer answers, the first option shown, and outputs that echo their own phrasing. Three habits keep them honest:

  • Rubric, not vibes. "Rate helpfulness 1–5" is useless. Anchor every point on the scale to observable behavior.
  • Calibrate on a human-labeled sample. Have humans grade 30–50 cases; measure judge-vs-human agreement (aim for Cohen's κ ≥ 0.6). If it disagrees, tighten the rubric.
  • Use a different model as judge. Grading with the same model that produced the output leaks bias in both directions.
  • Spot-check verdicts weekly. Read 10 random judge scores and their reasoning. It's the cheapest way to catch drift.

LLM-as-judge rubric template

You are grading an AI assistant's response against a rubric. Be strict. Cite exact evidence from the response.

<task>{task}</task>
<response>{response}</response>

Rubric (rate 1–5 per dimension):
- Task completion: 1 = ignored task; 3 = partial; 5 = fully done, no gaps.
- Faithfulness: 1 = contains false claims; 3 = mostly grounded, one soft claim; 5 = every claim traceable to input/tools.
- Efficiency: 1 = wandered/looped; 3 = extra steps; 5 = minimum viable path.

Output JSON only:
{"task_completion": N, "faithfulness": N, "efficiency": N, "evidence": "<quote>", "verdict": "pass"|"fail"}

Trajectory review prompt (Layer 2)

You are auditing an AI agent's tool-call trajectory. The goal was: {goal}
Expected minimum steps: {n_min}

<trajectory>
{list of tool_name(args) -> result, in order}
</trajectory>

Answer in JSON:
{"steps_taken": N, "wasted_steps": N, "wrong_tool_calls": [<indices>], "unsafe_actions": [<indices>], "verdict": "pass"|"fail", "reason": "<one sentence>"}

Adversarial case generator (grow the set)

Generate 5 new eval cases that are likely to break an agent whose current failures cluster around: {failure_pattern}.

For each case give: input, expected output OR pass criterion, ideal tool sequence, and why this case is hard.

Return YAML.

CI gate: fail the bad change before it ships​

The eval only pays off when it blocks regressions automatically. Wire it into CI as a check on every prompt / model / tool change:

# tests/eval_gate.py — runs on every PR
import json, sys
from anthropic import Anthropic
from my_agent import run_agent

client = Anthropic()
golden = json.load(open("evals/golden.v3.json"))

results = []
for case in golden:
    trace = run_agent(case["input"])
    layer1 = tool_calls_match(trace, case["expected_tools"])   # deterministic
    layer3 = judge(client, case, trace.final_output)           # LLM rubric
    results.append({"id": case["id"], "layer1": layer1, "layer3": layer3["verdict"]})

pass_rate = sum(r["layer3"] == "pass" for r in results) / len(results)
tool_acc  = sum(r["layer1"] for r in results) / len(results)

# Gates — tighten over time
assert pass_rate >= 0.85, f"Task success dropped to {pass_rate:.0%}"
assert tool_acc  >= 0.90, f"Tool-call accuracy dropped to {tool_acc:.0%}"
print(f"PASS: task={pass_rate:.0%} tools={tool_acc:.0%}")

Store per-run scores so you can chart the trend. A drop of 3+ points between merges is a real regression, not noise.

Anti-patterns that make evals under-deliver
  • Judging the final answer only — misses every trajectory bug. Score Layers 1 and 2 too.
  • Static golden set — if it doesn't grow with every prod failure it stops predicting prod. Budget time monthly.
  • Same model as agent and judge — bias in both directions. Rotate to a different model for grading.
  • No cost or latency in the gate — a prompt tweak that adds 8 tool calls can 'pass' the eval while 10×-ing the bill.
  • Vibes-only scoring — 'feels better' is not a metric. If you can't diff two numbers, you can't ship confidently.
Key takeaways
  • Agents produce trajectories, not answers — evaluate the path, not only the outcome
  • Layer cheapest-first: tool-call correctness → trajectory quality → task success → production drift
  • The five metrics that ship decisions: task success rate, cost per success, p50/p95 latency, tool-call accuracy, intervention rate
  • LLM-as-judge scales, but only with an explicit rubric, a different model, and calibration against human labels
  • A golden set that doesn't grow from prod failures stops predicting prod — grow it monthly
  • Wire the eval into CI as a hard gate — the check that catches a regression before users do

Check yourself​

Check yourself

0/4
  1. Why do agents need trajectory evals, not just final-answer evals?
  2. You're layering your evals. Which order is cheapest-to-most-expensive and correct?
  3. Which pair of habits actually keeps LLM-as-judge trustworthy over time?
  4. Your CI gate passes task success rate but latency and cost per task doubled. What's the right call?
اضغط Enter أو مفتاح المسافة لقلب البطاقة. استخدم مفتاحي السهمين الأيسر والأيمن للتنقل بين البطاقات.تم إظهار المصطلح.
1 / 7

Next​