Skip to main content
Intermediate

The Verification Bottleneck

What you'll learn
  • See the pattern: AI shifted the bottleneck from writing code to trusting it — and the industry data proving it
  • Understand the three forces (volume, context loss, plausibility bias) that make AI-generated PRs uniquely expensive to review
  • Learn the three-layer routing that separates baseline checks, AI triage, and focused human judgment
  • Set up a codebase-aware reviewer using AGENTS.md / CLAUDE.md + a per-service rule stack
  • Know the compounding loop — how monthly rule additions turn team taste into machine-checkable policy over 12 weeks

Here is a pattern that catches almost every team the first year they roll out AI coding agents:

Cursor / Copilot / Claude Code went in. Feature-branch throughput jumped. Then the review queue got worse, cycle time flattened, and nobody could figure out where the win went.

This is the verification bottleneck. Writing code was never the whole cost of shipping code — reviewing it, trusting it, and deciding whether it's safe to merge always was. AI dropped the price of the first part by an order of magnitude and left the rest of the system priced the same.

The pattern in one sentence​

AI reduces the cost of producing code before it reduces the cost of trusting code. The constraint moves. If you don't move your workflow with it, throughput at the top of the pipe just piles up at the review step.

Or, in the framing from Codacy's writeup of the phenomenon: "more code reaches the pull request queue, more changes need verification, and the people trusted to review risky work still have the same number of hours in the week."

The industry data (2025–2026)​

The shape shows up across multiple independent datasets:

SignalFindingSource
Feature-branch throughput+59% YoY for AI-adopting teamsCodacy (2026)
Main-branch throughput (median team)−7% YoY, success rate down to 70.8%Codacy (2026)
Pickup time for AI-assisted PRs2.47× longer wait before a reviewer startsCodacy (2026)
Pickup time for fully agentic PRs5.3× longer waitCodacy (2026)
PRs merging with zero review+31% among AI-heavy teamsCodacy (2026)
Size of AI-assisted PRs~18% larger than traditional PRsJellyfish, cited by MetaCTO (2026)
Trust in AI code accuracy (dev survey)29% — historic lowStack Overflow Developer Survey 2025

Notice what these numbers don't say. They don't say the AI writes bad code. They say the pipeline downstream of the AI wasn't designed for the volume, size, and context-loss that AI-generated changes bring.

Why AI-generated PRs are uniquely expensive to review​

Three forces stack:

1. Volume. One prompt can generate what used to take a day. If your team is producing 3× the PRs, your review capacity doesn't magically scale with it. The queue grows first; morale grows second.

2. Context loss. A human author remembers the failed approaches, the constraint that ruled out the "obvious" fix, the file they almost changed. An AI author leaves none of that in the diff. The reviewer has to reconstruct intent from scratch — for every PR — because the implementation journey isn't in the commit message.

3. Plausibility bias. AI output looks right. It follows conventions, formats cleanly, imports the expected libraries. The failure mode isn't syntax errors that jump off the page — it's subtle mismatches between intent and behavior that only surface when you read carefully. That reading is slow, and it's the exact reading humans skip when the queue is deep.

Put them together: more PRs, each one carrying less inherited context, each one demanding more careful reading than a human-authored diff of the same size. That's the bottleneck.

The three-layer routing that fixes it​

The mitigation that repeatedly shows up across Codacy, MetaCTO, Moderne, and the FreeCodeCamp handbook is the same shape: layer the review so humans only see what humans are irreplaceable for.

Guided walkthrough1 of 3
  1. Format, lint, type-check, security scan, dependency policy, secrets scan, coverage gate. Runs before a reviewer is even assigned. Blocks the boring, mechanical failures that eat 40% of a human's attention when review starts cold. If a change fails here, it doesn't reach anyone.

The first two layers should account for most of the diff you'd previously have eyeballed. The third layer stops being a queue and starts being a decision.

The codebase-aware reviewer, concretely​

A generic AI reviewer misses the one thing that matters: your architecture. The pattern that's crystallizing across teams is to move institutional knowledge into machine-readable files the reviewer can consume.

project-root/
├── AGENTS.md              # entry point — lean, high-signal
├── CLAUDE.md              # symlink → AGENTS.md
├── .claude/
│   ├── settings.json      # read-only guardrails
│   ├── pr-rules/
│   │   ├── common.md      # rules that apply everywhere
│   │   ├── frontend.md    # per-workspace rules
│   │   └── backend.md
│   └── commands/
│       └── review-pr.md   # the review slash command
├── frontend/AGENTS.md     # architecture, patterns, anti-patterns
└── backend/AGENTS.md      # business rules, contracts, test conventions

Two design choices in this layout are load-bearing:

  • Per-service AGENTS.md, not one giant file. Long instruction lists cause degraded compliance across all entries — the model starts skimming past line 200. Keep each file to one paragraph per topic, link out for detail.
  • Rules are imperative, not aspirational. "Controllers must not call repositories" is testable. "Try to keep controllers thin" is a coin flip.

Here's the shape of a review command that runs locally before a PR opens:

Local /review-pr command (Claude Code / codebase-aware reviewer)

You are reviewing a pull request against the main branch.

STEPS:
1. Fetch main, compute the merge base with the current branch, and read the full diff.
2. Read the PR title/body for stated intent. If intent is unclear, ask before reviewing.
3. Load rules in order: .claude/pr-rules/common.md, then any workspace-specific rule
 files whose path prefix matches the changed files (e.g. frontend/**, backend/**).
4. Read the nearest AGENTS.md for each changed file's service and note conventions.

OUTPUT — use exactly these sections and nothing else:

## Summary
One paragraph. What changed, and why (as stated in the PR).

## Blocking
Real defects, security issues, or rule violations that must be fixed before merge.
Format: `path:line — <problem>. <suggested fix>.`

## Should fix
Quality issues that would normally get pushback in review but aren't blockers.

## Nice to have
Minor improvements. The author may reasonably ignore these.

## Verified
Things you actively checked and confirmed correct. This section exists so the
human reviewer does not re-check them.

## Rule candidate (optional)
If you saw a recurring pattern this review, suggest ONE new rule for a human to
evaluate for pr-rules/. Do not modify any rule file yourself.

CONSTRAINTS:
- No praise, no manufactured concerns. If nothing is wrong in a section, write "None."
- Cite as `file:line`. No prose descriptions of location.
- You never modify rule files. You never approve or merge.

Guardrails: default to read-only​

A reviewer agent with write access to main, secrets, or workflow files is a supply-chain incident waiting to happen. The convention that's converging is a .claude/settings.json (or the equivalent in whatever tool you use) that hard-blocks:

  • Secrets: .env*, .npmrc, .pgpass, *.pem, **/credentials.json
  • Git write ops: push, commit, rebase, reset --hard — allow fetch, diff, log
  • PR mutation: creating, merging, or approving PRs; commenting is allowed only if you want the reviewer's output posted automatically
  • Workflow / secret mutation: gh workflow run, gh secret, gh variable

If the reviewer can only read, the worst case of a compromised or hallucinating agent is a bad review comment. That's a much cheaper failure mode than a bad merge.

Risk-based routing (not everything deserves the same review)​

The other mistake teams make is treating every AI-generated PR identically. Route by risk:

Risk tierExamplesReview
LowDoc-only changes, dependency patch bumps caught green by CI, generated snapshot updatesLayer 1 + AI triage; auto-merge if both pass
MediumFeature code in isolated modules, refactors within a serviceLayer 1 + AI triage + one focused human reviewer
HighAuth, billing, migrations, code that crosses service boundaries, anything touching production dataLayer 1 + AI triage + named senior reviewer + intent doc pre-coding

The point isn't to reduce review — it's to spend the human hours where they change outcomes.

PR decomposition: keep changes checkable​

A 10,000-line AI-generated PR is not reviewable in a meaningful sense. It gets rubber-stamped or it sits. The workflow move is to force decomposition:

  • Cap PR size with an explicit split-if-larger policy (many teams pick 400–900 lines, with generated migrations exempted).
  • Stacked PRs for features that legitimately need more scope — each layer is checkable, the whole set lands together.
  • AI plans, deterministic tools execute. For repo-wide changes, treat AI as the planner (which files, what recipe) and a deterministic tool as the executor (apply the recipe identically everywhere). Morgan Stanley's OpenRewrite-based program at scale used exactly this split.

The compounding loop​

Every recurring review comment is a candidate rule. The rule goes into pr-rules/, the AI reviewer picks it up on the next PR, and that comment never has to be written by a human again. The FreeCodeCamp handbook frames the timeline as:

  • Week 1: the reviewer catches 5–10% of your team's recurring nits.
  • Week 4: 20–30%, as the ruleset grows from real PR feedback.
  • Month 3+: the ruleset has matured into written-down team taste.

The compounding is the actual moat. Any team can install CodeRabbit or Greptile in an afternoon. The team that spent 100 PRs feeding review comments back into pr-rules/ has an AI reviewer that knows their codebase. The team that didn't has a generic one.

The maintenance discipline is small but non-negotiable:

  • Every catch: write one line into the right pr-rules/ file.
  • Monthly: trim stale rules (features removed, patterns retired).
  • Quarterly: re-read each per-service AGENTS.md and prune drift.

What still requires a human (and always will)​

The three-layer routing is aggressive, but there's a floor. Humans stay irreplaceable for:

  • Product judgment. Should this change exist at all? An AI reviewer measures against rules; it can't measure against strategy.
  • Cross-team consequences. The change looks correct in isolation but breaks a contract with another team's service.
  • Accountability. Someone has to own a post-mortem when this ships broken. The AI can't be paged.
  • AI-blind-spot review. Two AIs trained on the same data share the same blind spots. If both the author and the reviewer are AI, an entire class of bugs becomes systematically invisible. A human sees them precisely because their training set is different.

The healthy end state isn't "the AI reviews everything." It's "the AI clears the noise so humans review the things that need a human."

The honest framing​

Most of the AI productivity gain in coding is real. Feature-branch throughput at +59% is not a mirage. But throughput at the top of a pipe is not throughput out the bottom, and every measurement of shipped code — main-branch merges, cycle time, defect escape rate — tells the same story: the constraint moved, and teams that didn't move the workflow with it saw the gain disappear into the review queue.

The teams winning with AI coding agents in 2026 aren't the ones generating the most code. They're the ones with the shortest, cheapest, most trusted path from generated to merged.

Quiz​

Check yourself

0/5
  1. A team rolls out Claude Code. Feature-branch throughput jumps 60%, but main-branch throughput and cycle time barely move. What's the most likely explanation?
  2. Why is AI-generated code uniquely expensive to review compared to human-authored code of the same size?
  3. You're setting up the three-layer routing. Which layer should the AI reviewer live in, and what's its job?
  4. The FreeCodeCamp handbook and Codacy both call out the *compounding* effect of the workflow. What compounds?
  5. Your AI reviewer has permission to push branches, create PRs, and modify pr-rules/. What's the specific risk this creates?

Flashcards​

Press Enter or Space to flip the card. Use the left and right arrow keys to move between cards.Term shown.
1 / 9

Sources & further reading​