Skip to main content
Intermediate

Context Engineering

Prompt engineering is about the words you choose. Context engineering is about the workspace you hand the model — what's in it, what order it's in, and what you deliberately left out.

The distinction matters because a context window is not a notepad. It's a limited, expensive, attentional resource. How you fill it changes what the model focuses on, how much it costs you, and whether it stays useful as sessions grow.

What you'll learn
  • Tell context engineering apart from prompt engineering — and know why the field moved
  • Treat the context window as an attention budget, not a storage bin
  • Spot context rot and 'lost in the middle' before they wreck a long session
  • Place instructions where attention actually lands — top, end, never buried in the middle
  • Apply the three workhorse tactics: compaction, note-taking, and just-in-time retrieval

The context budget​

Every model has a maximum context size — a hard ceiling measured in tokens. Think of it as a budget. You spend it on:

  • Your system prompt and standing instructions
  • Retrieved documents, codebase snippets, tool definitions
  • Conversation history
  • The model's output (which also counts against the window in multi-turn sessions)

When you run out, something has to give. Either old content gets dropped, or the session hits a wall.

Most beginner guides treat the context window as "more is better." Context engineering treats it as a resource to allocate carefully: spend it on what the model actually needs for this turn, not on everything that might be relevant. Anthropic frames the whole discipline as a search for "the smallest possible set of high-signal tokens" — every token you add competes for a finite attention budget, a direct consequence of how a transformer relates every token to every other one.

Context rot and "lost in the middle"​

There is a well-documented phenomenon in long-context LLMs: models pay disproportionate attention to content near the beginning and end of their context, and their recall of content buried in the middle degrades. Researchers studying this effect called it "lost in the middle."

The practical consequence: if you stuff a 100,000-token context with documents and bury the most critical instruction at position 60,000, the model may effectively ignore it — not because it's incapable of reading that far, but because attention is not evenly distributed across the window.

"Context rot" is the broader pattern: as a session grows, the quality of responses tends to drift. Early instructions get diluted. Repeated back-and-forth crowds out the original task. The model starts hedging, repeating itself, or losing the thread of what you actually asked for.

These are not bugs you can fully fix with a better prompt. They are structural properties of how attention works at scale. The engineering response is to keep the context smaller and sharper, not to fill it and hope.

Ordering matters​

Where you place content is as important as what you include. Established good practice:

PositionWhat to put there
Very top (system prompt)Stable, durable instructions. Persona, rules, format requirements.
After system promptThe current task, in plain terms.
Just before the last user turnThe most critical, specific context for this exact request.
MiddleSupporting documents, retrieved chunks — ordered by relevance, not chronology.
Conversation historyOnly what's necessary for continuity. Prune aggressively.

The general rule: the closer to the current turn, the more attention it gets. Critical instructions that live only in the middle of a long history are at risk.

The "right altitude" for instructions​

A system prompt can fail in two opposite ways. Too low and you hardcode brittle, if-this-then-that logic that breaks the moment reality differs. Too high and you write vague guidance that assumes context the model doesn't have. Anthropic calls the target the "right altitude" — the Goldilocks zone that is "specific enough to guide behavior effectively, yet flexible enough to provide strong heuristics." Aim there: concrete rules and examples, not a decision tree, and not a vibe.

Retrieval over stuffing​

The temptation is to put everything in: all the docs, the full codebase, the entire conversation. Resist it.

The better approach is selective retrieval: identify what the model actually needs for this specific request, and inject only that. A well-retrieved 2,000-token chunk of the right document outperforms a 40,000-token dump where the answer is somewhere in the middle.

This is why retrieval-augmented generation (RAG) exists — not just to overcome context limits, but to improve quality by keeping the context curated. The agentic version is just-in-time retrieval: instead of pre-loading every document, the agent holds lightweight identifiers (file paths, IDs, queries) and pulls the actual content into context only at the moment it's needed.

For interactive sessions, the same logic applies: instead of accumulating everything, periodically compact or clear history to remove content that is no longer relevant to the current task. Claude Code's /compact and /clear commands are context engineering tools, not just session management. On the API, the same pattern is automated by memory and context editing — old tool results get pruned from the window while what matters is written to a persistent memory store.

The three workhorse tactics​

For long-horizon work, three techniques do most of the heavy lifting. They compose — most real agents use all three.

Guided walkthrough1 of 3
  1. When a session nears the window limit, summarize it and reinitialize with the distilled version. Preserve the load-bearing details — architectural decisions, unresolved bugs, key implementation choices — and drop the play-by-play. This is what /compact does in Claude Code.

The cost angle​

Tokens you send are tokens you pay for — both in money and latency. Stuffing context with loosely relevant material inflates both. Context engineering and cost efficiency are the same problem.

More concretely:

  • A bloated system prompt you copy-paste from a template is paid for on every single call.
  • Old conversation history you carry forward because "it might be useful" is paid for on every single call.
  • Documents you inject "just in case" are paid for on every single call.

Trimming what doesn't need to be there is simultaneously better for quality and cheaper to run.

Practical tactics for Claude users​

In Claude.ai:

  • Use distinct conversations for distinct tasks. Don't let an afternoon of tangents pollute the context of a focused project.
  • Summarize long threads before asking a complex question that depends on them. An explicit summary is often more useful than the raw history.
  • Put the specific thing you want at the end of a long message, not buried in the middle.

In Claude Code:

  • Keep your CLAUDE.md file lean. Every line in it is injected into every session. See CLAUDE.md and Context Management.
  • Use /clear when switching to a genuinely different task. Use /compact when you want to continue but the session is growing.
  • Reference files by path rather than pasting their contents when the full file isn't needed for the current step.

At the API level:

  • Design system prompts to contain only what every request truly needs. Move task-specific instructions into the user turn.
  • For document-heavy use cases, retrieve and inject the relevant chunks rather than uploading an entire corpus.
  • Structure the prompt so the stable, reusable prefix comes first — this also enables prompt caching, which is a natural companion to context engineering.

When you want to hand a long document to Claude, the placement rule beats raw volume every time:

Instruction-first, restated-last

Task: Find every clause where this contract caps our liability, and quote each one verbatim with its section number.

[... paste the full 40-page contract here ...]

Reminder of the task: list every liability-cap clause above, quoted exactly, with section numbers. If there are none, say so explicitly.

The same instruction sits at the top and the bottom — the two positions attention favors — so it survives even a very long middle.

The shift in mindset​

Prompt engineering asks: "What should I say?" Context engineering asks: "What should the model see, in what order, and what should I deliberately keep out?"

The second question is harder, but it's the one that actually determines quality at scale.

Check yourself

0/3
  1. What is the core difference between prompt engineering and context engineering?
  2. You bury the single most important instruction at token 60,000 of a 100,000-token context. What's the likely result?
  3. A coding agent is about to run out of context on a long task. Which tactic preserves progress best?
Lock in the vocabulary
Press Enter or Space to flip the card. Use the left and right arrow keys to move between cards.Term shown.
1 / 6
Key takeaways
  • The context window is an attention budget, not storage — spend it on high-signal tokens only.
  • Position beats volume: critical instructions go at the top and just before the final turn, never buried in the middle.
  • Compaction, note-taking, and just-in-time retrieval are the three tactics that keep long agents coherent.
  • Curating context is the same lever as cutting cost — fewer tokens, better answers, lower bill.