Tokens, Context & Memory

Beginner

Three ideas unlock a lot of "why did it do that?" moments: tokens, the context window, and memory. Get these and you'll stop being surprised by drift, forgetting, and surprise bills.

What you'll learn

• Read text the way a model does — in tokens, not words or characters
• Picture the context window as a finite desk, and predict when things fall off it
• Recognize 'context rot' — why models can lose the middle of a long input
• Know the four real sources of 'memory' and how to provide it on purpose

Tokens: the unit models think in

Models don't read characters or words — they read tokens, chunks of text roughly ¾ of a word in English. "Unbelievable" might be 3–4 tokens; common words are one each; a space, a comma, or a chunk of code each cost tokens too. Both your input and the model's output are counted, and tokens are exactly what pricing and limits are measured in.

You don't need to count by hand, but a rough feel helps: ~750 words ≈ ~1,000 tokens. Type something and watch:

Paste some text here to see roughly how many tokens it is.

12words

58characters

~15–16estimated tokens

≈ <1% of a 200K-token context window

A rough feel only (~chars ÷ 4, or words × 1.33). Token counts are model-specific — never use another model's tokenizer. For exact numbers use Anthropic's token-counting endpoint.

:::tip Why the ratio shifts Plain English lands near ¾ word per token. Code, JSON, non-Latin scripts, long URLs, and rare words split into more tokens — so a 500-line file or a Chinese paragraph costs more than its word count suggests. When a bill or a limit surprises you, this is usually why. :::

The context window: working memory

The context window is the maximum number of tokens the model can consider at once — your system prompt, the whole conversation so far, any attached files, and the reply it's writing, all together. Think of it as the model's desk: large, but finite. Window sizes differ by model and keep growing — see Models & Pricing for current numbers rather than memorizing one.

Everything the model "knows" in the moment lives on that desk:

When a conversation grows past the window, the oldest content falls off. That's why a very long chat can seem to "forget" how it started, or drift away from your original instruction.

Context rot: it's not just full vs empty

A subtler problem: even when everything still fits, models tend to use the beginning and end of a long input more reliably than the middle. Bury the one sentence that matters in the center of a 50-page paste and it may get under-weighted — a failure mode often called "lost in the middle."

Last verified: 2026-06-29. The "lost in the middle" effect — degraded use of information placed mid-context — was documented by Liu et al. (2023). Newer long-context models handle it better, but the practical habit below still pays off. Confirm against the official source.

Guided walkthrough1 of 4

1. Put the actual instruction or question first, before pasting a long document — not buried after it.

Here's the same request, structured so the instruction sits in the strong positions:

Instruction-first, restated-last

Task: Find every place this contract caps our liability, and quote the exact clause.

[... paste the full 40-page contract here ...]

Reminder of the task: list only the liability-cap clauses, with exact quotes and section numbers. Ignore everything else.

:::tip In Claude Code Long agent sessions hit the same ceiling. Claude Code manages it deliberately — compacting history and letting you steer what stays in view. See Context Management and Context Engineering. :::

Memory: there isn't any, unless you provide it

By default, each conversation is a blank slate. The model doesn't remember your last chat. Everything that looks like memory is one of four things:

Source	What it is	You control it by
Re-sent history	Chat apps resend the conversation each turn, until the window fills	Starting fresh chats; keeping threads focused
Memory features	Some Claude surfaces carry facts across chats	Memory Across Chats settings
Files you provide	Persistent context you attach on purpose	Projects, CLAUDE.md
Your own code	The API is stateless — you resend prior messages	First API Call

The throughline: if you want the model to remember something, you have to keep putting it on the desk.

Why this matters

Almost every "it ignored my earlier instruction" or "it lost track" issue traces back to one of three things: the window filled up, a new session started cold, or the key detail sat in the dead middle of a long paste. Knowing this, you'll structure prompts and sessions to keep the important stuff in view.

Check yourself

Check yourself

0/3

1. 1Roughly how many tokens is 750 words of plain English?
2. 2A long chat starts 'forgetting' how it began. The most likely cause is:
3. 3You must paste a huge document plus one key instruction. Best placement?

Key terms

Lock in the vocabulary

Press Enter or Space to flip the card. Use the left and right arrow keys to move between cards.Term shown.

1 / 4

:::note Takeaways

• Tokens are the unit of both thinking and billing — ~1,000 per 750 English words, more for code and other scripts.
• The context window is a finite desk; long chats forget because old content falls off it.
• Even within the window, lead with your instruction and restate it at the end — the middle gets under-used.
• There is no memory by default. Provide it deliberately with files, Projects, CLAUDE.md, or by resending history. :::

Next

• Prompting for Long Context — where to place documents, how to tag them, and the quote-extraction trick
• What Is an LLM?
• System, User & Assistant Roles
• Context Engineering
• Tokens, Context & Pricing (API)