Skip to main content

What AI Actually Costs (Across Providers)

Intermediate

"How much does AI cost?" has no single answer — it depends on which pricing archetype you're buying and how much you actually use it. The dollar figures churn every few months; the structure of the cost barely moves. This page teaches the durable shape: the three ways AI is priced, how to estimate a workload before you build it, the levers that cut the bill, and the real fork between renting an API and self-hosting.

What you'll learn
  • Tell apart the three cost archetypes: per-token API, flat subscription, and open/self-hosted
  • Estimate any workload's cost from tokens × rate × volume — before you ship it
  • Pull the big levers that cut spend without hurting quality
  • Know when open/self-hosted beats API on cost, and when API wins
  • Know which numbers go stale fast — and where to check today's truth

The three cost archetypes​

Almost every way you can pay for AI collapses into one of three shapes. Learn the shapes, not the prices.

  • Per-token API (usage-based). You call a hosted model over an API and pay per token — and input and output are priced separately, with output almost always the more expensive of the two. Bigger/smarter models cost more per token; smaller models cost far less. Most providers also offer discounts: a reduced rate for cached input you re-send, and a flat batch discount for non-urgent async jobs. This is how you pay when you build a product on Claude, GPT, or Gemini. Cost scales directly with usage — pennies at small scale, the dominant line item at large scale.
  • Flat consumer/seat subscription. A fixed monthly fee for a chat app or IDE assistant (the Claude / ChatGPT / Gemini consumer plans, Copilot-style seats). You don't see tokens; you get a usage allowance and rate limits. Predictable and cheap for a human at a keyboard — but it doesn't scale to programmatic/high-volume work, and per-seat costs add up across a team.
  • Open / local (self-hosted). You run an open-weight model (Llama, Mistral, Qwen, DeepSeek) on your own hardware or a rented GPU. There's no per-call price — you pay for the hardware (or hourly GPU rental) + electricity + ops, whether the model is busy or idle. The marginal cost of one more call is effectively near-zero at scale, but you carry fixed cost and operational burden. See Run Models Locally with Ollama.
Pro tip
  • The key split: API and subscriptions are MARGINAL cost (you pay per use); self-hosting is mostly FIXED cost (you pay to keep capacity up, used or not).
  • On the API, the single biggest surprise is that OUTPUT tokens usually cost several times more than input — verbose answers cost real money.
  • A flat subscription is the right tool for a human chatting; the per-token API is the right tool for an app calling the model in a loop.

How to estimate a workload's cost​

You can size an API bill before writing a line of production code. The whole model is one formula:

cost ≈ (input tokens × input rate) + (output tokens × output rate), per call × number of calls

Rates are quoted per million tokens, and input and output have different rates — so estimate the two halves separately and add them. A rough token rule of thumb: ~1 token ≈ 4 characters of English, or ~0.75 words. For exact counts, use the Token Estimator area and read Tokens & Pricing. The full mechanics of where tokens hide live in The Token Economy.

Guided walkthrough1 of 5
  1. Add up the input: system prompt + tools + retrieved context + history + the user turn. Then estimate the output you ask for. Count input and output separately — they're priced differently.

Back-of-envelope estimate (fill in today's rates)

Per call:
input_tokens  = system + tools + context + history + user_turn
output_tokens = the answer you ask for
call_cost = (input_tokens  * input_rate_per_million  / 1e6)
          + (output_tokens * output_rate_per_million / 1e6)

Per month:
monthly_cost = call_cost * calls_per_day * 30

Then adjust:
- cached repeated input -> use the (lower) cache-read rate for that slice
- offline / non-urgent  -> apply the batch discount to the whole job
Rates change monthly — pull today's numbers from the provider's pricing page.

The big levers to cut cost​

Most real workloads carry dead weight. These levers, roughly in order of leverage, cut spend without touching quality — pull the cheap structural ones first.

  • Right-size the model. Don't pay a flagship rate for a job a small/cheap model does fine. Classification, extraction, routing, and simple formatting usually run great on the smallest tier at a fraction of the per-token price. Reserve the big model for genuinely hard reasoning, and consider routing: cheap model handles the easy majority, escalate only the hard cases. See Choosing a Model.
  • Trim the context you send. The cheapest token is the one you never send. Prune bloated system prompts, compact long histories into a running summary, and expose only the tools the task needs — every one is input you re-pay each call.
  • RAG instead of stuffing huge context. Pasting a 50-page document to answer one question pays for 50 pages every call. Retrieval fetches only the few relevant passages — far fewer input tokens for the same (often better) answer. Reach for a giant context window only when you genuinely need the whole corpus in view at once.
  • Prompt caching. If many calls share a large unchanging prefix (system prompt, tool catalog, reference doc), caching processes it once and re-serves it at a steep discount on every later call. The single highest-leverage structural change for chat and agent workloads, because it pays back every turn.
  • Batch the non-urgent. Evals, bulk labeling, summarizing an archive — anything where you don't need the answer in seconds — runs through an async batch path at a flat discount on most providers. Trade immediacy for a materially lower bill.
  • Shorten the output. Output is the pricier side. Ask for JSON or a tight schema instead of a chatty paragraph: fewer output tokens and no downstream parsing guesswork.
  • Smaller / open models for the easy slice. For the high-volume, low-difficulty majority of calls, an open or small model can be dramatically cheaper per call — sometimes near-free if you already self-host. Keep the frontier model for the cases that actually need it.
Pro tip
  • These stack multiplicatively: cached input × right-sized model × terser output × batch discount compounds into a large total cut on an easy task — with quality unchanged.
  • Pull the structural levers (right-size, cache, RAG, batch) before micro-optimizing wording — they move the bill far more.
  • Always MEASURE the change against the real bill, not a guess. The deep playbook is The Token Economy.

When open/self-hosted beats API — and when API wins​

This is the decision that actually moves your AI budget. It's a fixed-vs-marginal-cost crossover, and volume is the variable that decides it.

  • Open / self-hosted wins at HIGH, steady volume. Once you're running enough calls to keep a GPU busy most of the time, the near-zero marginal cost per call beats paying per token forever. You amortize the fixed hardware/rental cost across a huge number of calls, gain data privacy and full customization — and accept the operational burden (provisioning, scaling, uptime, MLOps) as the price of admission.
  • API wins at LOW or SPIKY volume, or with no infra. If traffic is small, bursty, or unpredictable, paying per token means you pay only for what you use and nothing when idle — no GPU sitting warm at 3 a.m. You also skip all the ops. For most teams shipping a product, prototyping, or with traffic that comes in spikes, the API is both cheaper and far less work.
Watch out
  • Self-hosting is rarely 'free.' The model weights may be free, but GPUs, electricity, and the engineer-hours to keep it running are not — count them honestly before claiming a saving.
  • Idle GPUs are pure loss: fixed-cost infra only beats per-token pricing when utilization is HIGH. Low or spiky traffic favors the API every time.
  • The crossover point moves whenever API prices drop or hardware gets cheaper — re-run the math periodically, don't decide once and forget.
AI cost vocabulary
Press Enter or Space to flip the card. Use the left and right arrow keys to move between cards.Term shown.
1 / 6

Check yourself

0/3
  1. On a per-token API, which side of the bill is almost always more expensive?
  2. You have low, spiky, unpredictable traffic and no ML infrastructure. Which is usually cheaper?
  3. What's the most reliable way to cut an API bill on an easy, high-volume task?
Key takeaways
  • Three archetypes: per-token API (usage-based, input vs output priced separately), flat subscription (predictable, doesn't scale to high volume), and open/self-hosted (fixed cost, near-zero per call).
  • Estimate before you build: (input × input rate) + (output × output rate), per call × volume — then apply cache and batch discounts.
  • Biggest levers: right-size the model, trim context, RAG over huge context, prompt caching, batch the non-urgent, shorten output.
  • Self-hosting wins at HIGH steady volume (amortize fixed cost); API wins at LOW/spiky volume or with no infra (pay only for use).
  • Every dollar figure goes stale fast — verify at the provider's pricing page and a tracker, and run your own numbers in the Cost Calculator.

Sources & further reading​

Next​