Fine-tuning vs Prompting vs RAG
When the model doesn't do what you want, there are three levers — and people reach for the expensive one first. Here's the order that actually works.
Try in this order
1. Prompting — start here, always
Clearer instructions, examples, a role, output constraints (Prompting Basics). Fixes the majority of problems, costs nothing extra, and is instant to iterate. Most "the model is bad at X" turns out to be "the prompt was vague."
2. RAG — when it needs your knowledge
If the gap is missing or fresh information (your docs, your data, current facts), add RAG. Keeps knowledge updatable and citable without touching the model.
3. Fine-tuning — last resort, for behavior/format at scale
Fine-tuning further-trains a model on your examples. Reach for it only when prompting + RAG can't get consistent style, format, or task behavior and you have many high-quality examples and the volume to justify it.
The decision table
| Your problem | Reach for |
|---|---|
| Vague/wrong outputs, wrong format | Prompting |
| Doesn't know your data / needs current info | RAG |
| Needs a very specific style/behavior, consistently, at scale | Fine-tuning |
| Needs to take actions | (Not these — that's tool use/agents) |
Why people get it wrong
Fine-tuning sounds like "teaching the model," so it feels like the real fix. But it's the slowest, costliest, least flexible option, it doesn't add fresh knowledge well (RAG does that), and it's easy to do badly. Exhaust prompting and RAG first — you usually won't need step 3.
:::tip They combine A strong system is often a good prompt + RAG for knowledge, with fine-tuning reserved for a narrow behavioral need. They're not mutually exclusive. :::