LLM guide
Fine-tuning: when it helps, and when it does not
Fine-tuning changes how a model behaves. It is good at teaching style, format and specialised tasks. It is poor at teaching facts that change, such as policies, prices or product details. Most enterprise projects need retrieval first, and fine-tuning only sometimes.
First tryBetter prompts
For your factsRAG
For style and formatFine-tuning
Usual methodLoRA
Which approach fits your problem?
| The problem | Try first | Why |
|---|---|---|
| The model does not know our policies or products | RAG | Facts change. Looking them up at question time keeps answers current and shows sources. |
| Answers are in the wrong format or tone | Better prompts and examples, then fine-tuning | A few good examples in the prompt often fix it. Fine-tune if it still drifts. |
| A narrow, repeated task (classify, extract, route) | Fine-tune a small model | A small fine-tuned model can match a large general one at a fraction of the cost. |
| Specialised language (legal, medical, internal jargon) | RAG plus a glossary, then fine-tuning if needed | Fine-tuning helps the model use terms naturally once retrieval supplies the facts. |
What fine-tuning takes
- Data: hundreds to a few thousand high-quality examples of input and ideal output. Quality matters far more than quantity.
- Method: usually LoRA, which trains a small set of extra weights instead of the whole model. Cheaper, faster and easier to roll back.
- Compute: LoRA on a 70B-class model fits on one 8-GPU server; on an 8B model, a single GPU can be enough. See the GPU calculator.
- Evaluation: the same test set used for model selection, so you can prove the fine-tuned model is actually better.
- Maintenance: when the base model is updated, the fine-tuning usually has to be repeated.
More LLM guides: Choosing a model · Private hosting · Governance and LLMOps · LLM FAQ · Use case: claims summaries
Starting with LLMs, or stuck after a pilot?
Tell us the task you want AI to help with and any rules about where your data can go. We will come back with a plain recommendation: which kind of model, where to run it, roughly what it costs, and how to know if it is working.