What is the core idea behind fine-tuning?
Fine-tuning adapts a general model you did not have to build into a specialist for your task.
How does fine-tuning differ from related concepts?
| Concept | Difference |
|---|---|
| Fine-Tuning vs Training from Scratch | Training from scratch builds a model from nothing at huge cost. Fine-tuning starts from an existing model. |
| Fine-Tuning vs Prompt Engineering | Prompt engineering changes the instructions. Fine-tuning changes the model weights. |
| Fine-Tuning vs RAG | RAG gives a model external documents at query time. Fine-tuning bakes knowledge or behaviour into the model itself. |
How does fine-tuning work?
- Start with a pre-trained foundation model
- Prepare a labelled dataset of examples for the target task
- Continue training so the model weights adjust to the new data
- Methods like LoRA fine-tune only a small subset of parameters to cut cost
When should you fine-tune a model?
- To teach a consistent style, tone, or format
- To specialise in a domain such as law or medicine
- To improve accuracy on a narrow, repeatable task
- When prompting and RAG alone are not enough
Why is fine-tuning important?
Fine-tuning lets organisations get frontier-model capability tailored to their needs without the enormous cost of training a model from scratch. It is a key lever for building differentiated AI products on top of foundation models.
How is fine-tuning used in practice?
Fine-tuning is used for customer-specific assistants, code models tuned to a codebase, and domain experts in fields like healthcare and finance. Its limitations are the need for quality labelled data, the risk of overfitting, and the cost of retraining as the base model updates.
Frequently Asked Questions
Is fine-tuning better than prompting?
Not always. Prompting and RAG are cheaper and faster to iterate. Fine-tuning wins when you need consistent behaviour or specialised knowledge that prompting cannot reliably deliver.
How much data do you need to fine-tune?
It varies. Behavioural or style fine-tuning can work with hundreds to thousands of examples, while deep domain adaptation may need far more.