What is fine-tuning in one sentence?

Fine-tuning continues training a pretrained model on a smaller, task-specific dataset so it adapts its behavior, tone, or output format to your particular use case.

Should I fine-tune or use RAG?

Use RAG when you need the model to know specific or changing facts. Use fine-tuning when you need it to behave differently — a consistent format, tone, or domain style. Many systems use both.

LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method that trains a small set of added weights instead of the whole model, cutting cost and storage dramatically while keeping most of the benefit.

How much data do I need to fine-tune?

Often less than people expect — sometimes a few hundred to a few thousand high-quality, consistent examples. Quality and consistency of the examples matter far more than raw volume.

Can fine-tuning go wrong?

Yes. It can overfit narrow examples, cause the model to forget general skills, or bake in mistakes from low-quality data. Without an evaluation set you cannot tell improvement from regression.

Glossary · Data & AI

What is Fine-Tuning?

Fine-tuning takes a model that already learned general capabilities during pretraining and trains it a little further on a smaller, focused dataset — so it adopts a specific behavior, tone, or output format. It is how you teach a general-purpose model to act like a specialist, without training one from scratch.

Standing on a pretrained base

Training a capable large language model from zero costs millions of dollars and enormous datasets. Fine-tuning sidesteps that by starting from a model that already understands language, code, and reasoning, then nudging its weights with a few hundred to a few thousand task-specific examples. You inherit all the general capability and spend a tiny fraction of the compute to specialize it. The same idea applies to image and audio models, though the language-model case dominates current demand.

Behavior, not facts

The most common mistake is reaching for fine-tuning to make a model "know" your company's data. That is usually the wrong tool. Fine-tuning excels at shaping how a model responds — enforcing a rigid JSON format, adopting a brand voice, classifying into your categories, or speaking a niche domain dialect. For injecting facts that change over time, retrieval-augmented generation is the better fit, because you can update the knowledge instantly without retraining and can cite sources.

Fine-tuning vs. RAG vs. prompting

Think of three escalating levers. Prompt engineering is free, instant, and should always be tried first — a better prompt solves a surprising number of problems. RAG adds knowledge from an external store. Fine-tuning changes the model itself and is the heaviest lever: it costs money, takes time, and produces an artifact you must version and maintain. The right architecture often combines them — a fine-tuned model for consistent behavior, fed by RAG for current facts, steered by a tight prompt.

LoRA and parameter-efficient methods

Updating every weight in a multi-billion-parameter model is expensive and produces a full-size copy per task. Parameter-efficient fine-tuning avoids that. LoRA (Low-Rank Adaptation) freezes the original weights and trains a small set of added matrices, capturing most of the benefit for a fraction of the compute and storage — and letting you swap adapters per task. These techniques are why fine-tuning moved from a big-lab luxury to something a small team can do on modest hardware.

The data is the hard part

Fine-tuning is only as good as its examples. A few hundred clean, consistent, representative examples beat tens of thousands of noisy ones; the model faithfully learns whatever patterns — including mistakes — live in the data. The work is in curation, labeling, and deduplication, the same data engineering discipline that underpins any model. And you cannot tell whether a run helped without a held-out evaluation set measured before and after — which is squarely an MLOps concern.

At QUANT LAB

Our first move on an AI integration project is usually to talk teams out of fine-tuning — at least at first. A sharper prompt or a solid retrieval layer often delivers what they actually want without the cost and maintenance burden of a custom model. When fine-tuning is genuinely the right call — a hard format requirement, a specialized domain, a latency-sensitive narrow task — we invest in the data curation and the evaluation harness up front, because that is what separates a model that improves from one that quietly regresses.

Long-form deep-dives that use this term

All posts

Related terms

Not sure if you should fine-tune?

We help teams choose between prompting, RAG, and fine-tuning — and build the data and evaluation pipeline behind whichever wins. Book a 30-minute call.

AI integration services