Fine-tuning means continuing to train a pre-trained model on your own examples so it behaves the way you need. Full fine-tuning of a large model updates billions of parameters and needs large GPU clusters. Techniques such as LoRA and QLoRA changed that, making fine-tuning feasible on a single GPU.

When fine-tuning is the right tool

Fine-tuning shines at teaching behavior and form:

  • a consistent output format or style;
  • a specialized classification or extraction task;
  • domain-specific tone or terminology;
  • making a smaller model match a larger one on a narrow task, which cuts costs.

It is weaker at adding facts that change. For knowledge, retrieval-augmented generation is usually better. Before fine-tuning, check whether good prompting with examples already solves the problem.

How LoRA works

Low-Rank Adaptation, introduced by Edward Hu and colleagues at Microsoft in 2021, freezes the original model weights and adds small trainable matrices to selected layers. Instead of updating a huge weight matrix directly, LoRA learns a low-rank update — the product of two thin matrices — that is added to it.

The results are striking:

  • Far fewer trainable parameters — the paper reported reductions by orders of magnitude compared with full fine-tuning.
  • Lower memory use, since optimizer state is needed only for the small matrices.
  • Small adapter files, often megabytes rather than gigabytes, that can be swapped on top of the same base model.
  • No added inference latency once the adapter is merged into the weights.

QLoRA: fine-tuning on a single GPU

QLoRA (2023), by Tim Dettmers and colleagues, combined LoRA with a base model quantized to 4 bits, plus memory-saving techniques. The authors fine-tuned a 65-billion-parameter model on a single 48 GB GPU while matching the performance of 16-bit fine-tuning on their benchmarks. QLoRA made fine-tuning accessible to individuals and small teams.

The workflow

  1. Define the task and success metric. What exactly should improve, and how will you measure it?
  2. Build a dataset. Hundreds to a few thousand high-quality examples in the model’s chat format often suffice for narrow tasks. Quality beats quantity: remove inconsistent or wrong examples.
  3. Hold out a test set that the model never sees during training.
  4. Choose a base model whose license allows your use and whose size fits your hardware.
  5. Train with a library such as Hugging Face PEFT and TRL, or specialized tools. Key settings: LoRA rank, learning rate, number of epochs.
  6. Evaluate against the base model with prompting, on your test set.
  7. Deploy by loading the adapter on top of the base model or merging it.

Watch out for overfitting

Training too long on a small dataset makes the model memorize examples and lose general abilities — sometimes called catastrophic forgetting. Monitor validation loss, keep epochs low, and test general capabilities as well as the target task.

Common mistakes to avoid

  • Fine-tuning to add knowledge. Facts learned through fine-tuning are hard to update and easy to hallucinate; use retrieval for knowledge.
  • Skipping a baseline. Compare against the base model with good prompting; sometimes fine-tuning adds little.
  • Dirty data. Inconsistent labels or formats teach inconsistent behavior. Review samples manually.
  • Ignoring licenses. Check that the base model’s license allows fine-tuning and your intended use.

Hardware expectations

With 4-bit quantization (QLoRA), many 7–8B models can be fine-tuned on a single consumer GPU with around 16–24 GB of memory.

Related: build, buy or fine-tune?

Key takeaways

  • Fine-tuning is best for behavior, format and narrow tasks, not changing knowledge.
  • LoRA trains small low-rank adapters while freezing the base model.
  • QLoRA adds 4-bit quantization so large models can be tuned on a single GPU.
  • A few hundred to a few thousand high-quality examples often suffice for narrow tasks.
  • Always evaluate against a prompted baseline and watch for overfitting.

Frequently asked questions

How much data do I need?

For narrow tasks, often a few hundred to a few thousand good examples. Complex behaviors need more. Quality and consistency matter more than volume.

Can I fine-tune closed models?

Some providers offer fine-tuning of selected models through their APIs. With open-weight models you have full control over the process.

What is LoRA rank?

The size of the low-rank matrices. Higher ranks can capture more complex changes but use more memory; small ranks often work well.