A fine-tuning run that ends with a lower training loss has told you almost nothing about whether the model is better. Loss measures how well the model imitates your training examples. It does not say whether the model still handles tasks you did not train on, follows your output format, stays safe, or beats the base model with a good prompt. This guide gives a six-step checklist for testing a fine-tuned model or LoRA adapter before it goes into production. It assumes you have already trained something; if you have not, start with our LoRA practical overview and our guide to preparing a fine-tuning dataset.
Why fine-tuned models need their own tests
Fine-tuning changes weights, and changes have side effects. Research gives two reasons to be careful.
First, specializing a model can make it forget. The paper “LoRA Learns Less and Forgets Less” compares LoRA with full fine-tuning on programming and math. According to its abstract, LoRA substantially underperforms full fine-tuning on the target domains, but it better maintains the base model’s performance on tasks outside the target domain. So the trade-off is real in both directions: you may gain less than you hoped, and you may lose something you did not measure.
Second, safety behavior is not guaranteed to survive. In “Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!”, the authors report that a handful of adversarially designed examples can remove a model’s safety guardrails, and that fine-tuning on benign data can also degrade safety alignment. If you ship a chat product, that is a regression you need to check.
The upshot is simple: compare the tuned model with the base model, on more than the target task.
Step 1: Freeze a held-out test set before you look at results
Set aside prompts that were never used for training, validation or prompt tuning. Aim for at least 100 examples for a narrow task; more if outputs vary a lot. Check for leakage: near-duplicates of training examples (same customer ticket with a different ID, the same document chunked twice) inflate scores.
Write down, before running anything, what “good” means: exact match, a rubric, or a pass/fail test. Deciding afterward invites you to pick the metric that flatters the result. Our guide to building an evaluation set for LLM apps goes deeper on this.
Step 2: Run three systems, not one
Score these on the same test set with the same decoding settings:
- The base model with a plain prompt. This is your floor.
- The base model with your best prompt, including a few examples and clear instructions. This is the fair comparison. Many fine-tuning projects turn out to be matched by a better prompt.
- The fine-tuned model, with the shorter prompt it was trained for.
Fix temperature (zero for deterministic tasks) and the maximum output length. If the fine-tuned model only beats the base with a plain prompt but not with the best prompt, fine-tuning may not be worth its maintenance cost.
Step 3: Check for forgetting on general tasks
Pick a small regression suite that reflects what the model did well before: general question answering, basic reasoning, instruction following, and any other task your product relies on. Run the base model and the tuned model on it and compare.
You do not need a famous benchmark. Thirty to fifty representative prompts from real usage, with expected behaviors, will reveal most damage. The LoRA forgetting results above are why this step matters even when you used a parameter-efficient method.
If you see a drop, common remedies include mixing some general instruction data into the training set, lowering the learning rate, training for fewer epochs, or using a lower LoRA rank. The PEFT documentation’s LoRA guide explains the rank and scaling settings.
Step 4: Test format, length and failure modes
Fine-tuning often exists to enforce a format, so test it hard:
- Does the output parse every time? Run the full test set through your parser and count failures.
- Does it stop? Look for runaway generation, repeated phrases, or missing end-of-sequence behavior, a frequent bug when the chat template at inference differs from the one used in training.
- What happens with empty, very long, or off-topic inputs? Include a few on purpose.
- Does it refuse or comply sensibly on out-of-scope requests?
Template mismatch deserves a special check. Format your inference prompts with exactly the same template and special tokens as training, and confirm that the adapter is loaded onto the same base model version.
Step 5: Run a safety and policy check
Add a small set of prompts that your product should refuse or handle carefully, such as requests for harmful content, attempts to extract system prompts, and personal-data requests. Compare the tuned model with the base model. Because published research shows that even benign fine-tuning can erode safety behavior, treat this as a required step, not a nice-to-have.
Review the training data too: if it contains personal information, secrets or copyrighted text you cannot reproduce, test whether the model repeats it by prompting with prefixes of training examples.
Step 6: Use judges carefully, and read samples yourself
For open-ended outputs, many teams use a stronger model as an automatic judge. The MT-Bench paper found that strong LLM judges can agree with human preferences at a level similar to agreement between humans, but it also documents biases, including a preference for longer answers and sensitivity to the order in which answers are shown. Mitigate this by swapping answer positions, using a clear rubric, and calibrating the judge against a few dozen answers you graded yourself.
Whatever the score says, read at least 30 outputs from the tuned model by hand, including some of its worst. Numbers hide patterns that a human sees in a minute: a new verbal tic, a wrong tone, a consistent factual slip.
A simple decision rule
Ship when the tuned model beats the base-with-best-prompt on your held-out set by a margin that matters to your product, shows no meaningful drop on the regression suite, parses reliably, and is no worse than the base on safety checks. Otherwise, adjust the data first, since data problems cause most failures, then the hyperparameters. Keep the evaluation set and the scores with the adapter’s version so the next run is comparable.
What to watch next
- Newer base model releases that match your tuned model out of the box; rerun Step 2 against them.
- Drift in real traffic, which means your test set should be refreshed with fresh production examples.
- Improvements in tooling for adapter management and automated evaluation in libraries such as PEFT and TRL.
- Further research on forgetting and safety after parameter-efficient fine-tuning.
Key takeaways
- Training loss does not show whether a fine-tuned model is better; a held-out test set does.
- Always compare against the base model with a well-written prompt, not only a plain one.
- Check for forgetting and format regressions; LoRA reduces forgetting but does not remove the need to test.
- Fine-tuning can weaken safety behavior, so include a safety check.
- Use LLM judges with calibration and position swapping, and read samples yourself.
Frequently asked questions
How big should my test set be?
For a narrow task, at least 100 held-out examples is a reasonable floor, and more helps when outputs vary. For regression checks, a few dozen representative prompts usually reveal obvious damage.
Does LoRA prevent forgetting?
Not completely. Published results indicate LoRA forgets less than full fine-tuning on out-of-domain tasks, but it also learns less in some target domains. Test either way.
Can I just compare benchmark scores?
Public benchmarks can help as a sanity check, but your own held-out prompts matter more because they reflect your task and format. See our guide to building an evaluation set.
