AI Glossary
A fine-tuning method that freezes a model's weights and trains small low-rank matrices added to them, cutting memory and compute dramatically.
Introduced by Microsoft researchers in 2021, LoRA trains only a tiny fraction of parameters, and adapters can be swapped in and out of the same base model. QLoRA combines it with 4-bit quantization so mid-sized models can be tuned on consumer GPUs.
See the practical LoRA overview.
When fine-tuning helps, how LoRA and QLoRA make it affordable, how to prepare data, train, evaluate and deploy an adapter — and when not to fine-tune at all.
thelatentdaily.com Editorial Team3 min read