Area 01 // LLMsLesson 28 / 14016 min read

Fine-tuning with LoRA and QLoRA

Fine-tuning adapts an existing model to your task with your own examples; LoRA does it cheaply by freezing the original weights and training tiny add-on matrices, and QLoRA goes further by loading the frozen model in 4-bit so big models can be tuned on a single GPU.

On this page

Think of it like this
The problem it solves
How it works: fine-tuning with LoRA
How it works: QLoRA
See it in code
On the real platform
A real example
The easy way
Trade-offs
Common mistakes
Interview questions
Quiz
Key terms
Before and after

Think of it like this

Analogy

A professional pianist joins a jazz band. You don't retrain them from scratch (that's pretraining). You give them a few weeks of jazz practice: that's fine-tuning.

LoRA is like giving them a thin booklet of jazz adjustments they read alongside their classical training, instead of rewriting everything they know. The booklet is small, cheap to write, and you can hand a different booklet to the same pianist for a wedding gig. QLoRA is doing that practice on a compact digital keyboard instead of a concert grand: much cheaper, nearly as good.

The problem it solves

Sometimes prompting isn't enough: you need a very specific style or output format every time, a narrow skill the base model is mediocre at, or a small, fast model that behaves like a big one on your task. That calls for training on your own examples.

But full fine-tuning of a 7B+ model updates billions of weights: it needs many large GPUs, takes a full copy of the model per task, and risks breaking what the model already knew. Most teams can't afford that. LoRA and QLoRA made fine-tuning practical on one GPU and a modest budget.

How it works: fine-tuning with LoRA

Collect examples of the behaviour you want: usually a few hundred to a few thousand high-quality conversations in the SFT format (messages with user and assistant turns). Hold some back for evaluation.
Pick a base model: an open instruct model (Llama, Qwen, Mistral, Gemma) or a provider's hosted fine-tuning option.
Freeze the original weights W of the model. They won't change.
Add a pair of small matrices to chosen layers (usually the attention projections): A (r × d) and B (d × r). The rank r is small, like 8, 16 or 64. The layer now computes W·x + B·A·x.
Initialise B to zero, so at the start the model behaves exactly like the original.
Train only A and B with gradient descent on your examples. Typically under 1% of the parameters train, so memory and time drop sharply.
Save the adapter: just A and B, often a few megabytes to a few hundred megabytes. Load it on top of the base model at inference, or merge it into the weights.
Evaluate against the untuned model on your held-out set and on general tasks, to catch over-specialisation.

How it works: QLoRA

Load the frozen base model in 4-bit precision (quantization) instead of 16-bit, cutting its memory about 4×.
Keep the LoRA adapters in higher precision (16-bit) and train them as usual.
During training, the 4-bit weights are temporarily converted up for each calculation; gradients flow only into the adapters.
Result: models that would need several data-centre GPUs to fine-tune can be tuned on one large GPU, with quality close to 16-bit LoRA in many tasks.

LoRA keeps the big matrix frozen and learns a tiny add-on. QLoRA stores the frozen part in 4-bit.

	Full fine-tuning	LoRA	QLoRA
Weights trained	All	under 1% (adapters)	under 1% (adapters)
Base model memory	16-bit + optimiser states	16-bit	4-bit
Output	A full model copy	A small adapter file	A small adapter file
Typical hardware for ~8B	Several big GPUs	One big GPU	One mid-size GPU

See it in code

Install: pip install numpy
Needs: Python 3.10+. The training config below needs a GPU and the transformers, peft and trl libraries.

Why LoRA is cheap, and how the update works, in numbers.

lora_math.py

import numpy as np
 
d = 4096                       # one weight matrix in an 8B-class model is about 4096 x 4096
r = 8                          # LoRA rank: the size of the "bottleneck"
 
full_params = d * d            # fine-tuning updates every number in W
lora_params = d * r + r * d    # LoRA trains only A (r x d) and B (d x r)
print(f"full fine-tune of one matrix: {full_params:,} numbers")
print(f"LoRA (rank {r})              : {lora_params:,} numbers ({lora_params / full_params:.2%})")
 
# How LoRA changes the layer: W stays frozen, the small update B @ A is added on top.
rng = np.random.default_rng(0)
W = rng.normal(size=(6, 6))                   # frozen pretrained weights (tiny here)
A = rng.normal(size=(2, 6)) * 0.01            # trainable, rank 2
B = np.zeros((6, 2))                          # starts at zero, so training starts from W exactly
x = rng.normal(size=6)
print("before training, output unchanged:", np.allclose(W @ x, (W + B @ A) @ x))
 
B = rng.normal(size=(6, 2))                   # pretend training learned B
print("update B @ A has rank:", np.linalg.matrix_rank(B @ A), "(cheap, but it shifts the output)")

Output

full fine-tune of one matrix: 16,777,216 numbers
LoRA (rank 8)              : 65,536 numbers (0.39%)
before training, output unchanged: True
update B @ A has rank: 2 (cheap, but it shifts the output)

What a real QLoRA setup looks like with Hugging Face libraries (the key settings only):

qlora_config.py

import torch
from peft import LoraConfig
from transformers import AutoModelForCausalLM, BitsAndBytesConfig
 
model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-7B-Instruct",                     # any open instruct model
    quantization_config=BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                                           bnb_4bit_compute_dtype=torch.bfloat16),  # QLoRA: 4-bit base
)
lora = LoraConfig(r=16, lora_alpha=32, lora_dropout=0.05,
                  target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],  # attention layers
                  task_type="CAUSAL_LM")
# Then train with trl's SFTTrainer(model=model, peft_config=lora, train_dataset=...).

On the real platform

What this looks like in Hugging Face (model and adapter pages) · See the real screen in the official docs

A LoRA adapter repository on a model hub. Simplified view of a LoRA adapter repository on a model hub. — Simplified view of a LoRA adapter repository on a model hub.All About AI (simplified mock-up, not a real screenshot). Made for this course.

Fine-tuned adapters are shared on the Hugging Face Hub like models:

Model card of an adapter names its base model; you must load that exact base.
Files contain adapter_config.json (rank, target modules) and adapter_model.safetensors (the small weights).
Licence: an adapter inherits the base model's licence terms.
Hosted options also exist: OpenAI and Google offer fine-tuning of some of their models from a dashboard; you upload JSONL examples and get a new model ID.

A real example

Example

A support team teaches a small model its house style

A software company wanted support replies in a strict format: greeting, one-paragraph diagnosis, numbered steps, a link to docs, no apologies longer than one sentence. A big model followed the long style prompt about 85% of the time; a small model only 60%.

They fine-tuned a small open 8B model with QLoRA on 2,000 approved past replies (one GPU, about two hours). Format compliance reached 97%, latency halved and cost per reply dropped about 10×. Product knowledge still came from RAG, so the adapter only had to learn the style, not the facts. They retrain the adapter each quarter from newly approved replies.

The easy way

Start here

Before fine-tuning, push prompting and few-shot examples as far as they go and build an eval set. If you still need it, start with your provider's hosted fine-tuning (upload JSONL, get a model ID) or a QLoRA notebook on a rented GPU, with a few hundred excellent examples. Compare against the untuned model on your evals.

When you grow: the production setup

Teams keep versioned training data, run fine-tuning jobs as reproducible pipelines, track experiments (MLflow), evaluate task metrics plus general regression suites and safety tests, store adapters in a model registry, and serve many adapters on one base model (multi-LoRA serving in vLLM) to give each customer or task its own behaviour cheaply.

Trade-offs

What you gain

Fine-tuning gives consistent behaviour and format beyond what prompts achieve, often with a smaller, cheaper model.
LoRA trains under 1% of parameters: cheap, fast, small files, swappable adapters.
QLoRA makes large-model fine-tuning possible on a single GPU.

What it costs you

Needs good, representative training data and evals; bad data makes a worse model.
Can reduce general abilities (forgetting) or safety behaviour if done carelessly.
Adds an MLOps burden: training pipelines, versions, retraining when base models change.

When not to use it

The problem is missing or changing knowledge: use RAG.
Prompting and few-shot already meet your quality bar.
You only have a handful of examples or no way to evaluate.

Common mistakes

Fine-tuning to teach facts that change, then the model confidently states stale facts.
Training on a few hundred noisy or inconsistent examples.
Skipping a held-out eval and a general-ability check.
Loading an adapter on a different base model than it was trained on.
Training too many epochs and over-fitting to the examples' exact wording.

Interview questions

Try answering out loud first, then open the model answer.

Explain LoRA and why it's efficient.

Model answer

LoRA freezes the pretrained weight matrix W and learns a low-rank update ΔW = B·A, where A is r×d and B is d×r with r much smaller than d. Only A and B are trained, cutting trainable parameters and optimiser memory by orders of magnitude, with B initialised to zero so training starts from the original model. Adapters are tiny, can be merged into W for zero-latency serving, or swapped per task on one base model.

What does QLoRA add, and what are its trade-offs?

Model answer

QLoRA quantises the frozen base model to 4-bit (NF4, with double quantization) and trains LoRA adapters in 16-bit, with paged optimisers to handle memory spikes. Memory drops roughly 4× so large models can be tuned on a single GPU, with quality usually close to 16-bit LoRA. Trade-offs: slower steps due to dequantisation, slight quality loss on some tasks, and extra complexity in deployment if you want to serve the quantised base.

How do you decide between prompting, RAG and fine-tuning?

Model answer

Prompting first, it's cheapest and fastest to iterate. RAG when the model lacks knowledge that's large, private or changing, and you need citations. Fine-tuning when you need consistent behaviour, style, format or a narrow skill that prompting can't reliably achieve, or to make a smaller model match a larger one for cost and latency. They combine: a fine-tuned model can still use RAG. Decide with evals.

Quiz

Pick an answer to see whether it’s right and why.

0 of 6 answered

Key terms

Fine-tuning: Further training an existing model on your own examples to change its behaviour, style or narrow skills.
LoRA: Low-rank adaptation. Fine-tuning that freezes the original weights and trains small add-on matrices, usually under 1% of the parameters.
QLoRA: LoRA fine-tuning on top of a base model loaded in 4-bit precision, so large models can be tuned on a single GPU.
Adapter: The small set of trained weights produced by LoRA, loaded on top of a base model to change its behaviour.
Rank (LoRA): The size of LoRA's bottleneck (e.g. 8 or 16). Higher rank means more capacity and more trainable parameters.
Quantization: Storing a model's numbers with fewer bits (e.g. 4-bit instead of 16-bit) to save memory and speed it up, with a small quality loss.
Supervised fine-tuning (SFT): Training a model on examples of prompts paired with ideal responses, so it learns to follow instructions in a given style.

Before and after

Read first