Fine-Tuning LLMs and LoRA Explained Simply

Fine-tuning means training an existing model a little more on your own examples, so it behaves the way you want without being told every time. It is powerful, but it is not the first tool to reach for. Most problems are solved faster and more cheaply with good prompts and retrieval. This lesson covers when fine-tuning makes sense, how it works, and LoRA, the technique that made it affordable.

When to fine-tune, and when not to

Fine-tune when the problem is behaviour:

• A strict output format or style that prompts cannot hold reliably.
• A narrow, high-volume task, such as classifying tickets, where a small fine-tuned model can match a large general one at a fraction of the cost.
• A specialised way of writing or reasoning that is hard to describe in a prompt.

Do not fine-tune to add knowledge such as product facts or policies. The model cannot cite it, it goes out of date, and private information can leak to any user. Use retrieval for knowledge. See RAG vs fine-tuning.

How supervised fine-tuning works

1. Collect examples: inputs paired with the exact outputs you want, usually hundreds to a few thousand, all high quality and varied.
2. Split the data: keep a test set aside that is never used for training.
3. Train: continue training the model on the examples with a small learning rate, so it shifts its behaviour without forgetting what it knew.
4. Evaluate: compare the fine-tuned model with the original model plus a good prompt, on the same test set. Check general abilities and safety did not get worse.

LoRA: train a tiny fraction of the weights

A large model has billions of weights. Updating all of them needs a lot of GPU memory and produces a full copy of the model for every task. LoRA (low-rank adaptation) freezes the original weights and learns a small correction for selected layers instead. The correction is the product of two thin matrices, A and B, so it has far fewer numbers than the layer itself.

This example shows the idea on one layer of size 512 × 512. The behaviour we want differs from the original by a low-rank change, which is the situation LoRA assumes:

import numpy as np

rng = np.random.default_rng(42)
d = 512                                    # size of one layer's weight matrix: d x d
W = rng.normal(size=(d, d)) / np.sqrt(d)   # frozen pre-trained weights

# The change we want the layer to learn happens to be low-rank (rank 4), as LoRA assumes.
true_change = rng.normal(size=(d, 4)) @ rng.normal(size=(4, d)) / d
X = rng.normal(size=(2000, d))
Y = X @ (W + true_change)                  # the behaviour we want after fine-tuning

r = 4                                      # LoRA rank
A = rng.normal(size=(d, r)) * 0.01         # trainable
B = np.zeros((r, d))                       # trainable, starts at zero so training starts from W
lr = 0.5
for step in range(300):
    pred = X @ W + (X @ A) @ B             # W never changes; only A and B are trained
    grad_out = (pred - Y) / len(X)
    grad_B = (X @ A).T @ grad_out
    grad_A = X.T @ (grad_out @ B.T)
    A -= lr * grad_A
    B -= lr * grad_B

err_before = np.mean((X @ W - Y) ** 2)
err_after = np.mean((X @ W + X @ A @ B - Y) ** 2)
print(f"full fine-tuning would train {d * d:,} numbers; LoRA trains {A.size + B.size:,} ({(A.size + B.size) / (d * d):.1%})")
print(f"error before: {err_before:.2e}   after LoRA: {err_after:.2e}")

full fine-tuning would train 262,144 numbers; LoRA trains 4,096 (1.6%)
error before: 7.50e-03   after LoRA: 5.85e-31

Training only 1.6% of the numbers was enough to learn the change, while the original weights W never changed. In a real model, LoRA is applied to many layers at once, but the saving is similar: adapters are often only megabytes, compared with gigabytes for the full model. You can keep one base model and swap small adapters for different tasks.

QLoRA goes further by loading the frozen base model in 4-bit precision while training LoRA adapters, which lets surprisingly large models be fine-tuned on a single GPU.

Preparing good training data

• Quality beats quantity. A few hundred clean, consistent examples often beat thousands of messy ones.
• Cover the real variety: different lengths, edge cases, and examples where the right answer is “not enough information”.
• Remove personal or confidential data you are not allowed to use for training.
• Keep a test set that the model never sees during training.

Hosted or do-it-yourself

Several model providers offer hosted fine-tuning: you upload examples and get a custom model behind the same API. With open-weight models, you can fine-tune yourself with LoRA using open-source libraries, then serve the model and adapter on your own infrastructure. Hosted is simpler; self-managed gives more control.

Common mistakes

1. Fine-tuning before trying prompting and retrieval.
2. Training on knowledge that changes or must be cited.
3. No baseline comparison, so you cannot show the fine-tuned model is actually better.
4. Overfitting: training too long on too few examples, so the model copies them instead of generalising.

Key takeaway

Fine-tuning changes a model’s behaviour using your examples; it is the right tool for consistent formats, narrow high-volume tasks and specialised styles, not for adding facts. LoRA makes it affordable by freezing the original weights and training a small low-rank correction. Always compare against a prompting baseline on a held-out test set. Next: LLM as a judge, how to score those comparisons at scale.