Interactive Explainer

Freeze or Fine-tune?

You have a network pretrained on a giant dataset and a new task with your own data. Do you retrain the whole thing, freeze the backbone and only train a fresh head, or start from scratch? The right answer swings on one thing: how much data you have. Set the dataset size and cut point below and watch the accuracy curves — and the overfitting — respond.

~8 min Deep Learning · Transfer Learning · Fine-tuning

The Lab

Pick a strategy, watch it learn

The backbone is four convolutional blocks; the head is a fresh classifier for your task. Choose a training strategy, then set how many backbone blocks are frozen (kept at their pretrained weights) and how big your dataset is. The curves are illustrative but deterministic — they encode the standard bias–variance story, not random noise.

Strategy:

Frozen blocks (cut point) = 0 / 4

Dataset size = 100 imgs

The backbone (four conv blocks) and head. Blue = pretrained & trainable, grey = frozen, orange = trained from scratch, teal = the fresh head.

Illustrative learning curves for the chosen setting. Solid = training accuracy, dashed = validation accuracy. A wide gap between them is overfitting.

Best validation acc. —

Generalisation gap —

Trainable params —

The Payoff

Less data → freeze more. More data → fine-tune more.

Tiny dataset. Fine-tuning millions of weights on a few hundred images overfits: training accuracy shoots up, validation lags and even droops. Freezing the backbone slashes the number of trainable parameters, so the model can't memorise — the frozen pretrained features do the heavy lifting and validation holds up.
Large dataset. Now there is enough signal to safely move every weight. Fine-tuning adapts the backbone's features to your task and pulls ahead; a frozen backbone becomes a ceiling because those features can no longer specialise.
From scratch needs the most data of all — with no pretrained features it starts from noise, so with little data it is the worst option by a wide margin.
The cut point is a dial, not a switch. Freeze the early, generic layers (edges, textures) and fine-tune the later, task-specific ones. Slide the cut point and watch the trainable-parameter count — and the overfitting risk — move with it.

Why early layers transfer best. The first conv blocks of almost any vision model learn edge and texture detectors that are useful for essentially every image task. The later blocks are increasingly specialised to the original labels. So the safe default is: freeze early, fine-tune late, and let the amount of data decide how far down the fine-tuning reaches.