Reuse a representation
A pretrained model can transform inputs into features. A new task can train a small prediction head on those features, or fine-tune some or all of the original model.
When does it help?
Choose a strategy below. With a frozen feature extractor, only the new head changes during training. With fine-tuning, the pretrained extractor can change too. Both approaches reuse the starting weights; neither guarantees better results.
INTERACTIVE EXPLORATION
Choose what to update
Reuse a pretrained feature extractor and add a new head. Switch between freezing the extractor and fine-tuning it.
Check the right data
Keep target-task test data separate when selecting a model or tuning settings. Compare the strategies on held-out data: domain mismatch and overfitting can affect the result.
Check your understanding
Does pretraining guarantee better performance on every new task?
No. Performance depends on the tasks, data and evaluation.