Three different jobs
Training data adjusts model parameters. Validation data helps choose model settings and compare approaches. The test set is reserved for a final estimate after these choices. Repeatedly selecting models based on the test score can turn that test set into another validation set.
Prevent leakage
Preprocessing can leak information too. For example, fit a scaler using the training set, then apply that fitted scaler to validation and test data. Keep related examples from the same person or source together when a random split would place near-duplicates across sets. Compare the two timelines. In this forecasting example, a later training observation can give the model information that was unavailable at an earlier test time.
INTERACTIVE EXPLORATION
Choose a split for forecasting
The timeline contains eight observations. Compare a shuffled example with a chronological split when the goal is to predict the future.
Split for the real question
For future forecasting, a chronological split is often more realistic than mixing future and past examples at random. The right split depends on deployment: the test should represent the kind of data the model will actually face.
Check your understanding
Can we tune the learning rate repeatedly on the test set and still call it an untouched final test?
No. Those decisions use information from the test set; reserve separate data for the final evaluation.