Start with one prediction
Let x = 2, target y = 4 and weight w = 1. The model predicts ŷ = wx = 2. Squared loss is (ŷ − y)² = 4. Our question is: if we change w a little, which way will the loss move?
Follow the effect backward
The loss changes with the prediction at a rate dL/dŷ = 2(ŷ − y) = −4. The prediction changes with the weight at dŷ/dw = x = 2. Multiply the two local sensitivities: dL/dw = −4 × 2 = −8. Move w below and watch both the forward values and backward gradient change. Start at the forward loss, then trace the derivatives back to the weight. Change the weight until the prediction meets the target and inspect the gradient.
INTERACTIVE EXPLORATION
Trace the gradient backward
Adjust the weight for ŷ = wx and L = (ŷ − y)². Follow the local derivatives from the loss back to the weight.
A gradient is not an update
Backpropagation calculates derivatives. An optimizer uses them to choose an update. With learning rate 0.1, a gradient-descent step from w = 1 gives w = 1.8. Large networks reuse intermediate calculations and add contributions when a value affects the loss along several paths.
Check your understanding
In the example, what is the gradient dL/dw?
−8. The prediction is 2, the error is −2, and 2 × (−2) × 2 = −8.