One input
For f(w) = (w − 3)², the derivative f′(w) = 2(w − 3). At w = 4 the slope is 2: a small positive change in w increases f approximately twice as much. This is a local approximation, not a promise about a large step.
Several inputs
A loss may depend on many weights. A partial derivative changes one parameter while holding the others fixed. The gradient collects these partial derivatives into a vector pointing toward the direction of steepest local increase under the usual Euclidean geometry. Move from one side of the minimum to the other. The tangent changes from downward to upward; its slope is zero at the bottom.
INTERACTIVE EXPLORATION
Move along the curve
Move w along L(w) = (w − 3)². The orange line is the tangent: its slope describes the local rate of change.
Why learning moves downhill
For a differentiable loss, a small step in the negative-gradient direction reduces loss when the gradient is nonzero. A finite step can overshoot, and a zero gradient does not by itself guarantee a global minimum.
Check your understanding
At w = 4, which way should we move to reduce (w − 3)²?
Left: the derivative is positive, so a small negative step reduces the loss.