Project onto a direction
Imagine points clustered near a diagonal line. Project each point onto a chosen direction: its projection is a dot product with a unit vector. Different directions preserve different amounts of variation.
Choose the first component
Rotate the green direction. The dashed lines show each orange point projected onto it. When the direction lines up with the diagonal, the two projected points are farthest apart and their variance is greatest. This is the first PCA direction for this centered two-point example. Rotate the direction and watch each original point fall onto the line. The displayed variance describes the projected values, not their reconstruction error.
INTERACTIVE EXPLORATION
Project onto a direction
Two centered points are (−2, −2) and (2, 2). Rotate a unit direction and see the variance of their projections.
What is lost?
Dropping components loses information. PCA is linear and sensitive to feature scaling. The direction of greatest variance is not necessarily the best direction for a downstream label or prediction task.
Check your understanding
Does the highest-variance direction always predict a target best?
No. PCA does not use the target label; high variance may be unrelated to it.