A boundary with breathing room
Imagine two points on a number line, one from each class. Any boundary between them separates them, but the midpoint maximizes the distance to the nearest point. This is a simple picture of a margin.
Which points matter?
Move the boundary between the two points. Watch the distance to the nearer point: it is largest at the midpoint. In a higher-dimensional classifier the boundary is a hyperplane, and nearby training examples constrain its margin. Move the boundary between the two fixed points. The smaller of the two distances is widest at the midpoint in this one-dimensional example.
INTERACTIVE EXPLORATION
Find the widest gap
Class A is at 1 and class B is at 5. Move the boundary between them and compare its distance to the nearest point.
What if the data overlaps?
A soft-margin SVM allows violations in exchange for a wider margin. Kernels can express nonlinear decision boundaries. The one-dimensional experiment only shows a hard-margin case.
Check your understanding
For points at 1 and 5, where is the widest-margin boundary?
At 3: each point is two units from the boundary.