A positive pair
Two views of the same image can form a positive pair. A model is encouraged to make their representations similar. What counts as a good view depends on the task.
Compare with alternatives
A contrastive objective compares a positive similarity with alternatives. If the positive score rises while the other stays fixed, its softmax probability rises. A similarity score is not itself proof that two examples mean the same thing. Increase the positive-pair score while the alternative stays fixed. Its share of the two-score softmax grows; this example does not update an encoder.
INTERACTIVE EXPLORATION
Compare similarity scores
Change the positive score while the other score stays at zero.
Avoid a shortcut
Augmentations, negatives and data quality matter. The toy example uses two invented scores and does not train a representation.
Check your understanding
If both scores are equal, what is the softmax probability of the positive pair?
0.5.