Likelihood is a question about parameters
For a fixed observed outcome, likelihood asks how probable that outcome would be under each candidate parameter value. For two independent observed successes with predicted probability p, the likelihood is p². Independence is an assumption of this example, not a general property of data.
Why use a logarithm?
The log of a product equals the sum of the logs. Maximizing log likelihood gives the same optimum as maximizing a positive likelihood because log is increasing. In practice, adding log probabilities can also be numerically more stable than multiplying many tiny probabilities. Try a small success probability and then a large one. The likelihood of two successes changes as p²; negative log likelihood gives a larger penalty to unlikely observations.
INTERACTIVE EXPLORATION
Score an observed outcome
Two successes are independent in this toy model. Change p and compare likelihood with negative log likelihood.
Turning likelihood into loss
Negative log likelihood changes maximization into minimization. For one observed success it is −log(p): the closer p is to zero, the larger the penalty. This is one part of binary log loss; failures contribute −log(1 − p).
Check your understanding
If p = 0.5, what is the likelihood of two independent successes?
0.25. Their joint probability is 0.5 × 0.5.