✳ BETWEEN QUESTIONS ML STUDY ATLAS

A QUESTION-LED LESSON

Logistic Regression: From a Linear Score to a Probability

A linear classifier can produce any real-valued score. Logistic regression applies a sigmoid to turn that score into a value between zero and one.

A score has no natural limit

A linear model produces a score z = wx + b. That score can be negative or larger than one, so it is not itself a probability. The sigmoid function maps any real score to a number between zero and one.

Separate the probability from the decision

Move the score below and watch the point travel along the sigmoid curve. Then move only the threshold: the dashed line moves, and the predicted class can change even though the model probability stays the same. At score zero, the sigmoid gives 0.5. First move the score; then leave the score alone and move the threshold. The curve gives the probability, while the threshold decides the label.

INTERACTIVE EXPLORATION

From score to a decision

Move the score and the threshold independently. A probability does not change when you change only the decision threshold.

What does the number mean?

Logistic regression is commonly trained with binary log loss. A sigmoid output is a probability-shaped number; whether it is well calibrated must be checked on suitable data. The threshold is a decision rule and can depend on the costs of different errors.

Check your understanding

With z = 0 and threshold 0.5, what class is predicted if ties count as positive?

The sigmoid is 0.5, so the positive class is predicted under this tie rule.

Keep exploring

Linear classification ↗Logarithms and likelihood ↗Loss functions ↗

Further reading

Source and further reading ↗