Logistic Regression: Predicting a Yes or No Outcome

WHY IT EXISTS Many real-world questions have a binary answer: will a customer churn, is this email spam, will this loan default? Linear regression, which predicts continuous numbers, can't answer these questions because its output isn't bounded. Logistic regression was developed to model the probability of a 'yes' or 'no' outcome.

THE MENTAL MODEL Think of logistic regression as a two-step process. First, it creates a score by calculating a weighted sum of the input variables, just like linear regression. Second, it passes this score through a special 'squashing' function called the logistic function (or sigmoid). This function takes any real number and maps it to a value between 0 and 1, which can be interpreted as a probability.

HOW IT WORKS Technically, the model finds a linear relationship between your input variables and the 'log-odds' of the event occurring. The log-odds is a transformation of probability that isn't restricted to the 0-1 range. The model's core is a linear equation that predicts these log-odds. The logistic function then converts the log-odds prediction back into a familiar probability. The unit of measurement for the log-odds scale is called a 'logit'.

WHEN TO USE IT Use logistic regression for binary classification problems where you need a simple, interpretable model. It's a baseline for tasks like medical diagnosis (e.g., predicting the presence of a disease based on symptoms), fraud detection, and marketing analytics (e.g., predicting if a user will click an ad).

WHEN NOT TO USE IT Do not use it to predict a continuous quantity like a price or temperature; use linear regression instead. It also assumes a linear relationship between the input features and the log-odds of the outcome. If the underlying relationship is highly non-linear, more complex models may perform better.

ONE CANONICAL EXAMPLE Imagine predicting whether a student passes an exam (1) or fails (0) based on the number of hours they studied. The model takes 'hours studied' as an input variable. It learns a parameter (a weight) for this variable. For a new student who studied for 10 hours, the model calculates a score, then uses the logistic function to convert that score into a probability, such as 0.85. This means the model estimates an 85% probability of passing.