Logistic Regression: Predicting a Yes or No Outcome
Logistic regression predicts a binary outcome (yes/no, 1/0) by calculating the probability of an event. It's used for classifying things like spam email or fraudulent transactions. The footgun is mistaking its output for certainty instead of a probability.
Why it exists
Many real-world questions have a binary answer: will a customer churn, is this email spam, will this loan default? Linear regression, which predicts continuous numbers, can't answer these questions because its output isn't bounded. Logistic regression was developed to model the probability of a 'yes' or 'no' outcome.
The mental model
Think of logistic regression as a two-step process. First, it creates a score by calculating a weighted sum of the input variables, just like linear regression. Second, it passes this score through a special 'squashing' function called the logistic function (or sigmoid). This function takes any real number and maps it to a value between 0 and 1, which is a model-estimated probability; calibration determines how closely it matches observed frequencies.
How it works
Technically, the model finds a linear relationship between your input variables and the 'log-odds' of the event occurring. The log-odds is a transformation of probability that isn't restricted to the 0-1 range. The model's core is a linear equation that predicts these log-odds. The logistic function then converts the log-odds prediction back into a familiar probability. The unit of measurement for the log-odds scale is called a 'logit'.
When to use it
Use logistic regression for binary classification problems where you need a simple, interpretable model. It's a baseline for tasks like medical diagnosis (e.g., predicting the presence of a disease based on symptoms), fraud detection, and marketing analytics (e.g., predicting if a user will click an ad).
When not to use it
Do not use it to predict a continuous quantity like a price or temperature; use linear regression instead. It also assumes a linear relationship between the input features and the log-odds of the outcome. If the underlying relationship is highly non-linear, more complex models may perform better.
One canonical example
Imagine predicting whether a student passes an exam (1) or fails (0) based on the number of hours they studied. The model takes 'hours studied' as an input variable. It learns a parameter (a weight) for this variable. For a new student who studied for 10 hours, the model calculates a score, then uses the logistic function to convert that score into a probability, such as 0.85. This means the model estimates an 85% probability of passing.
Interview question
What is the primary role of the logistic (sigmoid) function within a logistic regression model?
- a.To transform the linear combination of input features into a probability score between 0 and 1.Correct
- b.To directly output a binary classification (e.g., 0 or 1) for the event.
- c.To determine the optimal weights for the input variables during training.
- d.To ensure the model can predict continuous values, similar to linear regression.
Why? this is the answer
The logistic function takes the weighted sum of inputs and maps it to a value between 0 and 1, which represents the probability of the event occurring. It does not directly output a binary classification; that requires an additional thresholding step.
Just read this? Test yourself on what you have been reading.
Read the original → en.wikipedia.org
- #machine learning
- #statistics
- #classification
- #data science
Put your scrolling time to good use
Learn one idea, try a quiz and save useful cards for revision. Tezvyn makes it easy to learn and stay current in your tech field, a few minutes at a time.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on machine learning — each one lists the topics its interview covers.
See open roles