Linear Regression: Finding the Line of Best Fit

WHY IT EXISTS Linear regression was developed to move beyond simply observing a correlation and to quantify the relationship between variables. It provides a simple, interpretable mathematical formula to make predictions about future outcomes based on past data, forming a foundational tool in statistics and machine learning.

THE MENTAL MODEL Think of a scatter plot of data points, like years of experience versus salary. Linear regression is the process of drawing a single straight line through those points that best represents the overall trend. This line, defined by the equation y = mx + b, becomes your predictive model. The 'm' is the slope (how much salary changes per year of experience) and 'b' is the intercept (the theoretical starting salary with zero experience).

HOW IT WORKS The model finds the optimal line by minimizing the sum of the squared vertical distances (called residuals) from each data point to the line itself. This common method is known as Ordinary Least Squares (OLS). When you have one explanatory variable, it's a simple linear regression. When you have two or more, it's a multiple linear regression, which fits a hyperplane instead of a line, but the core principle of minimizing error remains the same.

WHEN TO USE IT Use linear regression when you need to predict a continuous numerical value (the dependent variable) and you believe there's a linear relationship with one or more predictor variables. It is excellent for forecasting (e.g., future sales), understanding the strength of relationships (e.g., how much does rainfall affect crop yield?), and situations where model interpretability is critical.

WHEN NOT TO USE IT Do not use it for classification problems where the outcome is a category (e.g., 'spam' or 'not spam'). It performs poorly if the true relationship between variables is non-linear. The model is also sensitive to outliers, which can dramatically skew the line of best fit. Most importantly, never use a strong linear fit to claim one variable causes another; it only measures association.

ONE CANONICAL EXAMPLE Predicting a student's final exam score based on the number of hours they studied. Hours studied is the independent variable, and exam score is the dependent variable. A linear regression model might produce an equation like Score = 4.5 * Hours + 55. This means for every additional hour studied, the score is predicted to increase by 4.5 points, with a baseline score of 55 for zero hours of study. You could then use this model to predict the score for a student who studied for 8 hours.