MLE versus MAP estimation and the role of priors

What's really being asked

This probes your fluency with the frequentist-versus-Bayesian framing of estimation and your ability to see regularization as a prior. It separates people who memorized formulas from those who understand how data volume shifts the balance.

The full answer

Maximum Likelihood Estimation chooses the parameter theta that maximizes the likelihood P(data given theta); it asks which parameter makes the observed data most probable and uses no prior belief. Maximum A Posteriori instead maximizes the posterior, which by Bayes' rule is proportional to P(data given theta) times the prior P(theta). The prior encodes belief before seeing data. Because MAP multiplies the likelihood by the prior, the prior acts as regularization, shrinking estimates toward its high-density region. A Gaussian prior on parameters corresponds to L2 regularization; a Laplace prior corresponds to L1.

The mistakes people make

Saying MAP is always more accurate. Forgetting that with a uniform prior MAP reduces exactly to MLE. Claiming the prior never matters, when in fact it can dominate under scarce data. Confusing MAP, a point estimate, with full Bayesian inference that returns a posterior distribution.

What usually comes next

Show that a Gaussian prior gives ridge regression. What happens to both estimates as sample size goes to infinity. How does MAP differ from computing the posterior mean.

A concrete example

Estimate a coin's bias from three flips, all heads. MLE gives probability 1.0 of heads, an overconfident estimate from tiny data. With a Beta(2,2) prior favoring fairness, MAP pulls the estimate toward 0.5, yielding something like 0.6, which is far more reasonable. After a thousand flips the data dominates and both methods nearly agree, showing the prior matters most when evidence is thin.