MLE versus MAP estimation and the role of priors

WHAT THIS TESTS This probes your fluency with the frequentist-versus-Bayesian framing of estimation and your ability to see regularization as a prior. It separates people who memorized formulas from those who understand how data volume shifts the balance.

A GOOD ANSWER COVERS Maximum Likelihood Estimation chooses the parameter theta that maximizes the likelihood P(data given theta); it asks which parameter makes the observed data most probable and uses no prior belief. Maximum A Posteriori instead maximizes the posterior, which by Bayes' rule is proportional to P(data given theta) times the prior P(theta). The prior encodes belief before seeing data. Because MAP multiplies the likelihood by the prior, the prior acts as regularization, shrinking estimates toward its high-density region. A Gaussian prior on parameters corresponds to L2 regularization; a Laplace prior corresponds to L1.

COMMON WRONG ANSWERS Saying MAP is always more accurate. Forgetting that with a uniform prior MAP reduces exactly to MLE. Claiming the prior never matters, when in fact it can dominate under scarce data. Confusing MAP, a point estimate, with full Bayesian inference that returns a posterior distribution.

LIKELY FOLLOW-UPS Show that a Gaussian prior gives ridge regression. What happens to both estimates as sample size goes to infinity. How does MAP differ from computing the posterior mean.

ONE CONCRETE EXAMPLE Estimate a coin's bias from three flips, all heads. MLE gives probability 1.0 of heads, an overconfident estimate from tiny data. With a Beta(2,2) prior favoring fairness, MAP pulls the estimate toward 0.5, yielding something like 0.6, which is far more reasonable. After a thousand flips the data dominates and both methods nearly agree, showing the prior matters most when evidence is thin.