Propensity Score Matching: Simulating a Randomized Trial

WHY IT EXISTS In a perfect world, we would use randomized controlled trials (RCTs) to measure an intervention's effect. But it's often unethical or impractical to randomly assign people to, for example, a specific medical treatment or social program. Observational data is what we have, but simple comparisons are biased because the groups (e.g., treated vs. untreated) often differ in many ways from the start. PSM was invented to try and correct for these pre-existing differences.

THE MENTAL MODEL Think of Propensity Score Matching as building a 'statistical doppelgänger' for every person who received a treatment. Instead of comparing a whole treated group to a whole untreated group, you find an untreated individual who was just as likely to have received the treatment as a treated individual, based on their observable characteristics. By comparing outcomes only between these matched pairs, you get a cleaner estimate of the treatment's actual effect, as if you had been able to randomize.

HOW IT WORKS The process has two main steps. First, you build a statistical model (like logistic regression) to predict the probability of receiving the treatment for every single person in your dataset, based on their pre-treatment characteristics (covariates like age, income, health status). This probability is the 'propensity score.' Second, for each treated person, you find one or more untreated people with a very similar propensity score. This creates a new, smaller dataset of matched pairs. All subsequent analysis of the treatment's effect is performed on this matched sample, where the treatment and control groups are now balanced on the observed covariates.

WHEN TO USE IT Use PSM when you have observational data and want to estimate the causal effect of a non-randomized treatment, intervention, or exposure. It is common in epidemiology, public policy analysis, and economics for questions like, 'What is the effect of a job training program on future earnings?' or 'Does a specific medication reduce hospital readmissions?'. It's a standard method when an RCT is not feasible.

WHEN NOT TO USE IT Do not use PSM if you can run a randomized trial—the trial is always the gold standard. Critically, PSM is inappropriate if there are strong, unmeasured confounding variables that influence both the treatment assignment and the outcome. If 'doctor's intuition' influences who gets a drug and you can't measure that intuition, PSM can't fix the bias. It also requires a good overlap in propensity scores between the treated and untreated groups; if the groups are too different to begin with, you won't find good matches.

ONE CANONICAL EXAMPLE A researcher wants to know if a voluntary new teaching method improves student test scores. Simply comparing students in the new program to those not in it is biased, as more motivated students might have self-selected into the program. Using PSM, the researcher first calculates each student's propensity score for joining the program based on prior grades, attendance, and demographics. Then, they match each 'treated' student with an 'untreated' student who had a nearly identical propensity score. By comparing test scores only within these matched pairs, they get a less biased estimate of the teaching method's true impact.