Regression Discontinuity Design (RDD)
RDD finds a natural experiment at a cutoff point, like a test score threshold for a scholarship. By comparing people just above and below the score, you can estimate the program's effect. The footgun is assuming this local effect applies to everyone.
The mental model
Regression Discontinuity Design (RDD) is a clever way to find a cause-and-effect relationship when you can't run a true A/B test. It works by finding an arbitrary cutoff point and assuming the people just below it are nearly identical to the people just above it. The only real difference is that one group got a "treatment" and the other didn't. Any sharp jump in outcomes right at that cutoff is likely caused by the treatment itself.
How it works
You need two things: a continuous variable (like an exam score, age, or income) and a sharp cutoff for an intervention. For example, everyone with a score of 80 or above gets into a program. You then plot an outcome variable (like future salary) against the score. If you see a sudden jump, or "discontinuity," in the salary trend line right at the score of 80, that jump represents the estimated effect of the program for people near the cutoff.
When to use it
Use RDD when a randomized controlled trial is impossible, unethical, or impractical. Many real-world policies and programs, from financial aid to healthcare benefits, are already assigned based on a threshold. RDD allows you to analyze the causal impact of these existing programs using observational data, turning a policy rule into a quasi-experiment.
When not to use it
RDD fails if people can precisely manipulate their score to get just over the cutoff. If students can easily retake a test to barely pass, the groups on either side of the threshold are no longer comparable; one is now full of more determined individuals. Also, RDD only estimates the effect at the cutoff. It tells you nothing about the program's effect on people far from the threshold, like top performers or those who scored very low.
One canonical example
A company gives a performance bonus to all salespeople who exceed 100,000 in sales. To measure the bonus's effect on next year's sales, RDD compares salespeople who sold 101,000 with those who sold 99,000. These two groups are assumed to be very similar in skill and effort. If the 101k group shows a significantly higher sales increase the following year, that difference can be attributed to the motivational impact of receiving the bonus.
Interview question
What is a critical limitation regarding the generalizability of findings from a Regression Discontinuity Design (RDD)?
- a.RDD cannot establish a causal relationship, as it is based on observational data.
- b.It requires a perfectly continuous running variable, which is rarely found in real-world applications.
- c.The results of an RDD study are only valid for the specific population at the exact cutoff point.Correct
- d.RDD is rendered invalid if individuals can strategically influence their position relative to the cutoff.
Why? this is the answer
Option C is correct because the card explicitly states, "RDD only estimates the effect at the cutoff. It tells you nothing about the program's effect on people far from the threshold." Option D describes a condition that invalidates the RDD study itself (a threat to internal validity), rather than a limitation on the generalizability of its findings if it were valid.
Just read this? Test yourself on what you have been reading.
Read the original → en.wikipedia.org
- #analytics
- #statistics
- #causal inference
- #quasi-experiment
Put your scrolling time to good use
Learn one idea, try a quiz and save useful cards for revision. Tezvyn makes it easy to learn and stay current in your tech field, a few minutes at a time.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on analytics — each one lists the topics its interview covers.
See open roles