How would you estimate causal impact using a quasi-experimental method?

WHAT THIS TESTS: Your ability to identify and defend a quasi-experimental strategy when true randomization is unavailable. The interviewer wants to see if you understand that simple cross-sectional comparisons are biased and that you can articulate the identifying assumptions behind difference-in-differences, the standard tool for this scenario.

A GOOD ANSWER COVERS: First, name difference-in-differences as the primary method and set up the two-by-two structure: Canada is the treatment group, Australia is the control, and you compare the before-after change in Canada to the before-after change in Australia. Second, explicitly state the parallel trends assumption, which is that in the absence of the feature both countries would have experienced the same evolution in engagement metrics. Third, describe how to test this assumption using pre-treatment data, specifically by running an event study or plotting trends to verify no divergence before launch. Fourth, mention placebo tests such as assigning a fake launch date to Australia to confirm no spurious effect. Fifth, discuss robustness alternatives like synthetic control if parallel trends is questionable.

COMMON WRONG ANSWERS: A major red flag is proposing a simple post-launch comparison of means between Canada and Australia without any pre-period adjustment. Another weak answer is confusing correlation with causation by attributing all Canadian engagement changes to the feature while ignoring seasonality or macro trends. Some candidates mention A/B testing, which is operationally impossible here since the feature was launched nationally. Failing to name parallel trends or offering no way to validate it also signals shallow familiarity with causal inference.

LIKELY FOLLOW-UPS: The interviewer may ask what you would do if pre-trends diverge, in which case you should discuss synthetic control methods or finding a better control group. They might probe heterogeneous treatment effects, asking how the impact varies by user segment, which leads to interacting the DiD estimator with cohort indicators. Another follow-up is handling staggered rollout across multiple countries, which complicates the standard two-period DiD and requires recent advances in staggered difference-in-differences estimators.

ONE CONCRETE EXAMPLE: Suppose daily active user rate in Canada was ten percent before launch and twelve percent after, while Australia moved from nine percent to nine point five percent over the same period. The DiD estimate is the difference of differences, which is one point five percentage points. You would then run a placebo test by pretending the feature launched three months earlier and checking that the coefficient is near zero, and you would plot weekly engagement for both countries for six months pre-launch to visually confirm parallel trends.