How would you handle 10% nulls in a key column?
This tests your ability to diagnose data issues before solving. First, investigate why data is null. Then, discuss trade-offs of dropping vs. imputing with the mean or median, considering the impact on the dashboard's accuracy.
WHAT THIS TESTS: This question isn't about knowing a specific library function. It tests your data curiosity and analytical rigor. The interviewer wants to see if you diagnose before you prescribe. Can you move from a technical problem (nulls) to the business impact (a misleading dashboard)? They are evaluating your ability to consider the context, ask clarifying questions, and articulate the trade-offs of your chosen statistical method. For a senior role, simply stating "I'd fill with the mean" is a significant red flag.
A GOOD ANSWER COVERS: A strong answer has a clear, structured approach. First, investigate the source: are the nulls random, or are they correlated with another feature? Is it a data collection bug? Second, discuss the primary options and their trade-offs. Dropping the rows is simple but loses 10% of your data, which is significant. Imputing with a constant (like 0) is only valid if null truly means zero. Imputing with the mean is common but will reduce the column's variance and is sensitive to outliers. Imputing with the median is generally safer as it's robust to outliers. Third, state your recommendation based on a hypothetical assumption (e.g., "Assuming the data has outliers, I'd start with the median"). Finally, mention the importance of documenting the imputation and communicating it to the dashboard users.
COMMON WRONG ANSWERS: The most common mistake is jumping to a single solution without any investigation. For example: "I'd just use fillna with the column's mean." This shows a lack of curiosity and an inability to consider second-order effects. Another red flag is suggesting to drop 10% of the data without acknowledging the significant information loss. A weaker answer might list the methods (mean, median) but fail to explain why one would be better than another in the context of a dashboard for a numerical column. They miss the "pros and cons" part of the prompt.
LIKELY FOLLOW-UPS: "How would your answer change if the column was categorical instead of numerical?" (Impute with mode/most frequent value, or a new category like 'Unknown'). "What if the nulls weren't random, but correlated with another feature?" (This points towards multivariate imputation, where you use other columns to predict the missing value, like with scikit-learn's IterativeImputer). "At what percentage of nulls would you consider dropping the column entirely?" (No single right answer, but tests judgment. Above 40-50% is often a good starting point for discussion).
ONE CONCRETE EXAMPLE: Imagine a 'user_age' column where 10% of values are null. If we impute with the mean age of 35, we artificially shrink the variance of our age distribution. This could make a dashboard showing "Age Distribution of Users" misleading, as it would over-represent the 35-year-old bracket. If there were a few users with age > 90, the mean would be skewed high. Using the median age, say 31, would be more robust to these outliers and give a more representative picture of the central tendency, which is often a safer choice for visualization.
Read the original → scikit-learn.org
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.