How can EDA and visualization identify dataset bias before modeling?

WHAT THIS TESTS: This question tests whether you can operationalize statistical bias theory into a reproducible exploratory data analysis workflow. Interviewers want to see that you distinguish sampling bias, which is a systematic distortion between sample and population, from random noise or model error. They also want evidence that you validate data before training rather than treating bias as a post-modeling fairness problem.

A GOOD ANSWER COVERS: First, benchmark your sample against the population. Use cumulative distribution functions, QQ plots, or histogram overlays to compare age, income, or geographic distributions against census or known market data. Second, audit feature coverage and correlation structure across subgroups. Use heatmaps or mosaic plots to check whether certain demographics have sparsely populated features or different correlation patterns that signal undercoverage. Third, analyze missingness patterns. Plot missingness by subgroup and data collection source to detect nonresponse bias or attrition, since subjects with specific characteristics may be less likely to participate or complete surveys. Fourth, validate temporal, geographic, or source splits. If data is collected over time or from multiple vendors, visualize participation rates and feature drift across these dimensions to catch self-selection or convenience sampling artifacts.

COMMON WRONG ANSWERS: Treating class imbalance as synonymous with sampling bias. Class imbalance is a label distribution issue; sampling bias is a systematic misalignment between your sample and the target population. Another red flag is suggesting bias detection only after model training using fairness metrics like equalized odds. This misses the point because biased data propagates into features before any model is fit. A third error is ignoring the sampling frame entirely, such as assuming an online survey represents an entire city without comparing respondents to census demographics.

LIKELY FOLLOW-UPS: How would you weight or resample to correct for a known underrepresented subgroup? What would you do if population benchmarks do not exist? How do you distinguish representation bias from label noise or feature measurement error? When is it acceptable to intentionally oversample a subgroup, and how does that change your analysis?

ONE CONCRETE EXAMPLE: Suppose you are building a healthcare model using patient survey data collected at clinics. You plot age and income distributions against regional census data and notice that respondents skew ten years older and thirty percent higher income than the general population. A heatmap of missingness by clinic location reveals that rural clinics have forty percent more missing chronic condition fields than urban clinics, suggesting nonresponse or ascertainment bias. A mosaic plot of insurance type by clinic shows that uninsured patients are underrepresented by a factor of three relative to the county population. Before modeling, you flag that the sample frame excludes homebound patients and weight rural responses upward, or you restrict model generalization to populations resembling the biased sample.

Source: Statistics By Jim