How do you train and evaluate on imbalanced defect data?

WHAT THIS TESTS The interviewer wants both training tactics for extreme imbalance and, critically, the right metrics, since accuracy is meaningless when defects are rare.

A GOOD ANSWER COVERS On the data and training side: oversample the minority defect class, for example with augmentation or synthetic samples, or undersample the majority; apply class weights so defects contribute more to the loss; use focal loss to focus learning on hard, rare examples; and consider reframing the problem as anomaly or one-class detection that models normal product and flags deviations, which suits cases where defects are diverse and scarce. On evaluation: never trust raw accuracy, because predicting always-good can score 99 percent while catching zero defects. Instead use precision and recall on the defect class, the precision-recall curve and its area, and an F-beta score where beta is chosen by the relative cost of misses versus false alarms; a confusion matrix makes the error types explicit. PR-AUC is preferred over ROC-AUC under heavy imbalance because it focuses on the rare positive class.

COMMON WRONG ANSWERS Reporting accuracy as the headline metric. Ignoring recall, so real defects slip through. Naive oversampling that leaks duplicates into validation, or evaluating on a rebalanced rather than real-distribution test set.

LIKELY FOLLOW-UPS Why PR-AUC beats ROC-AUC here, threshold selection by cost, SMOTE versus augmentation, and keeping the test set at the true class ratio.

ONE CONCRETE EXAMPLE If 1 in 1000 parts is defective, a model that labels everything good is 99.9 percent accurate yet useless. You instead weight the loss toward defects, augment defect images, and report that the system catches, say, most defects at an acceptable false-alarm rate, summarized by PR-AUC and a cost-tuned F-beta.