How do you train and evaluate on imbalanced defect data?

What's really being asked

The interviewer wants both training tactics for extreme imbalance and, critically, the right metrics, since accuracy is meaningless when defects are rare.

The full answer

On the data and training side: oversample the minority defect class, for example with augmentation or synthetic samples, or undersample the majority; apply class weights so defects contribute more to the loss; use focal loss to focus learning on hard, rare examples; and consider reframing the problem as anomaly or one-class detection that models normal product and flags deviations, which suits cases where defects are diverse and scarce. On evaluation: never trust raw accuracy, because predicting always-good can score 99 percent while catching zero defects. Instead use precision and recall on the defect class, the precision-recall curve and its area, and an F-beta score where beta is chosen by the relative cost of misses versus false alarms; a confusion matrix makes the error types explicit. PR-AUC is preferred over ROC-AUC under heavy imbalance because it focuses on the rare positive class.

The mistakes people make

Reporting accuracy as the headline metric. Ignoring recall, so real defects slip through. Naive oversampling that leaks duplicates into validation, or evaluating on a rebalanced rather than real-distribution test set.

What usually comes next

Why PR-AUC beats ROC-AUC here, threshold selection by cost, SMOTE versus augmentation, and keeping the test set at the true class ratio.

A concrete example

If 1 in 1000 parts is defective, a model that labels everything good is 99.9 percent accurate yet useless. You instead weight the loss toward defects, augment defect images, and report that the system catches, say, most defects at an acceptable false-alarm rate, summarized by PR-AUC and a cost-tuned F-beta.