High ROC-AUC but low PR-AUC: what does this imply?

WHAT THIS TESTS: This question probes whether you understand how ROC curves and precision-recall curves behave differently under class imbalance. A senior candidate should know that ROC-AUC evaluates ranking quality across all thresholds using true positive rate and false positive rate, while PR-AUC tracks the trade-off between precision and recall. The gap between these two metrics is a classic signature of a heavily imbalanced dataset.

A GOOD ANSWER COVERS: First, state clearly that the dataset is severely imbalanced with a large majority of negative examples and very few positives. Second, explain that ROC-AUC remains high because false positive rate is calculated as false positives divided by all negatives; with millions of negatives, even thousands of false positives produce a tiny FPR, so the ROC curve stays close to the top-left corner. Third, explain that PR-AUC collapses because precision equals true positives divided by true positives plus false positives; when positives are rare, even a modest number of false positives destroys precision, dragging the precision-recall curve down. Fourth, connect this to business impact by noting this pattern is critical when the positive class is rare and false positives are expensive, such as fraud detection, anomaly detection, or rare disease screening, where acting on a positive prediction triggers costly investigations or treatments.

COMMON WRONG ANSWERS: A major red flag is claiming the model is excellent based on ROC-AUC alone without mentioning the precision collapse. Another mistake is saying the model has high recall but low precision without explaining why the metrics diverge structurally. Some candidates blame threshold choice; while threshold selection matters for a single operating point, it does not explain why the area under the PR curve is low across all thresholds.

LIKELY FOLLOW-UPS: The interviewer may ask how you would fix this, so be ready to discuss class weighting, oversampling techniques like SMOTE, or using focal loss. They might also ask which metric you would optimize for production; the correct answer is to choose based on business cost rather than defaulting to ROC-AUC. You could also be asked to sketch what the curves look like or to explain why accuracy would be misleading here.

ONE CONCRETE EXAMPLE: Imagine a credit card fraud dataset with one million transactions and only one thousand fraudulent positives. Your model scores a ROC-AUC of 0.92 because it ranks most frauds above legitimate transactions, giving a low false positive rate. However, if the model produces five thousand false positives, precision drops to roughly one thousand divided by six thousand, or about seventeen percent, yielding a PR-AUC near 0.45. In production, this means investigators waste time on thousands of false alarms, making the system unusable despite the flashy ROC number.

Source: developers.google.com