High ROC-AUC but low PR-AUC: what does this imply?
Tests if ROC-AUC hides imbalance while PR-AUC exposes it. Severe imbalance dilutes FPR across many negatives, inflating ROC-AUC, but precision crashes. Critical for rare positives with costly false positives. Praising the model on ROC-AUC alone fails.
What's really being asked
This question probes whether you understand how ROC curves and precision-recall curves behave differently under class imbalance. A senior candidate should know that ROC-AUC evaluates ranking quality across all thresholds using true positive rate and false positive rate, while PR-AUC tracks the trade-off between precision and recall. The gap between these two metrics is a classic signature of a heavily imbalanced dataset.
The full answer
First, state clearly that the dataset is severely imbalanced with a large majority of negative examples and very few positives. Second, explain that ROC-AUC remains high because false positive rate is calculated as false positives divided by all negatives; with millions of negatives, even thousands of false positives produce a tiny FPR, so the ROC curve stays close to the top-left corner. Third, explain that PR-AUC collapses because precision equals true positives divided by true positives plus false positives; when positives are rare, even a modest number of false positives destroys precision, dragging the precision-recall curve down. Fourth, connect this to business impact by noting this pattern is critical when the positive class is rare and false positives are expensive, such as fraud detection, anomaly detection, or rare disease screening, where acting on a positive prediction triggers costly investigations or treatments.
The mistakes people make
A major red flag is claiming the model is excellent based on ROC-AUC alone without mentioning the precision collapse. Another mistake is saying the model has high recall but low precision without explaining why the metrics diverge structurally. Some candidates blame threshold choice; while threshold selection matters for a single operating point, it does not explain why the area under the PR curve is low across all thresholds.
What usually comes next
The interviewer may ask how you would fix this, so be ready to discuss class weighting, oversampling techniques like SMOTE, or using focal loss. They might also ask which metric you would optimize for production; the correct answer is to choose based on business cost rather than defaulting to ROC-AUC. You could also be asked to sketch what the curves look like or to explain why accuracy would be misleading here.
A concrete example
Imagine a credit card fraud dataset with one million transactions and only one thousand fraudulent positives. Your model scores a ROC-AUC of 0.92 because it ranks most frauds above legitimate transactions, giving a low false positive rate. However, if the model produces five thousand false positives, precision drops to roughly one thousand divided by six thousand, or about seventeen percent, yielding a PR-AUC near 0.45. In production, this means investigators waste time on thousands of false alarms, making the system unusable despite the flashy ROC number.
Interview question
A fraud model scores ROC-AUC 0.92 and PR-AUC 0.45 on a dataset with one million negatives and one thousand positives. What best explains this divergence?
- a.ROC-AUC evaluates ranking quality across all thresholds, while PR-AUC only evaluates the default threshold, so their values naturally differ under imbalance.
- b.The decision threshold is set too aggressively, producing many false positives that crush precision at that operating point, even though ROC-AUC remains high because it averages across all thresholds.
- c.The massive negative majority shrinks the false positive rate, keeping ROC-AUC high, while the same false positives swamp the rare positives and devastate precision.Correct
- d.The baseline PR-AUC is inherently low in imbalanced data, so the gap reflects metric sensitivity to class distribution rather than a meaningful performance difference.
Why? this is the answer
ROC-AUC stays high because false positive rate divides by the huge number of negatives, making even thousands of false positives appear negligible; precision divides by predicted positives, so those same false positives dominate the rare positives and collapse PR-AUC. The threshold distractor is wrong because PR-AUC is an area across all thresholds, not a single operating point.
Just read this? Test yourself on what you have been reading.
Read the original → developers.google.com
- #classification metrics
- #class imbalance
- #roc-auc
- #pr-auc
- #model evaluation
You just looked this up. Could you explain it out loud?
That is the part interviews actually test. Tezvyn takes questions like this one and gives you what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles