Risks of optimizing recommendations only for CTR

WHAT THIS TESTS This checks whether you understand Goodhart's law applied to recommender systems: when a proxy metric becomes the sole target, it gets gamed and stops reflecting the real goal, which is sustained user value.

A GOOD ANSWER COVERS Click-through rate is only a proxy for value, so optimizing it alone produces predictable harms. The system learns to surface clickbait and sensational or outrage-driven content that earns clicks but disappoints after the click, eroding trust. Short-term clicks can rise while satisfaction and long-term engagement fall, because users feel manipulated and eventually use the product less or churn. Recommendation diversity can collapse into a narrow, repetitive feed, and the model may favor low-quality but catchy content over substantive items. To guard against this, pair CTR with counter-metrics that capture post-click value and the long horizon: dwell time or content completion rate to confirm clicks lead to real consumption, explicit satisfaction signals such as ratings, thumbs, or surveys, long-term retention and session quality measured over weeks not minutes, content diversity and novelty metrics, and negative signals like complaints, hides, unfollows, or rapid bounce-backs. Frame the objective as a balanced scorecard rather than a single number, and use long-window holdout experiments to see whether CTR gains actually help retention.

COMMON WRONG ANSWERS Treating CTR as a clean, complete measure of value. Listing only more engagement metrics that are themselves gameable. Ignoring the long-term and trust dimensions. Proposing no concrete counter-metrics.

LIKELY FOLLOW-UPS How would you weight these metrics into a single objective? How do you detect clickbait specifically? Why do short-term and long-term metrics sometimes conflict?

ONE CONCRETE EXAMPLE An A/B test of a new CTR-maximizing ranker shows clicks up eight percent, looking like a win. But the counter-metrics tell a darker story: average dwell time per click drops, satisfaction surveys decline, and four-week retention is down two points, revealing the model learned to push clickbait. The team rejects the launch and adds dwell-weighted CTR plus a diversity constraint to the objective.