Cohort Analysis: Comparing User Groups Over Time

THE MENTAL MODEL: Stop analyzing all your users in one giant, misleading bucket. Cohort analysis groups users by a shared characteristic, most often their sign-up date, to see how their behavior evolves over time. It's like comparing the graduating class of 2024 to the class of 2023, rather than averaging the performance of all students from freshmen to seniors. Each group's journey is unique and should be judged against its peers.

HOW IT WORKS: The process involves two main steps. First, you define the cohort. The most common type is an acquisition cohort, where users are grouped by the week or month they were acquired (e.g., "January 2024 Sign-ups"). Other cohorts can be defined by a shared behavior, like making a first purchase. Second, you track a key metric for each cohort over its lifecycle. This is often visualized in a table where each row is a cohort and columns represent time elapsed (Day 1, Day 7, Day 30), showing how a metric like retention rate changes for each group.

WHEN TO USE IT: Cohort analysis is essential for understanding product health and user behavior over the long term. Three key use cases are: first, measuring user retention to see if your product is getting better or worse at keeping users engaged; second, analyzing the long-term impact of product changes; and third, understanding the customer lifecycle to pinpoint when and why users churn.

WHEN NOT TO USE IT: This method is less effective for real-time, up-to-the-second operational monitoring. It's also not ideal for products with very few users, as the cohorts will be too small to provide statistically significant insights. If you need a simple, aggregate snapshot of current activity (like "users online right now"), other metrics are more appropriate. Cohort analysis is for trend analysis, not instantaneous measurement.

ONE CANONICAL EXAMPLE: A SaaS company wants to know if a major onboarding redesign in March was successful. They create cohorts for users who signed up in January, February, March, and April. They then track the percentage of each cohort that converts to a paid plan within their first 30 days. If the March and April cohorts show a significantly higher conversion rate than the January and February cohorts, it's strong evidence the redesign worked. This isolates the change's impact, which would be hidden in a site-wide average conversion rate.