Metrics
400 bites tagged Metrics — interview questions with model answers, and 60-second explainers.
Pull vs. Push: How Your Metrics Get to the Collector
Push vs. Pull metrics is about who starts the conversation. In a push model (like OTLP), the app sends metrics to a collector. In a pull model (like Prometheus), the collector scrapes metrics from the app. The footgun: pull can miss short-lived jobs.
The Four Golden Signals of Service Monitoring
The Four Golden Signals provide a simple framework for service health: Latency, Traffic, Errors, and Saturation. Use them for top-level dashboards and alerts. The biggest footgun is ignoring saturation, the leading indicator of future latency issues.
Service Level Indicators: Measuring What Matters
An SLI is a direct measurement of your service's performance, like request latency or error rate. It's the raw data that forms the basis for reliability goals (SLOs). The main footgun is measuring system internals instead of the actual user experience.
Inception Score: Judging AI Art for Quality and Variety
Inception Score judges an AI image generator on quality and variety, using a classifier to check if images are distinct and the overall set is diverse. Its main footgun is that it only measures what another AI can classify, not what a human finds appealing.
Fréchet Inception Distance (FID): Grading AI Art
FID grades AI-generated images by comparing their statistical "vibe" to real ones. It uses a pre-trained network (InceptionV3) to see if a batch of generated images has similar feature distributions to a real dataset. A lower score is better.
Vanity vs. Actionable Metrics: Measure What Matters
Actionable metrics explain *why* and guide your next move, while vanity metrics just look good on a chart. Use actionable metrics for setting KPIs to understand user behavior and business health.
Quantitative Growth Model: The Spreadsheet That Runs Your Business
A quantitative growth model is a spreadsheet that maps your business, showing how inputs like ad spend turn into revenue. It's used to forecast growth, simulate strategy changes, and set goals. The footgun: your model is only as good as its assumptions.
Expansion MRR: Growing Revenue from Existing Customers
Expansion MRR measures new monthly recurring revenue from your existing customers. It's how you grow through upgrades, add-ons, or cross-sells, not just new signups. The footgun is mixing it with new business MRR, which hides your product's retention power.
Guardrail Metrics: Don't Win the Battle to Lose the War
Guardrail metrics are your experiment's safety net, ensuring a win in one area doesn't cause a loss elsewhere. During A/B tests, you monitor them to catch negative side effects on site speed or revenue.
The K-Factor: Measuring Your Product's Viral Growth
The Viral Coefficient (K-factor) measures how many new users each existing user generates. It's key for products with referrals or sharing. The common mistake is chasing K > 1; even K=0.5 is valuable, as it effectively halves your acquisition cost.
AARRR: Pirate Metrics for the Customer Lifecycle
The AARRR framework maps your customer's journey through five key stages: Acquisition, Activation, Retention, Referral, and Revenue. It helps startups diagnose leaks in their growth funnel, from first visit to final sale.
The North Star Metric: Your Company's One True Focus
A North Star Metric (NSM) is your company's compass: a single metric capturing the core value you deliver to customers. Spotify uses 'time spent listening' to align teams on user engagement.
OKRs: Frame Growth Goals as Measurable Outcomes
OKRs separate your ambitious goal (Objective) from the measurable results that prove you're there (Key Results). Growth teams use this to align on what success looks like.
Average Revenue Per User (ARPU)
ARPU is the average revenue each user generates, a pulse check for a subscription business's health. It's used by media and SaaS companies to track if users are becoming more valuable, but it can be skewed by a few high-paying "whales."
Customer Health Score: A Predictive Churn Signal
A Customer Health Score is like a credit score for customer loyalty, predicting churn risk. SaaS companies use it to focus retention efforts on at-risk accounts before they cancel. The footgun is using vanity metrics like logins over true value signals.
Negative Churn: When Losing Customers Still Means Growth
Negative churn means existing customers upgrade faster than others leave, growing your revenue even if you lose logos. It's a key SaaS metric for variable pricing models. The footgun: you can have negative revenue churn while still losing many customers.
Customer Retention: Are Users Sticking Around?
Customer retention measures if users return to your product. It's often more valuable to keep an existing customer than acquire a new one. This is critical for any business relying on repeat engagement, from SaaS to e-commerce.
Cost Per Mille (CPM): The Price of 1,000 Ad Impressions
CPM is the price for 1,000 ad impressions, a common metric for brand awareness campaigns. It's like buying billboard space online. The footgun is confusing low cost with high value; CPM measures exposure, not engagement or conversion.
Novelty and Learning Effects in A/B Testing
The novelty effect is a temporary metrics lift from curious users exploring a new feature. The learning effect is the opposite: a dip as users struggle with a change. Both can mislead A/B tests if you don't run them long enough to see the true.
RARRA Framework: Retention Over Acquisition
The RARRA framework flips the classic AARRR funnel, prioritizing Retention over Acquisition. In crowded markets, it's cheaper to keep a user than to find a new one. Use it when acquisition costs are high.
Customer Lifetime Value (LTV): A Customer's Future Net Profit
LTV predicts the total net profit a customer will generate over their entire relationship. It's used to set marketing budgets and prioritize features for high-value users.
Churn Rate: How Fast Your Business is Leaking
Churn rate is your business's leak rate—the percentage of customers lost over a period. It's a vital health metric for subscription services like SaaS or streaming. The footgun?
User Activation: Engineering the 'Aha' Moment
User activation engineers the 'aha' moment when a new user first experiences your product's value. It's measured by a key action, like sending a message in Slack, that predicts retention.
The Customer Lifecycle: From Prospect to Advocate
The customer lifecycle maps the journey from first contact to loyal advocate. It's used to decide when to send a welcome email (acquisition), a loyalty discount (retention), or a 'we miss you' offer (win-back). The footgun is focusing only on acquisition.
Get Metrics bites daily.
Five a day, five minutes, offline. With quizzes so it sticks.
Open testing — you’ll join as an early tester.