Review of Reviews (Postmortem Metrics)

WHY IT EXISTS Many teams write postmortems diligently but never verify they produce change. Action items rot, the same outage recurs, and the document becomes ceremony. A review of reviews exists to measure the postmortem process itself, closing the gap between writing lessons and applying them.

THE MENTAL MODEL Think of postmortems as a manufacturing line for reliability improvements. Individual postmortems are the output; the review of reviews is quality control on the line. You sample the output, measure defects, and adjust the process when quality drops.

HOW IT WORKS On a regular cadence, often quarterly, a group samples recent postmortems and scores them against a rubric. Common metrics include action-item completion rate, median time to close action items, percentage of repeat incidents, depth of root-cause analysis, and whether the writing stayed blameless. Trends matter more than single numbers. Findings feed back as coaching, template changes, or staffing for unaddressed action items.

WHEN IT MATTERS It matters most at scale, where dozens of postmortems per quarter make individual oversight impossible, and in cultures where postmortems risk becoming compliance theater. It also matters when the same class of incident keeps recurring, signaling that fixes are superficial.

ONE CONCRETE EXAMPLE A platform team notices three database-saturation outages in two quarters. The review of reviews surfaces that each postmortem listed add capacity as the action item, none of which was prioritized, and none addressed the missing query-rate limiting that caused saturation. The meta-review escalates the systemic fix, assigns an owner with a deadline, and updates the template to require a why-will-this-not-recur section. The next quarter shows zero repeat saturation events, validating that the process correction worked.