Threat Modeling for Reliability: Find Failures Before They Happen

Why it exists

To move reliability from a reactive, incident-response activity to a proactive, design-time discipline. Instead of waiting for systems to fail in production and then writing a postmortem, we can anticipate and mitigate many failures before they ever happen, improving availability and user trust.

The mental model

Think like a storm looking for a weak point. Security threat modeling looks for malicious actors; reliability threat modeling looks for any potential cause of failure, malicious or not. It's a structured process for asking "What are all the ways this system could fail or degrade?" and "What have we done about it?". You analyze the system from the perspective of its own fragility and dependencies.

How it works

The process is structured and repeatable. First, you model the system, creating a diagram of components, data flows, and dependencies. Second, you identify threats based on this model. For reliability, threats aren't hackers but failure modes like a database overload, a slow network link, a third-party API outage, or a cascading failure from a downstream service. Third, you determine responses for each threat. This could be adding a circuit breaker, implementing retries with exponential backoff, creating a fallback to a static cache, or simply improving monitoring and alerting.

When to use it

Use it early in the Software Development Lifecycle (SDLC), especially during the design phase of a new service or a major feature. It should also be a continuous process, revisited whenever the system's architecture, dependencies, or usage patterns change significantly. It's a key practice for any system where high availability is a core requirement.

When not to use it

It's likely overkill for non-critical internal tools or throwaway prototypes where downtime has minimal impact. The process requires engineering time, so it's best focused on production services where the cost of an outage is high. Don't let the process become bureaucratic and slow down development for low-stakes applications.

One canonical example

A team is building a new user profile service. During threat modeling, they diagram the service, which calls a database and an external avatar-rendering service. They identify a threat: what happens if the avatar service becomes slow or goes down? The user profile page would hang or fail to load. The mitigation they choose is to implement a 100ms timeout for the avatar service call. If it times out, they log the error and display a default placeholder avatar. This prevents a non-critical dependency from causing a full outage of the profile service.