Serverless Cold Starts: The Price of 'Pay-per-Use'

Why it exists

Serverless platforms promise 'pay for what you use' by not running your code 24/7. To save costs, they automatically shut down function instances that are idle. A cold start is the necessary consequence of bringing an instance back online from a 'cold' (non-running) state to serve a new request.

The mental model

Think of a serverless function like a food truck. A 'warm' function is a truck that's parked with the engine running and grill hot, ready to serve immediately. A 'cold' function is a truck parked in a garage. When an order comes in, the driver has to travel to the location, start the engine, and heat the grill before they can cook. That initial setup time is the cold start.

How it works

When a request triggers a cold function, the cloud platform performs several steps: it allocates a server, downloads your function code package, creates a new container environment, starts the language runtime (e.g., Node.js, Python), and finally runs your function's initialization code. Only then does it process the actual request. Subsequent requests to the same, now 'warm', instance skip these steps, resulting in much lower latency.

When to use it

The concept applies to any serverless function, but you need to actively manage it for latency-sensitive workloads. This includes API backends for web or mobile apps, real-time data processing pipelines, or any system where a user is waiting for an immediate response. These are situations where a multi-second delay is unacceptable.

When not to use it

You can often ignore cold starts for asynchronous, background tasks. Examples include batch processing jobs, generating nightly reports, or processing uploads to a storage bucket after the fact. In these cases, a startup delay of a few seconds is usually acceptable and doesn't impact the user experience.

One canonical example

An e-commerce site uses a serverless function to handle checkout payments. If no one has checked out for 15 minutes, the function instance is shut down. The next customer who clicks 'Pay' experiences a 3-second delay while the function cold starts, creating a poor user experience. To fix this, the developer might use 'provisioned concurrency' to keep one instance always warm, or switch to a plan that guarantees warm instances, like Azure's Premium plan.