Idempotency in infrastructure provisioning scripts

What's really being asked

Whether you can build provisioning automation that survives retries, network blips, and reruns without creating duplicate or corrupted resources.

The full answer

Idempotency means running the operation once or many times yields the same final state. If a network timeout makes you unsure whether a create succeeded, you can safely retry without spinning up a second instance. Technique one is declarative desired-state reconciliation: describe the target state, read the current state, compute the diff, and apply only what is missing. Tools like Terraform and Kubernetes controllers work this way. Technique two is idempotency keys: the client generates a unique token per logical operation and sends it with each request; the server records the token and returns the original result for duplicates instead of acting again. A simpler variant is read-before-write or an upsert that creates only if absent.

The mistakes people make

Saying idempotency just means retrying, claiming POST is inherently safe, or relying on naming alone without checking existing state, which races under concurrency.

What usually comes next

How long do you store idempotency keys? How do you handle partial failures mid-provision? What about concurrent runs racing on the same resource? How does this interact with eventual consistency in the cloud API?

A concrete example

A script provisions a virtual machine. With desired-state reconciliation, it lists existing VMs by a stable tag, finds none, and creates one; on rerun it finds the VM and does nothing. Alternatively, it sends the create call with an idempotency key derived from a request UUID; if a timeout triggers a retry, the cloud API recognizes the same key and returns the existing VM rather than launching a duplicate, so a flaky network never doubles your fleet or your bill.