How do you inject secrets from an external store at runtime?

What's really being asked

Whether you can design a secrets architecture that satisfies a no-secrets-in-etcd policy while still delivering credentials to workloads, and whether you understand workload identity.

The full answer

The core challenge is bootstrapping trust without a static credential. The pattern uses the Pod's projected ServiceAccount token as its identity. With Vault, the Kubernetes auth method validates that token against the cluster's TokenReview API and returns a short-lived Vault token scoped by policy. A Vault Agent Injector mutating webhook adds an init and sidecar container that authenticates, fetches secrets, and renders them to a shared in-memory tmpfs volume that the app container reads as files, with automatic renewal and re-rendering on rotation. An alternative is the Secrets Store CSI driver, which mounts secrets from Vault, AWS Secrets Manager, or Azure Key Vault directly as a volume; on AWS, IRSA maps the ServiceAccount to an IAM role so no static keys exist. Crucially, secrets live only in memory in the Pod, never persisted to etcd.

The mistakes people make

Using External Secrets Operator to sync values into native Kubernetes Secrets, which reintroduces plaintext to etcd and violates the policy. Embedding a long-lived Vault token or AWS access key in the manifest. Writing secrets to a hostPath or persistent disk.

What usually comes next

How does the ServiceAccount token get validated? How are secrets refreshed without restarting the Pod? Trade-offs between sidecar injection and the CSI driver? How do you audit access in Vault?

A concrete example

Annotate a Deployment with vault.hashicorp.com/agent-inject and a role; the injector adds a sidecar that logs into Vault using the Pod's ServiceAccount token, reads database creds, and writes them to /vault/secrets/db on tmpfs, which the app reads at startup and which is re-rendered when the lease renews.