How do you manage secrets for hundreds of services in centralized CI/CD?

WHAT THIS TESTS: This question evaluates whether you understand that secret management at scale is fundamentally an identity and access problem rather than a simple encryption exercise. The interviewer wants to see that you consider the full lifecycle of credentials from creation and distribution to rotation and revocation, and that you actively prevent secrets from leaking into build artifacts, logs, or source control.

A GOOD ANSWER COVERS: First, adopt a dedicated secrets manager such as HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, or Google Secret Manager instead of building a custom solution. Second, enforce role-based access control so that each CI/CD pipeline or service identity can fetch only the specific secrets it needs at deployment time. Third, prefer short-lived dynamic credentials over static long-lived passwords, and inject them at runtime through sidecars, init containers, or ephemeral environment variables that are never persisted to disk or build logs. Fourth, require comprehensive audit logging for every read and write operation so you can detect anomalous access patterns. Fifth, implement automatic rotation and versioning so that a compromise of one credential does not trigger a global fire drill.

COMMON WRONG ANSWERS: Storing secrets in Git repositories is a critical red flag because history is forever and deletion does not remove them from forks or clones. Another red flag is using a single shared master key or admin token for all services, which creates a blast radius that spans the entire fleet. Passing secrets through standard environment variables on shared CI runners is also dangerous because those variables can be scraped by other jobs or printed in logs. Finally, suggesting manual rotation or spreadsheet tracking shows you have not operated at scale and lack automation discipline.

LIKELY FOLLOW-UPS: The interviewer may ask how you would handle secret rotation without downtime, how you would recover if the secrets manager itself were compromised, or how you would manage secrets across multiple cloud providers. They might also drill into how you authenticate the CI/CD system to the secrets manager in the first place, for example through workload identity, OIDC, or mutual TLS rather than long-lived tokens stored on the runner.

ONE CONCRETE EXAMPLE: Imagine a microservice deployed by GitHub Actions to a Kubernetes cluster. Instead of placing a database password in the repository, you configure the Actions workflow to authenticate to HashiCorp Vault using OIDC. Vault issues a short-lived database credential valid for one hour. The credential is injected into the pod through a CSI secrets store driver at startup. If the pod restarts, it receives a fresh credential. Audit logs show exactly which workflow run requested which credential, and the database password itself was never visible to the CI runner or stored in the container image.