Configuration Hydration: From Template to Manifest

Why it exists

To solve the "invisible change" problem in GitOps. When you only change a variable in a Helm values.yaml file, the pull request diff is tiny, but the resulting change to the running system could be massive. Hydration makes the full impact visible before deployment.

The mental model

Think of it as a "render-and-commit" step in your CI pipeline. Instead of letting a deployment tool like Argo CD render templates just-in-time, you do it ahead of time. The source of truth for the cluster then becomes the fully rendered manifests in Git, not the abstract templates.

How it works

A CI pipeline triggers on a change to source templates (e.g., a Helm chart or its values). The pipeline runs a tool like helm template or kustomize build to generate the raw Kubernetes YAML. It then commits these generated YAML files to a separate branch or repository. The GitOps tool (like Argo CD) is configured to watch this repository of rendered manifests, not the original templates. The final deployment is based on these static, fully-rendered files.

When to use it

Use it when you need strict auditability and review for all configuration changes. It's ideal for production environments or regulated industries where every change must be explicitly reviewed as raw YAML. It also helps when debugging complex template interactions, as you can see the final output directly.

When not to use it

Avoid it for simple projects or fast-moving development environments where the overhead isn't justified. It adds complexity to the CI/CD pipeline and can slow down the development loop. If your team is small and comfortable reviewing template changes directly, hydration might be overkill.

One canonical example

A team manages a complex application with a Helm chart. Instead of pointing Argo CD at the chart repository, they set up a GitHub Action. When a developer changes values.yaml in a pull request, the Action runs helm template and commits the resulting deployment.yaml and service.yaml files to a manifests/production directory in the same repo. The PR reviewer can now see the exact diff of the generated YAML, ensuring no unexpected side effects before merging.