I have different environments, and passing variables into my Terraform code is getting messy. I have been using tfvars files, but it feels like there should be a better way to handle environment-specific configurations.
What are you using to manage environment secrets and configs? Should I be using a central configuration store like GCP Secret Manager or a dedicated tool like HashiCorp Vault?
The most effective method for managing environment-specific variables and secrets in GCP is to store sensitive data in GCP Secret Manager and utilize dynamic runtime injection to decouple infrastructure code from configuration.
6 answers
Using tfvars for secrets is a security disaster waiting to happen because it inevitably leads to plain-text credential sprawl in your version control history.
- Use GCP Secret Manager for all sensitive variables to ensure proper IAM-based access control.
- Store non-sensitive environment configuration in a structured format like JSON or YAML within a centralized repo.
- Inject these values at runtime using a provider data source rather than hard-coding them into your pipeline definitions.
Cory, I’ve tried explaining this to my team a dozen times. Your points are entirely correct, though it remains a constant struggle to get everyone to follow these basic security practices.
Sorry to bother you with this, Cory, but I'm honestly just trying to keep my head above water. Your breakdown on using provider data sources is helpful; I will try that today.
You should adopt a hierarchy where non-sensitive configuration is held in YAML files organized by environment and secrets are fetched dynamically from Google Secret Manager.
- Define environment-specific blocks in separate directory structures
- Use the gcp_secret_manager_secret_version data source to inject secrets at runtime
- Leverage Terraform workspace tagging to avoid cross-environment contamination
Nolan Vargas, I am worried about this hierarchy. What if the workspace tagging fails or someone misconfigures the runtime injection? It sounds risky to rely on those specific data sources.
Stop relying on static tfvars files for sensitive environment configurations if you want to maintain a scalable infrastructure lifecycle. The most robust approach for GCP is to utilize Terraform workspaces or multiple directory structures combined with a centralized secrets manager for sensitive values and an external configuration store like Google Cloud Secret Manager or HashiCorp Vault for non-sensitive parameters.
I remember struggling with this exact sprawl during a migration project where every single environment had a dozen unique files that eventually drifted from the source of truth. We solved it by externalizing the configuration into a key-value store, which allowed us to decouple our infrastructure code from the environment-specific data, effectively eliminating the messy variable injection issues that plagued our earlier deployments.
I am so sorry to ask, Aishwarya, but does moving toward workspaces make the state management feel more fragile? I’m quite nervous about breaking our current production deployment by changing the structure.
Aishwarya, help! Our tfvars are an absolute disaster. If I move to a key-value store, how do I sync existing secrets without taking down the production environment right now?
Aishwarya, your suggestion to externalize configurations into a key-value store is clear. I will implement this to stop the environment drift we are seeing in our current Terraform lifecycle.
Managing environment-specific configurations is best achieved by moving away from raw tfvars and adopting a hierarchical approach using Terraform Terragrunt or a structured environment-based directory layout that leverages GCP Secret Manager for sensitive data.
I remember when I helped a client transition from a sprawling mess of tfvars files to this architecture, which drastically reduced our manual configuration drift.
We ultimately defined core variables in base modules and then only injected the minimal necessary overrides per environment, keeping the codebase clean and audit-ready.
Terraform variable files are not a security control, so stop trying to force them to be one. You have to evaluate the trade-offs between static configuration and dynamic secrets management: static variables work fine for non-sensitive environment tagging but create maintenance overhead at scale, while dynamic secret injection via Secret Manager is vastly superior when you need strict audit logging and fine-grained IAM access control.
If you have compliance requirements, you must move away from flat files and utilize a centralized store to ensure that access to production secrets is logged and restricted.
Agreed, Anna. Keeping secrets out of files is just more efficient. It saves me so much time on compliance reviews even if my current setup is still a total disaster.
Anna Knight, you are a lifesaver! I have been drowning in tfvars and my audit logs are a mess. This switch to Secret Manager is exactly the fix I need right now.
Stop overcomplicating it with HashiCorp Vault unless you are already running it elsewhere, because Secret Manager is the native, lower-friction choice for GCP environments.
Variable injection via tfvars is a security nightmare that inevitably leads to secrets leaking into your source control, which is why I advise everyone to scrub their repositories immediately. Keep your architecture simple, use environment-specific buckets for config, and lean on IAM-bound secrets to keep your production environment from becoming an incident waiting to happen.
Shruti, I’m drowning in documentation here. If I switch to Secret Manager now, is it actually easier to set up, or am I just going to break everything else in my GCP setup?
Sorry to bother you, Shruti Uchil, but scrubbing the repositories sounds like a huge task. I am a bit overwhelmed by the prospect, but I agree that secrets definitely do not belong there.
I apologize for asking again, Shruti Uchil, but is it really that much simpler? I worry about messing up the IAM bindings, but I suppose that is better than a massive leak.
I worry about Secret Manager just being another point of failure. Is there really no way to keep these secrets locally without them ending up in the repo? It feels too complex.