I'm worried about security in our CI/CD process. We currently have API keys and DB passwords stored in the CI settings, but I’ve heard horror stories about these being leaked through build logs or compromised runner environments. What is the most secure way to inject production secrets into a running pipeline without exposing them to everyone who has access to the repo?
3 answers
The first rule is to never print secrets to the console. Most modern CI tools (GitHub Actions, CircleCI) will "mask" anything recognized as a secret, but it's not 100% foolproof. For enterprise security, you should use an external "Secret Vault" like AWS Secrets Manager or HashiCorp Vault. Instead of storing the actual password in the CI, you store a "Vault Token." During the build, the pipeline uses that token to fetch the real secrets directly into memory. They never touch the disk and are never written to logs. Also, use "Environment Scoping"—ensure that only the "Production" pipeline has access to the "Production" secrets, while developers only have access to "Staging" keys.
Do you recommend using "Branch Protection" rules to prevent people from editing the CI YAML to print secrets?
Rotate your secrets regularly! Even if a key is leaked, if it expires every 30 days, the window of opportunity for an attacker is significantly smaller.
Good point, Karen. Automating secret rotation via your cloud provider is one of the best "set it and forget it" security wins.
Absolutely, Larry. That is a very common attack vector. An attacker (or a disgruntled employee) could modify the .github/workflows/main.yml to include echo $PROD_PASSWORD | base64 to bypass the log masker. You must use branch protection on your main branch and require a code review for any change to the CI/CD configuration files. Additionally, use "Deployment Protection Rules" that require a different set of eyes to approve any job that accesses production-level secrets. Security in CI/CD is all about limiting the "Blast Radius" of a single compromised account.