Exposed Credentials Across the DevSecOps Pipeline: 5 Places Secrets Hide in Plain Sight

Jan 17, 2023
7 minutes
... views

Developers leverage hardcoded credentials to seamlessly access or authenticate the services needed to build and deploy applications. While this practice streamlines development, it also introduces risk.

When credentials — such as passwords, API keys and access tokens — in source code are publicly exposed, bad actors can leverage them to gain unauthorized access and instigate attacks involving data leaks, code tampering, stolen sensitive information, or service disruptions. Avoiding secrets exposure is critical, but is not always easy.

Secrets can hide anywhere — in infrastructure as code (IaC) and application code files, repo config files, delivery pipeline config files and more — and if those files become public, for whatever reason, those secrets can fall into the wrong hands. And even if you‘re confident that your hardcoded credentials are safe in a private repo or behind a firewall, mistakes happen. Files can still become public.

Eliminating hardcoded credentials and adopting best practices, such as employing secrets vaults, is the best way to adopt proactive secrets security and avoid the risks of credentials exposure.

Let’s dive into the five places secrets hide in plain sight. Along the way, we’ll cover some best practices to help you implement a comprehensive secrets management strategy and get ahead of your exposed secrets risk.

1. IaC and Application Code Files

Even though it’s a best practice never to hardcode credentials in source code, secrets can still sneak into your environment during development. Accidents happen, and developers sometimes temporarily hardcode a credential — with the intention of removing it later — but then forget. The secret becomes stored within the source code in the version control system (VCS) and then falls off the radar.

Secrets can also sneak in via IaC templates and container images, which is why any complete secrets management strategy must leverage a secrets scanning solution that scans and surfaces exposed credentials in all file types.

But in reality, not all secrets scanners offer the depth and breadth you need to secure your secrets. Many one-dimensional scanning tools won’t scan all file types and don’t surface all exposed credentials because they only leverage regular expression and keyword scanning.

Regular expression scanning looks for specific patterns related to one credential type, such as AWS access keys, whereas keyword scanning looks for phrases that are likely to contain secrets, such as “password.” While these scanning techniques are valuable, they’re likely to miss secrets or introduce noisy alerts if they aren’t paired with the more advanced scanning techniques, such as entropy-based scanning, that multidimensional secrets scanners employ.

2. Version Control Systems

When your team hardcodes credentials in source code and those secrets don’t get picked up by your secrets scanner, your secrets can make their way into your VCS. Public repositories present the more obvious risk, as bad actors pay special attention to public repos to find exposed credentials.

But even files stored in private repos can be downloaded by those with access or can be copied into public repos, like in the Toyota data breach last year. Weak VCS access control can also leave you vulnerable. That’s why it’s important to adopt some best practices to proactively remove and secure any credentials that may exist in your VCS.

To start, you’ll want to ensure you’re scanning your IaC and application code as early in the development lifecycle as possible. Early scanning will enable developers to get notified about secrets they may have missed or forgotten to remove. This then allows them to take the steps needed to move the secret out of the source code and into a secrets manager, or simply have it ignored in the commit. That way, you can minimize the likelihood that hardcoded credentials are uploaded to your VCS.

You’ll also want to set up guardrails within your VCS so that when a new file or new code is committed, you can immediately identify all hardcoded secrets. Additionally, it’s a best practice to leverage environment variables or a secrets manager to ensure that any secrets you must use are protected against bad actors.

3. Delivery Pipelines

Secrets can hide in two places in delivery pipelines — in both the code that’s about to be deployed and in the delivery pipelines themselves — within CI/CD configuration files. Scanning your code as early in the development lifecycle as possible is always a best practice, but it’s important to scan for exposed credentials as a part of your CI/CD builds as well. And if you’re using IaC and CI/CD pipelines to pull in container images — which will run as application components or testing environments — it’s a best practice to employ a secrets security solution that identifies all images pulled from IaC and CI/CD files so you can then scan them for exposed credentials.

To get complete protection, you’ll also need to identify and remediate exposed credentials in the pipelines themselves. By leveraging a multidimensional secrets scanner that supports CI/CD config file scanning, you can get visibility into your publicly exposed credentials and remove or secure them as needed.

4. Runtime Environments

If you're employing a basic secrets scanner, secrets can be introduced into runtime environments from two sources. Basic scanners do not provide complete secrets scanning coverage, which means they may not flag all the secrets in source code or the secrets embedded in container images before those secrets are deployed.

Let’s say your scanner overlooks a hardcoded API token in your source code and the code gets deployed. If your team doesn’t realize this until after the fact, they’ll have to manually remediate the issue by removing the credentials from the running application and source code, and then both revoking and rotating the key, which may break other apps.

Alternatively, it’s possible that you have secrets embedded in your container images. Although it’s a bad practice to modify running containers, secrets may have been manually added to images. Secrets can also sneak into your runtime environments if they’re not spotted early because they were managed through the pipeline (e.g. in Vault) and then added to the image. It’s a best practice to inject secrets just in time for access. Without looking deep into container filesystems, these secrets could be missed by you — but not by attackers.

5. In Your Long List of Alerts

Secrets scanning solutions have a bad reputation for generating noisy alerts. Too many false positives can introduce alert fatigue, distract your team and result in overlooked alerts that flagged legitimate exposed secrets.

To reduce the number of alerts your team must field, you’ll need to find a comprehensive secrets scanner that’s multidimensional and finely-tuned. When your scanner combines keyword, regular expression and entropy-based scanning in a multidimensional approach, it will identify more secrets, so being able to validate secrets is important to reduce total alert volume, lower your false positive rate and give you the complete coverage you need.

The cascading nature of secrets exposure means that overlooked secrets in IaC files and application code can get uploaded to public repos and deployed by delivery pipelines — which contain their own exposed secrets — to then become runtime security issues that teams must remediate. Getting ahead of this cascading effect requires that you shift your secrets security left and adopt a comprehensive, prevention-first approach to secrets management.

Want to see secrets security in action? Watch our on-demand webinar and learn how to get full-stack, multidimensional secrets security with Prisma Cloud.


Subscribe to Cloud Native Security Blogs!

Sign up to receive must-read articles, Playbooks of the Week, new feature announcements, and more.