Secret Scanning in CI: TruffleHog, Gitleaks, and Pre-Commit Strategies

Cybersecurity

Every secret scanner finds two kinds of things, false positives and the AWS key your engineer pasted into a Jupyter notebook three years ago. The challenge is wiring the scanner into the workflow so that real leaks get caught at commit time, not after the rotation cost has already been incurred.

By Arjun Raghavan, Security & Systems Lead, BIPI · August 14, 2023 · 9 min read

Secret scanning is the security control that everyone agrees with and almost nobody runs well. The tools are mature, the patterns are well documented, and the integration points are clear. The hard part is making the scanner part of the developer workflow rather than a quarterly cleanup exercise.

Three layers, three jobs

Pre-commit is the cheapest layer. It runs on the developer machine before the commit lands, and a well-tuned hook stops most leaks before they enter the repo. CI is the safety net. It runs on every push and every PR, catches what pre-commit missed, and blocks the merge. Periodic full-history scans are the audit layer. They find the secrets that were committed before the program started.

• Pre-commit: catch leaks before they enter history, fast feedback, optional bypass
• CI on PR: blocking gate, no bypass, scoped to the diff
• Periodic full-history: monthly job that catches legacy leaks and pattern drift

TruffleHog vs Gitleaks vs the built-in scanners

TruffleHog v3 has the broadest detector set and supports verification, which actually calls the API behind a suspected key to check if it is live. That cuts false positives dramatically. Gitleaks is faster, easier to configure, and ships better default rules for the common patterns. GitHub and GitLab both ship native secret scanning that catches the obvious AWS, GCP, and Stripe formats but misses internal patterns.

Our default is Gitleaks at pre-commit for speed, TruffleHog verified mode in CI for accuracy, and the native scanner enabled as a third net.

Pre-commit done right

1. Install pre-commit framework, hook gitleaks into .pre-commit-config.yaml
2. Pin the gitleaks version, do not float to latest
3. Scope the scan to the staged diff, not the full repo, to keep it under one second
4. Provide a documented bypass for emergencies, audit the bypass log monthly
5. Distribute the hook through a repo template, do not rely on every engineer to install it

CI configuration that works

In CI, run the scanner against the PR diff, not the full history. Full-history scans on every PR are slow and produce repeated findings for legacy leaks that are already tracked. The diff scan is fast, focused, and blocks the right thing. Keep the full-history scan as a scheduled job that runs nightly or weekly.

The scanner that produces fifty false positives per PR is the scanner that gets disabled by the platform team within a quarter.

Verification mode is the unlock

TruffleHog verified mode is the feature that takes secret scanning from noisy to operationally useful. For supported providers, it calls the API with the suspected key. A live key triggers a high-severity finding. A dead or malformed string is suppressed. The signal-to-noise improvement is dramatic, often ten to one.

When the leak is real

Rotation, not deletion, is the response. A git rewrite to remove the file does not help, because the secret was already in the build cache, the runner logs, and possibly a third-party mirror within minutes of being committed. Treat any verified leak as compromised, rotate immediately, and investigate the access window with provider audit logs.

• Rotate the credential within the hour, document the rotation in a ticket
• Pull provider audit logs for the access window between commit and rotation
• Notify the owning team and the security on-call
• Add a regression rule to the scanner so the same pattern is caught in future
• Run a postmortem if the rotation took longer than the SLO

Closing

Secret scanning works when it is fast, scoped, and paired with a rotation playbook. The teams that struggle have either a scanner producing noise that gets ignored, or no scanner and a quarterly cleanup that finds the same five legacy leaks every quarter. Pick the tools, tune the rules, and put the on-call in place.

Read more field notes, explore our services, or get in touch at info@bipi.in. Privacy Policy · Terms.