Why secret scanning noise matters
Secret scanning is a critical defense for catching exposed credentials before they turn into incidents. But at GitHub’s scale, even small inefficiencies create real friction. Too many false positives make alerts harder to trust, and when alerts feel noisy, developers spend more time triaging and less time fixing actual issues. Over time, this slows remediation and erodes confidence in the system.
To address this, GitHub collaborated with Microsoft Security & AI’s Agents Offense team to bring contextual reasoning into GitHub’s secret scanning verification. The work applies an approach from Agentic Secret Finder, a detection and verification system designed to understand potential secrets in context,—not just whether they match a secret-like pattern. The goal: reduce low-value alerts while preserving detection coverage.
How detection and verification fit together
GitHub secret scanning combines two layers: pattern-based detection and AI-based detection. Pattern-based detection catches known formats like partner tokens and API keys. AI-powered generic detection expands coverage to unstructured secrets such as passwords that don’t match a known provider pattern.
GitHub already has industry-leading precision for provider-pattern detection at massive scale, processing billions of pushes across millions of repositories. The next challenge was raising the precision of AI-detected secrets to a comparable standard. The solution combines GitHub’s large-scale detection pipeline with LLM-based contextual verification.
Detection continues to generate candidates; the verification step evaluates them. Adding more context-awareness improves the system’s ability to distinguish real exposures from values that merely resemble secrets—without changing upstream detection logic or reducing coverage.
Better context, not more code
A key challenge in verification is deciding what context to provide. A short code snippet often isn’t enough to judge whether something is a real secret. However, passing entire files or repositories introduces noise and increases cost and latency.
Instead of giving the model more context, the system gives it better context. Rather than send vast amounts of code, it extracts a small set of high-signal information that explains how the detected value is used. The system looks for cases where a value is assigned to a variable and later passed into an API request, authentication header, database client, or cloud SDK call. Pattern matching can identify that a value looks like a secret; the surrounding usage context helps determine whether it is actually being used as one. This filters out false alarms like random UUIDs or opaque strings without reviewing the full file or repository.
Most false positives can be resolved with focused, file-level context. If a value resembles a placeholder, test data, or unused configuration, it can often be filtered out without deeper analysis. What matters is not how much code the model sees, but whether it has the right signals. This keeps the system accurate, fast, and scalable across large codebases.
Measured impact on alert quality
The approach was evaluated on hundreds of customer-confirmed false positive alerts. The target was a 65% reduction; the result was 75.76%, exceeding the goal while maintaining strong detection performance.
In practice, this means significantly less noise and a higher proportion of alerts that require action. Developers see fewer irrelevant alerts, trust what they see more readily, and spend less time triaging. Real issues can be prioritized and fixed faster.
Looking ahead
GitHub continues to evaluate the approach on larger datasets and live traffic while refining how context is extracted and used for verification. Reducing false positives remains a consistent need at scale, and this work focuses on improving signal quality where it matters most.
The goal is straightforward: fewer distractions, clearer signals, and faster action on real risks. Organizations can run the risk assessment to see how their current setup would benefit, or learn more about secret scanning in the GitHub documentation.
