How Copilot secret scanning finds passwords in code

GitHub's Copilot secret scanning, generally available since October 2024, uses an LLM to detect generic passwords in codebases. The feature is part of GitHub Secret Protection and supplements the pattern-based detectors that already identify hundreds of provider-minted secret types. Regular expressions work well for secrets with strict formats, but generic passwords have too much structural variation, which historically produced noisy results. AI-based scanning adds contextual analysis—examining where and how a potential secret appears—to reduce false positives.

The detection pipeline is built around an LLM prompt that contains three elements: general information about password vulnerabilities, the source code location and file contents where a potential issue exists, and a strict JSON output format for automated processing. The approach sounds straightforward, but getting password precision to a trustworthy level required iterating through test cases, prompt strategies, and model choices.

Early failures exposed context blind spots

The first implementation used few-shot prompting with GPT-3.5-Turbo, chosen for resource efficiency at scale. Alongside it, an offline evaluation framework with manually curated positive and negative test cases validated the concept before customer deployment.

Private preview results told a different story. Detection worked acceptably on offline evaluation but failed badly in real customer repositories. The model struggled with file types and structures outside the conventional coding patterns common in LLM training data. The gap between synthetic evaluation and production reality made the team rethink the entire approach.

Better evaluation, better prompting

The offline framework grew in three ways. Reports from private preview participants brought real-world diversity to the test set. Visual analysis capabilities let the team spot deviations caused by model or prompt changes. And a data collection pipeline, built using the GitHub Code Security team's evaluation processes, used GPT-4 to generate new test cases from existing secret scanning alerts in open source repositories.

This gave the team enough breadth to measure both precision (accurate detection with few false positives) and recall (reliable detection with few false negatives). The testing process examined several questions: whether a different model performed better, whether running prompts multiple times and combining responses helped, and how to handle the inherent nondeterminism of LLM responses.

Several specific mechanisms were tried:

  • Voting—asking the model the same question repeatedly—produced more deterministic responses but didn't improve precision.
  • A confirming scanner using GPT-4 validated candidates found by GPT-3.5-Turbo, improving precision without reducing recall at the cost of more resources.
  • Prompt strategies including Fill-in-the-Middle, Zero-Shot, and Chain-of-Thought were compared. The team ultimately adopted MetaReflection, an offline reinforcement learning technique from Microsoft Research that derives a hybrid Chain-of-Thought and few-shot prompt from past trial experiences. This improved precision with a small recall penalty.

The final public preview version combined all of these techniques.

Managing capacity at scale

Secret scanning covers not just incoming Git pushes but entire Git history on all branches, so resources scale linearly with each new customer. Before expanding LLM capacity, the team looked for ways to reduce resource consumption itself:

  • Excluding file classes unlikely to contain credentials or comprehensible to the model, such as media files and test, mock, or spec files, from scanning.
  • Evaluating newer models including GPT-4-Turbo and GPT-4o-mini for lower resource demands without performance loss.
  • Experimenting with context window sizes to reduce resources without raising latency.
  • Improving tokenization, including retaining memory of previous tokenizations while processing new parts of a file.

Some changes helped, others didn't—smaller content chunks had little impact, while model choice mattered more. The decisive fix came from a workload-aware request management system that maximizes and fairly shares LLM capacity across scan workloads.

The core capacity problem was that fixed rate limits per workload were inefficient. Git commit scanning correlates with working hours, while full history scanning spikes when an administrator enables the feature on a new organization. A workload could hit its limit within its operational context while other workloads had idle capacity. The team designed an algorithm inspired by existing solutions like Doorman, GitHub's own Freno, and weighted fair-priority queue algorithms. It assigns a range of limits per workload—enough to prevent any single workload from overwhelming the LLM, while allowing workloads to borrow unused capacity from others. The approach proved effective enough that it was adopted for Copilot Autofix and security campaigns as well.

Mirror testing before GA

Confidence for general availability came from a mirror testing framework. The team ran prompt and filtering changes against a subset of public preview repositories, rescanning with the latest improvements to measure changes in alert volume and false positive resolutions without affecting real users.

The results showed a significant drop in detections and false positives with very few genuine passwords missed, including a 94% reduction in false positives across some organizations. That before-and-after comparison confirmed that the cumulative changes increased precision without sacrificing recall.

Ongoing lessons

Copilot secret scanning now detects passwords on nearly 35% of all GitHub Secret Protection repositories. The operational principles from this work carry forward into Copilot Autofix and future development:

  • Precision first: Security teams need actionable alerts without noise.
  • Diverse test cases: Customer feedback continues to feed into the test bed for detection refinement.
  • Resource management: Scalability and performance must be balanced continuously.
  • Collaborative innovation: Cross-team partnerships with GitHub and Microsoft teams push Copilot capabilities forward.

Since the GA launch, Copilot secret scanning enablement is included in security configurations, giving organizations control over which repositories use secret detection. Ongoing monitoring, mirror testing, and feedback loops continue to drive improvements to detection quality.