AI-Era Code Modeling: How GitHub Automates Sink Discovery for CodeQL
GitHub is increasingly leaning on AI to improve both the speed of development and the security of the code developers produce. This push extends beyond code generation features like Copilot’s security filters for Python and JavaScript—which already target patterns such as hardcoded credentials, SQL injections, and path injections—into the deeper infrastructure of its security analysis tools.
One of the most resource-intensive tasks for the team behind CodeQL, GitHub’s semantic code analysis engine, is manually creating models for the thousands of frameworks and APIs the tool must understand. In order to detect vulnerabilities, CodeQL must identify where untrusted user data enters a program (sources), where it reaches sensitive operations (sinks), and how it travels through the code (taint propagators). With the vast number of open source packages evolving daily, any gaps in these definitions can create false negatives—vulnerabilities that go undetected.
From Manual Reviews to LLM-Generated Models
Previously, modeling these APIs required painstaking manual effort. Over the last six months, however, the CodeQL team has switched to using Large Language Models (LLMs) to automate the creation of these API models. This shift has accelerated the process significantly and reduced the tool’s false negative rate by allowing it to recognize a wider set of sinks out of the box.
To validate the improvements, the team relied on variant analysis, a technique used to find new types of security vulnerabilities by running queries against a broad pool of code. They applied the queries using the AI-generated models across the most impactful repositories hosted on GitHub.
Discovery of CVE-2023-35947
This combination of LLM-driven modeling and large-scale variant analysis directly led to the discovery of a real-world vulnerability: CVE-2023-35947, a path traversal flaw in the Gradle build tool. The finding was a direct result of the expanded coverage provided by the automatically generated models.
For technical details on the vulnerability, GitHub points researchers to the Security Lab’s CodeQL Wall of Fame and the GitHub Advisory Database.
Toward Scaled Security Research
The discovery illustrates the potential of merging AI-assisted tooling with GitHub Advanced Security capabilities. To help other security teams adopt a similar workflow, GitHub released Multi-Repository Variant Analysis (MRVA) in March 2023. This feature allows security researchers to run CodeQL queries across thousands of repositories at scale from within Visual Studio Code.
CodeQL remains free to use on open source repositories. Developers interested in learning how to perform MRVA can review GitHub’s documentation for code scanning and multi-repository variant analysis.
