Bugs that survive the heat of continuous fuzzing

A False Sense of Security: Why Long-Term Fuzzing Still Misses Critical Bugs

OSS-Fuzz, run in collaboration with the OpenSSF Foundation, has become one of the most impactful security initiatives in open source, helping to uncover thousands of vulnerabilities across more than 1,300 projects at no cost to maintainers. Yet continuous fuzzing is not a silver bullet. Even mature projects enrolled for years can harbor serious vulnerabilities that evade detection. Recent audits by GitHub Security Lab have revealed critical bugs in several well-established projects, highlighting persistent gaps in automated testing approaches.

GStreamer: The Coverage Gap

GStreamer, the default multimedia framework for GNOME and Ubuntu, is invoked whenever a user opens a multimedia file in Totem, accesses file metadata, or generates thumbnails. In December 2024, researchers discovered 29 new vulnerabilities in the framework, including several high-risk issues—a surprising finding for software that has undergone continuous fuzzing for seven years.

The public OSS-Fuzz statistics explain why: GStreamer has only two active fuzzers and code coverage of approximately 19%. For comparison, a heavily researched project like OpenSSL operates 139 fuzzers, while the compression library bzip2 achieves 93.03% code coverage—nearly five times GStreamer's rate.

The disparity underscores a fundamental limitation of OSS-Fuzz: the platform requires active human supervision to monitor coverage metrics and author new fuzzers for untested code paths. While AI agents may eventually help fill this gap, a human currently must do this work manually.

There is also a non-technical dimension to the problem. Many developers view enrollment in OSS-Fuzz as a completion checkbox rather than an ongoing commitment. Once their project is "being fuzzed," they assume it is "protected by Google" and move on—even in cases where the project actually fails during the build stage and is not being fuzzed at all. This false sense of confidence means that human security expertise remains essential for maintaining each enrolled project, a requirement that does not scale well with OSS-Fuzz's success.

Poppler: Blind Spots in Dependencies

Poppler, the default PDF parsing library in Ubuntu, renders documents in Evince and Papers. Its OSS-Fuzz profile shows 16 fuzzers and approximately 60% code coverage—solid numbers that are above average. Despite this, a 1-click remote code execution affecting Evince in Ubuntu was recently demonstrated; a victim only needs to open a malicious file to be compromised.

The vulnerability survived not because of weak coverage in Poppler itself, but because of external dependencies. Poppler relies on several libraries—freetype, cairo, libpng—that, according to Fuzz Introspector data, have not been instrumented by libFuzzer. Without instrumentation, the fuzzer receives no feedback from these libraries, leaving many execution paths completely untested.

The situation is worse for some of Evince's default dependencies that are not included in the OSS-Fuzz build at all. DjVuLibre, which implements support for the DjVu document format popular in the late 1990s and early 2000s for compressed scanned documents, is one such case. Although DjVuLibre has become less widely used since PDF standardization in 2008, it is still shipped by default with Evince and Papers—meaning millions of systems include a dependency that has never been fuzzed. This demonstrates that software is only as secure as the weakest dependency in its graph.

Exiv2: Neglecting the Encoding Side

Exiv2, a C++ library for reading and modifying Exif, IPTC, XMP, and ICC metadata in images, is used by major projects like GIMP and LibreOffice. The project was enrolled in OSS-Fuzz in 2021 as part of a security improvement effort, which uncovered multiple vulnerabilities including CVE-2024-39695, CVE-2024-24826, and CVE-2023-44398. Yet despite more than three years of continuous fuzzing, external researchers have continued to report new issues, including CVE-2025-26623 and CVE-2025-54080.

The recurring pattern here is a common one in media format fuzzing: researchers focus overwhelmingly on the decoding attack surface, which is the most obviously exploitable, while the encoding logic receives far less attention. Vulnerabilities in encoding functions can therefore remain unnoticed for years. While encoding flaws may seem less dangerous from a user's perspective, these libraries are frequently used in background workflows such as thumbnail generation, file conversions, cloud processing pipelines, and automated media handling—where a successful exploit in an encoding function becomes critical.

Across all three projects, the lesson is consistent: fuzzing is a valuable tool, but not a substitute for ongoing human analysis. Coverage monitoring, dependency auditing, and expanding attention beyond the most obvious attack surfaces are all still necessary to keep mature, heavily fuzzed software secure.

Coverage is not enough: the fuzzing workflow that finds the rest

Fuzzing has clear limits when used as a blanket safety net, as the earlier examples in this series show. To get dependable results from it, you need a disciplined process. The workflow that has worked best for me over the past year is a five-step cycle: preparation, coverage, context, value, and triaging.

Preparation and coverage first

The first step is code preparation: making the target fuzzable by removing checksums, reducing randomness, dropping unnecessary delays, and handling signals properly. After that, the focus shifts to maximizing code coverage, which is an iterative loop of running fuzzers, checking an LCOV report, and improving coverage by writing new harnesses or adding input cases for corner cases. For an automated approach to coverage improvement, the Plunger module in my FRFuzz framework tries to handle some of this work; more details on that project will appear in a future post.

When is coverage good enough to move on? In my experience fuzzing a range of projects, you want to see more than 90% before trying other strategies—or even before turning on tools like ASAN or UBSAN. Reaching that figure means fuzzing not just the obvious attack surfaces like decoders, demuxers, socket receivers, and file readers, but also encoders, muxers, senders, and file-writing routines. Two advanced techniques help close the gap:

• Fault injection, which intentionally introduces unexpected conditions—failed allocations, partial reads or writes, missing files, timeouts—to exercise rare paths like error handling. The Linux kernel's fault injection framework is a good reference.
• Snapshot fuzzing, which restores the target to a chosen state before each test case, useful for stateful software like network services or VMs. AFL++'s QEMU mode and Nyx mode are both examples.

Moving beyond edge coverage

Mainstream fuzzers like AFL++, libFuzzer, and honggfuzz track coverage at the edge level: a transition from one basic block to another in the control-flow graph. That approach is simple and effective, but it doesn't record the order in which blocks execute.

Consider a program with a plugin pipeline where global state is modified by each plugin. Different execution orders can lead to completely different states while the edge coverage bitmap stays the same. When that happens, the fuzzer sees no new paths, its guidance stalls, and bugs go undiscovered. Context-sensitive coverage addresses this by recording, for each edge, some information about what ran immediately before it.

AFL++ offers two such options:

• Context-sensitive branch coverage: each function gets a unique ID; when an edge executes, the fuzzer hashes the current call stack IDs with the edge identifier to produce the coverage entry.
• N-Gram branch coverage: combines the current location with the previous N locations—1-gram looks at just the prior location, 2-gram the prior two, 4-gram the prior four—to form a larger coverage context.

With context-sensitive coverage, the 90% target no longer applies. The practical number depends on the architecture and on how deep into the call stack you go; anything above 60% is a solid outcome in my experience.

Value coverage finds the bugs path coverage misses

Even total code coverage misses certain vulnerabilities. This webserver snippet illustrates the point:

The function unicode_frame_size executed 1,910 times during fuzzing, and no bug surfaced. But there's a clear div-by-zero when r.padding == FRAME_SIZE * 2:

Because padding is client-controlled, an attacker can trigger a denial of service with a request carrying a padding size of 2156 * 2, or 4312 bytes. The fuzzer never got there despite all those executions. Value coverage targets exactly such cases: instead of guiding on control-flow paths alone, it guides on the range of values a variable takes. If the fuzzer had generated that 4312 value for r.padding, it would have hit the bug.

One way to make a variable value-visible to a coverage-guided fuzzer is to map distinct numeric ranges onto separate execution paths. A simple helper function can do that:

inline uint32_t value_coverage(uint32_t num) {

   uint32_t no_optimize = 0;
  
   if (num < UINT_MAX / 2) {
       no_optimize += 1;
       if(num < UINT_MAX / 4){
           no_optimize += 2;
           ...
       }else{
           no_optimize += 3
           ...
       }

   }else{
       no_optimize += 4;
       if(num < (UINT_MAX / 4) * 3){
           no_optimize += 5;
           ...
       }else{
           no_optimize += 6;
           ...
       }
   }

   return no_optimize;
}

It uses a no_optimize variable to keep the compiler from optimizing the paths away. You then call it for any variable you want to track:

static volatile uint32_t vc_noopt;

uint32_t webserver::unicode_frame_size(const HttpRequest& r) {

   //A Unicode character requires two bytes
   vc_noopt = value_coverage(r.padding); //VALUE_COVERAGE
   uint32_t size = r.content_length / (FRAME_SIZE * 2 - r.padding);

   return size;
}

That approach yields a vast number of paths, so it's only practical for "strategic" variables: input-controlled values involved in sensitive operations. Selecting which variables qualify is a matter of developer and researcher intuition. To keep the path count manageable, you can group values into buckets—each bucket mapping to one path instead of testing all 2^32 values of a 32-bit integer. Buckets don't need to be uniform, either; you can use smaller buckets in critical subranges and larger ones where precision matters less.

The standard tooling offers partial support for value coverage, but with caveats:

• AFL++ CmpLog and Clang trace-cmp track values used in comparison instructions only. They would not spot the div-by-zero above, since the division value isn't compared.
• Clang trace-div combined with libFuzzer's -use_value_profile=1 does trace values involved in divisions and would catch the example, but it gives no variable-level granularity—only function or file scope, which isn't fine-grained enough to focus solely on the variables that matter.

Because existing options fell short, I implemented a custom LLVM FunctionPass for value coverage with the level of control I needed; that implementation is in FRFuzz. The remaining steps of the workflow—context and triaging—will be covered in the final part of this series.

The Long Tail of Fuzzing Blind Spots

Even with a fully resourced, continuously running fuzzing setup, certain vulnerability classes will consistently slip through the net. These are the bugs that exist at the outer edges of what fuzzing can achieve, and they typically fall into two distinct categories.

Size-Dependent Bugs

Some vulnerabilities only manifest when processing extremely large inputs—on the order of megabytes or gigabytes. This is a brutal barrier for fuzzers due to two fundamental constraints:

• Most fuzzers enforce a hard cap on input size (e.g., 1 MB for AFL) to maintain speed and efficiency, as larger inputs inherently lead to longer execution times.
• The theoretical input space grows exponentially at O(256ⁿ), where n is the byte size of the input. Even coverage-guided heuristics operate as a sub-exponential solver relative to input size, making the probability of hitting a deep bug diminish rapidly as the input grows.

A concrete example is CVE-2022-40303, an integer overflow in libxml2 that requires a trigger input larger than 2GB. Such cases are essentially out of reach for conventional fuzzing tools.

Time-Dependent Bugs

The second blind spot is the time dimension. Fuzzers are performance-obsessed by design, often executing thousands of test cases per second. This forces per-execution timeouts in the 1–10 millisecond range. Any vulnerability that requires even a few seconds of continuous execution to bloom is immediately out of scope.

My colleague Kevin Backhouse demonstrated this concept with a reference-count overflow in Poppler. In that code, counters tracking pointer references were implemented as 32-bit integers. An attacker who could trigger 2^32 increments would cause the counter to wrap to zero, leading to a use-after-free. Kevin wrote a proof of concept for the flaw, but it required 12 hours of runtime to complete. This is an extreme case, yet many vulnerabilities still need at least several seconds of runtime to emerge—well beyond the typical sub-second fuzzer timeouts.

It is worth noting, however, that when fuzzers do flag a timeout, it is often a false positive. Still, those anomalies deserve inspection. Occasionally they uncover genuine performance-related DoS flaws, such as quadratic loops.

Where to Go From Here

We don't yet have effective fuzzing strategies to conquer these edge cases. Mainstream coverage-guided fuzzers simply cannot grasp vulnerabilities that depend on massive inputs or extended execution windows. To find them, security researchers must fall back on alternative approaches: static analysis, concolic testing (a blend of symbolic and concrete execution), and the still-highly-effective practice of manual code review.

Fuzzing is not a fire-and-forget solution. It is a powerful tool, but it has demonstrable structural gaps. Without human intervention and supplementary analysis, entire classes of bugs have survived years of continuous fuzzing in critical open-source projects. Moving beyond simple code coverage to incorporate context-sensitive and value-based coverage metrics helps close some gaps, but the discipline still requires patience and human insight. For those looking to improve their fuzzing methodology, a structured approach that accounts for these blind spots is essential to uncovering the vulnerabilities that automated processes will continue to miss.