Making the WAF 40% faster

A Faster Path Through the WAF

Cloudflare's Web Application Firewall (WAF) inspects every HTTP request that crosses its edge, applying thousands of rules designed to catch malicious payloads while keeping false positives low. The security team’s mandate is to do this without adding noticeable latency. Recent work on the rule execution pipeline has delivered exactly that: a 40% reduction in average request processing time, achieved through a combination of cache optimization and rule rewriting.

From Backtracking to Linear Time

The foundation for these improvements was laid in July 2019, when the WAF switched its regular expression engine from PCRE to one inspired by RE2. PCRE relies on backtracking, which can cause exponential execution time on certain inputs—a problem that led to an outage when a newly deployed rule matched a request in a pathological way. RE2, built around a deterministic finite automaton (DFA), guarantees linear execution time relative to input size.

After the migration, CPU consumption at the edge was unchanged, but the 95th and 99th percentile execution-time outliers dropped noticeably. Since the WAF engine uses a thread pool, the team also had to implement and tune a shared regex cache to prevent excessive memory use. The reliability gains from that switch gave the team room to look for further optimizations in the rest of the engine.

How Rules Are Executed

An HTTP request reaching the WAF is split into logical variables—method, path, headers, and body—stored in Lua. Before a request is matched against attack signatures, transformations run against those variables. These may be simple operations like lowercasing or more complex tokenizers and parsers built to fingerprint malicious payloads.

A typical rule, using a ModSecurity-like syntax, might take the request body, apply urlDecode() and lowercase(), then compare the result against a signature such as \x00+evil:

SecRule REQUEST_BODY "@rx /\x00+evil" "drop, t:urlDecode, t:lowercase"

In pseudo-code, this becomes:

rx( "/\x00+evil", lowercase( urlDecode( REQUEST_BODY ) ) )

Requests matching this rule have a percent-encoded NULL byte followed by the word "evil" in the body:

POST /cms/admin?action=post HTTP/1.1
Host: example.com
Content-Type: text/plain; charset=utf-8
Content-Length: 16

thiSis%2F%00eVil

The challenge is that nearly all requests are benign and no rules will match. The engine must still assume the worst and check everything. To avoid running expensive regular expressions on every request, the first matching step for many rules is pre-filtering. A quick byte scan looks for a marker such as the NULL byte (\x00) and skips the entire rule if it's absent:

contains( "\x00", REQUEST_BODY )
and
rx( "/\x00+evil", lowercase( urlDecode( REQUEST_BODY ) ) )

These checks are cheap and since most requests don't trigger any rule, the added tests don't increase the total amount of work—they reduce it. Executing less is often the easiest way to make a system faster.

Caching Intermediate Results

Memoization—caching a function's output and reusing it later—is the second major technique in play. Consider the following set of expressions:

1. rx( "\x00+evil", lowercase( url_decode( body ) ) )
2. rx( "\x00+EVIL", remove_spaces( url_decode( body ) ) )
3. rx( "\x00+evil", lowercase( url_decode( headers ) ) )
4. streq( "\x00evil", lowercase( url_decode( body ) ) )

Here, the result of the nested calls in (1) can be reused directly in (4), and intermediate results, such as the output of url_decode(body), can be shared across multiple expressions. A simple hash table with the function name and arguments as the key and the output as the value is sufficient for an initial implementation. Some transformations are expensive, so caching yields significant savings: one particular rule saw a 95% reduction in execution time once it was modified to take advantage of memoization:

However, as Cloudflare adds rules and new functions to the Managed Rulesets, the memoization cache hit rate naturally degrades.

Rewriting Rules for Cache Hits

The team started by profiling the rules that consumed the most wall-clock time:

Then cross-referenced those against the rules that were missing cache opportunities. Output truncated with [...]:

$ ./parse.py --profile
Hit Ratio:
-------------
0.5608

Hot entries:
-------------
[urlDecode, replaceComments, REQUEST_URI, REQUEST_HEADERS, ARGS_POST]
[urlDecode, REQUEST_URI]
[urlDecode, htmlEntityDecode, jsDecode, replaceNulls, removeWhitespace, REQUEST_URI, REQUEST_HEADERS]
[urlDecode, lowercase, REQUEST_FILENAME]
[urlDecode, REQUEST_FILENAME]
[urlDecode, lowercase, replaceComments, compressWhitespace, ARGS, REQUEST_FILENAME]
[urlDecode, replaceNulls, removeWhitespace, REQUEST_URI, REQUEST_HEADERS, ARGS_POST]
[...]

Candidates:
-------------
100152A - replace t:removeWhitespace with t:compressWhitespace,t:removeWhitespace
100214 - replace t:lowercase with (?i)
100215 - replace t:lowercase with (?i)
100300 - consider REQUEST_URI over REQUEST_FILENAME
100137D - invert order of t:replaceNulls,t:lowercase
[...]

More than 40 rules were identified and subsequently rewritten to better use memoization, with pre-filter checks added wherever possible. The rewrites weren't obvious, which is why the team is also building tooling to help analysts write efficient rules that stay within latency budgets. The outcome: the cache hit rate jumped from 56% to 74%, and importantly the most expensive transformations were among the newly cached ones.

The operational results were equally dramatic. The average time to process and analyze a request at the edge fell by 40%:

Similar drops appeared at the 95th and 99th percentiles. CPU consumption at the edge also decreased by 4.3%.

Changes That Carry Over

The current Lua-based WAF is being ported to the same engine that powers Firewall Rules, built on Cloudflare's open-source wirefilter execution engine. That engine uses a filter syntax inspired by Wireshark and is designed for more flexible expressions with better performance and safety. The optimizations described here were implemented to be engine-agnostic—deliberately not tied to a Lua-specific behavior—so they won't be lost in the migration. While the team routinely profiles and benchmarks the Firewall stack, this round was a reminder that relatively simple changes—better caching, lighter rules—can have an outsized effect.