Cutting the Cost of Kernel-Memory IP Lists
Magic Firewall evaluates layer 3 traffic against customer-defined rules that can reference millions of IP addresses. IP lists make those rules readable, but the initial implementation had a scaling problem: every network namespace on a server held its own full copy of every list its rules referenced. With enough customers sharing a list, kernel memory ballooned and nft, the nftables configuration tool, spent noticeable CPU rendering huge configurations.
The team's fix moves list storage out of per-namespace nftables sets and into shared eBPF maps, then patches nftables so a rule can call a pinned eBPF program without losing atomic rule replacement.
Why the Per-Namespace Design Hurt
Magic Firewall runs inside the network namespaces that Cloudflare creates for Magic Transit and Magic WAN traffic. A daemon on each server translates API-driven configuration changes into nftables rules, and because nftables state lives inside the namespace, one customer's firewall is isolated from another's. Early on, IP lists were built as nftables sets, letting a wirefilter expression like ip.src in $anonymizer_list become a set lookup inside the ruleset.
That was fast to ship but expensive at scale. Each network namespace that referenced a list got its own kernel copy. Ten customers using the same list meant ten copies on every server. The memory cost was predictable, but the CPU cost from nft rendering those large sets during updates was a surprise—it became the primary contributor to a monitoring alert for another team's service. Incremental set updates and splitting changes across multiple nft statements helped at the margins but added complexity for modest gains.
Sharing Data with eBPF Maps
eBPF maps are not tied to a network namespace, so they offer a way to store IP lists once and have every namespace reference the same data. A simple hash or array map won't work for lists that contain CIDR ranges, but Linux ships BPF_MAP_TYPE_LPM_TRIE, a longest-prefix-match trie whose bpf_map_lookup_elem() returns a hit when an IP falls inside any inserted range. The eBPF lookup code is concise: given an IPv4 or IPv6 address plus a prefix length key, the kernel trie handles the range matching.
The resulting kernel memory profile changes character. Instead of per-customer copies of each list, the maps are populated once and shared. The memory that does remain is attributed to the cgroup of the Go daemon that manages the firewall, which makes it straightforward to report and monitor.
Restoring Atomic Rule Replacement
Earlier eBPF integration into Magic Firewall had a structural drawback: the nft command-line tool had no syntax for invoking a BPF program from a rule, so the daemon formatted netlink messages itself. That worked, but it meant eBPF-based rules had to be inserted after the native nftables rules, in a separate transaction. Failure handling became ambiguous: retry, roll back the nftables ruleset, or risk partial application?
The team patched nftables to add a bpf keyword to its configuration grammar. The work had two main pieces. The first was implementing struct expr_ops and its required methods so nft's parser understands the new keyword. The second was building the netlink messages through the nftnl library rather than manually formatting struct xt_bpf_info_v1 into a byte array the way iptables does.
With the patch in place, a configuration can name a pinned eBPF program by path, and that program executes as part of the normal nftables ruleset. Native nftables matches and eBPF programs can be mixed in one atomic transaction, which also clears the way for more advanced packet inspection in eBPF later.
Measured Impact
Deployment rolled out across namespaces over a few hours. Kernel memory dropped as the per-namespace sets were replaced by the shared eBPF maps, and the memory profile has stayed flat since the maps were first populated—customer count no longer multiplies the cost of this feature. The change also puts the service on firmer footing as list sizes and the number of customers using them continue to grow.
