How we saved 100 terabytes of memory by optimizing 1.1.1.1’s DNS cache

Rethinking the DNS cache entry

Big Pineapple, the platform behind 1.1.1.1, Gateway DNS, DNS Firewall, and related Cloudflare DNS services, holds over 250 billion cache entries at any moment. At that scale, a single wasted byte per entry costs more than 250 gigabytes of memory fleet-wide.

Five successive storage optimizations cut the per-entry footprint by more than half, freeing roughly 100 terabytes of RAM across the fleet — equivalent to the memory in 130 Gen 13 servers. The cache also got faster: insert throughput rose 43%, and lookup latency dropped 19%, because fewer allocations and tighter memory locality meant the space savings did not come at the expense of speed.

What gets cached

Big Pineapple starts each cold boot with an empty cache. As DNS queries arrive, entries accumulate until the maximum entry count is reached, at which point older or less-popular items are evicted. Cache size varies by data center. With EDNS Client Subnet (ECS), authoritative servers return different answers based on the client network, so we cache multiple variants of the same query — increasing both entry count and per-entry memory, making ECS-heavy locations benefit most from these changes.

Each cache item is a key-value pair. The key identifies the query, and the value holds the DNS response: the answer, authority, and additional sections, plus metadata such as creation time, a hit counter, and the TTL. Both data structures carry fields whose type overhead is unnecessary once an entry is stored.

pub struct CacheKey {
    qname: Name,
    qtype: Rtype,
    authenticated: bool,
    tag: Vec<u8>,
}

pub struct CacheEntry {
    timestamp: UnixTimeStamp,
    pub inception: Instant,
    pub ttl: Ttl,
    pub hits: u32,
    pub answers: Vec<Record>,
    pub authority: Vec<Record>,
    pub additional: Vec<Record>,
    pub errors: Vec<ExtendedError>,
    ...
}

How memory is measured

To gauge each change, we benchmark by filling the cache with randomly generated entries matching typical production traffic: 56% A records, 25% AAAA, and 19% TXT, with one to four records per entry. TXT stands in for all variable-length record types, with sizes randomized between 64 and 224 bytes.

Memory usage is tracked with a custom allocator wrapping Rust's System allocator, recording the number and size of allocations per cache entry. Alongside memory, insert throughput and lookup latency are measured across the full cache flow. Resident memory is also sampled across production instances during rollout, since process memory depends on traffic mix, cache occupancy, allocator state, and non-cache usage.

Trimming capacity overhead

A Vec<T> stores a pointer to heap data, the current length, and the total capacity. Pushing an item checks length against capacity, reallocating only when full. But once a DNS response is cached, it is never modified; the capacity field is dead weight at 8 bytes per Vec, and over-allocated heap space goes unused.

Switching to Box<[T]> removes the capacity field and the reserved-but-empty heap slots. The same applies to String, whose capacity field disappears with Box<str>. Each cache entry has 8 such Vec and String fields, so the swap saves 8 bytes per field — 64 bytes per entry — plus the excess heap memory Vec holds in reserve. Combined savings exceed 15 terabytes across 250 billion entries.

Consolidating record sections

Instead of separate lists for the answer, authority, and additional sections, we store a single list with u16 offsets marking each section's start. DNS record counts per section fit in a u16, so each offset costs 2 bytes versus the 8-byte pointer plus 8-byte length that a standalone Box<[T]> needs.

Eliminating two lists' worth of pointers and lengths replaces 32 bytes with 4 bytes of offsets, saving 28 bytes per entry. These savings can exceed the raw field sizes because Rust inserts padding to satisfy alignment, rounding a struct's size up to the alignment multiple. Packing several booleans into a single bitflag shrank surrounding padding, making the struct smaller by more than the booleans themselves.

Inferring the owner

Each DNS record carries an owner — the domain it belongs to. Often that matches the domain queried; example.com A returns records whose owner is example.com. But CNAME chains produce records with differing owners. The DNS wire format handles repetition with name compression per RFC 1035, encoding a 2-byte pointer to a previous occurrence instead of repeating the name. Our cache stores the full owner with each record, since dereferencing compression pointers per lookup is too slow for the hot path.

Most records share the queried domain as their owner, so we can drop the stored owner entirely for them, restoring the queried domain from the cache key at read time. When an owner differs — such as the A records behind a CNAME — the entry stores the full name.

$ dig example.com A

;; ANSWER SECTION:
example.com.        300    IN    A        198.51.100.1
example.com.        300    IN    A        198.51.100.2

$ dig example.com A

;; ANSWER SECTION:
example.com.        300    IN    CNAME    cdn.example.com.
cdn.example.com.    300    IN    A        198.51.100.1
cdn.example.com.    300    IN    A        198.51.100.2

pub struct Record {
    owner: Option<Box<Name>>,
    class: Class,
    ttl: Ttl,
    rtype: Rtype,
    data: RecordData,
}

When the owner is None, response construction pulls the queried domain from the cache key without a heap allocation. The record is no longer self-contained, but the cache key is always available during a lookup. When the owner differs, Some points to the heap-allocated name. In practice, most cached records share their owner with the queried domain, so the majority require no heap allocation for the owner field.

Right-sizing record enums

Rust enums are sum types, always as large as their largest variant. Option stores a tag plus room for the biggest variant, even when the active variant is smaller.

pub enum Option<T> {
    Some(T),
    None,
}

Representing each DNS record type as an enum variant seems natural:

pub enum RecordData {
    A(Ipv4Addr),
    Aaaa(Ipv6Addr),
    Txt(Txt),
    Naptr(Naptr),
    Svcb(Svcb),
    // ...
}

But the enum is sized to its largest variant — NAPTR, at 136 bytes for three variable-length text fields, a domain name, and two integers. With the variant tag and padding, the whole enum becomes 144 bytes.

An A record needs only 4 bytes; AAAA needs 16. Together they comprise over 80% of traffic, so most records waste over 120 bytes of padding each — a heavy cost across entries holding many records.

Boxing the big variants

Moving the larger variants to a separate heap allocation shrinks the enum to an 8-byte pointer. The heap object takes only the size it needs.

pub enum RecordData {
    // Small and common variants are stored inline
    A(Ipv4Addr),
    Aaaa(Ipv6Addr),
    // Large variants are stored on the heap
    Txt(Box<Txt>),
    Naptr(Box<Naptr>),
    Svcb(Box<Svcb>),
    // ...
}

For A and AAAA, this saves 120 bytes per record. Smaller types like TXT and CNAME still occupy the 24-byte enum but allocate exactly their data size rather than padding to 144 bytes. The largest variant, NAPTR, pays slightly more — a heap pointer plus allocation overhead — but NAPTR is rare in practice.

The price of boxing

Boxing carries two costs. First, allocator overhead: each boxed variant becomes a separate heap allocation, rounded to an allocator size class. Big Pineapple uses jemalloc, which groups similar-sized allocations into bins. A 32-byte TXT record fits exactly into a 32-byte bin, but a 40-byte MX record rounds to 48, wasting 8 bytes.

Second, memory locality degrades. Un-boxed record enum values for an entry live in one contiguous allocation; boxed variants scatter across the heap, each requiring a pointer dereference that may fetch a new cache line. With millions of entries, boxed data spreads out instead of packing together.

Neither cost is prohibitive alone, but eliminating both — addressed next — produces measurable gains in memory use and lookup latency. The boxing trade-off still holds, but it is not the final answer.

Record storage: the wire-format middle ground

Storing complete DNS responses in wire format, patching only per-client fields like the message ID on each lookup, is an obvious possibility. But it has problems. DNSSEC records are only included when the client sets the DO (DNSSEC OK) flag, so a wire-format cache would need two variants per entry — one with DNSSEC and one without — or would have to filter records out of an already-built message. And parsing the whole message on every lookup defeats the purpose of the enum approach, which stores already-parsed records.

The middle ground we settled on: keep the rest of the cache entry as structured fields, but store record data as raw bytes. Instead of a list of parsed enum variants, each entry stores its records as a single Box<[u8]> containing each record encoded as a 2-byte length prefix followed by its raw bytes.

This removes the per-variant enum overhead and the boxed heap allocations from the earlier optimization. Data becomes contiguous, which improves CPU cache locality. The tradeoff: records can no longer be randomly indexed; we must iterate through the buffer sequentially. That adds complexity for round-robin rotation of A/AAAA records, but record counts per entry are small enough that the cost is negligible.

When building a response from cached records, most record types can be copied straight from the buffer into the outgoing message. Previously each parsed record had to be serialized field by field back into DNS wire format. Now A, AAAA, TXT, and all DNSSEC record types skip that work by copying their encoded bytes directly. Only records containing domain names — CNAME, NS, MX, SOA — still need parsing so DNS name compression can be applied. Because directly copyable records dominate our traffic, this reduces work on the lookup path; combined with better memory locality, it cut cache lookup latency by 5% in benchmarks.

Building the record buffer uses a reusable scratchspace buffer that persists across cache insertions. Since previous writes have already grown it, reallocation is rare. Record sizes vary, so we don't know the exact buffer size until serialization is done; once the records are in the scratchspace buffer, we allocate a Box<[u8]> and memcpy the data into it. That replaces one allocation per boxed record with a single allocation for all record data, and avoids the waste of shrinking a Vec<u8>, where the allocator may not reclaim the unused tail. This change alone increased cache insert throughput by 13% in benchmarks.

Production results

Production measurements show how the per-entry savings translated to whole-process resident memory. The graph below tracks p90, p98, and p99 memory usage across Big Pineapple instances. The first dashed line marks the rollout start on May 18, 2026; the second marks completion across all services on July 6, 2026. Each release introduced one or more optimizations, so memory usage dropped in steps rather than all at once.

As each release rolled out, restarted instances began with empty caches and consumed more memory as those caches filled. The stable plateaus therefore reflect steady-state memory usage better than the initial dips do.

Per-instance memory dropped across all percentiles. At p99, resident memory fell from 9.3 GB to 5.3 GB, a 43% reduction. At p90 it fell from 6.5 GB to 3.8 GB, a 42% reduction. Instances with fuller caches saw the largest absolute savings.

In benchmarks the five optimizations reduced per-entry memory footprints from 953 bytes to 420 bytes — a 56% reduction. Per-entry allocations dropped from 1.1 KB to 461 bytes. Production reductions are smaller because resident memory includes the cache plus all other process data. After the rollouts settled, aggregate working-set memory across the fleet was roughly 100 terabytes lower.

Performance also improved. Cache insert throughput rose 43% while lookup latency fell 19%.

The freed memory is earmarked for increasing cache capacity without increasing memory usage, which benefits cache hit rates and reduces upstream query volume. We are also exploring further cache optimizations.

For more on Big Pineapple, see How Rust and Wasm power Cloudflare’s 1.1.1.1. If you work on DNS or other large systems, share your own optimization experiences in the Cloudflare Community or on Cloudflare Developers Discord.