How We Cut DNS Cache Memory by 56% 5 Rust Optimizations That Saved 100 TB

Why DNS Cache Memory Matters

At Cloudflare's scale, the DNS cache stores over 250 billion entries at any moment. A single wasted byte per entry translates to 250 GB of memory across the fleet. This isn't just about cost—it's about performance. More memory-efficient caching means higher hit rates, lower latency, and better user experience.

We identified five successive optimizations that cut per-entry memory usage from 953 bytes to 420 bytes—a 56% reduction. The result: roughly 100 TB of freed RAM, equivalent to 130 Gen 13 servers. But we didn't sacrifice speed. Insert throughput rose 43% and lookup latency dropped 19%.

The First Win: Replace Vec with Box<[T]>

The Rust Vec type includes a capacity field—8 bytes per vector. But once we store a DNS response, we never modify it. Switching to Box<[T]> eliminates the capacity field and over-allocation. We applied this to all 8 vector fields in each cache entry, saving 64 bytes per entry. This alone freed over 15 TB across the fleet.

// Before: Vec with capacity overhead
pub struct CacheEntry {
    answers: Vec<Record>,
    authority: Vec<Record>,
    additional: Vec<Record>,
    // ...
}

// After: Box<[T]> with no extra capacity
pub struct CacheEntry {
    answers: Box<[Record]>,
    authority: Box<[Record]>,
    additional: Box<[Record]>,
    // ...
}

Consolidating Lists and Dropping Owners

Instead of storing answer, authority, and additional sections as separate lists, we store a single list with 2-byte offsets to each section. This removes two pointers and two lengths per entry—saving 28 bytes.

// Before: Three separate boxes
pub struct CacheEntry {
    answers: Box<[Record]>,
    authority: Box<[Record]>,
    additional: Box<[Record]>,
}

// After: Single buffer with offsets
pub struct CacheEntry {
    records: Box<[u8]>,
    answer_offset: u16,
    authority_offset: u16,
    // ...
}

We also noticed that most DNS records have an owner identical to the queried domain. By storing Option<Name> for the owner field, we can infer the domain from the cache key when it's None, avoiding a heap allocation for the majority of records.

Enum Sizing and Wire Format Storage

Rust enums are sized to their largest variant. Our RecordData enum had a NAPTR variant at 136 bytes, so every A record (4 bytes) wasted 120 bytes. Boxing large variants moved them to the heap, but introduced allocator overhead and poor memory locality.

The breakthrough came from storing record data in wire format as raw bytes with a 2-byte length prefix. This eliminated per-variant overhead and improved cache locality.

// Before: Enum with large variants
pub enum RecordData {
    A(Ipv4Addr),
    Aaaa(Ipv6Addr),
    Naptr(Naptr),
    // ...
}

// After: Boxed large variants
pub enum RecordData {
    A(Ipv4Addr),
    Aaaa(Ipv6Addr),
    Txt(Box<Txt>),
    Naptr(Box<Naptr>),
    // ...
}

Benchmarks and Production Results

Our benchmarks showed per-entry footprint dropped from 953 to 420 bytes, and allocations from 1.1 KB to 461 bytes. In production, p99 memory usage fell from 9.3 GB to 5.3 GB per instance. Aggregate savings reached 100 TB.

Limitations and Considerations

These optimizations aren't free. Wire format storage requires sequential iteration, complicating features like round-robin rotation. Boxing large variants can hurt memory locality if not carefully managed. Also, the results depend on traffic mix—ECS-heavy locations benefit more.

Next Steps for Your Codebase

If you're working on high-performance systems, consider these takeaways:

• Profile your data structures: Identify fields that carry unnecessary overhead.
• Use Box<[T]> over Vec for immutable data.
• Store data in compact formats (e.g., wire format) when random access isn't critical.
• Measure both memory and performance to avoid trade-offs.

For deeper insights into resilient engineering, check out our Cloudflare Code Orange fail small engineering insights. And if you're exploring edge AI platforms, see our NVIDIA IGX Thor industrial edge AI platform analysis.

Source: Cloudflare Blog