Why caching is getting harder to scale
Caching is what keeps large-scale services responsive: an email client can show recent messages without re-fetching them, and a social feed can render without repeated lookups to the database. The engineering challenge is that every cache layer must balance speed against cost and power consumption. For years, organizations built separate, bespoke caches for different services, each maintained by a different team. That approach duplicates effort and overlooks the problems all caching systems face, from deployment to maintenance.
Cost is driving the architectural shift. Traditional DRAM-based caches are expensive and power-hungry, so operators are increasingly mixing DRAM with non-volatile memory (NVM) drives. But this hybrid model only pays off with cache designs that decide what to store and for how long. Plain LRU-style eviction is no longer enough; the useful content is the content that gets requested again soon, and that determination must be made across two very different storage tiers.
One engine for in-process caching
Facebook is tackling this with CacheLib, an open source, in-process caching engine built as a C++ library. The engine provides a simple API so developers can add a concurrent, scalable cache to their own services without writing a new caching layer from scratch. It is not a standalone server; it is a library that runs inside the application process, and it abstracts over both DRAM and NVM storage tiers.
Around the same core, Facebook is open-sourcing CacheBench, a benchmarking tool designed to measure caching performance against production-like workloads. The idea is that both the engine and the benchmark become the shared substrate for innovations that would otherwise be siloed inside one company.
This is not a greenfield project. CacheLib runs in more than 70 large-scale systems at Facebook, including the social graph, content delivery, storage, and look-aside key-value caches. With that production footprint, the project has a repository of diverse access patterns to test against. CacheBench turns those patterns into replayable workloads, letting a new cache design be validated far more accurately than a synthetic microbenchmark could.
Adoption across industry and academia
The open source release is backed by collaborations that were already underway before this announcement. Twitter, for instance, is working with Facebook to integrate CacheLib into Pelikan, Twitter's caching framework, in order to offload cache objects to SSDs. Pinterest is evaluating CacheLib inside its machine learning infrastructure, where the goal is better prediction performance without sacrificing stability.
Academic work is also plugged into the project. Researchers at Carnegie Mellon, Princeton, and Yale are using CacheLib and CacheBench to prototype new caching heuristics. Because they can benchmark their experimental designs against real industry workloads rather than lab-built traffic, they can iterate on their prototypes much faster. That is a significant change from the usual academic workflow, where the gap between simulation and production code is wide.
Hardware vendors are in the loop as well. Intel, KIOXIA, Samsung, and Western Digital have collaborated with Facebook on features to make SSDs serve caching workloads better. That work has fed into the Open Compute Project’s NVMe Cloud SSD Specification. The convergence of CacheLib's software layer with the OCP specification is meant to push the next generation of NVM drives to be cache-aware from the device up.
CacheLib's value is its position at the intersection of a production-ready codebase, a wide set of benchmark workloads, and an open ecosystem of contributors. For operators, it shortens the path to a hybrid DRAM/NVM cache; for researchers and hardware vendors, it provides a common frame of reference. Everything about this release is oriented toward making caching innovation a shared effort rather than one company's internal engineering problem.
