Social workloads at Meta scale meet a public benchmark

Meta has open-sourced TAOBench, a benchmark designed to reproduce the social graph workload served by TAO, the company’s geographically distributed graph data store. Along with the framework, Meta has published workload configurations and the full research paper. TAOBench is intended to fill a gap in the database research and engineering toolkit: few existing benchmarks realistically simulate the request patterns produced by large social networks, which limits work on improving the systems that support them.

The benchmark’s request patterns are derived from profiling TAO, which serves more than 10 billion requests per second against a multi-petabyte, frequently changing dataset. TAOBench characterizes these patterns through a small set of parameters, including transaction size, key-to-shard mapping, and operation type frequency. Those parameters support both downscaling Meta’s production workload for testing and generating emergent application behavior. The framework is extensible so that it can model a range of request patterns beyond the original configuration.

What a social network benchmark must capture

TAOBench is built around five properties that Meta argues any comprehensive social network workload should exhibit:

  1. Accurate emulation of social network request patterns
  2. Capture of transactional requirements
  3. Expression of data colocation preferences and constraints
  4. Modeling of distributions without prescriptive query types
  5. Exhibition of multitenant behavior on shared data

The underlying TAO workload also highlights the reading and writing asymmetry typical of social graphs. For example, more than 99 percent of data items that are frequently written are read, on average, less than once per day.

Use cases and early results

Meta describes four internal uses of TAOBench that would otherwise be difficult or infeasible to evaluate in production:

  1. Analyzing new transactional use cases
  2. Assessing contention under longer lock hold times
  3. Evaluating new APIs
  4. Quantifying performance of high fan-out transactions

The benchmark has also been run against five widely used distributed database systems: Cloud Spanner, CockroachDB, PlanetScale, TiDB, and YugabyteDB. Meta reports these results demonstrate the utility of TAOBench for studying performance trade-offs and spotting optimization opportunities across systems.

The absence of realistic, publicly available workloads has made it difficult for both academics probing database limits and practitioners evaluating new features or troubleshooting issues. By matching TAOBench’s generated workloads against their production equivalents, Meta says it has validated the workloads’ integrity. TAOBench provides a means to reproduce the demands of a real social network workload without access to Meta’s internal systems.