ZooKeeper practitioners gather for third annual meetup
Facebook recently hosted the third annual ZooKeeper Meetup@Facebook in Menlo Park, bringing together engineers from Cloudera, Salesforce, Twitter, San José State University, and Facebook to discuss performance, scalability, and security work on Apache ZooKeeper. The talks covered production experience with large ZooKeeper fleets, new features, and ongoing research.
Facebook: write throughput, scalability, and containerization
Facebook engineers presented several patches aimed at improving ZooKeeper performance and operations:
- Write throughput: Fangmin Lyu described patches that improve write throughput by up to 10x through changes to the snapshot scheduler and support for batch commits.
- Semaphore optimization: Jie Huang discussed a new lock API that reduces server round trips and queries per second (QPS) for semaphore use cases.
On the scalability side, Yisong Yue and Vladimir Ivic covered demand control, which introduces data size and request rate quotas for ZooKeeper namespaces and client IDs, helping to reduce noisy-neighbor situations in multitenant deployments. Vladimir also walked through migrating users between ensembles with zero downtime.
Christopher Bunn concluded the Facebook portion with a look at the fleet-wide migration to Facebook's in-house containerization system, replacing bare-metal deployments and simplifying fleet management.
Cloudera: upgrading to ZooKeeper 3.5
Máté Szalay-Bekő presented Cloudera's experience integrating and testing ZooKeeper 3.5.5 across more than 13 open source components as part of the new Cloudera Data Platform, which combines the best of the former Cloudera and Hortonworks distributions.
Salesforce: FIPS 140-2 compliance and C client enhancements
Fiaz Hossain and Suhas Dantkale shared their work on Federal Information Processing Standard (FIPS) 140-2 compliance and enhancements to the ZooKeeper C client binding at Salesforce.
Twitter: running on the master branch
Michael Han discussed the experience and lessons learned from upgrading, running, and maintaining Twitter's entire ZooKeeper fleet on the Apache ZooKeeper master branch, as well as the systems Twitter built around ZooKeeper.
SJSU: tracking performance over time
Students from San José State University's systems group, led by ZooKeeper founder Ben Reed, presented their work on benchmarking ZooKeeper performance changes across releases. The project reproduces the reference benchmark with the latest codebase (updated from 3.2) and runs it on different releases to understand performance changes. By automatically benchmarking new pull requests, the team aims to detect the performance impact of proposed changes before they land.
