Apache ZooKeeper: A Coordinator for Distributed Systems
Think of ZooKeeper as a reliable key-value store for metadata. It provides distributed systems with essentials like configuration management, leader election, and service discovery, ensuring all nodes agree on the system's state.
Why it exists
Building distributed systems is hard because nodes need to agree on state. Who is the leader? What is the current configuration? Where are the other services? ZooKeeper was built to provide a robust, off-the-shelf solution for this coordination problem, so every application doesn't have to reinvent consensus.
The mental model
ZooKeeper is like a special-purpose, highly-available file system for your cluster's metadata. You don't store large application data in it. Instead, you store small, critical pieces of information (called znodes) that all your services need to see consistently, like a configuration flag or the address of the current primary database. It's a source of truth.
How it works
A ZooKeeper cluster is called an ensemble, typically of 3 or 5 servers. To ensure reliability, data is replicated across the ensemble. Writes go through a leader and are committed only after a majority of servers acknowledge them, using a consensus protocol called ZAB (ZooKeeper Atomic Broadcast). Clients maintain a persistent session and can set "watches" on data nodes to be notified of changes.
When to use it
It's a foundational component in many distributed data systems. Use it for service discovery (finding other services), leader election (deciding which node is in charge), and distributed configuration management. Systems like Apache Kafka, HBase, and Solr historically relied heavily on it.
When not to use it
Don't use it as a general-purpose database. It's designed for low-volume, small-payload metadata, not high-throughput application data. For new projects, consider lighter-weight alternatives like etcd or Consul, as ZooKeeper is known for being operationally complex to manage. Many modern systems are also removing their ZooKeeper dependency.
One canonical example
Apache Kafka brokers historically used ZooKeeper to elect a "controller" broker. The first broker to create a special, temporary znode /controller becomes the leader. If that broker fails, its session expires, the znode is automatically deleted, and other brokers race to create it again, ensuring a new controller is always elected.
Interview question
What core problem does Apache ZooKeeper primarily aim to solve for distributed systems?
- a.Providing a high-performance, scalable database for large application datasets.
- b.Distributing computational workloads and managing task queues across a cluster.
- c.Ensuring consistent agreement on shared metadata like configuration, leadership, and service locations.Correct
- d.Facilitating asynchronous message passing and event streaming between services.
Why? this is the answer
ZooKeeper's primary role is to provide a consistent, highly available service for managing critical metadata, enabling distributed systems to agree on shared state for tasks like leader election and configuration. It is explicitly not designed for large application datasets or high-throughput operations, making option A incorrect.
Just read this? Test yourself on what you have been reading.
Read the original → en.wikipedia.org
- #distributed systems
- #coordination
- #apache
- #consensus
Put your scrolling time to good use
Learn one idea, try a quiz and save useful cards for revision. Tezvyn makes it easy to learn and stay current in your tech field, a few minutes at a time.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on distributed systems — each one lists the topics its interview covers.
See open roles