Implementing Raft: Part 0 - Introduction - Eli Bendersky's website

What Raft Solves

Raft is a distributed consensus algorithm first published in 2014. It is already widely used in industry, most notably in Kubernetes through the etcd distributed key-value store. This series of posts walks through a complete, rigorously tested implementation of Raft in Go, and provides intuition for how the algorithm works along the way. That said, this series is not meant as a sole learning resource: you should read the Raft paper at least once, and ideally also explore the resources on the Raft website, watch a talk by the creators, play with the visualization, or skim Ongaro's PhD thesis.

Do not expect to fully grasp Raft in a single day. Even though it was designed to be easier to understand than Paxos, Raft is still a fairly complicated algorithm. Distributed consensus is a hard problem, and there is a natural lower bound on the complexity of any correct solution.

Replicated state machines

At a high level, distributed consensus algorithms solve the problem of replicating a deterministic state machine across multiple servers. A state machine here just represents an arbitrary service — databases, file servers, lock servers, and so on can all be modeled this way. Multiple clients connect to the service, issue requests, and expect responses:

This works fine as long as the server is reliable. If it crashes, the service becomes unavailable, and the whole system is only as reliable as that one server. Replication increases reliability: run several instances of the service on different servers, forming a cluster where no single server crash brings the service down. Isolating servers from each other also removes common failure modes that would affect multiple servers simultaneously.

Instead of contacting one server, clients contact the cluster as a whole. The service replicas also communicate among themselves to keep state in sync:

Each state machine in this diagram is a replica of the service. The idea is that all replicas execute in lockstep, taking the same client requests and performing the same state transitions, so they return the same results even if some servers fail. Raft is an algorithm that achieves this.

Some terminology used throughout this series:

• Service: the logical task of the distributed system, e.g. a key-value database.
• Server or Replica: one instance of the Raft-enabled service running on an isolated machine with network connections to other replicas and clients.
• Cluster: the set of collaborating Raft servers that implement the distributed service. Typical cluster sizes are 3 or 5.

Consensus module and the Raft log

Raft is generic about how the service itself is built. What it provides is a reliable, deterministic way to record and reproduce the sequence of commands (inputs) that a state machine receives. Given an initial state and the full command sequence, a state machine can be replayed with complete fidelity: two separate replicas fed the same inputs from the same initial state will reach the same state and produce the same outputs.

Here is the internal structure of a single Raft-enabled server:

The components are:

• The state machine represents the actual service, e.g. a key-value store.
• The log stores all client commands. Commands are not applied to the state machine immediately; Raft only applies them once they have been replicated to a majority of servers. The log is persistent, surviving crashes, and can be used to replay the state machine after a restart.
• The consensus module is the core of Raft. It accepts commands from clients, appends them to the log, replicates them to other servers in the cluster, and commits them to the state machine when it is safe to do so. Committing notifies clients of the change.

If this feels abstract, that is expected — the remainder of this series fills in the details.

Leader and follower roles

Raft uses a strong leadership model. Each cluster has a single leader, and the rest are followers. The leader handles all client requests, replicates commands to followers, and returns responses to clients. During normal operation, followers simply replicate the leader's log. If the leader fails or is partitioned away, one of the followers can take over leadership to keep the service available.

This model has notable trade-offs. The key advantage is simplicity: data flows in one direction (leader to followers), only the leader serves clients, and the system is easier to analyze, test, and debug. The main disadvantage is performance — a single leader can become a bottleneck under heavy client traffic. Raft is therefore not a good fit for traffic-heavy services; it is better suited to low-traffic scenarios where consistency is critical, even if that costs some availability.

How clients find the cluster

The phrase "contact the cluster" can be misleading — a cluster is just a group of networked servers. In practice:

• The client knows the network addresses of all replicas (through service discovery or similar, though that is out of scope here).
• The client sends its request to an arbitrary replica. If that replica is the leader, it acknowledges the request, and the client waits for the response. The client then remembers which replica is the leader, so it does not have to search again unless the leader fails.
• If the replica is not the leader, the client tries another replica. A useful optimization: since replicas communicate regularly, a follower can tell the client which replica is the leader, saving the client a few guesses.
• The client may also detect it is talking to a non-leader if its request is not committed within a timeout. This can happen when the replica it contacted was partitioned from the rest of the cluster, though the replica still thinks it is the leader. When the timeout fires, the client resumes searching for a leader.

The timeout-based detection and the leader-redirection optimization only matter during fault scenarios. In normal operation a Raft cluster spends well over 99.9% of its time with clients who already know the leader from first contact. Fault scenarios — covered in the next section — create a temporary blip of uncertainty, but a Raft cluster recovers from a server crash or network partition very quickly, typically in well under a second, after which normal operation resumes.

Fault tolerance and CAP

Consider a cluster of three Raft replicas with no attached clients:

What failures can occur? Treating each server as an atomic unit, there are two main failure kinds:

1. Server crash: a server stops responding to all traffic, then is typically restarted and comes back online.
2. Network partition: one or more servers are cut off from other servers or clients by faulty networking equipment or media.

To a server A, a crash of server B is indistinguishable from a network partition between A and B — both look like A stops receiving messages from B. Partitions are more insidious system-wide because they affect multiple servers simultaneously, and this series covers several such tricky scenarios in later parts.

To keep making progress despite arbitrary crashes and partitions, Raft requires that a majority of the cluster's servers be up and reachable by the leader at any given moment. A 3-server cluster tolerates one failure, a 5-server cluster tolerates two, and in general a 2N+1 server cluster tolerates N failures.

This directly reflects the CAP theorem. Since network partitions are unavoidable, the real trade-off is between availability and consistency. Raft firmly sides with consistency: its invariants prevent the cluster from ever reaching an inconsistent state where different clients get different answers, even if that means the cluster becomes briefly unavailable during a partition.

That choice has performance consequences. Every client request triggers a fair amount of work — replicating the command to a majority of servers and persisting it — before the client gets a response. You would not build a high-throughput replicated database on Raft; it would be too slow. Raft is far better suited to coarse-grained distributed primitives: lock servers, leader election for higher-level protocols, critical configuration replication, and similar tasks where consistency matters more than raw throughput.

Why Go Fits Raft Implementation

The Raft code in this series is written in Go. Three aspects of the language make it especially suited for this kind of work and for networked services broadly:

1. Concurrency: Raft is a fundamentally concurrent algorithm. Every replica juggles its own ongoing operations, fires timed events, and responds to asynchronous messages from peers and clients. Go’s concurrency model matches this shape of problem well.
2. Standard library: For Raft specifically, the first practical question is how replicas exchange messages. Go’s net/rpc package provides a straightforward answer without dragging in third-party dependencies, custom serialization, or protocol design work. The rest of the standard library is similarly robust for building networked services quickly.
3. Simplicity: Distributed consensus is hard enough on its own. Go’s idioms push toward clear, minimal code, which matters more as the algorithm’s complexity piles up.

Next Steps

The conceptual model of Raft can look deceptively tidy, but implementation detail is where the traps hide. The upcoming parts of this series dig into each piece of the algorithm with concrete code.

Part 1 starts building the core election mechanics, and it serves as the foundation for everything that follows.