Distributed Systems

Leader epoch

Monotonically-increasing term number for each elected leader.

In plain terms

Kafka KIP-101. New leader's first action is to fence out the old leader by epoch. Closes the "ghost record" data-loss window.

Origin

Kafka KIP-101 (2017) introduced leader epochs to close the "ghost record" data-loss window. The general pattern — fence the old leader by term — appears in Raft and ZooKeeper too.

Where it shows up in production

Kafka Every leader gets a monotonically-increasing epoch. New leader's first action: fence out the old leader by epoch.
Raft term Same concept under a different name. Term is part of every RPC.

On Semicolony

Guide kafka

Sources & further reading

Docs KIP-101 — Kafka leader epoch

Found this useful?