Distributed Systems also: HW

High water mark

Highest offset acknowledged by all in-sync replicas.

In plain terms

Kafka concept. Consumers can only read up to the HW; this is what makes the log durable. HW lags the leader by one round-trip.

Origin

Term comes from the Kafka KIP-101 work (2017). Generalises a much older durability concept — the highest log offset safely replicated to a quorum.

Where it shows up in production

Apache Kafka Consumers can only read up to the HW. HW lags the leader by one round-trip to the slowest ISR.
Raft commitIndex plays the same role — the highest entry replicated to a majority.

On Semicolony

Sources & further reading

Found this useful?