When reads outgrow one database node and it can't get any bigger, the answer is more copies of the data. Replication lets several machines share the reading, at the price of copies that are a moment behind.

Read replicas

A read replica is a copy of the database that serves reads. All writes still go to one node, the primary. After each write, the primary sends the change to every replica. You'll also hear the primary called the leader and the replicas followers; the labs use those names.

Writes go to the primary; reads spread across the replicas.

Reads are spread across the replicas. Two replicas share the reads between them, while the primary keeps the writes.

Replicas don't scale writes

Every write still lands on the primary, and every replica has to apply it too:

Two replicas share the reads, but each one replays every edit.

With two replicas, Inkwell Blogs' 600 edits per second become 1 800 applied writes across the three machines. Reads get spread out; writes get multiplied.

A replica replays a write for far less than the primary paid to make it: it skips parsing, planning and talking to a client, and only applies the change from the log, about a tenth of a millisecond of CPU here. At 600 edits a second that's 6% of one core. Two things can still leave it behind. A PostgreSQL replica replays on a single process, one write at a time, so however many cores it has, its replay has a ceiling: this simulator puts it at 10 000 simple writes a second, and real ones land in the thousands to tens of thousands, fewer with indexes to update and on slow disks: a big primary in the middle of a bulk import can outrun it. And the replay shares the replica's cores with its reads, so a replica kept busy by readers replays each write later.

That's the right trade for Inkwell Blogs, where reads far outnumber edits. For a write-heavy system, adding replicas adds work rather than removing it, and you need a different tool: splitting the data across machines, which you'll meet in the Partitioning chapter.

Replication lag

For 20 ms after the edit, the replica still serves the old version.

Copying a change to the replicas takes time: shipping it, then replaying it. That delay is replication lag, usually a few to a few hundred milliseconds. During it, a replica still has the old version of the post.

Usually that's fine. A reader who sees a post 50 ms late will never know. This is eventual consistency: replicas catch up, eventually.

Lag isn't the same for every replica, and it isn't steady. It stays small while a replica keeps up. Once writes arrive faster than it can replay them, or its reads keep every core busy so the replay waits its turn, it falls further behind every second, and every read that lands on it is that stale. A replica far away adds the distance on top.

Read your own writes

It's not fine for the author who just made the change. They save an edit, the page reloads, and the read lands on a replica that hasn't caught up: their edit seems to have vanished. That's a read-your-writes violation.

The common fix is to send reads of recently changed data to the primary for a little longer than the usual lag. You can do this per row (any read of a row written in the last moment) or, more often, per user (the author's own reads, right after they write). The price is load on the primary: every read in that window lands on the one machine that also takes every write, so the window should be just longer than the lag, not much more. A fixed window breaks when the lag spikes past it. The sturdier fix is a version: the app remembers how far into the log the author's last write was, and reads from a replica only once it has replayed that far.

In the wild

The ideas in this chapter, and where you'll meet them in real systems and interviews:

Your practice

Scale Reads with Replicas: keep Inkwell Blogs up with replicas, then make sure authors always see their own edits.