When a database can't keep up with its reads, there are two common fixes: a cache in front of it, or read replicas behind it. Both take reads off the primary, but they suit very different traffic.
Two ways to scale reads
A cache keeps the answers to recent reads and serves repeats without asking the database. A read replica is a full copy of the database that serves reads itself.
| Cache | Read replicas | |
|---|---|---|
| Holds | the items read recently | everything |
| Takes off the database | only the reads it hits | every read, from the primary |
| Cost grows with | the working set | the number of reads |
| Stale reads come from | entries not yet invalidated or expired | replication lag |
The deciding question is the one from sizing a cache: how many of the reads would the cache actually hit?
Hit rate decides
A cache only helps with reads it has seen before. When a few items get most of the traffic, a small cache holds them and catches most reads. When every item is read about as often as any other, a cache that holds 5% of the items catches about 5% of the reads, and the database still sees the other 95%.
When the cache wins
Skewed traffic flips the answer. A cache of a few thousand items catches half the reads for $0.10 an hour, while every extra replica costs as much as the primary. For the same result, the cache is far cheaper.
So before choosing, look at the traffic: how concentrated it is decides the hit rate, and the hit rate decides whether a cache is worth its price.
Freshness and speed
The two also differ in what readers see. A replica applies each write a little after the primary, so a read right after an edit can get the old version. A cache that's invalidated on every write drops the old copy at once, as long as its misses read the primary, apart from the small race from Caching: a miss already on its way when the write lands can put the old value back, which a lease prevents. A miss filled from a lagging replica can put the old value back, and it stays until its TTL runs out. And a cache hit skips the database entirely, so it's faster than any database read.
Neither scales writes
Every write that must not be lost still lands on the primary. A write-back cache can batch writes you could afford to lose, like view counters, but orders and edits still go to the database one by one, and a replica only adds the work of copying them. When writes are the problem, the fix is elsewhere: partitioning, or a store built for them.
| The reads are... | Usually choose |
|---|---|
| Concentrated on a few items | a cache |
| Spread evenly over everything | read replicas |
| Both, at large scale | a cache for the popular items, replicas for the rest |
| Mostly writes | neither: fix the writes |
In the wild
The ideas in this chapter, and where you'll meet them in real systems and interviews:
| Idea | Where you'll meet it |
|---|---|
| Caches | Redis, Memcached |
| Replicas | Amazon Aurora, with up to 15 read replicas sharing one storage layer |
Your practice
Inkwell Blogs now has an archive of 200 000 old posts, and readers from search engines land on a different one every time. The database can't keep up, and the database team wants a big cache in front of it. Make the archive fast, and spend the money only where it takes load off the database.