Invalidation keeps a cache fresh only when every write passes through it. When data changes somewhere the cache can't see, the only thing that limits how old its copies get is how long it keeps them.
Writes the cache never sees
In the caching chapter, every write went through the app server and the cache, so the cache could drop or update its copy. Real systems have other writers: a nightly import, a pricing job, an admin tool, another team's service. They write straight to the database.
With no TTL, a copy of a popular product can stay in the cache for as long as the cache runs, showing a price that changed long ago.
What a TTL bounds
A TTL (time to live) is how long the cache keeps a copy before fetching it again. It doesn't make every read fresh: a price that changed one second after it was cached is wrong for the rest of the TTL. What it guarantees is a ceiling. No copy is ever older than the TTL, plus the moment it took to fetch.
| TTL | A price can be out of date for up to |
|---|---|
| none | as long as the cache runs |
| 60 s | about a minute |
| 5 s | about 5 seconds |
| 0.5 s | about half a second |
So a TTL turns "how stale is acceptable?" from a hope into a number you can promise.
The price of a short TTL is the reads it stops catching. Below, items are read at 200 requests a second, and every cached copy lives for one TTL.
The cost of a short TTL
Every expiry is a miss, and every miss goes to the database. Halve the TTL and each popular product is fetched twice as often. The database's total load grows by less than that, since a product viewed less than once per TTL misses either way, but it grows. For a database that was sized for a high hit rate, a short TTL can be the difference between coping and falling over.
When a popular copy expires
A TTL expires every copy, including the most popular one. The moment the bestseller's copy expires, every request for it misses at once, and hundreds of identical queries hit the database before the first answer refills the cache. That's a cache stampede (or thundering herd).
| Fix | How it works |
|---|---|
| Request coalescing | the first miss fetches; the others wait for its answer |
| Jittered TTLs | each copy gets a slightly random TTL, so they don't expire together |
| Refresh ahead | refresh a popular copy just before it expires, or serve the old one while refreshing |
Redis and Memcached don't coalesce for you. The app does it, letting one request per key fetch while the others wait, or the cache hands out leases, as Facebook's memcache and Memcached's newer meta commands do. In retrystorm it's a setting on the cache.
The shorter the TTL, the more often this happens, which is one more reason to keep TTLs as long as the product allows.
Picking a number
Start from the product, not the cache: how old may this data be before someone is harmed or annoyed? Then check that the database can take the misses that TTL causes. If it can't, the answer isn't a longer TTL than the product allows; it's a bigger database, or getting the writes to go through the cache. For writes the cache never sees, the common answer is change data capture: a process reads the database's change log and drops the cached copy of every row that changes.
| Data | Acceptable age | TTL |
|---|---|---|
| Product descriptions | minutes | 5 to 10 min |
| Product prices | seconds | a few seconds |
| Stock left | none | don't cache, or invalidate on every write |
Different data in the same system usually gets different TTLs.
TTLs and invalidation together
The two aren't rivals. Invalidate on the writes you do see, so most copies are fresh right away, and keep a TTL as a safety net for the writes you don't, so nothing can stay wrong for long.
Invalidation for the writes you see; a TTL for the ones you don't.
In the wild
The ideas in this chapter, and where you'll meet them in real systems and interviews:
| Idea | Where you'll meet it |
|---|---|
| TTLs | Redis EXPIRE, HTTP Cache-Control: max-age |
| Serving stale while refreshing | HTTP stale-while-revalidate |
| Stampede protection | request coalescing in CDNs and in Facebook's memcache leases |
Your practice
Inkwell Shop's prices now change all day, written by a pricing job straight into a small database. Finance allows a price to be a few seconds out of date, never more than 5 seconds. Pick a TTL that keeps prices inside that limit without sending the database more misses than it can take.