retrystorm · Learn · Interview practice · Glossary · Components

Home › Interview practice › Spread the Feed

Spread the Feed

Thousands of new posts a second and one viral post, on a store with small partitions.

Inkwell keeps every post in a key-value store. Writers publish about 3 000 posts a second, and one post has gone viral: it draws about a fifth of all reads. The store's plan gives small partitions (5 000 reads and 1 000 writes a second each), and it keeps throttling.

What it does

  • Write: a writer publishes a new post (a new key every time).
  • Read: a reader opens a post. Most reads go to a handful of popular posts, one far more than the rest.

Workload

A steady 30 000 rps, 90% reads / 10% writes, for 20 seconds, against 100 000 posts.

How to build it

  1. Build client, load balancer, app server, KV store. The app servers are fixed and plenty.
  2. Run it with one partition and look at what fails.
  3. Add partitions. Then look again: is anything still throttling, and would more partitions fix it?

Part of Chapter 27: Partitioning.

About · Contact · Privacy · Terms of use