Inkwell keeps every post in a key-value store. Writers publish about 3 000 posts a second, and one post has gone viral: it draws about a fifth of all reads. The store's plan gives small partitions (5 000 reads and 1 000 writes a second each), and it keeps throttling.
What it does
- Write: a writer publishes a new post (a new key every time).
- Read: a reader opens a post. Most reads go to a handful of popular posts, one far more than the rest.
Workload
A steady 30 000 rps, 90% reads / 10% writes, for 20 seconds, against 100 000 posts.
How to build it
- Build client, load balancer, app server, KV store. The app servers are fixed and plenty.
- Run it with one partition and look at what fails.
- Add partitions. Then look again: is anything still throttling, and would more partitions fix it?