A morning TV show featured Inkwell's daily writing prompt. All day, about a third more people open the app than the two app servers can take, and new servers won't be approved until tomorrow.
The app gives up on a request after one second and doesn't retry. Each app replica lets 2 000 requests wait in its queue, so the wait grows past a second and almost every request times out.
What it does
- Read: a user opens today's prompt or a note.
- Write: a user saves a note.
Workload
A steady 3 600 rps for 15 seconds, 90% reads / 10% writes, against 10 000 notes. Two medium app replicas take about 2 700 rps between them.
How to build it
The servers are fixed: two app replicas behind a load balancer, and a database big enough for all of it. The app's timeout is fixed too. What you can change is how many requests each app replica lets wait: select the app server and change its Max queue / replica.
- Build the path: client, load balancer, app server, database. Run it as it is.
- Compare the throughput with what the servers can take. Where did the rest go?
- Change the queue until the requests that get in are fast.