Every time someone opens a note in Inkwell, the same short journey happens. Almost every online system, from a to-do app to a bank, is built from the same three pieces. Learn them once and you'll recognise them everywhere.

Clients, app servers and databases

• A client is whatever the user holds: the Inkwell app on their phone, or a browser. It sends requests and waits for responses.
• An app server runs Inkwell's code. For each request it checks who is asking and whether the request makes sense, decides which data it needs, talks to storage, and builds the response.
• A database remembers. It keeps every note safe on disk so it survives restarts, and answers questions about them.

One request, from the phone to the data and back.

Clients never talk to the database directly. The database would be exposed to the whole internet, and every rule about who may read or change which note would have to live in every copy of the app. The app server is the gatekeeper.

App servers usually keep nothing between requests: they are stateless. Everything that must last lives in the database. Hold on to that idea; it's what will let you add more app servers on launch day.

Reads and writes

• A read fetches data without changing it: opening a note, scrolling the list of notes.
• A write changes data: saving a note, deleting one.

Both kinds of request take the same path, in very different numbers.

Inkwell's users open notes far more often than they save them, and most systems are like that: 10 to 100 reads for every write. Many of the techniques in later chapters only work because of that imbalance.

In retrystorm, each connection out of an app server carries R (reads), W (writes) or both. That's how you decide where each kind of request goes.

Latency: where the time goes

Opening a note pays for every step it takes: the hop to the app server, the app server's own work, the hop to the database, the database's work, and the same hops back. Inside one datacenter a hop costs a millisecond or two in this simulator (a real datacenter is faster, about half a millisecond for a round trip); a database lookup a few milliseconds more.

Requests don't all take the same time, so latency is given as percentiles: half the requests are faster than the p50 (the median), and 99% are faster than the p99.

The slow tail matters more than it looks: at 1 000 requests per second, 10 requests every second land beyond the p99.

Throughput and errors

Latency tells you how long one request takes. Two more numbers tell you how the whole system is doing:

• Throughput is how many requests complete per second. If 200 Inkwell users open a note every second and the backend keeps up, throughput is 200 requests per second. If it can only finish 150, the other 50 pile up somewhere.
• Errors are requests that failed. Real systems split them by whose fault it is. A client error (an HTTP 4xx status, like 404 Not Found) means the request itself was bad. A server error (a 5xx, like 503 Service Unavailable) means the backend couldn't answer. The failures you'll meet in retrystorm, all of them the backend's fault:
  • Rejected: something was full (no free thread, no free database connection) and turned the request away.
  • Timed out: the client gave up waiting. That one has no status code, since the client stopped listening; when a gateway gives up on the server behind it, it answers 504 Gateway Timeout.
  • Unanswerable: nothing in the path can answer it, like opening a note when no database is connected.

A design can be fast and still fail every request that needs data.

A healthy system keeps throughput equal to the traffic coming in, latency low, and errors near zero. In retrystorm, the Results panel shows all three after every run.

In the wild

The ideas in this chapter, and where you'll meet them in real systems and interviews:

Your first design

In Hello, Backend you'll give Inkwell its first backend: client → app server → database. The Must pass checks make sure saved notes really reach the database and opened notes really come from it. An app server with nothing behind it can answer quickly, but it can't remember a single note.