System design glossary: 196 terms defined

Home › Reference › Glossary

System design glossary

196 terms, each defined in a line and linked to the chapter that teaches it.

Access pattern: One way the application reads or writes data, with how often it happens. Data is laid out to make the common ones cheap.
App server: Runs the application's code: checks each request, talks to storage and builds the response.
At-least-once delivery: A queue's promise that every job is delivered, possibly more than once.
Availability: The share of time or requests a system serves correctly, usually counted in nines.
B-tree: The sorted tree most databases use for an index. A lookup walks a few levels, and the tree gains a level only when the table grows by hundreds of times.
Back-of-the-envelope estimate: Rough arithmetic from users to requests per second, storage and servers, good to a factor of two.
Backlog: The jobs waiting in a queue. If the oldest keeps getting older, the workers can't keep up.
Bandwidth: How many bits a second a machine's network card can send.
Base62: Writing numbers with digits and both cases of letters, 62 symbols, so codes stay short.
Bloom filter: A small bit array that says a key is definitely not in a set, or maybe is. A maybe is sometimes wrong; a no is right every time.
Bottleneck: The resource that is full and limits the whole system. Relieve it and the next one appears.
Burn rate: How fast the error budget is being spent: 1 uses it exactly over the period, 10 empties it ten times sooner.
Cache: Keeps copies of recent answers in memory to serve repeats without asking the database.
Cache key: What a cache files each copy under, usually the URL.
Cache stampede: Many requests missing at once when a popular copy expires, all hitting the database together.
Cache-aside: The app checks the cache first and, on a miss, reads the database and fills the cache.
CAP theorem: During a network partition, a store can stay consistent or available, not both.
Causal consistency: If one change could have caused another, every copy shows them in that order; unrelated changes may appear in any order.
CDN: A content delivery network: caches in cities around the world, close to users.
Celebrity problem: A few writers with huge audiences make one post cost millions of copies, so they're handled differently from everyone else.
Change data capture: Reading a database's own log of committed changes and publishing them, so other systems hear of every change without the app sending it twice.
Checkpoint: The last moment a database's tables were known to be fully on disk. Recovery replays the log from there.
Circuit breaker: Stops calling a failing service for a while, failing fast or using a fallback instead.
Client: Whatever the user holds, an app or a browser, that sends requests and waits for responses.
Compaction: An LSM tree's background merge of its sorted files, which drops overwritten and deleted values.
Conditional write: A write that applies only if the data is still as the writer last saw it, such as a version number, so a repeat does nothing.
Connection: A database's slot for one conversation with an app server; each running query holds one.
Connection gateway: A component that holds each client's connection open on a lightweight slot, so one instance carries tens of thousands.
Connection pool: Open connections an app server keeps and reuses, counted against the database's limit.
Consensus: A protocol by which nodes agree on a value, such as who the leader is, by majority vote.
Consistency model: What a store promises a read can return while copies catch up.
Consistent hashing: Maps keys to machines on a ring, so the same key always lands on the same machine and adding one moves only its share of keys.
Consistent prefix: Seeing changes in the order they happened, so a reply never shows up before the comment it answers.
Consumer group: A service reading a stream, with its own position in each partition.
Consumer lag: How far a subscriber is behind: the messages waiting for it, or the age of the oldest one.
Count-min sketch: A small fixed table of approximate counts. It can count a key a little high, never low, and can't list its keys: a top K pairs it with a heap.
Covering index: An index that holds every column a query needs, so the database answers it without reading the table.
CRDT: A data type built so concurrent changes always merge the same way, such as a counter that adds every region's increments.
Cursor pagination: Paging by a marker for the last item seen, so new items arriving mid-scroll don't repeat or skip any.
Dead-letter queue: Where a job goes after failing every attempt, for someone to look at.
Deadline: How long a caller waits for a call before it stops waiting and goes on without it, or fails.
Deadlock: Two transactions each waiting for a lock the other holds. The database aborts one so the other can go on.
Denormalisation: Copying data next to where it's read, so a read skips a lookup. Every copy is another write when the fact changes.
Dirty read: Reading another transaction's change before it commits, when it may still be rolled back.
Double-entry bookkeeping: Recording each movement of money as two entries that sum to zero, so a total that doesn't add up exposes a bug.
Durable: A write that no crash, power cut or failover can lose once it was acknowledged.
Edge and origin: The edge is a CDN location near users; the origin is your own servers behind it.
Effectively once: Work that may be repeated but changes nothing the second time, because it's idempotent or its offset commits with its output.
Egress: The bytes a service sends out to readers, and what the cloud charges for them.
Embedding: Storing child items inside their parent, so one read returns both. Safe only for a list that can't grow without limit.
Error budget: The share of requests an SLO allows to fail, such as 0.1% for 99.9%.
Eventual consistency: Copies catch up eventually; right after a write, a read can return the old value.
Exactly-once effect: What at-least-once delivery plus a receiver that skips repeats gives: each message's effect happens once.
Exponential backoff: Waiting longer before each new retry: 1 s, 2 s, 4 s and so on.
Fail open and fail closed: What a limiter does when its shared store is down: let everything through, or refuse everything.
Failover: Promoting a replica to take over when the primary dies, then sending writes to it.
Fan-out (calls): One request that calls several services at once, and waits for the slowest.
Fan-out on read: Building a feed when it's opened, from the latest posts of everyone followed.
Fan-out on write: Copying each new post into every follower's feed when it's published.
Fencing: Cutting the old primary off before promoting a replica, so two nodes can't both accept writes.
Fencing token: A version number that grows with each new hold or lock. A write carrying an older one is refused.
Fixed window: Counts requests per calendar second or minute; simple, but lets up to twice the limit through at a boundary.
fsync: Asking the operating system to flush a file's writes to disk and waiting until it has.
Functional requirements: What a system does: the features.
Geohash: A latitude and longitude encoded as a string in which nearby points mostly share a prefix, so a cell can be found with an ordinary index.
Golden signals: Latency, traffic, errors and saturation: four numbers to start with for any service.
Goodput: The answers someone actually received in time, as opposed to work done for callers who already left.
Graceful degradation: Showing less rather than nothing when part of the system is slow or down.
Handshake: The work of setting up a new connection, which takes a share of an instance's cores before the client is connected.
Headroom: Spare capacity kept for bursts, usually by running at 60 to 75% of the limit.
Health check: A load balancer's regular probe of each target; failing ones are taken out of rotation.
Heartbeat: A small periodic message that lets each side notice a connection that has died.
Hedged request: A second copy of a slow call, sent after a short wait, taking whichever answers first. Only for calls that are safe to repeat.
Hit rate: The share of requests a cache answers itself.
Hold: A short-lived reservation, such as a seat kept for one buyer for ten minutes, taken in one transaction and freed when it expires.
Hot key: One record that draws more traffic than one machine can serve.
Hot partition: A partition that throttles while the others sit idle, because a hot key lives on it.
HTTP 429: Too Many Requests: the answer to a caller over its rate limit, usually with a Retry-After header.
Hybrid feed: Pushing posts of ordinary writers into followers' feeds, and pulling the posts of writers with huge audiences when a feed is opened.
Idempotency key: A unique id a client sends with a write, so the server can spot and skip a repeat.
Idempotent: A request that leaves the same result whether it runs once or twice.
Index: A sorted lookup structure beside a table, so the database finds rows without scanning them all.
Invalidate: On a write, drop the cached copy so the next read fetches the new value.
Inverted index: For every word, the list of documents that contain it. What full-text search reads instead of scanning.
Isolation level: How far a transaction is shielded from the others running at the same time.
Jitter: A random part of each retry's wait, so callers that failed together don't come back together.
Key-value store: A store that gets and puts values by a single key, and splits its keys across machines.
Last writer wins: Settling conflicting writes by keeping the one with the newest timestamp. The other is dropped without an error.
Latency: How long one request takes, end to end.
Latency budget: A target for the whole request, with a deadline for each call inside it.
Leaky bucket: A rate limiter that queues requests and lets them out at a fixed rate, refusing only when the queue is full.
Lease: A time-limited right. A cache uses one to refuse an answer fetched before the latest invalidation; a database primary uses one as its permission to accept writes.
Least connections: A load-balancing strategy that picks the replica with the fewest requests in flight.
Linearizable: A store that behaves as if there were only one copy: every read returns the latest acknowledged write. The C in CAP.
Little's Law: Requests in progress equal the arrival rate times how long each one takes.
Load balancer: Sits in front of replicas and picks one for each request.
Load shedding: Turning excess requests away on purpose, early and cheaply, so the rest are served well.
Lock: A claim a transaction holds on a row until it commits or aborts. Another transaction that needs the row waits for it.
Log (stream): An append-only sequence of messages, kept in order within each partition for a retention window. Each consumer group reads it at its own pace and can replay it.
Long polling: The client asks, and the server holds the request open until there's news.
Lost update: Two overlapping read-modify-writes on one row, where the second overwrites the first without seeing it.
LRU: Least recently used: evicts the item nobody has asked for in the longest time.
LSM tree: A store that writes to memory and appends sorted files, merging them in the background. Cheap writes, reads that may check several files.
Message queue: Stores jobs durably and acknowledges them at once, so workers can do the work later.
Message TTL: How long a copy may wait before it is dropped instead of delivered late.
Monotonic reads: A promise that a user never sees data go back in time between reads.
Multi-leader replication: Each region accepts writes for the same data and copies them to the others, so two regions can take conflicting writes that must be settled.
Multipart upload: Sending a big file in parts, so a lost part is sent again on its own instead of the whole file.
Non-functional requirements: How well a system does it: scale, latency, consistency, durability, availability.
Normalisation: Storing each fact in one place and pointing to it by key, so a change is one write and joins rebuild the views.
Object storage: Stores files (objects) in buckets, with no sizing, priced per request and per byte.
Optimistic concurrency: Taking no lock, and checking at commit that the row is unchanged, restarting when it isn't.
Origin shield: A CDN layer between the edges and the origin, so many edges missing the same file cost the origin one fetch.
p50 and p99: Percentiles of latency: half the requests are faster than the p50, 99% faster than the p99.
PACELC: Extends CAP: without a partition, the everyday trade is latency against consistency.
Partition: A slice of a store's keys, served by one node at a time (with copies on others), with its own throughput budget.
Partition key: The part of a key that decides which partition holds an item. Items with the same partition key are read together.
Phantom: A row that matches a query appears between two runs of it inside one transaction.
Point-in-time recovery: Restoring a backup plus its log up to a chosen moment, such as just before a bad delete.
Presigned URL: A short-lived link that lets a client upload or download one object directly.
Primary: The database node that takes every write and copies it to the replicas.
Publish/subscribe: Sends each message to every subscriber, so several services react to one event.
Purge: Removing a CDN's cached copy before its TTL runs out, usually after the content changed. It takes from a fraction of a second to minutes to reach every edge, depending on the CDN.
Quadtree: A map split into four, and the busy parts split again, so a dense city gets small cells and an ocean one large one.
Query planner: The part of a database that chooses how to run a search, such as a table scan or an index, from statistics about the data.
Quorum: The number of copies an operation must reach to count. A write waits for W of the N copies and a read asks R; when R + W > N, every read overlaps the latest acknowledged write.
Raft: A consensus protocol: nodes elect one leader per term, and an entry is committed once a majority has stored it.
Range partitioning: Giving each partition a range of keys, which keeps range scans together and risks hot ranges.
Rate limiter: Caps how many requests each caller may send, refusing the rest at once with HTTP 429.
Read and write: A read fetches data without changing it; a write changes it. Most systems see 10 to 100 reads per write.
Read committed: An isolation level that shows only committed data, as of each read. The usual default, and it still allows lost updates.
Read repair: Fixing a replica that returned an old value by writing the newest one back to it during a read.
Read replica: A copy of the database that serves reads; writes go to the primary.
Read your writes: A promise that a user always sees their own changes.
Read-through cache: A cache that sits between the app and the database and fetches its own misses, so the app only ever asks the cache.
Rebalance: Handing a consumer group's partitions out again when a consumer joins or leaves. Each partition resumes from its committed offset.
Reconnect storm: Many clients retrying a connection at once after an outage, queueing for the same handshakes.
Redundancy: Running spares of each part, so one can fail while another serves.
Replay: Reading a stream from the beginning of what it still holds, to fill a new service or redo work.
Replication lag: The delay before a replica has a write the primary already has.
Retention: How long a stream keeps a message, whether or not it has been read.
Retry: Sending a failed request again, ideally after a backoff.
Retry storm: Retries of timed-out requests piling onto an overloaded server, keeping it down after the rush ends.
Round robin: A load-balancing strategy that hands requests to replicas in turn.
RPO and RTO: How much recent data may be lost in a failure, and how long recovery may take.
Saga: A change across several databases made as a series of local transactions, each with a compensation that undoes it if a later step fails.
Saturation: How full a component is, such as its queue depth, busy threads or connections in use. It moves before errors do.
Scale out: Adding more copies of a server, called replicas (horizontal scaling).
Scale up: Moving to a bigger machine (vertical scaling).
Selectivity: The share of a table's rows a search returns. The smaller the share, the more an index is worth having.
Serializable: An isolation level that behaves as if transactions ran one at a time, by locking or by aborting one of two that conflict.
Single point of failure: Any part whose loss takes the whole system down.
SLI, SLO and SLA: What you measure, the goal for it, and a promise to customers with a penalty attached.
Sliding window: Counts the requests in the last full window wherever it ends, so no span holds more than the limit.
Sloppy quorum: Sending writes to stand-in nodes outside the N while a copy is down. Writes keep working, but reads no longer surely overlap them.
Snapshot isolation: An isolation level that shows the data as it was when the transaction began, and aborts it if it changes a row someone else changed since.
Snowflake ID: A 64-bit id made of time, machine number and sequence, roughly in time order.
Sort key: The part of a key that orders items within a partition, so a range of them comes back in order in one read.
Split brain: Two nodes both acting as primary and accepting writes; prevented by fencing the old one.
Stale read: A read that returns a value which has since changed.
Stateless: A server that keeps nothing between requests, so any copy of it can serve any request.
Static and dynamic content: Static content is the same for everyone and caches well; dynamic content is built per user.
Sticky sessions: Sending every request from a user to the same server, because that server holds their session.
Stream: An ordered, partitioned log that keeps its messages until retention ends, so any number of groups can read them at their own pace.
Strict serializable: Serializable and linearizable at once: transactions behave as if run one at a time, in an order that matches real time.
Strong read: A read that always returns the latest acknowledged write, at a higher cost.
Sub-counters: One counter split across several keys, summed on read, so writes spread out.
Subscriber: A service that gets its own copy of every message on a topic, and works through them at its own pace.
Synchronous commit: Acknowledging a write only once its log entry is flushed to disk.
Synchronous replication: Acknowledging a write only once at least one replica has it.
Table scan: Answering a query by reading every row of a table, at a cost that grows with its size.
Tail latency: The slowest requests, the p99 and beyond. It grows with every call a page waits on.
Throughput: How many requests complete per second.
Timeout: How long a caller waits for an answer before giving up.
Token bucket: A rate-limiting algorithm: tokens refill at the limit, each request takes one, and the bucket size allows short bursts.
Top K: The K largest items of a set, such as the ten most-liked posts of the hour.
Topic: A component that takes each published message once and puts a copy on every subscription's own backlog.
Transaction: A group of reads and writes that take effect together or not at all.
Transactional outbox: Saving a message in an outbox table in the same transaction as the data, for a relay to publish afterwards, so the save and the send can't come apart.
TTL: Time to live: how long a cached copy is kept before it's fetched again.
Tumbling window: A fixed slice of time a stream is counted in, such as every five minutes, one after the other.
Two-phase commit: A coordinator asks every database to prepare a change, then tells them all to commit or all to roll back. It blocks if the coordinator fails in between.
Utilization: How much of a limit (CPU, threads, connections) is in use. Waiting grows sharply as it nears 100%.
UUID: A 128-bit id that needs no coordination; the common version 4 is random. 36 characters in its usual form, 22 in base62.
Virtual nodes: Many small stretches of the hash ring per machine, which spread the keys evenly under consistent hashing. A hot key still lands on one machine.
Visibility timeout: How long a queue waits for a worker to finish a job before handing it out again.
Watermark: A stream consumer's running estimate of how far event time has got: no event older than it is expected any more. A window closes once the watermark passes its end.
WebSocket: One connection that stays open both ways, so the server can send the moment something happens.
Worker: A process that takes jobs from a queue and does them in the background.
Working set: The items asked for often enough to be worth caching.
Write coalescing: Combining the writes to one key that arrive within a short window into a single write, so a hot key takes far fewer updates at the cost of a short delay.
Write skew: Two transactions each check something the other then changes, so both commit and together break a rule.
Write-ahead log: An append-only log of changes, flushed before a write is acknowledged and replayed after a crash.
Write-back: Write to the cache only and save to the database later: fast, but a crash can lose writes.
Write-through: On a write, update the cached copy along with the database.
Zipfian distribution: Traffic where a few items get most of the requests and most get almost none.