Home › Reference › Components › Cache

Cache

In-memory LRU cache sitting between app server and store.

Settings

Cache tier

Sets the default cache size (entries held) and how many lookups per second one cache node can serve. Past that, calls queue, and any call waiting over a second is refused. A bigger tier does not make lookups faster.

Capacity (entries)

How many entries the cache holds before it drops the least recently used one. Larger means a higher hit rate and less load on the store. Leave blank to use the tier's default.

Hit latency

How long the cache takes to answer a read it already has, drawn at random per hit. Shapes: constant (always the same), lognormal (mostly near the mean, some slow), exponential, uniform (between min and max).

Miss overhead

Extra time a read spends in the cache when it misses, before being passed on to the store. Adds to total latency for every miss. Shapes: constant, lognormal, exponential, uniform.

Write strategy

What happens to the cached copy when a key is written. Invalidate removes it so the next read refetches. Write-through updates it once the store accepts the write. Bypass leaves it alone, so it can serve stale data until it expires.

TTL (ms)

How long an entry stays valid after it is stored. Expired entries count as misses and are refetched. Shorter means fresher data but a lower hit rate. Blank means entries never expire.

Misses on one key share a fetch

When several reads miss on the same key at once, only one goes to the store and the others wait for its answer. This stops a popular key from flooding the store when its entry expires or is evicted.

Crashes

Scheduled cache crashes. While down, requests through it fail. When it restarts it is empty, so the store takes a burst of load while the cache warms up again.