System Design - Caching Strategies

Wed, 15 Apr 2026 18:40:00 +0200

What is cache?#

Intermediary layer between client and data source
Stores results from heavy/expensive operations
Reduces latency and dependency load
Cheap but hard to do it right :)
Needs to be reconstructible – can only be cached what can be retrieved from the origin

Weak consistency
Eventual consistency
Extra layer of the golden source – cheap layer from the expensive layer
Challeges of guaranteeing aligment of cache and the data source
Writes need to update or invalidate cache
Question: how many times can the registration data for a single user change?
We need to ensure cache data remains up to date, as non-updated data can be a problem (i.e. returning an old profile picture when the user already updated)

Defines which items to remove once the cache storage reach its limit
LRU: removes the least recently used
- Assumes that if the item hasn’t been used recently, it will probably not be used in the future
LFU: removes the least frequently used
- Counter; counts the items that hasn’t been used frequently
FIFO: removes the oldest
- Removes the oldest key (the first key that was created)
- Not very performatic
RR: Random replacements
- Randomly removes a key
Mechnism of defense, but it costs a lot to perform; we try to prevent this using good cache validation approaches

Caching metrics
Cache Hit: data is in cache; so response is immediate
Cache Miss: data is not in cache; so we need to fetch from the source
- We want to decrease cache miss and increase cache hits
Hit Rate: cache hits / number of requests (miss + hits)
High hit rates = efficient system
Low hit rate = inneficient system

Memory cache (In-Memory)
Distributed cache (Redis, Memcached, etc.)
Database cache and data layers
- Cache-Aside (Lazy Loading)
- Write-Through (Double Write)
- Write-Behind (Lazy Writing)
- Distributed Content Cache (CDN Cache)

Write is done simultanously – once the data is stored in the DB, its immediatly written to cache
DB and cache
Ideal for when read operations needs to be done fast since the beginning
Consistency challenge
Combined with Cache-Aside (Fallback)