Opscraft
Caching & CDNs · Lesson 1 of 6 · 3 min read

Why and where to cache

Caching is the cheapest performance and resilience win in most systems, and the source of some of the strangest bugs: users seeing old pages, one user seeing another's data, or a database falling over the moment a cache restarts. Interviewers ask about it in system design rounds and in troubleshooting questions. This topic covers the layers of caching, the HTTP headers that control browsers and CDNs, how CDNs such as CloudFront work, application caching with Redis or Valkey, and the failure modes you need to design against.

The layers of caching

A request can be answered from a cache at several points before it reaches the database:

Enlarge diagrambrowserHTTP cacheCDN edgenearest POPproxyNginx, Varnishapp cacheRedis,Valkeydatabasesource oftruthLeft: fastest, hardest to take back. Right: slowest, always correct.
Figure 1. A request can be answered at any layer. Each hit saves every layer behind it.
Swipe sideways to see all of it, or tap to open it full screen.
Layer Example Typical TTL Who controls it
Browser HTTP cache, service worker Seconds to a year Cache-Control from your server
CDN edge CloudFront, Cloudflare, Fastly Seconds to days Headers plus CDN cache policy
Reverse proxy Nginx proxy_cache, Varnish Seconds to minutes Proxy config
Application Redis, Valkey, Memcached, in-process maps Seconds to hours Your code
Database Buffer pool, query cache, read replicas Automatic The database

The closer to the user, the faster and cheaper the hit, and the harder it is to take back a wrong answer: you can delete a Redis key instantly, but a response cached in a million browsers for a year stays there.

Hit ratio, freshness and what to cache

  • Hit ratio = hits ÷ (hits + misses). A CDN at 95% sends one request in twenty to the origin; dropping to 80% quadruples origin traffic. Track it per path type.
  • TTL (time to live) decides how long a copy counts as fresh. After that it is stale: the cache must check with the origin (revalidate) or fetch again.
  • Cache what is read often, changes rarely, and is the same for many users: images, scripts, product catalogues, rendered public pages, configuration, the results of expensive queries.
  • Don't share-cache personalised or sensitive responses (account pages, anything after login) unless the user is part of the cache key, and never cache Set-Cookie responses at a shared layer.
  • Every cache is a copy, so every cache is a consistency decision: how stale can this data be before it hurts someone?
Enlarge diagram99% hits: 1 in 100 reach the origin95% hits: 5 in 100 reach the origin80% hits: 20 in 100 reach the originBar length: requests the origin has to serve.
Figure 2. Small changes in hit ratio make big changes in origin load.
Swipe sideways to see all of it, or tap to open it full screen.