Caching Strategies and Distributed Caching Questions
Using caches to reduce latency and load: cache-aside, read-through, write-through, and write-behind patterns, TTLs, eviction policies, and distributed caches such as Redis or Memcached. Covers cache invalidation, stampede and thundering-herd protection, and the consistency tradeoffs of caching. Focuses on where and how to cache across tiers.
At extreme scale, a single cache miss for a hot key can overload the origin. Propose a comprehensive defense-in-depth strategy to prevent stampedes: singleflight, background regeneration, early recompute, probabilistic TTLs, prewarmed hot key paths, and rate limiting. Explain how to orchestrate these across many app instances.
Sample Answer
Direct answer
At extreme scale you cannot rely on a single stampede defense. Combine request coalescing (only one request repopulates a hot key while others wait), background/proactive refresh before expiry, probabilistic early expiration, jittered time-to-live (TTL) values, and origin rate limiting as a last-resort backstop, coordinated so all application instances agree on who is allowed to refresh a given key at once.
Structured elaboration
- Request coalescing (singleflight): on a cache miss, the first request acquires a short-lived lock (e.g.,
SETNXin Redis) for that key and fetches from origin; concurrent requests for the same key either block briefly on a notification channel (pub/sub or polling with backoff) or serve a stale value if one exists. This bounds concurrent origin load per key to roughly one in-flight fetch, regardless of instance count, because the lock lives in the shared cache, not in any one process. - Probabilistic early expiration: instead of a hard expiry, each read close to TTL end recomputes with a small, increasing probability (a common formula is P(refresh)=e−β⋅(texpiry−tnow)/δ where δ is the time it took to compute the value and β tunes aggressiveness). This spreads refreshes across many requests instead of concentrating them at the exact expiry instant.
- Jittered TTLs: add randomized jitter (e.g., base TTL plus/minus 10 to 20 percent) so keys written around the same time do not all expire in the same millisecond, which is what turns an ordinary cache miss into a correlated stampede across thousands of keys at once (a cache avalanche).
- Background/prewarmed refresh: a scheduled worker (or the request that detects "close to expiry") refreshes hot keys proactively so the TTL rarely actually lapses for high-traffic keys; this trades a small amount of continuous background load for eliminating stampede risk on the hottest paths.
- Origin rate limiting as a backstop: even with the above, cap concurrent origin requests per key (or per origin endpoint) so a defense-in-depth failure degrades gracefully into serving stale data or a fast error instead of taking the origin down.
- Orchestrating across many app instances: the lock, the "who refreshes next" decision, and the notification of waiters must live in the shared cache (Redis) or a coordination service, not in in-process state, because coalescing only works if every instance agrees on a single winner per key.
Worked example
Say a hot key normally takes 200 ms to recompute and serves 5,000 requests per second (RPS) at peak. Without any defense, if it expires with no coalescing, the next ~1,000 requests in that 200 ms window (5,000 RPS times 0.2 s) would all miss and hit the origin simultaneously; a database that comfortably serves single-digit concurrent queries per second for that expensive query falls over. With coalescing, exactly 1 request recomputes and the other ~999 either wait ~200 ms for the notification or receive the previous (slightly stale) value immediately; origin load for that key stays at 1 concurrent request regardless of RPS.
Trade-offs and pitfalls
A lock that never expires on a crashed refresher permanently blocks that key; always set a lock TTL slightly longer than the expected recompute time, plus a fallback path that lets a waiter give up and fetch directly after a bounded wait. Serving stale-while-refreshing is a correctness trade-off, not a free win: it is the right default for read-heavy, staleness-tolerant data, and the wrong default for a low-latency-but-must-be-fresh field like an account balance. Jitter alone does not help an already-hot key that is legitimately read far more than others; that is a hot-key sharding problem, not a stampede problem, and needs a different fix (splitting the key, adding a replica-backed local cache).
Describe how HTTP caching works using Cache-Control, ETag, and Last-Modified headers. Explain how a CDN and a Service Worker might interact with those headers and describe a conditional GET flow including a 304 response. Provide one example where a Service Worker should bypass CDN semantics.
Sample Answer
Direct answer
HTTP caching is coordinated through response headers: Cache-Control tells any cache (browser, content delivery network, CDN) how long and under what conditions a response can be reused, and ETag/Last-Modified let a client revalidate a possibly-stale cached copy cheaply via a conditional request instead of re-downloading the full response.
Structured elaboration
- Cache-Control: directives like
max-age=<seconds>(how long the response is fresh),no-cache(must revalidate before use, but can still be stored),no-store(never cache at all), andprivate/public(whether shared caches like a CDN may store it) give fine-grained control over caching behavior. - ETag: an opaque identifier (often a hash) for a specific version of a resource; a client that has a cached copy sends
If-None-Match: <etag>on its next request. - Last-Modified: a timestamp-based alternative to ETag; a client sends
If-Modified-Since: <timestamp>. - Conditional GET / 304 flow: the client's cached copy has expired (its
max-ageelapsed) but might still be valid; instead of re-fetching the full body, it sends a conditional request withIf-None-MatchorIf-Modified-Since. If the resource has not actually changed, the server responds304 Not Modifiedwith no body, and the client's existing cached copy is revalidated as fresh; if it has changed, the server responds normally with the new body and headers. - CDN and Service Worker interaction: a CDN typically respects
Cache-Controldirectly, serving from its edge cache without contacting the origin untilmax-ageexpires. A Service Worker sits in the browser and can implement its own caching logic (including bypassing normal HTTP semantics entirely) via the Cache API andfetchevent interception, which is useful when the app needs offline support or caching behavior more sophisticated than standard headers allow. - When a Service Worker should bypass CDN semantics: for content the app knows is safe to serve instantly from a local cache regardless of what a content delivery network (CDN)'s time-to-live (TTL) says (e.g., app shell assets for instant repeat-visit loads), a Service Worker can serve from its own cache first and revalidate in the background (a stale-while-revalidate pattern implemented at the application layer), rather than waiting on the CDN's standard freshness check.
Worked example
A JS bundle served with Cache-Control: max-age=31536000, immutable and a content-hashed filename (app.a1b2c3.js) never needs revalidation at all, since a new deploy produces a new filename/URL; "invalidation" is simply pointing to the new URL. A frequently-changing API response served with Cache-Control: max-age=0, must-revalidate plus an ETag lets the client always check freshness cheaply (a 304 response, no body) without re-downloading the full payload on every request when nothing has actually changed.
Trade-offs and pitfalls
Setting Cache-Control: public on a response containing per-user or sensitive data lets shared caches (a CDN, a corporate proxy) serve one user's private data to another; always use private or no-store for personalized responses. Relying on Last-Modified timestamp granularity (often only second-level precision) can miss legitimately distinct versions of a resource that changed within the same second; ETag avoids this by tying validation to actual content, not a coarse timestamp.
Design a graceful degradation mechanism for when the caching layer becomes unavailable or overloaded. Propose fallback patterns, thresholds and protection mechanisms to prevent origin overload, and how to surface degraded behavior to users and operators.
Sample Answer
Direct answer
Graceful degradation means the application keeps functioning, at reduced quality or freshness, when the cache is unavailable, rather than the cache's failure becoming the application's failure; that requires a fast, automatic fallback path plus protection so the fallback itself does not overwhelm the origin.
Structured elaboration
- Direct origin fetch with a circuit breaker: when cache calls start failing or timing out, a circuit breaker trips after a threshold of failures and stops attempting cache calls for a cooldown period, falling back straight to the origin; this avoids adding cache-timeout latency on top of every request during an outage.
- Serve stale content: if a recently-expired or slightly-stale cached value is still available (even if technically past its time-to-live, TTL), serving it is often better than a full origin round-trip or an error, for data where a few extra minutes of staleness is an acceptable trade for availability.
- Limited origin admission (protection against overload): because the cache's job is normally absorbing load, a cache outage means the origin suddenly sees a much larger fraction of traffic; apply rate limiting or admission control at the origin's edge so it degrades gracefully (serving a fraction of requests well) rather than falling over entirely trying to serve all of them.
- Thresholds and protection mechanisms: define explicit thresholds for when to trip the circuit breaker (e.g., error rate or timeout rate exceeding X percent over a rolling window) and when to attempt recovery (a periodic "half-open" probe to check if the cache has recovered, per the standard circuit-breaker pattern).
- Surfacing degraded behavior: expose a clear operational signal (a metric, a status indicator) that the system is currently in degraded mode, both for on-call visibility and, where appropriate, for user-facing messaging (e.g., "results may be slightly delayed") rather than degrading silently.
Worked example
A product-listing page normally reads from cache with a 20ms budget; when the cache becomes unavailable, a circuit breaker trips after 5 consecutive timeouts within 10 seconds, and subsequent requests skip the cache call entirely, going straight to the origin with a rate limiter capping concurrent origin requests at a level the database can sustain (say, 500 concurrent queries), while excess requests receive a fast, clearly-labeled degraded response (a cached-but-stale value, or a simplified fallback) rather than queueing behind an overwhelmed origin.
Trade-offs and pitfalls
A circuit breaker with too low a failure threshold trips on ordinary transient blips, causing unnecessary degradation; too high a threshold delays the fallback until real damage has already occurred. Serving stale content is a correctness trade-off that must be explicitly acceptable for that specific data; do not apply it uniformly to data where staleness during an outage is actually harmful (e.g., real-time inventory during checkout).
You must choose between Redis and Memcached to implement a session store for a web app. List trade-offs and recommend one choice. Consider persistence, data types, replication/HA, memory efficiency, eviction semantics, and operational features such as monitoring and backup.
Sample Answer
Direct answer
Choose Redis when you need richer data structures, persistence, or replication/high-availability (HA) built in; choose Memcached when you need the simplest possible pure key-value cache with the lowest per-operation overhead and multi-threaded read scaling out of the box.
Structured elaboration
- Data types: Redis supports strings, hashes, lists, sets, sorted sets, and more, useful when the cache itself needs to do more than store opaque blobs (e.g., a sorted set for a leaderboard). Memcached only stores simple key-value byte strings.
- Persistence: Redis can persist to disk (RDB snapshots, append-only file, AOF) so data can survive a restart; Memcached is purely in-memory with no persistence, so a restart is always a full cache flush.
- Replication / high availability: Redis has built-in replication and clustering (Sentinel for failover, Cluster for sharding); Memcached has no native replication, relying on the client or an external layer for HA.
- Memory efficiency: Memcached's simpler data model generally has lower per-key memory overhead for pure key-value use cases; Redis's richer data structures and features carry some additional overhead.
- Eviction semantics: both support least-recently-used (LRU) style eviction, but Redis offers more configurable policies (
allkeys-lru,volatile-ttl,allkeys-lfu, and others) versus Memcached's simpler slab-based LRU. - Operational features: Redis has a larger ecosystem for observability, Lua scripting for atomic multi-step operations, and pub/sub; Memcached's multi-threaded architecture can give it an edge on raw throughput for simple get/set at very high concurrency on a single node.
Worked example
For a session store: if sessions are pure key-value blobs, Memcached is a perfectly reasonable, simpler choice. If sessions need persistence across a restart (avoiding logging every user out simultaneously) or replication for HA, Redis with AOF persistence and Sentinel-managed failover is the safer choice, at the cost of a slightly more complex operational footprint.
Trade-offs and pitfalls
Choosing Redis "because it can do more" for a workload that is genuinely simple key-value adds operational surface area (persistence tuning, replication topology) without benefit; match the tool to the actual requirement. Memcached's lack of native replication means a node loss is a hard cache-miss event for everything that node held, with no automatic failover; that must be an explicit, accepted trade-off, not an oversight.
Describe how to implement negative caching safely for non-existent resources and how to set TTLs to balance reduced DB load with the risk of false negatives. Explain mechanisms to detect and recover from incorrectly cached negatives and how to prevent poisoning of the cache.
Sample Answer
Direct answer
Negative caching (caching the fact that something does NOT exist) reduces repeated backend load for lookups that legitimately miss often, but needs a shorter time-to-live (TTL) than positive caching and explicit protection against caching a TRANSIENT error as if it were a permanent absence.
Structured elaboration
- When negative caching helps: an endpoint frequently queried for resources that legitimately do not exist (a sparse product search, a lookup by an ID that is often invalid or not-yet-created) benefits from caching "not found" so repeated identical misses do not all hit the backend.
- TTL sizing: negative entries should generally use a SHORTER TTL than positive entries, since a resource that does not exist now might be created moments later (a product about to be listed, a user about to be created), and a long negative-cache TTL would delay legitimate new data from becoming visible.
- Avoiding poisoning from transient errors: a backend timeout or a temporary error must NOT be cached as "not found"; only cache an explicit, confirmed "this does not exist" response (e.g., an actual 404 from a healthy backend), never an ambiguous failure, or a temporary outage will look like every queried resource permanently vanished.
- Detecting and recovering from an incorrectly cached negative: monitor the negative-cache hit rate for anomalies (a sudden spike suggesting something is being wrongly cached as absent), and provide an explicit purge mechanism for a specific key if a false negative is identified.
- Preventing cache poisoning attacks: without care, an attacker (or a misbehaving client) could probe many nonexistent IDs to fill the negative cache with entries, which is generally low-risk for negative caching specifically (it just wastes some memory) but worth being aware of if negative-cache capacity is shared with more sensitive cached data.
Worked example
A product-search endpoint receiving many queries for discontinued or misspelled product IDs: caching "not found" for 30 seconds (much shorter than the 5-minute TTL used for actual product data) absorbs repeated identical misses for the same nonexistent ID within that window, while still allowing a newly-listed product with that same ID to become visible within 30 seconds rather than being stuck behind a long negative-cache TTL.
Trade-offs and pitfalls
Caching a timeout or a 500-level error as a negative ("not found") result is the single most damaging mistake in negative caching; it turns a transient backend problem into an apparent, cache-durable "this doesn't exist" for every subsequent request until the negative TTL expires, actively making an outage worse and longer-lasting than it needed to be. Setting a negative TTL as long as the positive TTL "for simplicity" delays visibility of genuinely new data unnecessarily; size them independently based on how quickly a false negative needs to self-correct.
Unlock Full Question Bank
Get access to all Caching Strategies and Distributed Caching interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.