Caching Strategies and Distributed Caching Questions
Using caches to reduce latency and load: cache-aside, read-through, write-through, and write-behind patterns, TTLs, eviction policies, and distributed caches such as Redis or Memcached. Covers cache invalidation, stampede and thundering-herd protection, and the consistency tradeoffs of caching. Focuses on where and how to cache across tiers.
For a system containing user consent and privacy-sensitive fields, discuss TTLs, explicit purge, versioned keys, and event-driven invalidation for ensuring regulatory correctness. Compare these strategies and recommend a design that minimizes legal risk while balancing latency and cost.
Sample Answer
Direct answer
For consent and privacy-sensitive fields, favor explicit invalidation and versioned keys over a passive time-to-live (TTL), and treat a long TTL as a direct legal-risk decision, not just a performance tuning knob, because a stale consent state can mean processing data the user has explicitly withdrawn permission for.
Structured elaboration
- TTL alone is insufficient: a TTL bounds staleness but does not guarantee it; a user withdrawing consent expects that to take effect promptly, and "eventually, within the TTL window" is not a defensible position if that window is, say, an hour.
- Explicit purge on consent change: a consent withdrawal should trigger immediate, explicit invalidation of the cached consent state (and ideally, propagate to every system that reads it), rather than relying on the next natural TTL expiry.
- Versioned keys: bump a version on consent change so any cached read of the OLD version becomes unreachable immediately, avoiding a race where a slow, in-flight read repopulates the cache with the just-withdrawn consent state after the invalidation (the same race pattern seen elsewhere in this topic, here with legal consequences instead of just a UX inconvenience).
- Event-driven invalidation for regulatory correctness: propagate consent changes as events to every downstream consumer that might have cached consent state, since a consent decision often needs to be honored by multiple systems (advertising, analytics, data processing pipelines), not just the primary service.
- Recommended design minimizing legal risk while balancing latency and cost: a short TTL (as a safety-net backstop, not the primary mechanism) combined with event-driven, explicit invalidation on any consent change gives near-immediate correctness for the common case while still self-healing if an invalidation event is somehow lost, rather than choosing one mechanism alone.
Worked example
A user withdraws marketing-communications consent; an event fires immediately, invalidating their cached consent state across every consuming service (the marketing service, an analytics pipeline, a personalization engine) within roughly the propagation latency of the event system (typically well under a second), rather than each of those systems independently waiting out its own TTL (which, if inconsistently configured across systems, could mean some honor the withdrawal in seconds and others in minutes).
Trade-offs and pitfalls
Treating regulatory correctness as "just another caching performance trade-off" and choosing a TTL purely for latency/cost reasons is a serious mistake; involve legal/compliance stakeholders in defining the acceptable staleness window for consent data, do not decide it unilaterally as an engineering optimization. Relying purely on event-driven invalidation with no TTL backstop reintroduces the risk that a lost event produces unbounded (rather than bounded) staleness, which is a worse regulatory posture than a short, explicit TTL ceiling.
How do you decide whether to introduce a cache for a given service endpoint? Describe the signals and measurements you would collect, the tests you would run (load, latency, profiling), and the criteria that justify adding an in-process cache, a shared cache (Redis), or a CDN. Include considerations for cost, operational complexity, and correctness.
Sample Answer
Direct answer
Decide whether to add a cache by measuring the actual read pattern (how often the same value is requested, how expensive it is to produce, and how much staleness is tolerable), not by defaulting to caching every endpoint; a cache with a low repeat-read rate or zero staleness tolerance is a cost with no real benefit.
Structured elaboration
- Signals to collect: request rate for the same key/query (does the same data actually get read repeatedly, or is nearly every read unique), the cost of producing the value (a fast, cheap lookup gains little from caching even if repeated), and the data's staleness tolerance (how quickly must a change be visible).
- Tests to run: a load test comparing latency and backend load with and without a proposed cache, and a profile of the actual query/computation to confirm it is genuinely a meaningful cost worth caching against.
- Criteria for choosing a cache tier: an in-process cache fits data that is cheap to duplicate per instance and does not need cross-instance consistency; a shared cache (Redis) fits data that benefits from being consistent across instances or too large to duplicate per instance; a content delivery network (CDN) fits public, non-personalized content that benefits from being close to users geographically.
- Cost: weigh the infrastructure and operational cost of adding a caching layer (a new dependency to monitor, secure, and keep available) against the actual load/latency benefit measured above; a marginal benefit may not justify the added complexity.
- Operational complexity: caching adds invalidation logic, a new failure mode (cache unavailable), and another thing to monitor; these costs are real even when the caching decision is otherwise sound, and should be weighed explicitly.
- Correctness: if the data's staleness tolerance is effectively zero (a value that must always reflect the absolute latest state, with no acceptable delay), caching adds risk without benefit, since any caching mechanism introduces at least a small window of potential staleness.
Worked example
An endpoint returning a real-time stock quote, requested uniquely per symbol per user with essentially no repeat reads within any meaningful window, and requiring zero staleness tolerance: this fails on both the "does the same value get read repeatedly" test and the "can staleness be tolerated" test, making it a poor caching candidate regardless of how expensive the underlying computation is. Contrast with a product description, read thousands of times per hour by different users for the same handful of popular items, changing rarely: this passes both tests clearly.
Trade-offs and pitfalls
Caching by default, without measuring the actual read-repetition rate, either wastes cache capacity on data that gets no benefit or, worse, introduces a staleness risk on data that could not tolerate it; always start from measurement, not habit. The decision is not binary per endpoint; the same service can have some data that benefits enormously from caching and other data (even on the same page) that should never be cached, and treating the whole endpoint uniformly misses that nuance.
Compare cache placement options: client-side, CDN/edge, reverse-proxy (e.g., Varnish), application-level in-memory (e.g., Redis/Memcached), and database-side (materialized views or DB-level caching). For each option describe pros, cons, typical use cases, security/privacy considerations, and how TTLs and invalidation differ by placement.
Sample Answer
Direct answer
Cache placement is a ladder from "closest to the user, cheapest, hardest to invalidate precisely" to "closest to the source of truth, most expensive per request, easiest to keep correct": client-side, content delivery network (CDN)/edge, reverse proxy, application in-memory, and database-side.
Structured elaboration
- Client-side: browser cache, local storage, or a mobile app's local store. Zero network cost on a hit, but you have essentially no control once data leaves your servers; invalidation means waiting out a time-to-live (TTL) or changing a versioned URL.
- CDN/edge: shared across all users near a given geography, ideal for static or long-TTL, non-personalized content. Purge/invalidation is slower (can take seconds to propagate globally) and typically coarser-grained than a server-side cache.
- Reverse proxy (e.g., Varnish): sits in front of your application servers, caching full HTTP responses; good for reducing application-server load for cacheable pages without needing application code changes, but still shared and public unless carefully scoped per-user.
- Application-level in-memory (Redis/Memcached, or in-process): shared across your own service's instances (Redis/Memcached) or private to one instance (in-process); this is where most business-logic caching happens, because you have full control over invalidation and can cache personalized data safely.
- Database-side (materialized views, query result caching): closest to the source of truth, so almost always the most consistent option, at the cost of doing the least to reduce load on the database itself.
- Redis vs. CDN by use case: for static assets, a CDN wins outright (no reason to burn application-tier memory on data that never changes per-request). For personalized HTML fragments, a CDN only works with edge compute; otherwise application-tier Redis is the safe default. For frequently-read configuration flags, an in-process or small Redis cache with a short TTL beats a CDN, since the data is tiny and needs low latency, not global edge distribution. For large objects with varying TTLs (e.g., images), a CDN with per-object cache-control headers is the natural fit.
Worked example
A product page has a static hero image (CDN, long TTL, content-hashed URL so invalidation is just a new URL), a shared "similar products" block computed the same for all users (reverse proxy or application cache, medium TTL), and a personalized "recently viewed" section (application-level cache keyed per user, short TTL, never placed at a shared CDN/proxy layer).
Trade-offs and pitfalls
Placing personalized content at a shared caching layer (CDN or reverse proxy) without per-user cache keys is a serious privacy bug, not just a staleness inconvenience; one user's private data can be served to another. The further a cache sits from the source of truth, the cheaper it is per request and the harder it is to invalidate precisely; choose the placement based on how quickly and precisely that specific data needs to be corrected on write.
Design a caching architecture for expensive analytics queries where results can be up to 5 minutes stale. Consider materialized views, result caching layers, cache invalidation on upstream changes, multi-tenancy isolation, and eviction strategies for large result sets.
Sample Answer
Direct answer
Analytics and business intelligence (BI) queries tolerate minutes of staleness in exchange for large latency and cost wins, so lean on materialized views and result caching aggressively, choosing the caching granularity (whole report, per-tile, per-query-result) based on how the dashboard is actually consumed.
Structured elaboration
- Materialized views: precompute and store the results of expensive aggregations on a schedule (or triggered by upstream data changes), so a dashboard read is a fast lookup against already-computed results rather than a live, expensive query against raw data.
- Result caching layers: cache the results of specific, frequently-run queries (a query-result cache keyed by the query and its parameters) for dashboards where the underlying data does not change often enough to justify a full materialized-view pipeline.
- Cache granularity: report-level caching (the whole dashboard's output) is simplest but coarse (any change forces a full recompute); tile-level (each widget/chart cached independently) allows partial invalidation when only some underlying data changed; query-result-level is the finest grain, useful when many different reports share underlying queries.
- Invalidation strategies for BI: time-to-live (TTL) is often sufficient here, since most BI use cases genuinely tolerate a bounded staleness window (minutes, sometimes hours); event-based invalidation (triggered by an upstream data-pipeline completion) is worth the added complexity specifically for dashboards where "as fresh as the last data load" matters more than a fixed time window; manual invalidation (an explicit refresh button) suits ad-hoc analysis tools where users want on-demand control.
- Cache warming/pre-computation: for dashboards viewed at predictable times (a morning operations review, a weekly business report), precomputing results just before that predictable access window avoids making the first viewer of the day pay the full, uncached computation cost.
- Balancing freshness, latency, and cost: the right TTL and caching granularity should map directly to how the business actually uses the dashboard, an operational dashboard checked continuously wants near-real-time and can justify more compute cost; a monthly strategic report tolerates hours of staleness and should be cached aggressively to save cost.
Worked example
A BI platform serving semantic-layer queries: for a query-result cache keyed by the query and its parameters, a report combining multiple underlying queries can serve most of its content from cache (queries that have not changed) while only recomputing the specific queries whose underlying data actually changed, rather than invalidating and recomputing the entire report on any single data update; this per-query granularity captures much of the tile-level caching benefit without needing the dashboard rendering layer itself to be cache-aware.
Trade-offs and pitfalls
Caching at the coarsest (whole-report) granularity for convenience, when the underlying data actually changes at different rates for different parts of the report, wastes the caching opportunity for the parts that rarely change and forces unnecessary staleness or unnecessary recomputation for the rest; choose granularity deliberately based on the actual update-rate heterogeneity within the report. Setting one TTL policy across every dashboard regardless of how it is actually used (operational versus strategic) either wastes freshness-driving compute cost where it is not needed, or under-serves freshness where it genuinely matters; tie the TTL choice to actual usage patterns.
What is a cache stampede or thundering herd problem and how can it affect reliability? Name and briefly describe at least four practical prevention techniques.
Sample Answer
Direct answer
A cache stampede (also called a thundering herd) happens when a popular cache entry expires or is invalidated and many concurrent requests for that same key all miss at once, sending a burst of simultaneous load to the origin that it was never sized to absorb directly.
Structured elaboration
- Why it happens: caching's whole benefit is deduplicating repeated work; a stampede is exactly the moment that deduplication briefly stops working, because every one of the concurrent requests independently decides "I need to recompute this" at the same instant.
- Why it is dangerous: the origin (a database, an expensive computation, a third-party API) is usually sized assuming the cache absorbs most traffic; a stampede can multiply its load by the number of concurrent requesters for that one key, which for a genuinely hot key can be thousands of requests in a fraction of a second.
- Prevention techniques (name and describe at least four): locking (only the first requester acquires a lock and recomputes, others wait or serve stale data), request coalescing/singleflight (deduplicate concurrent identical fetches into one in-flight request), jittered expirations (randomize time-to-live, TTL, slightly so many keys written together do not expire at the exact same instant), and background/proactive refresh (recompute before expiry so the cache rarely actually goes empty for a hot key).
Worked example
A homepage configuration cached with a 60-second TTL, read 5,000 times per second, expires at exactly the 60-second mark with no protection: in the recomputation window (say 200ms), roughly 1,000 requests (5,000 times 0.2s) would all miss simultaneously and hit the origin. With locking or coalescing, exactly one of those 1,000 requests recomputes; the rest wait briefly or receive the prior value.
Trade-offs and pitfalls
Jitter alone does not fully solve a single very-hot key's stampede risk (it helps most when MANY keys expire together, i.e., a cache avalanche); locking or coalescing is needed for a single key read at very high concurrency. A lock without a timeout can deadlock the system if the process holding it crashes mid-recompute.
Unlock Full Question Bank
Get access to all 10 Caching Strategies and Distributed Caching interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.