Caching Strategies and Distributed Caching Questions
Using caches to reduce latency and load: cache-aside, read-through, write-through, and write-behind patterns, TTLs, eviction policies, and distributed caches such as Redis or Memcached. Covers cache invalidation, stampede and thundering-herd protection, and the consistency tradeoffs of caching. Focuses on where and how to cache across tiers.
In a system with caches, a primary database, and a search index (e.g., Elasticsearch), describe common consistency pitfalls when updating entities (for example user profile changes). Propose an ordered update workflow that minimizes stale reads across layers and supports failure recovery, and explain trade-offs involved.
Sample Answer
Direct answer
When a cache, a primary database, and a search index all hold a copy of the same entity, an update has to reach all three, and the order they are updated in determines what inconsistent intermediate state a concurrent reader might see; design the update sequence deliberately rather than firing updates to all three independently.
Structured elaboration
- The core pitfall: three independent copies means three independent chances to be out of sync at any given moment; a naive "update DB, then update cache, then update search index" sequence has a window after each step where a reader hitting a different layer sees a different, momentarily-inconsistent view of the same entity.
- An ordered update workflow: write to the primary database first (the source of truth), then propagate to the cache and search index, ideally via the same mechanism (an event derived from the database's own write, e.g., change data capture, CDC) rather than the application independently, and possibly inconsistently, updating each one itself.
- Minimizing stale reads across layers: invalidate (or update) the cache promptly after the database write; the search index, which is often inherently eventually consistent by nature (indexing takes measurable time), should be treated with a wider, explicitly acknowledged staleness tolerance rather than expected to match the database's freshness.
- Failure recovery: if the propagation to the cache or search index fails partway (the database write succeeded but the cache invalidation did not), the system needs to detect and correct that drift, either via a reconciliation job that periodically compares layers, or by treating the CDC-based propagation as the single source of truth for keeping all downstream copies in sync (so a failure there is a well-understood, monitorable gap, not a silent one).
- Trade-offs involved: driving every downstream update from the database's own change stream (rather than each application code path independently updating cache and search index) is more work to set up but eliminates entire classes of "we updated the DB and cache but forgot the search index in this one code path" bugs.
Worked example
A user profile update: the database write succeeds; a CDC-derived event then triggers both a cache invalidation and a search-index update. A reader hitting the cache immediately after the database write (before the CDC event has propagated) sees a brief stale read from the cache, bounded by the CDC pipeline's typical propagation latency (often under a second); a search query against the index during that same window might return the OLD profile data in search results for slightly longer, since indexing itself typically has its own additional latency beyond simple cache invalidation, an acceptable, well-understood difference in staleness across layers as long as it is explicit and monitored.
Trade-offs and pitfalls
Independently updating cache and search index from application code (rather than from a single CDC-derived event stream) is a common source of "we forgot this one code path" bugs, where some write paths correctly update all three layers and others silently miss one; centralizing propagation removes that class of bug at the cost of needing CDC infrastructure. Treating all downstream layers as needing the SAME freshness guarantee is usually wrong; a search index's inherent indexing latency is a different, and typically wider, staleness budget than a cache's invalidation latency, and conflating the two leads to either wasted effort over-optimizing the index or an unrealistic freshness expectation for it.
A distributed cache cluster shows inconsistent data across nodes after a network partition. Explain the plausible root causes and design a recovery plan that minimizes downtime and data loss. Discuss the trade-off between restoring availability quickly and reconciling consistency.
Sample Answer
Direct answer
Inconsistent data across cache nodes after a network partition means two sides of the partition may have kept accepting writes independently (split-brain) or replication simply fell behind (partial replication/stale writes); recovery has to first stop the bleeding (restore a single source of truth) before reconciling the divergence, and that choice trades availability against correctness.
Structured elaboration
- Plausible root causes: split-brain (both sides of a partition believed they were the primary and accepted writes independently, producing two diverging histories), partial replication (replication was in progress but not complete when the partition occurred, so some nodes have older data than others), and stale writes (a client wrote to a node that was, unknown to it, no longer the authoritative primary).
- Restoring availability quickly versus reconciling consistency: choosing to bring the cluster back online immediately (picking one side's data as authoritative and discarding the other's divergent writes) restores service fast but can silently lose data from the discarded side; choosing to carefully reconcile both sides first (comparing and merging where possible) is safer for correctness but extends the outage.
- A recovery plan: first, stop accepting further writes on whichever side is NOT chosen as authoritative (preventing further divergence); second, decide the authoritative side (usually the side with quorum, or the side that was serving the majority of traffic, per your topology's own quorum rules); third, reconcile or discard the non-authoritative side's diverged writes, documenting what was lost if anything; fourth, resync the non-authoritative side from the authoritative one before bringing it back into service.
- Minimizing downtime and data loss together: this is rarely a clean binary choice; a common middle ground is bringing read traffic back online quickly from the authoritative side (restoring availability for reads) while writes stay paused or degraded until reconciliation completes, rather than an all-or-nothing restoration.
Worked example
A 6-node Redis cluster splits into a 4-node majority partition and a 2-node minority partition due to a network issue; if the cluster's quorum configuration correctly prevented the minority side from accepting writes during the partition (a properly configured min-replicas-to-write or Sentinel quorum), the majority side is unambiguously authoritative and the minority side simply needs to resync from it once the network heals, with no actual data divergence to reconcile. If quorum protection was misconfigured and the minority side DID accept writes, those writes are now in conflict with the majority's history and must be explicitly reconciled or, if reconciliation is not possible, discarded and documented as lost.
Trade-offs and pitfalls
The best outcome here comes from PREVENTING split-brain in the first place (properly configured quorum requirements), not from having a great recovery plan for when it happens; a recovery plan is a backstop, not a substitute for prevention. Discarding a non-authoritative side's writes without first checking whether any of them can be safely reconciled (a value that was never touched on the authoritative side, for example) is faster but can lose more than strictly necessary; weigh the reconciliation effort against how much data is actually at stake.
Discuss how to handle cache serialization and deserialization safely and efficiently. Consider versioning serialized formats, schema evolution, backward compatibility, and lazy migration strategies during rolling upgrades.
Sample Answer
Direct answer
Cache serialization needs explicit schema versioning so that a deployment change (a field added, removed, or retyped) does not produce a runtime error or silent data corruption when new code reads an old cached format, or old code reads a new one, during a rolling upgrade.
Structured elaboration
- Versioning the serialized format: embed a schema version alongside the serialized payload (either in the value itself, or in the cache key, as covered by the key-versioning pattern) so a reader can detect which shape it is dealing with.
- Schema evolution: prefer additive, backward-compatible changes (new optional fields default sensibly if absent) over breaking changes (renaming or retyping an existing field) wherever possible, since additive changes let old and new code coexist safely without any special handling.
- Backward compatibility during rolling upgrades: during a deploy, old and new application code run simultaneously for some window; if new code writes a new schema version and old code cannot deserialize it, old-code instances will error or silently misbehave on every cache read for that key until the deploy completes, unless the new schema is designed to be backward-readable, or the version is used to explicitly route old code away from new-format entries.
- Lazy migration strategies: rather than migrating every cached entry to a new format immediately, let entries migrate lazily: on a cache miss (or an explicit read-and-rewrite), the new format is written; old-format entries simply age out via their normal time-to-live (TTL), avoiding a disruptive bulk-migration pass.
- Choosing a serialization format: a format with strong built-in support for optional/default fields (protocol buffers, for example) makes additive schema evolution far more natural than a format that requires exact structural matching to deserialize correctly.
Worked example
Adding a new optional field to a cached user-profile object: with a backward-compatible serialization format, old code (not yet aware of the new field) simply ignores it when reading a new-format entry, and new code supplies a sensible default when reading an old-format entry that lacks the field; no special versioning logic is needed at all, because the format itself tolerates the difference. Contrast with renaming an existing field, which is NOT safely backward-compatible under most formats and would require either a version-gated read path or accepting a brief window of cache misses/errors during the rolling deploy.
Trade-offs and pitfalls
Assuming a serialization format is "safe" without checking its specific behavior on missing or extra fields is a common way this bites during a rolling deploy; verify the format's actual compatibility guarantees, do not assume. Skipping explicit versioning "because most changes are additive" works until the first genuinely breaking change arrives unplanned; building the versioning mechanism in from the start costs little and avoids an emergency retrofit later.
You operate a high-read service with current cache hit ratio 70% and database cost that scales linearly with queries. Present a cost-benefit analysis framework for deciding whether to (a) increase cache capacity, (b) tune cache TTLs, or (c) scale the database. What measurements, estimation models, and experiments would you run to make a data-driven decision?
Sample Answer
Direct answer
Compare the three options (grow cache capacity, tune TTLs, scale the database) by estimating the marginal cost and marginal latency/load benefit of each, since they are not mutually exclusive and the cheapest lever that closes the gap is usually the right first move.
Structured elaboration
- Growing cache capacity: if evictions are the bottleneck (the working set does not fit), more memory directly raises hit ratio; the cost is roughly linear in additional cache node memory/count, and the benefit is bounded once the cache is large enough to hold the full working set (beyond that point, more memory buys nothing).
- Tuning TTLs: if the working set fits but TTLs are shorter than necessary for the actual staleness tolerance, extending TTLs raises effective hit ratio for close to zero additional infrastructure cost; the ceiling here is the data's genuine staleness tolerance, not technical capacity.
- Scaling the database: if the cache is already well-sized and well-tuned but query volume is still too high (e.g., a low cache-hit-ratio workload with lots of genuinely unique reads), the database itself needs more capacity (read replicas, bigger instances); this is usually the most expensive lever, both in infrastructure cost and operational complexity.
- A framework for choosing: first check eviction rate (tells you if more cache memory would help), then check whether current TTLs are shorter than the data's actual staleness tolerance (tells you if a free tuning win exists), and only after those are exhausted, model out database scaling cost against the marginal load reduction remaining.
Worked example
A service with a 70 percent cache hit ratio and database cost scaling linearly with query volume: modeling the current state at, say, Q queries per second to the database and cost C=k⋅Q. Raising hit ratio to 85 percent (a plausible outcome of either more cache memory or longer TTLs, depending on root cause) cuts database query volume by 1−0.701−0.85=0.300.15=0.5, a 50 percent reduction in database cost, versus scaling the database itself, which would only add capacity without reducing the query volume driving that cost at all. If eviction rate is low (cache is not memory-constrained) but TTLs are conservative relative to the data's real staleness tolerance, extending TTLs achieves this improvement essentially for free; if eviction rate is high, the same improvement requires paying for more cache capacity instead.
Trade-offs and pitfalls
Scaling the database without first checking whether cache tuning could achieve the same load reduction for less money is a common and expensive mistake; always rule out the cheaper levers first with actual data (eviction rate, staleness tolerance), not assumption. Pushing TTLs longer than the data's genuine staleness tolerance to chase a cost win trades a real user-facing correctness problem for an infrastructure savings; the framework only works if the staleness ceiling is respected as a hard constraint, not a dial.
Describe common cache eviction policies (LRU, LFU, FIFO, and TTL-based expiry). For each policy explain: how it decides what to evict, what access patterns it suits best, and give a short concrete example scenario (e.g., session caching, hot-working set, analytics counters).
Sample Answer
Direct answer
Least Recently Used (LRU) evicts by recency, Least Frequently Used (LFU) evicts by access count, First-In-First-Out (FIFO) evicts by insertion order regardless of access pattern, and time-to-live (TTL) based expiry evicts by a fixed age regardless of how popular the item is; each fits a different assumption about what predicts future value.
Structured elaboration
- LRU: evicts the item that has gone longest without being accessed. Fits workloads with strong temporal locality (a session that is hot right after login, then cold); weak spot is a one-time scan flushing genuinely popular items.
- LFU: evicts the item with the fewest accesses (often with decay). Fits a stable popularity distribution (a small set of perennially popular catalog items); weak spot is that brand-new items start at zero and can be evicted before they get a chance to prove their popularity.
- FIFO: evicts the oldest-inserted item regardless of how often it has been accessed since. It is the simplest and cheapest to implement (no access-order bookkeeping needed on every read), and fits workloads where recency of insertion genuinely correlates with relevance (e.g., a queue of recent events), but performs poorly for general-purpose caching where popular older items get evicted just because they were inserted first.
- TTL-based expiry: evicts (expires) an item after a fixed duration regardless of access pattern; it is really answering a different question ("how stale is too stale") rather than "what should we keep when we're full," and is often combined with one of the other policies as a ceiling on staleness rather than used as the sole eviction mechanism.
- Choosing for a workload: session caching (LRU: recency-driven), hot-working-set analytics (LFU or a hybrid: stable popularity), analytics counters with a natural expiry (TTL: staleness is the real constraint, not memory pressure).
Worked example
A cache of user session objects with occasional heavy churn (many users logging in and out in bursts) fits LRU well: sessions are hot right after login and cold afterward, and LRU naturally keeps the currently-active sessions warm through the churn. LFU would be a poor fit here because it would take time to "learn" a new burst of sessions are popular, during which it might evict genuinely active sessions in favor of older sessions that accumulated more historical hits.
Trade-offs and pitfalls
FIFO is cheap but usually the wrong default for a general-purpose cache; it is easy to reach for because it is simple to implement, but it ignores the one signal (access pattern) that actually predicts value. Do not conflate TTL (a staleness ceiling) with an eviction policy chosen for memory pressure; a cache can need both simultaneously for different reasons.
Unlock Full Question Bank
Get access to all Caching Strategies and Distributed Caching interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.