Caching Strategies and Distributed Caching Questions
Using caches to reduce latency and load: cache-aside, read-through, write-through, and write-behind patterns, TTLs, eviction policies, and distributed caches such as Redis or Memcached. Covers cache invalidation, stampede and thundering-herd protection, and the consistency tradeoffs of caching. Focuses on where and how to cache across tiers.
Discuss how to handle cache serialization and deserialization safely and efficiently. Consider versioning serialized formats, schema evolution, backward compatibility, and lazy migration strategies during rolling upgrades.
Sample Answer
Direct answer
Cache serialization needs explicit schema versioning so that a deployment change (a field added, removed, or retyped) does not produce a runtime error or silent data corruption when new code reads an old cached format, or old code reads a new one, during a rolling upgrade.
Structured elaboration
- Versioning the serialized format: embed a schema version alongside the serialized payload (either in the value itself, or in the cache key, as covered by the key-versioning pattern) so a reader can detect which shape it is dealing with.
- Schema evolution: prefer additive, backward-compatible changes (new optional fields default sensibly if absent) over breaking changes (renaming or retyping an existing field) wherever possible, since additive changes let old and new code coexist safely without any special handling.
- Backward compatibility during rolling upgrades: during a deploy, old and new application code run simultaneously for some window; if new code writes a new schema version and old code cannot deserialize it, old-code instances will error or silently misbehave on every cache read for that key until the deploy completes, unless the new schema is designed to be backward-readable, or the version is used to explicitly route old code away from new-format entries.
- Lazy migration strategies: rather than migrating every cached entry to a new format immediately, let entries migrate lazily: on a cache miss (or an explicit read-and-rewrite), the new format is written; old-format entries simply age out via their normal time-to-live (TTL), avoiding a disruptive bulk-migration pass.
- Choosing a serialization format: a format with strong built-in support for optional/default fields (protocol buffers, for example) makes additive schema evolution far more natural than a format that requires exact structural matching to deserialize correctly.
Worked example
Adding a new optional field to a cached user-profile object: with a backward-compatible serialization format, old code (not yet aware of the new field) simply ignores it when reading a new-format entry, and new code supplies a sensible default when reading an old-format entry that lacks the field; no special versioning logic is needed at all, because the format itself tolerates the difference. Contrast with renaming an existing field, which is NOT safely backward-compatible under most formats and would require either a version-gated read path or accepting a brief window of cache misses/errors during the rolling deploy.
Trade-offs and pitfalls
Assuming a serialization format is "safe" without checking its specific behavior on missing or extra fields is a common way this bites during a rolling deploy; verify the format's actual compatibility guarantees, do not assume. Skipping explicit versioning "because most changes are additive" works until the first genuinely breaking change arrives unplanned; building the versioning mechanism in from the start costs little and avoids an emergency retrofit later.
Design a graceful degradation mechanism for when the caching layer becomes unavailable or overloaded. Propose fallback patterns, thresholds and protection mechanisms to prevent origin overload, and how to surface degraded behavior to users and operators.
Sample Answer
Direct answer
Graceful degradation means the application keeps functioning, at reduced quality or freshness, when the cache is unavailable, rather than the cache's failure becoming the application's failure; that requires a fast, automatic fallback path plus protection so the fallback itself does not overwhelm the origin.
Structured elaboration
- Direct origin fetch with a circuit breaker: when cache calls start failing or timing out, a circuit breaker trips after a threshold of failures and stops attempting cache calls for a cooldown period, falling back straight to the origin; this avoids adding cache-timeout latency on top of every request during an outage.
- Serve stale content: if a recently-expired or slightly-stale cached value is still available (even if technically past its time-to-live, TTL), serving it is often better than a full origin round-trip or an error, for data where a few extra minutes of staleness is an acceptable trade for availability.
- Limited origin admission (protection against overload): because the cache's job is normally absorbing load, a cache outage means the origin suddenly sees a much larger fraction of traffic; apply rate limiting or admission control at the origin's edge so it degrades gracefully (serving a fraction of requests well) rather than falling over entirely trying to serve all of them.
- Thresholds and protection mechanisms: define explicit thresholds for when to trip the circuit breaker (e.g., error rate or timeout rate exceeding X percent over a rolling window) and when to attempt recovery (a periodic "half-open" probe to check if the cache has recovered, per the standard circuit-breaker pattern).
- Surfacing degraded behavior: expose a clear operational signal (a metric, a status indicator) that the system is currently in degraded mode, both for on-call visibility and, where appropriate, for user-facing messaging (e.g., "results may be slightly delayed") rather than degrading silently.
Worked example
A product-listing page normally reads from cache with a 20ms budget; when the cache becomes unavailable, a circuit breaker trips after 5 consecutive timeouts within 10 seconds, and subsequent requests skip the cache call entirely, going straight to the origin with a rate limiter capping concurrent origin requests at a level the database can sustain (say, 500 concurrent queries), while excess requests receive a fast, clearly-labeled degraded response (a cached-but-stale value, or a simplified fallback) rather than queueing behind an overwhelmed origin.
Trade-offs and pitfalls
A circuit breaker with too low a failure threshold trips on ordinary transient blips, causing unnecessary degradation; too high a threshold delays the fallback until real damage has already occurred. Serving stale content is a correctness trade-off that must be explicitly acceptable for that specific data; do not apply it uniformly to data where staleness during an outage is actually harmful (e.g., real-time inventory during checkout).
In a system with caches, a primary database, and a search index (e.g., Elasticsearch), describe common consistency pitfalls when updating entities (for example user profile changes). Propose an ordered update workflow that minimizes stale reads across layers and supports failure recovery, and explain trade-offs involved.
Sample Answer
Direct answer
When a cache, a primary database, and a search index all hold a copy of the same entity, an update has to reach all three, and the order they are updated in determines what inconsistent intermediate state a concurrent reader might see; design the update sequence deliberately rather than firing updates to all three independently.
Structured elaboration
- The core pitfall: three independent copies means three independent chances to be out of sync at any given moment; a naive "update DB, then update cache, then update search index" sequence has a window after each step where a reader hitting a different layer sees a different, momentarily-inconsistent view of the same entity.
- An ordered update workflow: write to the primary database first (the source of truth), then propagate to the cache and search index, ideally via the same mechanism (an event derived from the database's own write, e.g., change data capture, CDC) rather than the application independently, and possibly inconsistently, updating each one itself.
- Minimizing stale reads across layers: invalidate (or update) the cache promptly after the database write; the search index, which is often inherently eventually consistent by nature (indexing takes measurable time), should be treated with a wider, explicitly acknowledged staleness tolerance rather than expected to match the database's freshness.
- Failure recovery: if the propagation to the cache or search index fails partway (the database write succeeded but the cache invalidation did not), the system needs to detect and correct that drift, either via a reconciliation job that periodically compares layers, or by treating the CDC-based propagation as the single source of truth for keeping all downstream copies in sync (so a failure there is a well-understood, monitorable gap, not a silent one).
- Trade-offs involved: driving every downstream update from the database's own change stream (rather than each application code path independently updating cache and search index) is more work to set up but eliminates entire classes of "we updated the DB and cache but forgot the search index in this one code path" bugs.
Worked example
A user profile update: the database write succeeds; a CDC-derived event then triggers both a cache invalidation and a search-index update. A reader hitting the cache immediately after the database write (before the CDC event has propagated) sees a brief stale read from the cache, bounded by the CDC pipeline's typical propagation latency (often under a second); a search query against the index during that same window might return the OLD profile data in search results for slightly longer, since indexing itself typically has its own additional latency beyond simple cache invalidation, an acceptable, well-understood difference in staleness across layers as long as it is explicit and monitored.
Trade-offs and pitfalls
Independently updating cache and search index from application code (rather than from a single CDC-derived event stream) is a common source of "we forgot this one code path" bugs, where some write paths correctly update all three layers and others silently miss one; centralizing propagation removes that class of bug at the cost of needing CDC infrastructure. Treating all downstream layers as needing the SAME freshness guarantee is usually wrong; a search index's inherent indexing latency is a different, and typically wider, staleness budget than a cache's invalidation latency, and conflating the two leads to either wasted effort over-optimizing the index or an unrealistic freshness expectation for it.
You must choose between Redis and Memcached to implement a session store for a web app. List trade-offs and recommend one choice. Consider persistence, data types, replication/HA, memory efficiency, eviction semantics, and operational features such as monitoring and backup.
Sample Answer
Direct answer
Choose Redis when you need richer data structures, persistence, or replication/high-availability (HA) built in; choose Memcached when you need the simplest possible pure key-value cache with the lowest per-operation overhead and multi-threaded read scaling out of the box.
Structured elaboration
- Data types: Redis supports strings, hashes, lists, sets, sorted sets, and more, useful when the cache itself needs to do more than store opaque blobs (e.g., a sorted set for a leaderboard). Memcached only stores simple key-value byte strings.
- Persistence: Redis can persist to disk (RDB snapshots, append-only file, AOF) so data can survive a restart; Memcached is purely in-memory with no persistence, so a restart is always a full cache flush.
- Replication / high availability: Redis has built-in replication and clustering (Sentinel for failover, Cluster for sharding); Memcached has no native replication, relying on the client or an external layer for HA.
- Memory efficiency: Memcached's simpler data model generally has lower per-key memory overhead for pure key-value use cases; Redis's richer data structures and features carry some additional overhead.
- Eviction semantics: both support least-recently-used (LRU) style eviction, but Redis offers more configurable policies (
allkeys-lru,volatile-ttl,allkeys-lfu, and others) versus Memcached's simpler slab-based LRU. - Operational features: Redis has a larger ecosystem for observability, Lua scripting for atomic multi-step operations, and pub/sub; Memcached's multi-threaded architecture can give it an edge on raw throughput for simple get/set at very high concurrency on a single node.
Worked example
For a session store: if sessions are pure key-value blobs, Memcached is a perfectly reasonable, simpler choice. If sessions need persistence across a restart (avoiding logging every user out simultaneously) or replication for HA, Redis with AOF persistence and Sentinel-managed failover is the safer choice, at the cost of a slightly more complex operational footprint.
Trade-offs and pitfalls
Choosing Redis "because it can do more" for a workload that is genuinely simple key-value adds operational surface area (persistence tuning, replication topology) without benefit; match the tool to the actual requirement. Memcached's lack of native replication means a node loss is a hard cache-miss event for everything that node held, with no automatic failover; that must be an explicit, accepted trade-off, not an oversight.
Design a mechanism to guarantee read-your-writes consistency for an authenticated user who may write in one region and read in another. Propose concrete approaches and explain the scalability and operational trade-offs of each.
Sample Answer
Direct answer
Guaranteeing read-your-writes when a user may write in one region and read in another means routing that user's reads to wherever their write is guaranteed visible, rather than trying to make every region's cache instantly consistent for everyone.
Structured elaboration
- Session affinity / sticky routing: after a write, route that user's subsequent reads to the same region (or the write's authoritative region) for a bounded window, so they always see their own change even while other regions are still catching up via replication. Simple to reason about, at the cost of some added routing complexity and potentially higher latency for that user during the sticky window.
- Redirect reads to the write region for a bounded time: similar to session affinity but implemented at the application/routing layer rather than infrastructure-level stickiness; the client (or a token it carries) signals "I just wrote, read from the authoritative source" for the next N seconds.
- Per-user replication priority: prioritize replicating a user's own write to the region they are currently reading from, ahead of the normal replication queue, so their own subsequent read is more likely to already reflect it without needing sticky routing at all.
- Tokens indicating last-update timestamp: the client carries a token (e.g., the write's timestamp or a version number) with subsequent requests; the reading region checks whether its local replica has caught up to at least that token before serving from cache, falling back to the authoritative region if not yet caught up.
- Causal metadata: more general than a single timestamp, causal metadata tracks the dependency chain of a write (which other writes it causally depends on) so a read can be guaranteed to see everything the user's own actions causally depended on, not just their literal last write; this is more complex to implement but generalizes better to multi-step user actions.
Worked example
A user updates their profile picture while in Europe, then immediately opens the app again while roaming and connecting through an Asia-Pacific region. With a version-token approach, their client carries the write's version number; the Asia-Pacific region checks its replica's version for that user against the token, and if it has not caught up yet (a race that can genuinely happen within a second or two of a write), it redirects that one read to the authoritative region rather than serving a stale cached profile picture.
Trade-offs and pitfalls
Session affinity and read-redirection both add operational complexity to routing and can increase latency for the affected user during the sticky window; they should be scoped narrowly (per-user, per-session, time-bounded) rather than applied broadly, or you lose most of the multi-region latency benefit for everyone. Causal metadata is powerful but expensive to implement correctly; do not reach for it unless simpler mechanisms (session affinity, version tokens) have proven insufficient for the actual product requirement.
Unlock Full Question Bank
Get access to all Caching Strategies and Distributed Caching interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.