Caching Strategies and Distributed Caching Questions
Using caches to reduce latency and load: cache-aside, read-through, write-through, and write-behind patterns, TTLs, eviction policies, and distributed caches such as Redis or Memcached. Covers cache invalidation, stampede and thundering-herd protection, and the consistency tradeoffs of caching. Focuses on where and how to cache across tiers.
You must choose between Redis and Memcached to implement a session store for a web app. List trade-offs and recommend one choice. Consider persistence, data types, replication/HA, memory efficiency, eviction semantics, and operational features such as monitoring and backup.
Sample Answer
Direct answer
Choose Redis when you need richer data structures, persistence, or replication/high-availability (HA) built in; choose Memcached when you need the simplest possible pure key-value cache with the lowest per-operation overhead and multi-threaded read scaling out of the box.
Structured elaboration
- Data types: Redis supports strings, hashes, lists, sets, sorted sets, and more, useful when the cache itself needs to do more than store opaque blobs (e.g., a sorted set for a leaderboard). Memcached only stores simple key-value byte strings.
- Persistence: Redis can persist to disk (RDB snapshots, append-only file, AOF) so data can survive a restart; Memcached is purely in-memory with no persistence, so a restart is always a full cache flush.
- Replication / high availability: Redis has built-in replication and clustering (Sentinel for failover, Cluster for sharding); Memcached has no native replication, relying on the client or an external layer for HA.
- Memory efficiency: Memcached's simpler data model generally has lower per-key memory overhead for pure key-value use cases; Redis's richer data structures and features carry some additional overhead.
- Eviction semantics: both support least-recently-used (LRU) style eviction, but Redis offers more configurable policies (
allkeys-lru,volatile-ttl,allkeys-lfu, and others) versus Memcached's simpler slab-based LRU. - Operational features: Redis has a larger ecosystem for observability, Lua scripting for atomic multi-step operations, and pub/sub; Memcached's multi-threaded architecture can give it an edge on raw throughput for simple get/set at very high concurrency on a single node.
Worked example
For a session store: if sessions are pure key-value blobs, Memcached is a perfectly reasonable, simpler choice. If sessions need persistence across a restart (avoiding logging every user out simultaneously) or replication for HA, Redis with AOF persistence and Sentinel-managed failover is the safer choice, at the cost of a slightly more complex operational footprint.
Trade-offs and pitfalls
Choosing Redis "because it can do more" for a workload that is genuinely simple key-value adds operational surface area (persistence tuning, replication topology) without benefit; match the tool to the actual requirement. Memcached's lack of native replication means a node loss is a hard cache-miss event for everything that node held, with no automatic failover; that must be an explicit, accepted trade-off, not an oversight.
Describe how HTTP caching works using Cache-Control, ETag, and Last-Modified headers. Explain how a CDN and a Service Worker might interact with those headers and describe a conditional GET flow including a 304 response. Provide one example where a Service Worker should bypass CDN semantics.
Sample Answer
Direct answer
HTTP caching is coordinated through response headers: Cache-Control tells any cache (browser, content delivery network, CDN) how long and under what conditions a response can be reused, and ETag/Last-Modified let a client revalidate a possibly-stale cached copy cheaply via a conditional request instead of re-downloading the full response.
Structured elaboration
- Cache-Control: directives like
max-age=<seconds>(how long the response is fresh),no-cache(must revalidate before use, but can still be stored),no-store(never cache at all), andprivate/public(whether shared caches like a CDN may store it) give fine-grained control over caching behavior. - ETag: an opaque identifier (often a hash) for a specific version of a resource; a client that has a cached copy sends
If-None-Match: <etag>on its next request. - Last-Modified: a timestamp-based alternative to ETag; a client sends
If-Modified-Since: <timestamp>. - Conditional GET / 304 flow: the client's cached copy has expired (its
max-ageelapsed) but might still be valid; instead of re-fetching the full body, it sends a conditional request withIf-None-MatchorIf-Modified-Since. If the resource has not actually changed, the server responds304 Not Modifiedwith no body, and the client's existing cached copy is revalidated as fresh; if it has changed, the server responds normally with the new body and headers. - CDN and Service Worker interaction: a CDN typically respects
Cache-Controldirectly, serving from its edge cache without contacting the origin untilmax-ageexpires. A Service Worker sits in the browser and can implement its own caching logic (including bypassing normal HTTP semantics entirely) via the Cache API andfetchevent interception, which is useful when the app needs offline support or caching behavior more sophisticated than standard headers allow. - When a Service Worker should bypass CDN semantics: for content the app knows is safe to serve instantly from a local cache regardless of what a content delivery network (CDN)'s time-to-live (TTL) says (e.g., app shell assets for instant repeat-visit loads), a Service Worker can serve from its own cache first and revalidate in the background (a stale-while-revalidate pattern implemented at the application layer), rather than waiting on the CDN's standard freshness check.
Worked example
A JS bundle served with Cache-Control: max-age=31536000, immutable and a content-hashed filename (app.a1b2c3.js) never needs revalidation at all, since a new deploy produces a new filename/URL; "invalidation" is simply pointing to the new URL. A frequently-changing API response served with Cache-Control: max-age=0, must-revalidate plus an ETag lets the client always check freshness cheaply (a 304 response, no body) without re-downloading the full payload on every request when nothing has actually changed.
Trade-offs and pitfalls
Setting Cache-Control: public on a response containing per-user or sensitive data lets shared caches (a CDN, a corporate proxy) serve one user's private data to another; always use private or no-store for personalized responses. Relying on Last-Modified timestamp granularity (often only second-level precision) can miss legitimately distinct versions of a resource that changed within the same second; ETag avoids this by tying validation to actual content, not a coarse timestamp.
Explain and compare strategies to partition (shard) cache data across multiple cache nodes. Discuss rebalancing costs, techniques to reduce key movement, how to mitigate hot shards, and failover behavior when nodes are added or removed.
Sample Answer
Direct answer
Consistent hashing places both cache nodes and keys on a conceptual ring (by hashing each to a point on the ring) and routes a key to the next node clockwise from it, so adding or removing a node only remaps the keys between it and its neighbor, not the entire keyspace the way naive modulo hashing would.
Structured elaboration
- Why modulo hashing fails at scale: routing a key with
hash(key) % Nmeans every node-count change (N) reshuffles almost every key's destination node, since the modulo result changes for nearly all keys; that means a near-total cache flush (and a stampede of misses to the origin) on every scale-out or node failure. - Consistent hashing's improvement: only the keys that fell between the added/removed node and its next neighbor on the ring need to move; on average, adding the Nth node to a ring of N nodes remaps roughly 1/N of the keyspace, not close to 100 percent.
- Virtual nodes: mapping each physical node to many points on the ring (e.g., 100 to 200 virtual nodes per physical node) smooths out the otherwise-uneven distribution you'd get from a small number of random hash points, and makes the fraction of keys remapped on a node change proportional to that node's actual share of capacity rather than being lumpy.
- Alternatives: rendezvous hashing (highest-random-weight) computes, for each key, a weighted score against every node and picks the highest, which achieves similar minimal-remapping properties without needing an explicit ring structure. A proxy layer (client-side hashing vs a routing proxy like Twemproxy) changes WHERE the hashing decision is made but not the underlying algorithm choice.
- Hot shards: consistent hashing distributes keys evenly in aggregate but does not know that some keys are read far more than others; a node can still become hot if it happens to own a popular key, which is a separate problem from rebalancing cost (see hot-key mitigation).
- Failover behavior: when a node fails, its portion of the ring is picked up by its neighbor (or, with virtual nodes, spread across several neighbors); this is graceful for read-through/cache-aside caches (a "miss" for a remapped key just refetches from origin) but can look like data loss for a cache that was the system of record for anything.
Worked example
A cluster grows from 10 nodes to 11. Under naive modulo hashing, roughly 91 percent of keys change their target node (1−10/11), effectively invalidating almost the whole cache at once. Under consistent hashing with virtual nodes, only the keys that fall in the ring segment now owned by the new node move, roughly 1/11≈9% of the keyspace, leaving the other 91 percent of cached entries untouched and avoiding a stampede.
Trade-offs and pitfalls
Too few virtual nodes per physical node produces an uneven distribution (some nodes end up owning noticeably more of the ring than others by chance); 100+ virtual nodes per physical node is a common starting point to smooth this out. Consistent hashing solves REBALANCING cost, not hot-key load; do not expect it to fix a single overloaded key. Client-side hashing requires every client to agree on the exact same ring state, which becomes an operational hazard during a rolling deploy where old and new client versions might briefly disagree on routing; a proxy-side approach centralizes that decision at the cost of an extra network hop.
Design an invalidation pipeline using Change Data Capture (CDC) (for example Debezium into Kafka) to keep caches updated across multiple services. Discuss topic design, ordering guarantees per key, consumer group design, retry semantics, and how to avoid over-invalidation or event storms.
Sample Answer
Direct answer
An event-driven or change-data-capture (CDC) invalidation pipeline turns every write in the source of truth into a message that downstream consumers use to correct their caches, which requires deliberate topic design, ordering guarantees per key, and idempotent handlers so a redelivered or reordered event does not corrupt cache state.
Structured elaboration
- CDC as the source of events: a tool like Debezium reads the database's own write-ahead log and publishes a change event (insert/update/delete) to a message broker (e.g., Kafka) automatically, which avoids the risk of the application forgetting to publish an invalidation on some code path; the database itself becomes the single source of truth for what changed.
- Message schema: include the key, a version or timestamp, and for deletes a tombstone marker rather than omitting the message; consumers need the version to detect and discard out-of-order or duplicate events, and an explicit tombstone rather than a missing message for deletes.
- Ordering guarantees per key: partition the topic by entity key so all events for the same entity land in the same partition and are processed in order by a single consumer; this avoids a newer update being overwritten by a late-arriving older event.
- Idempotency handling: because most message delivery is at-least-once, a consumer must be able to safely apply the same event twice without harm; using the event's version as a conditional check ("only apply if this version is newer than what's cached") makes redelivery safe.
- Consumer group design: scale consumers horizontally by partition, keeping the per-key ordering guarantee intact since Kafka guarantees order within a partition, not across the whole topic.
- Avoiding over-invalidation or event storms: a bulk update touching millions of rows produces millions of invalidation events; batch or debounce invalidations for the same key within a short window, and consider whether a coarser signal (e.g., "namespace version bumped") is more appropriate than a firehose of individual key invalidations for that specific case.
Worked example
A price-update job that touches 2 million product rows in a batch would, naively, produce 2 million invalidation events in a burst; consumers applying them all immediately would generate 2 million cache misses in a short window, an invalidation-triggered stampede against the very origin the cache exists to protect. Batching invalidations per consumer (apply up to N per second, or coalesce multiple updates to the same key within a short window into one) smooths this into a manageable, sustained rate instead of a spike.
Trade-offs and pitfalls
Relying purely on event-driven invalidation with no time-to-live (TTL) backstop means a lost or unprocessed event becomes permanent staleness with no self-correction; always pair this with a TTL ceiling. Partitioning by key for ordering can create a hot partition if one key is updated far more often than others, which needs the same hot-key mitigations as any other hot-partition problem.
Explain the trade-offs between client-side caching (in-process or browser) and a server-side shared cache (Redis). From an operations standpoint, what concerns differ between the two approaches?
Sample Answer
Direct answer
Client-side (in-process or browser) caching is faster and needs no network hop but is invisible and hard to invalidate from the server; a server-side shared cache (Redis) is slower per read (a network hop) but is centrally observable, controllable, and consistent across every consumer.
Structured elaboration
- In-process caching: fastest possible access (no network at all), but exists as N independent copies across N instances, invisible to any centralized monitoring or invalidation mechanism unless you specifically build one (e.g., pub/sub to every instance).
- Browser caching: similarly fast for the end user, but the server has essentially no direct control once data leaves it; "invalidation" really means waiting out a time-to-live (TTL) or changing a versioned resource identifier.
- Server-side shared cache (Redis): a single, centrally-managed copy; invalidation is a single operation that immediately affects every consumer, and its state is directly observable via standard monitoring, at the cost of a network round-trip on every access.
- Operational concerns for a shared server-side cache: capacity planning, replication/failover for availability, and security (access control, encryption) are all first-class operational concerns that a purely in-process or browser cache does not have in the same way, since those are either per-instance (in-process, trivially "available" as long as the instance is up) or entirely outside your infrastructure (browser).
- Service-mesh sidecar caching: a variant worth naming, a sidecar proxy running alongside each service instance can cache responses transparently at the network layer; this gets some of in-process caching's speed benefit without embedding caching logic in application code, but invalidation and consistency questions are similar to any other local-cache design (each sidecar has its own copy).
- A hybrid, fast-tier-plus-persistent-tier design: combining a fast in-memory tier for hot items with a persistent, larger tier for warm items (promotion/demotion between them based on access) gets some of both worlds' benefits, at the cost of managing two tiers' worth of operational complexity.
Worked example
An application caching a rarely-changing configuration value both in-process (near-zero latency access on every request) and in Redis (as the source the in-process cache warms from, and as what gets explicitly invalidated on a config change) gets the speed benefit of in-process access for the common case while still having a centrally controllable invalidation point; the in-process copies use a short TTL as a backstop in case a pub/sub-based invalidation signal is missed by a given instance.
Trade-offs and pitfalls
Relying purely on in-process caching for data that changes and needs prompt, reliable invalidation across many instances is a common design mistake; without a centralized signal (or at least a short backstop TTL), some instances can serve stale data indefinitely after a change. From an operations standpoint, a shared server-side cache is a dependency you must monitor, secure, and plan capacity/failover for, in a way an in-process cache simply is not; that operational surface is a real cost, not just a technical detail.
Unlock Full Question Bank
Get access to all Caching Strategies and Distributed Caching interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.