Comprehensive knowledge of caching principles, architectures, patterns, and operational practices used to improve latency, throughput, and scalability. Covers multi level caching across browser or client, edge content delivery networks, application in memory caches, dedicated distributed caches such as Redis and Memcached, and database or query caches. Includes cache design and selection of technologies, defining cache boundaries to match access patterns, and deciding when caching is appropriate such as read heavy workloads or expensive computations versus when it is harmful such as highly write heavy or rapidly changing data. Candidates should understand and compare cache patterns including cache aside, read through, write through, write behind, lazy loading, proactive refresh, and prepopulation. Invalidation and freshness strategies include time to live based expiration, explicit eviction and purge, versioned keys, event driven or messaging based invalidation, background refresh, and cache warming. Discuss consistency and correctness trade offs such as stale reads, race conditions, eventual consistency versus strong consistency, and tactics to maintain correctness including invalidate on write, versioning, conditional updates, and careful ordering of writes. Operational concerns include eviction policies such as least recently used and least frequently used, hot key mitigation, partitioning and sharding of cache data, replication, cache stampede prevention techniques such as request coalescing and locking, fallback to origin and graceful degradation, monitoring and metrics such as hit ratio, eviction rates, and tail latency, alerting and instrumentation, and failure and recovery strategies. At senior levels interviewers may probe distributed cache design, cross layer consistency trade offs, global versus regional content delivery choices, measuring end to end impact on user facing latency and backend load, incident handling, rollbacks and migrations, and operational runbooks.
EasyTechnical
95 practiced
Compare TTL (time-to-live) based expiration with explicit eviction/purge. Provide examples of use-cases where TTL is sufficient and where explicit invalidation is necessary. Also list drawbacks of using long TTL values.
Sample Answer
**Answer (Full‑Stack Developer perspective)****Short answer**TTL-based expiration automatically lets cached items become stale after a set time; explicit eviction/purge invalidates specific items on-demand. Choose TTL when acceptable eventual staleness exists; choose explicit invalidation when correctness requires immediate updates.**When TTL is sufficient**- CDN or browser caching of static assets (JS/CSS/images) with versioned filenames — simple TTL works.- Session tokens / rate‑limit windows where slight delay is OK.- Analytics or metrics dashboards where data can be slightly out-of-date.**When explicit invalidation is necessary**- User profile or permission changes that must take effect immediately (e.g., revoke access).- Inventory/stock counts during checkout — need immediate consistency to avoid oversell.- Feature flags or pricing updates that must roll out instantly.**Drawbacks of long TTL**- Stale data served longer; consistency issues for critical flows.- Harder to roll out urgent fixes (UI/text, security headers).- Cache poisoning/incorrect state persists until expiry.- Requires extra invalidation mechanisms (cache-busting, versioning) increasing complexity.**Guidance**Use short TTLs plus explicit invalidation for mutable, critical data; use longer TTLs for immutable/versioned assets. Combine techniques: e.g., set TTL but also support targeted purge for emergency updates.
HardSystem Design
76 practiced
Design a global distributed cache for a web application deployed in multiple regions serving 1M reads/sec globally. The system must balance low read latency, reasonable freshness (a few seconds acceptable), and minimal cross-region traffic. Describe architecture (regional caches, invalidation propagation, replication), cache key design, consistency model choices (strong vs eventual), and how you'd measure and tune end-to-end user latency and backend load.
Sample Answer
**Overview / Goals**Design regional caches so reads hit local region (low latency), allow ~seconds freshness, and minimize cross-region traffic by async replication and targeted invalidation.**Architecture**- Regional cache clusters (e.g., Redis/ElastiCache per region) fronted by an edge layer (CDN or regional API gateway). App servers read from local cache; cache miss -> regional origin service.- Global coordination via a lightweight control plane: topic-based invalidation service (Kafka or cloud pub/sub) and optional global metadata store (small, strongly-consistent) for critical keys.- Replication: primary-write-to-origin; propagate updates as events to other regions. Use per-key version/timestamp.**Invalidation & Propagation**- On update: write to DB -> publish change event with key, version, timestamp.- Regional subscribers receive events and either invalidate or refresh cache. Use batching and jitter to reduce thundering herd.- For very hot keys, use background proactive replication (push new value) rather than invalidate.**Cache Key Design**- Keys = service:resource:id:versionTag (e.g., user:123:etag-456) to enable quick invalidation by version. Include region suffix for region-specific variants.**Consistency Model**- Default: eventual consistency (seconds) using versioned updates; accept slightly stale reads.- Provide strong consistency option per-key: proxy reads to origin or use read-through with synchronous global validation (for payments/profile-critical flows).**Metrics & Tuning**- Measure: P95/P99 read latency, cache hit rate (per-region), cross-region traffic volume, backend QPS, staleness (time difference between latest DB write and served value).- Tune: adjust TTLs, batching window for invalidations, replicate hot keys proactively, increase regional cache size or use LRU tuning, add edge CDN for static content.- Simulate: chaos tests (drop invalidation messages) and load tests to validate freshness vs traffic trade-offs.This balances low latency for most reads, bounded staleness, and minimal cross-region bandwidth while allowing per-key strong consistency where required.
EasyTechnical
89 practiced
What operational metrics would you track to evaluate a cache's health and effectiveness? Explain why each metric matters and propose reasonable alert thresholds or SLO-driven targets for a read-heavy API cache.
Sample Answer
**Overview — role perspective**As a full‑stack developer I track cache metrics that show correctness, efficiency, and user‑perceived latency for a read‑heavy API.**Key metrics and why they matter**- Cache Hit Ratio (hits / total requests) - Why: Directly reduces backend load and response time. - Target / alert: SLO ≥ 90–95%; alert if < 85% for 5m.- Effective Hit Rate (hits for hot keys / total) - Why: Shows whether common endpoints benefit from cache. - Target: top 10 keys ≥ 95% hit; alert if drops >10%.- Eviction Rate (evictions per second) & Eviction Ratio - Why: High evictions imply undersized cache or bad TTLs, leading to thrashing. - Alert: sustained evictions > 5% of requests or sudden spike (>3x baseline).- TTL / Age Distribution & Staleness Incidents - Why: Ensures freshness vs performance tradeoff. - SLO: 99% of cached items younger than configured TTL; alert on >1% stale serves.- Backend Origin Load (requests/sec to DB/service) - Why: Confirms cache reduces origin pressure. - Target: origin traffic reduced by expected % (e.g., 80% vs no‑cache); alert on unexpected increase.- Latency (p50, p95, p99 for cached vs origin) - Why: User experience; tail latency matters. - SLO: cached p99 < 50 ms; alert if p99 cached > 100 ms or gap to origin shrinks.- Error Rate & Miss Type (cold miss vs capacity miss) - Why: Troubleshooting cause of misses. - Alert: error rate increase >0.5% or miss-type shift.**Practical notes**- Combine alerts with dashboards: hit ratio trends, eviction spikes, origin traffic. - Tune thresholds based on traffic and cost; run canary changes and monitor SLO burn rates.
HardSystem Design
145 practiced
Design a caching solution for a real-time leaderboard that supports 100k updates/sec and millions of reads/sec with low tail latency. Requirements: near-real-time ranking, ability to show top-K globally and per-region, and acceptable eventual consistency within 1 second. Discuss data structures, cache layering, update fan-out, and trade-offs between precision and performance.
Sample Answer
**Clarify requirements**- 100k updates/s, millions reads/s, <1s eventual consistency, top-K global and per-region, low tail latency.- I’d build for read scalability and cheap, fast ranking updates with acceptable 1s staleness.**High-level architecture**- Ingest stream → stream processor (Kafka + Flink/KS) → ranking store + cache layer.- Serving tier: CDN + edge cache (Redis Cluster / Twemproxy) + regional in-memory ranking.**Data structures**- Per-shard sorted sets (Redis ZSET) for precise top-K; maintain global and per-region ZSETs.- Approximate layer: Count-Min Sketch + Heavy Hitters (Space-Saving) for ultra-high write throughput and memory savings — used to prefilter candidates for exact ZSET inserts.**Cache layering**- Hot edge caches (read-through Redis replicas) store top- K lists per region and global for O(1) reads.- Longer tail served from microservice that queries Redis ZSETs; fall back to approximate candidates if needed.- TTL ~500ms and background refresh every 200–800ms to meet <1s consistency.**Update fan-out**- Stream processor aggregates micro-batches (10ms–100ms) and emits delta updates to: - Exact ranking updater (batched ZADD/ZINCRBY) - Edge cache invalidation/push (pub/sub)- Use batching + pipelined Redis commands to handle 100k/s.**Trade-offs**- Precision vs perf: keeping every update in ZSETs costs CPU/memory; using sketches reduces cost but requires periodic reconciliation (exacting top candidates).- Consistency vs latency: aggressive background refresh gives fresher views but higher write/CPU. Use 1s window to batch writes.- Sharding: per-region and per-score-range shards to parallelize; merge top-K on read for global view (fast since K small).**Role fit / implementation notes**- As a full-stack dev I’d implement the API endpoints to read cached top-K, client WebSocket pushes for live updates, and a small admin UI to tune TTLs and observe metrics (Redis latency, cache hit-rate).- Monitoring: emit staleness, p50/p99 latency, cache hit rate; adjust batch windows and approximation thresholds accordingly.
EasyTechnical
71 practiced
Briefly explain eventual consistency in the context of caches. Describe three practical tactics to reduce correctness issues caused by stale cache reads: invalidate-on-write, versioned keys, and conditional updates. For each tactic, state a downside.
Sample Answer
**Brief definition (role context)** As a full‑stack developer I treat eventual consistency in caches as the guarantee that after an update, different readers may see stale values for a short time until cache and/or replicas converge. That’s fine for many read‑heavy web flows but can cause correctness issues for writes or transactional reads.**Tactic 1 — Invalidate-on-write** - What: On any update, remove/expire the related cache entries so next read hits the source of truth. - Example: After updating user profile in DB, send a Redis DEL key for user:id. - Downside: Window for race conditions (read between DB write and invalidate) and higher latency/DB load immediately after invalidation.**Tactic 2 — Versioned keys** - What: Include a version or timestamp in cache keys (user:123:v42). Writes bump version so old cache is ignored. - Example: Increment profile version in DB or a versions store, then writes use new key. - Downside: Cache memory grows (old versions remain until TTL) and requires coordination to bump versions.**Tactic 3 — Conditional updates (compare-and-swap)** - What: When writing cache based on a read, only replace if the current cached version matches expected (or use optimistic locking). - Example: Use Redis WATCH/MULTI or ETag/If-Match on HTTP cache layers to avoid overwriting newer data. - Downside: More complex logic and extra round‑trips; higher chance of retry/failure under contention.I prefer combining tactics: invalidate-on-write plus versioned keys for strong practical results, and conditional updates for high‑contention hotspots.
Unlock Full Question Bank
Get access to hundreds of Caching Strategies and Patterns interview questions and detailed answers.
