Caching Strategies and In Memory Storage Questions

Caching strategies for improving performance and reducing latency: HTTP caching semantics (Cache-Control, ETag, conditional requests, Vary), application-level caching with Redis and Memcached, in-memory data structures for caching, cache eviction policies (LRU, LFU, FIFO), cache invalidation strategies, TTL selection and trade-offs, and the consistency and performance implications of deciding what and when to cache.

MediumSystem Design

50 practiced

Design a caching layer for product detail pages in an e-commerce application serving 10,000 requests per second. Describe cache placement across CDN, edge, application cache, and DB; cache key design including variants (currency, locale, logged-in state); invalidation strategy when product details or inventory change; choosing TTLs; and fallback behavior for cache misses.

Sample Answer

**Clarify goals & constraints**- 10k RPS read-heavy product pages; low-latency (<200ms) for most users, correct product details, eventual inventory consistency acceptable for some flows (cart checkout uses strong check).

**High-level placement**- CDN (CloudFront/Cloudflare): cache full product-page HTML or product JSON responses for anonymous users; global low-latency edge.- Edge workers (Lambda@Edge/Cloudflare Workers): handle locale/currency rewrites, A/B variants, and quick routing to origin or app cache.- Application cache (Redis cluster / ElastiCache): store personalized variants (logged-in specific pricing, offers), and authoritative JSON for origin.- DB (Postgres/Mongo): source of truth; updates emit events to invalidate/refresh caches.

**Cache key design**- Base key: product:{productId}- Include orthogonal variants as suffixes: - product:{id}:locale:{en-US}:currency:{USD}:auth:{anon|userTierA|userId?} - For heavy personalization, separate fragment caches: product:{id}:public and product:{id}:private:user:{userId}:segments:{tier}- Keep keys composable so public cache is reused; private data stored only in app cache.

**Invalidation strategy**- On product metadata update (title, images, price promotions): - Write to DB → publish an event (Kafka/SNS). - Consumers: - CDN purge API for specific URL patterns (or send surrogate-control purge) for affected routes. - App-layer listener evicts Redis keys or writes updated JSON to Redis (cache-warming).- For inventory: - Use two-tier: inventory service with its own short-ttl cache (5–15s) and DB updates push inventory events. - Avoid aggressive global invalidation; instead update inventory store and let app read-on-write for carts/checkout.- For promotion or price changes that must be immediate: immediate purge + write-through refresh to Redis.

**TTLs**- CDN: 30s–5min for dynamic product JSON/HTML depending on volatility; use Cache-Control + stale-while-revalidate for smoother UX.- Edge workers: mirror CDN TTL logic.- App Redis (public product): 10s–60s; for stable fields (images, description) can be longer (5–30 min) in separate keys.- Inventory: 5–15s; or no-cache with event-driven updates for accuracy in checkout.

**Fallback & cache-miss behavior**- Cache-aside pattern: - On miss: check app cache → miss → fetch from DB/service → populate Redis → return. - Backpressure: use request coalescing / locking (single-flight) to prevent stampede.- If origin/db is slow: - Serve stale cached entry if available (stale-while-revalidate) and trigger background refresh. - If nothing available, return origin-generated minimal payload (product id + “unavailable”) with 503 for non-critical fields, and surface graceful UI message.- Checkout path always validates inventory with the authoritative inventory service before confirming.

**Operational & reliability**- Monitor cache hit ratio, purge latencies, and event delivery.- Implement metrics and dashboards; alert on low hit-rates or failed invalidations.- Run periodic cache-warm jobs for top N products.

This design balances latency (CDN/edge), personalization (Redis/app cache), and correctness (DB + event-driven invalidation), with short TTLs for volatile fields and robust fallbacks to avoid outages.

EasyTechnical

46 practiced

What is cache warming (preloading) and why would a backend service use it? Describe practical strategies to warm caches during deployment or scale-up events: background prefetch workers, replaying recent traffic, precomputing popular keys, and selectively priming edges. Discuss trade-offs including additional load and potential stale priming.

HardTechnical

44 practiced

Your Redis nodes are showing high memory fragmentation and periodic latency spikes under heavy write load. Outline a detailed diagnostic and remediation plan: which Redis INFO and system metrics to check, how allocator settings like jemalloc affect fragmentation, strategies such as reworking key/value sizes, enabling lazyfree or defrag, tuning AOF/RDB settings, and application-level batching to reduce churn.

Sample Answer

**High-level approach**1) Diagnose whether fragmentation is Redis/allocator-level or OS-level. 2) Mitigate quickly (config tweaks, lazy free, batching) to stop spikes. 3) Long-term fixes: data-model changes, allocator tuning, AOF/RDB policy.

**What to check (Redis INFO & commands)**- redis-cli INFO memory — inspect: - used_memory - used_memory_rss - used_memory_peak - mem_fragmentation_ratio (used_memory_rss / used_memory) - mem_allocator (e.g., jemalloc)- redis-cli MEMORY STATS and MEMORY USAGE <key> for hot keys- redis-cli MEMORY DOCTOR for hints- INFO stats and clients to correlate clients/ops/sec with spikes- MONITOR / slowlog / latency doctor to correlate latency spikes with write bursts

**System metrics**- top/htop for CPU; vmstat for page faults; iostat/dstat for disk I/O (AOF rewrite)- /proc/<pid>/smaps and pmap to inspect RSS vs mappings- free -m and /proc/meminfo for overall memory pressure- perf/top or eBPF traces to find stop-the-world events- OS OOM or swapping indicators; avoid swap

**How jemalloc affects fragmentation**- jemalloc splits memory into arenas and bins; many small allocations + frees cause internal fragmentation and cross-arena waste.- Fragmentation may be visible as used_memory_rss >> used_memory.- You can tune jemalloc using MALLOC_CONF or build-time options (e.g., background_thread:true, metadata_thp:false) — background threads can release pages back to OS; MALLOC_CONF=background_thread:true,dirty_decay_ms:15000,muzzy_decay_ms:15000 can help.- If Redis built with libc malloc fragmentation patterns differ — check mem_allocator.

**Quick remediation steps**- Enable lazy free to avoid blocking frees on main thread: - configure lazyfree-lazy-eviction, lazyfree-lazy-server-del, lazyfree-lazy-user-del = yes- Enable active defrag (if Redis >= 4.0): - CONFIG SET activedefrag yes - Tune active-defrag-cycle-min/max and thresholds (start modest values, monitor CPU)- Tune jemalloc via MALLOC_CONF environment; test in staging before prod.

**AOF/RDB tuning**- If AOF fsync=always, heavy writes block — set appendfsync everysec (trade durability).- Use AOF rewrite settings; schedule rewrites to avoid coinciding with peaks; consider aof-use-rdb-preamble or RDB snapshots for faster rewrites.- For RDB persistence, avoid very frequent full snapshots under heavy write churn.

**Data/model strategies to reduce churn**- Reduce number of keys/objects: compact multiple small keys into hashes or JSON blobs when appropriate.- Normalize value sizes: avoid many slightly-different-sized strings that increase allocator churn.- Use pipelining/batching writes from application to reduce write bursts (batch multiple SETs/HSETs in a pipeline).- Prefer HSET for multiple fields vs multiple SET calls to reduce allocations.- TTL-friendly patterns: avoid mass expirations at once; spread expirations (jitter).

**Profiling & validation**- Reproduce in staging with workload generator (redis-benchmark, custom script) to measure fragmentation_ratio and latency.- Use jemalloc/prof tools (mallctl, jeprof) to inspect allocation patterns if available.- Monitor after each change: fragmentation_ratio, latency percentiles, CPU during defrag.

**Trade-offs**- Active defrag and jemalloc background threads use CPU — increases CPU but reduces RSS/latency spikes.- Lazyfree reduces pause time but defers memory release: transient memory use rises, and background free work consumes CPU.- AOF durability vs latency trade-offs (fsync policy).

Summary: start by measuring memory stats & system metrics, enable lazy free + active defrag carefully, tune jemalloc background decay, reduce allocation churn via data-model and batching, and adjust AOF/RDB to reduce I/O stalls. Validate via staged repro and iterate.

EasyTechnical

60 practiced

Explain cache-aside (lazy loading), write-through, and write-back caching patterns. For each pattern describe read and write flows, consistency guarantees, failure scenarios (power loss, crashes), and typical backend use-cases such as user profile reads or inventory updates. Which pattern would you choose for high write intensity and why?

Sample Answer

**Overview (short)** I'll describe cache-aside (lazy), write-through, and write-back with read/write flows, consistency, failure risks, example backend use-cases, then recommend for high write intensity.

**Cache-aside (lazy loading)** - Read flow: check cache → miss → read DB → populate cache → return. - Write flow: write DB → invalidate or delete cache (or update cache). - Consistency: eventual if you invalidate correctly; simple to reason about. - Failures: if DB write succeeds but app crashes before cache invalidation you may serve stale cache until TTL expires; power loss affects DB durability as usual. - Use-cases: user profile reads, product detail pages where reads >> writes.

**Write-through** - Read flow: cache hit returns; on miss read DB and populate cache. - Write flow: write both cache and DB synchronously (cache updated/written through). - Consistency: strong — cache reflects DB after write returns. - Failures: higher write latency; if cache write succeeds but DB fails you must rollback cache to avoid divergence. Power loss risks depend on DB durability; in-memory cache loss can be reloaded from DB. - Use-cases: session stores, user settings where immediate consistency matters.

**Write-back (write-behind)** - Read flow: read cache; miss may load from DB. - Write flow: write only to cache; asynchronously flush dirty entries to DB later. - Consistency: window of inconsistency; risk of data loss if cache (in-memory) crashes before flush. - Failures: highest risk—power loss/crash can lose recent writes unless cache persists/writes WAL. Complexity with eviction and crash recovery. - Use-cases: high-throughput counters, analytics buffering, or when DB write cost is high and some delay is acceptable.

**Which for high write intensity?** Choose write-back when write throughput and low latency are primary and occasional data-loss window is acceptable, but only if you add durability measures (replicated/persistent cache, write-ahead log). If you cannot tolerate data loss, prefer write-through with optimized DB (batching, faster storage) or use append-only logs/streaming to decouple writes.

EasyTechnical

50 practiced

List and compare common cache eviction policies such as LRU, LFU, FIFO, random, and TTL-based eviction. Explain which policy is preferable under different workload patterns (temporal locality, skewed popularity, uniform access) and give examples of how hybrid policies like Redis's volatile-lru help in practice.

Unlock Full Question Bank

Get access to hundreds of Caching Strategies and In Memory Storage interview questions and detailed answers.

Join thousands of developers preparing for their dream job.