Comprehensive knowledge of caching principles, architectures, patterns, and operational practices used to improve latency, throughput, and scalability. Covers multi level caching across browser or client, edge content delivery networks, application in memory caches, dedicated distributed caches such as Redis and Memcached, and database or query caches. Includes cache design and selection of technologies, defining cache boundaries to match access patterns, and deciding when caching is appropriate such as read heavy workloads or expensive computations versus when it is harmful such as highly write heavy or rapidly changing data. Candidates should understand and compare cache patterns including cache aside, read through, write through, write behind, lazy loading, proactive refresh, and prepopulation. Invalidation and freshness strategies include time to live based expiration, explicit eviction and purge, versioned keys, event driven or messaging based invalidation, background refresh, and cache warming. Discuss consistency and correctness trade offs such as stale reads, race conditions, eventual consistency versus strong consistency, and tactics to maintain correctness including invalidate on write, versioning, conditional updates, and careful ordering of writes. Operational concerns include eviction policies such as least recently used and least frequently used, hot key mitigation, partitioning and sharding of cache data, replication, cache stampede prevention techniques such as request coalescing and locking, fallback to origin and graceful degradation, monitoring and metrics such as hit ratio, eviction rates, and tail latency, alerting and instrumentation, and failure and recovery strategies. At senior levels interviewers may probe distributed cache design, cross layer consistency trade offs, global versus regional content delivery choices, measuring end to end impact on user facing latency and backend load, incident handling, rollbacks and migrations, and operational runbooks.
MediumTechnical
74 practiced
Design a cache key naming and versioning strategy for a microservice whose response schema changes frequently. Explain how you would support rolling upgrades, avoid stale data breaking clients, and manage key explosion over time with examples of key formats and migration steps.
Sample Answer
Overview: Use explicit, discoverable versioning in keys plus runtime compatibility rules so old and new clients/servers coexist during rolling upgrades. Combine versioned keys, read-fallbacks, controlled migration and eviction to avoid stale-breakage and key explosion.Key format examples:- Primary: svc:{resource}:{id}:v{schemaVersion} (e.g. orders:12345:v3)- With content type/format: svc:orders:12345:v3:json- For derived caches (index/secondary): svc:orders:index:{userId}:v3:hashPrinciples:- Version in key name (not just value) so different schemas never collide.- Backward-compatible reads: server tries newest key then falls back to older versions with an adapter/transform.- Writes: during migration use dual-write (write vN and vN+1) behind a feature flag, then switch read preference.- TTLs and tombstones prevent indefinite key growth.Rolling upgrade strategy:1. Deploy new code supporting both vN and vN+1 read paths.2. Enable dual-write for a percentage of traffic (canary) — write both vN and vN+1 keys.3. Read preference: new instances prefer vN+1, fallback to vN if miss, then populate vN+1 lazily (read-through).4. Gradually increase traffic to new code; monitor errors/latency.5. When stable, stop writing vN and set a finalization timeout.Avoiding stale data breaking clients:- Never delete vN keys until all clients relying on vN are off or until TTL elapsed.- Use adapter layer at service boundary to present stable API contract: transform cached payloads on read to requested version.- Conservative TTLs during transition; add schema version header in responses to detect mismatches.Managing key explosion:- Rotate namespaces: once vN is retired, run batch migration/compaction to convert remaining vN keys to vN+1 or purge if obsolete.- Use TTLs for short-lived caches and explicit purge jobs for long-lived.- Aggregate or hash large index keys (store lists in a single key with pagination) to reduce cardinality.- Maintain a small metadata store mapping deprecated versions and migration progress (not in cache).Migration steps (practical sequence):- Step 0: Add transform adapters and feature flag.- Step 1: Enable dual-write for 5% traffic; add monitoring.- Step 2: Increase to 100% dual-write once stable.- Step 3: Switch reads to prefer vN+1; on miss, fallback to vN and write-through to vN+1.- Step 4: After sufficient time, run background job to scan vN keys and migrate or delete according to retention policy.- Step 5: Remove vN read-path and reclaim namespace.Trade-offs:- Dual-write increases write cost and complexity but minimizes downtime.- Lazy migration reduces upfront migration load but prolongs key retention.- Aggressive TTLs reduce key explosion risk but may increase cache misses.Monitoring & safety:- Track key counts per version, hit/miss ratios, error rates during transforms.- Implement alarms for unexpected growth or conversions failing.This strategy ensures safe rolling upgrades, prevents schema collisions, and keeps cache cardinality manageable.
EasyTechnical
81 practiced
List the essential metrics and alerts you would instrument for a distributed caching layer. Explain why each metric matters and suggested alert thresholds or guardrails for hit ratio, eviction rate, tail latency, backend QPS, and error rates.
Sample Answer
Essential metrics to instrument for a distributed caching layer (and why):- Cache hit ratio (global and per-population): primary indicator of cache effectiveness; low hit-rate means more load to backends.- Eviction rate (items/sec, per-node): signals capacity pressure or TTL issues; high evictions may indicate under-provisioning or workload churn.- Tail latency (p50/p95/p99 for get/put): user-visible performance; tail spikes impact UX.- Backend QPS from cache misses: shows load offloaded to origin; helps capacity planning.- Error rates (cache errors, timeouts, connection failures): operational health of cache nodes and network.- Memory usage and free memory %, CPU, GC pauses: resource bottlenecks causing evictions/latency.- Per-key hot-spot metrics / top keys and request skew: detect thundering-herd patterns.- Replication/sync lag and cluster health (node joins/fails): consistency risk indicators.Suggested alert thresholds / guardrails:- Hit ratio: alert if global drops >10% absolute (e.g., from 85%→75%) or <70% sustained 5m; per-service threshold may be higher (≥80%).- Eviction rate: warning if >1% of cache population evicted per minute or sustained >X items/sec relative to dataset (tunable); critical if sudden 3x spike.- Tail latency: p99 > 100–200ms for reads (depending on SLO); alert on sustained p95 breach for 5m.- Backend QPS: alert if miss-driven backend QPS increases >30% over baseline or approaches backend capacity % (e.g., 70–80%).- Error rates: cache error rate >0.5–1% sustained or sudden spike of 5x baseline triggers incident.- Resource guards: node memory >85–90% or GC pause >500ms → warning/auto-scale.- Hot-key detection: single key >1% of requests or causing backend bursts → alert and mitigation (rate-limit, sharding).Rationale: combine effectiveness (hit ratio, evictions) with reliability/performance (latency, errors, resource usage) and operational triggers (backend QPS, hot-keys) so you detect both degradation of value and failures that propagate to origin systems. Tune thresholds to workload and SLOs; use multi-window alerts (instant spike vs sustained) to reduce noise.
MediumTechnical
81 practiced
You propose adding a cache in front of a user profile DB. Define an experiment (A/B or canary) to quantify the impact of caching. Specify which metrics you will collect, duration, required sample size considerations, how to avoid confounders, and how to interpret results for backend cost and user latency.
Sample Answer
Experiment design (Canary + controlled A/B):- Roll out a canary to 10% of production traffic for 1 week to validate safety, then run a 50/50 A/B for 2–4 weeks to measure steady-state effects across traffic patterns.Metrics to collect:- Cache: hit rate, miss rate, TTL expirations, evictions, load-on-miss.- Backend DB: RPS, CPU, memory, disk I/O, query latency, number of read replicas used.- End-user latency: API latency (P50, P90, P95, P99) for profile endpoints; client-perceived metrics if available (page load or API-to-render time).- Errors: 5xx rate, cache-related errors, stale-read incidents.- Cost proxies: DB instance-hours, read-replica counts, egress, cache instance-hours.- Business/SLO: % requests meeting latency SLO, user-facing error rate.Sample size & duration:- Power for detecting the primary lift (e.g., 30% DB read reduction or 10% P95 latency drop). Use standard sample-size formula for means/proportions: n ≈ (Z_{1-α/2} * σ / δ)^2, where δ = minimum detectable effect and σ is baseline stddev of the metric. Estimate σ from recent telemetry; choose α=0.05, power=0.8. For high-variance P99, increase sample or extend duration.- Ensure duration covers weekly seasonality (minimum 2 weeks; prefer 4) and cache warm-up (exclude initial warm-up window or use warmed canary).Avoiding confounders:- Randomize users or requests consistently (sticky assignment) so user experience isn’t mixed across cohorts.- Use feature flags and identical code paths except caching logic.- Warm caches before measuring (prime with background job or run a short warm-up).- Keep deployment/environment identical (same DB pool sizes, replicas) to avoid resource noise.- Monitor and exclude incident windows and rollout-related throttling.- Segment results by user/device/region to detect heterogeneous effects.Interpretation & decision rules:- Backend cost: translate DB RPS reduction into cost savings (saved instance-hours or reduced read-replica needs). Example: if reads drop 40%, run cost model: saved RPS * cost_per_RPS_hour → projected monthly savings. Factor cache cost (instance-hours + memory).- Latency: compare P95/P99 between cohorts; if SLO compliance improves (statistically significant) and no increase in error/staleness, treat as positive.- Safety thresholds: no >0.1% increase in error rate, no stale-read incidents above acceptable rate.- Final decision: enable if statistically significant DB-cost reduction net of cache cost and no degradation of user latency or reliability; otherwise iterate (tune TTL, sizing, or selective caching).Example quick calculation:- Baseline DB reads = 10k RPS, after caching 6k RPS (40% reduction). If DB cost = $0.0001 per read => monthly savings ≈ (4k RPS * 3600 * 24 * 30) * 0.0001 minus cache cost. Show confidence intervals for all estimates.
MediumTechnical
86 practiced
Compare Redis master-replica replication and Redis Cluster sharding. Explain trade-offs for high availability, read scaling, write scaling, multi-key ops, slot migrations, and propose which to use for a low-latency read-heavy service that also has frequent writes.
Sample Answer
Requirements & constraints:- Low-latency, read-heavy service but with frequent writes.- Need HA, horizontal read & write scaling, and reasonable multi-key operation behavior.High-level comparison1) Master-replica replication- Architecture: single master accepts writes; replicas serve reads (async replication).- High availability: simple failover possible with Sentinel or orchestrator; failover incurs leader election and possible data loss (async) unless using disk-based AOF/RDB sync strategies.- Read scaling: good — add replicas to scale reads and offload master.- Write scaling: none — single master is a write bottleneck.- Multi-key ops: fully supported when keys on same instance.- Slot migrations: N/A (no sharding).- Trade-offs: simple to operate, low-latency reads from replicas, but master write bottleneck and potential replication lag causing stale reads.2) Redis Cluster (sharding)- Architecture: data partitioned into 16,384 slots across multiple masters; each master can have replicas.- High availability: built-in failover per shard; still possible cross-shard complexity during failures.- Read scaling: good across cluster by distributing reads to replicas, but client must be cluster-aware.- Write scaling: horizontal — writes distribute across shard masters (improves throughput).- Multi-key ops: limited — multi-key commands require keys in same slot (or use client-side transaction patterns); cross-slot ops either fail or require client-side coordination.- Slot migrations: resharding supported but involves moving keys and can transiently increase latency; online rebalancing possible but operationally heavier.- Trade-offs: better write scalability and shard-level HA; more operational complexity and constraints for multi-key transactions.Recommendation for this service- Use Redis Cluster with replicas per master. Rationale: you need read low-latency and frequent writes — cluster gives horizontal write throughput plus replica reads. Mitigate stale reads by: - Directing critical reads to masters or using client-side read-after-write routing. - Using smaller shard key cardinality to co-locate related keys when you need multi-key ops. - Provisioning sufficient replica count and monitoring replication lag.- If multi-key transactions across many keys are frequent and correctness trumps throughput, prefer master-replica single-shard (or an application-level aggregator) to keep atomic multi-key semantics, accepting write scaling limits.Operational best practices- Use cluster-aware clients, enable TLS/auth, monitor replication lag, set up automated failover and backup, use key design to minimize cross-slot ops, and perform rolling resharding during low traffic windows.
MediumSystem Design
95 practiced
Design a multi-level caching architecture for an e commerce product page that serves 200K RPS globally across 10 regions. Requirements: p95 page load under 100ms, product metadata is read-heavy, inventory must be accurate at checkout, personalization required for logged-in users, and support for 100M SKUs. Sketch tiers, where to place caches, invalidation strategies, and consistency trade-offs.
Sample Answer
Requirements & constraints:- 200K RPS globally across 10 regions; p95 page load <100ms- Product metadata: read-heavy, 100M SKUs- Inventory must be accurate at checkout (strong consistency)- Personalization for logged-in users (per-user data)- Cost/scale and regional latency considerationsHigh-level multi-level cache tiers:1. CDN + Edge Compute (global) - Cache full product page HTML or pre-rendered JSON for anonymous users (TTL + stale-while-revalidate). - Use Lambda@Edge / Edge Functions to merge small personalized fragments (price badge, recommendations) into cached pages.2. Regional Edge Cache (per-region POP) - Large-memory LRU caches (Redis Cluster or in-memory KV) holding product metadata and rendered fragments to meet p95 latency. - Partition by SKU range/hash; use replication across AZs in region.3. Application / Service-level cache (per-service instance) - Local in-process LRU for hottest keys, for microsecond access.4. Read caches / Nearline DB cache - Redis/KeyDB clusters for product metadata, pricing, promotion rules; Materialized views in read-optimized stores (DynamoDB DAX / Memcached).5. Authoritative Stores - Product catalog and inventory in strongly-consistent stores: inventory in a transactional DB/service (e.g., RDBMS or strongly-consistent NoSQL with conditional writes) with regional leaders or global transactions for checkout.Where to place caches & personalization:- CDN/Edge: anonymous pages, static assets, image CDN.- Regional Redis clusters: product metadata, SKU-level aggregated data, price snapshots.- Per-user personalization: store user profile + session in regional Redis; compute personalized recommendations via feature service — cache recommendations per user-session with short TTL.- Use edge compute to apply personalization tokens to cached pages to avoid regenerating full page.Invalidation & freshness strategies:- Cache-aside for metadata: on read, check cache; on miss, read DB and populate cache.- Writes/updates (price, catalog changes): publish invalidation messages via global pub/sub (Kafka/Global PubSub) to regional caches; consumers evict or update entries (soft invalidate + background refresh).- Inventory: no long-lived cache for available quantity used at checkout. Use a short-lived inventory snapshot for display (e.g., “In stock”), but at checkout perform a strong transactional reserve (optimistic lock/compare-and-swap) against authoritative inventory.- Stale-while-revalidate for metadata to guarantee sub-100ms p95 while revalidating in background.- Use versioned keys (sku:v123) to simplify invalidation and prevent race conditions.Consistency trade-offs:- Metadata & personalization: eventual consistency acceptable; favor low latency and cache hit rate. Use short TTLs for dynamic fields (price/promotion) and publish updates to reduce staleness window.- Inventory at checkout: strong consistency required. Always perform final availability check and atomic reservation in authoritative inventory service; implement idempotency, two-phase commit or distributed locks for multi-step flows (reserve -> payment -> commit).- For high-scale writes (catalog updates), use batch windowing and versioning to limit invalidation storm.Scalability & operational considerations:- Shard caches by SKU hash; use consistent hashing for cluster membership changes.- Hot-key mitigation: detect top SKUs, pin them on larger nodes or use read replicas; use request coalescing to prevent thundering herd.- Bloom filters at edge to avoid unnecessary origin requests for non-existent SKUs.- Cache warming for launches and A/B tests.- Observability: latency, hit/miss rates, invalidation lag, inventory reservation latencies. SLOs: cache hit target (e.g., >95% regional) to meet p95 <100ms.- Disaster mode: if cache unavailable, fall back to regionally replicated read store; if inventory service degraded, place time-limited holds or reduce purchase concurrency.Why this meets requirements:- CDN + regional caches minimize round trips and ensure sub-100ms p95 for most traffic.- Personalization done as small edge-merged fragments keeps per-user latency low while reusing cached heavy content.- Strong transactional inventory check at checkout preserves correctness while allowing relaxed consistency for display-level data.- Sharding, pub/sub invalidation, and hot-key strategies scale to 100M SKUs and 200K RPS.
Unlock Full Question Bank
Get access to hundreds of Caching Strategies and Patterns interview questions and detailed answers.
