Caching Strategies and Cache Aside Pattern Questions
Understand caching layers (Redis, Memcached) to reduce database load. Discuss cache-aside (lazy loading), write-through, write-behind patterns, cache invalidation strategies, and consistency between cache and database.[4]
MediumTechnical
82 practiced
Explain how cache-aside interacts with database transactions and isolation levels. Provide a scenario where a read-after-write within a transaction could return stale data if improperly cached, and how to prevent it.
Sample Answer
Cache-aside (lazy loading) means application code first checks the cache, on miss reads the DB, then populates the cache. That pattern is orthogonal to database transactions and isolation levels, but they interact importantly: a transaction’s uncommitted writes are invisible to other transactions depending on isolation (e.g., Read Committed hides them; Serializable/Repeatable Read may show different behaviors). If you populate or read cache incorrectly during a transaction, you can expose stale or inconsistent data.Scenario: Transaction T1 updates a row r (UPDATE r = v1) but has not yet committed. Immediately afterwards, within the same transaction T1 there’s a read of r. If application uses cache-aside and, on the UPDATE, invalidates the cache but a racing process repopulates the cache from the DB (reading the pre-commit value v0 because T1’s update is not visible yet), the cache now contains stale v0. When T1 later reads from cache (or the app reads from cache expecting its own update), it sees v0 — stale relative to T1’s uncommitted write.Prevention strategies:- Within the same transaction, always read from the database (use the same DB connection/transaction context) rather than cache. Ensure your DB client provides transaction-scoped reads.- Perform cache writes/invalidation after commit (post-commit hooks). Only populate cache with committed state.- If low latency requires immediate cache visibility, use a transaction coordinator to apply cache updates atomically with DB commit (e.g., write-through with two-phase commit or outbox pattern that updates cache after commit).- Use versioning/ETags or logical timestamps in cache entries so readers can detect stale entries and fall back to DB.- For strong consistency needs, prefer Read Committed + cache populated post-commit, or avoid cache for transactional read-after-write paths.Key operational note: instrument cache-miss/invalidation latencies and monitor for cache-staleness errors; add alerts if high rate of transaction-read inconsistencies appears.
HardTechnical
96 practiced
Design an autoscaling strategy for a Redis caching tier that experiences diurnal traffic spikes. Include metrics to trigger scaling, approaches for scaling out/in safely (rebalancing keys, draining), and cost vs performance trade-offs.
Sample Answer
Requirements & constraints:- Serve diurnal traffic with predictable morning/evening spikes; SLOs: 99.9% read latency < 5ms, cache hit ratio ≥ 90%. Minimize cost while avoiding eviction storms and high latency.High-level approach:- Use a horizontally scalable Redis architecture (Redis Cluster or managed Redis with clustering) behind a proxy/sidecar client library that understands sharding.- Autoscale capacity (nodes or shard replicas) based on resource + workload metrics, with safe rebalancing and draining steps automated.Metrics & triggers (primary + safety):- Primary triggers: - 95th/99th percentile read latency > threshold (e.g., >5ms sustained 2m) - Cache miss rate increase causing origin load > threshold (miss rate up by X% or origin RPS > baseline) - Memory usage per node > 75–80% for 5+ minutes (avoid sudden evictions) - Eviction rate > 0 or growing trend- Secondary/safety: - CPU usage > 70–80% sustained - Network bandwidth nearing NIC limits - Replica lag > acceptable (e.g., >100ms)- Policies: - Minimum and maximum cluster size - Cooldown windows (scale-out: 2–5min evaluation, scale-in: longer 15–30min with hysteresis) - Rate limit on concurrent scaling operations (one shard/node at a time)Scaling-out procedure (safe):1. Provision node(s) and join to cluster (automated via infra tools).2. For Redis Cluster: use controlled slot migration (redis-cli --cluster reshard or cluster-rebalance tool) to move slots to new nodes. For managed services: use provider reshard API.3. Maintain write availability: perform incremental slot migration so traffic is rebalanced gradually; ensure clients honor MOVED/ASK redirects.4. For master/replica topology: add replicas first to increase read capacity, then promote or add masters for write scale.5. Monitor rebalancing progress; only proceed to next node after metrics stabilize and replica lag is low.6. Post-reshard: update monitoring, routing metadata.Draining & scaling-in (safe):1. Choose victim node using least-impact heuristic (lowest key count, lowest traffic).2. Mark node read-only / remove from client routing (if using proxy) to stop new connections/writes.3. For cluster: migrate slots away incrementally to other nodes.4. Wait until key migration completes and replica sync/lag zero.5. Remove node from cluster and deprovision.6. Ensure scale-in only if memory/cpu/latency comfortably below thresholds and recent traffic trend confirms lower demand (≥30–60min).Key rebalancing approaches (trade-offs):- Native Redis Cluster slot migration: minimal client changes, supports online resharding, but slot movement is O(keys moved) and can be slow.- Proxy/client-side consistent hashing (e.g., Twemproxy, client lib): faster node add/remove with consistent-hash ring—requires client/proxy support and may rehash many keys unless using virtual nodes.- Use read replicas to absorb read spikes rather than immediate master resharding (cheaper and faster).Performance vs cost trade-offs:- Overprovisioning (keep headroom): simplest, lowest operational complexity, higher cost.- Reactive autoscaling with fast rebalancing: cheaper but higher complexity and risk of transient latency during resharding; mitigate with small, frequent scale steps and warming.- Use replica read-scaling where possible to avoid moving master slots (good for read-heavy diurnal spikes).- Use instance families: scale with burst-capable instances for short spikes (cheaper than full nodes), but risk throttling.Operational safeguards & automation:- Automated playbooks for resharding with canary runs and pre-checks (replica health, disk, network).- Circuit breaker: if migration causes error spikes, abort and roll back.- Pre-warm cache before spike (scheduled jobs to repopulate hot keys) to reduce cold-start misses.- Canary and chaos testing of reshard flows in staging.- Alerting for failed migrations, high eviction rates, increasing latency.- Runbook and automatic rollback (re-add node or stop reshard, promote replicas).Example automation sequence (pseudo):- On trigger: if read-heavy -> add replicas; if memory-bound -> add masters and reshard- Provision node -> join cluster -> start incremental slot migration -> verify metrics -> mark healthy -> update routing -> remove old node when scale-in triggeredThis strategy balances reliability (incremental, observable resharding; replicas for read spikes) and cost (prefer replicas & scheduled pre-warm for predictable diurnal spikes; reserve scale-in cooldown to avoid thrash).
HardTechnical
77 practiced
You're asked to reduce memory usage of cached JSON documents in Redis without harming access latency. Propose practical techniques (serialization options, compression, partial caching, hashing), the trade-offs each introduces, and how you'd measure success.
Sample Answer
Approach: treat Redis-cached JSON as a trade-off between memory, CPU, and latency. I'll list practical techniques, the trade-offs, and concrete success metrics and measurement methods.Techniques and trade-offs- Binary serialization (MessagePack, CBOR, protobuf): - Benefit: smaller than JSON, faster parse for typed schemas. - Trade-off: needs schema management; less human-readable; client changes required.- Schema-based compression (strip optional fields, normalize): - Benefit: reduces per-object overhead by removing sparsity and duplicating strings. - Trade-off: added client-side logic and possible incompatibility with older clients; complexity in versioning.- In-place field hashing / interning (store repeated large strings as keys): - Benefit: deduplicates repeated values across documents, huge wins for common texts. - Trade-off: extra lookup indirection and maintain/invalidate logic; memory moved to intern table.- Lightweight compression (LZ4, Snappy) at store/transfer: - Benefit: good memory reduction with very low CPU and decompression latency. - Trade-off: still adds CPU cost on writes/reads; best for larger docs; choose compression level carefully.- Partial / tiered caching: - Benefit: cache only hot fields (IDs, small metadata) in Redis; keep heavy payloads in blob store (S3) or object DB. - Trade-off: increases application complexity and additional network calls when full doc required.- Redis data-model adjustments: - Use HASHes with fields for frequently accessed keys to avoid full-doc read/write. - Use Redis Modules (RedisJSON) to store compact JSON and manipulate partial JSON server-side. - Trade-off: operational complexity and potential client-library constraints.- Eviction & TTL strategies: - Benefit: automatic memory control; prioritize hot items. - Trade-off: possible increased miss rate if mis-tuned.- Sampling + adaptive caching: - Benefit: dynamically cache popular docs; reduce footprint. - Trade-off: requires monitoring and control loops.Measurement and success criteria- Baseline metrics: current Redis memory usage, cache hit ratio, p95/p99 read latency, CPU utilization on Redis and app, network bandwidth.- Post-change targets: - Memory reduction target (e.g., 30–60% for payloads). - Maintain p99 read latency within SLO (e.g., < X ms). - Keep cache hit ratio drop within acceptable delta (e.g., < 3%). - Acceptable CPU increase on clients/Redis (e.g., < 20%).- How to measure: - Use controlled A/B rollout: route a % of traffic to new caching strategy. - Collect metrics: Redis INFO memory, latency (redis-cli latency), application histograms (p50/p95/p99), CPU profiles, network IO. - Trace requests to measure extra RPCs/latency for partial fetches. - Monitor error budget and user-visible latencies.- Safety: start with LZ4 or Snappy on a sample set; validate CPU/latency; then expand. Add feature flags, telemetry, and automated rollback if SLOs breach.Conclusion: prioritize low-latency-friendly options first (binary serialization + LZ4 + field hashing) and combine with partial caching and eviction policies. Validate via canary tests, compare memory vs latency and CPU trade-offs, and iterate until SLOs and cost targets are met.
EasyTechnical
95 practiced
Describe the primary metrics you would capture to evaluate a cache layer's effectiveness (e.g., hit ratio, miss latency, eviction rate). For each metric explain why it matters and one alert condition you might create.
Sample Answer
Key metrics to evaluate a cache layer’s effectiveness, why each matters, and one example alert condition:1. Hit ratio (hits / total requests)- Why: Primary measure of cache usefulness; higher ratio reduces backend load and latency.- Alert: Hit ratio < 85% sustained for 5 minutes.2. Miss rate (misses / sec)- Why: Shows load on origin services and helps detect regression in caching logic or key churn.- Alert: Miss rate > 1000/sec or 2× baseline for 3 minutes.3. Miss latency (avg time to satisfy a miss from origin)- Why: Misses are expensive; high miss latency directly affects user-perceived latency and SLOs.- Alert: 95th-percentile miss latency > 200 ms for 5 minutes.4. Eviction rate (evictions / sec)- Why: High evictions indicate under-provisioning or poor key TTLs/patterns causing thrashing.- Alert: Eviction rate > 500/sec or >10% of requests as evictions over 5 minutes.5. Capacity/utilization (used memory vs provisioned)- Why: Ensures cache sizing is adequate; approaching capacity increases evictions and instability.- Alert: Utilization > 90% for 10 minutes.6. TTL-expiry vs explicit-eviction ratio- Why: Distinguishes healthy natural expiration from active eviction due to pressure.- Alert: Evictions > TTL expiries by a factor of 3 for 5 minutes.7. Cache error rate (cache node errors / total ops)- Why: Node errors/connection failures bypass cache and harm availability.- Alert: Error rate > 1% or >50 errors/minute.Implement these with percentiles, per-cluster and per-service dimensions, and correlate alerts to backend load to avoid noise.
EasyTechnical
72 practiced
List and briefly explain common cache eviction policies (LRU, LFU, FIFO, TTL-based, random). For each give an example workload where it performs well and one where it performs poorly.
Sample Answer
LRU (Least Recently Used)- What: Evict the item not used for the longest time.- Good workload: Web-session caches or CDN edge caches with strong temporal locality (recent items likely re-requested).- Poor workload: Sequential scans over a dataset larger than cache (one-pass reads) — causes cache thrashing.LFU (Least Frequently Used)- What: Evict the item with the lowest access count over a window.- Good workload: Hot-object caches where a small subset is repeatedly accessed (e.g., product catalog with popular items).- Poor workload: Workloads with changing popularity (temporal bursts) or one-off spikes — stale counters keep evicting newer useful items.FIFO (First-In First-Out)- What: Evict the oldest inserted item, regardless of access pattern.- Good workload: Simple streaming buffers where insertion order approximates usefulness (e.g., log batching).- Poor workload: Any workload with non-uniform reuse; ignores recency/frequency, so hot items may be evicted.TTL-based (Time-to-Live)- What: Items expire after a fixed time since insertion/refresh.- Good workload: Time-sensitive data (e.g., DNS records, cached API responses with known freshness).- Poor workload: Highly variable access patterns where fixed TTL either evicts still-useful items too soon or keeps stale items too long.Random Eviction- What: Evict a random item when space needed.- Good workload: Highly-contended caches where eviction overhead must be minimal, or uniform-access workloads.- Poor workload: Skewed-access workloads (hot keys) — risk evicting hot items and increasing miss rate.Notes: In distributed SRE contexts combine strategies (e.g., LRU with TTL, or LFU with aging) to balance adaptiveness and operational cost.
Unlock Full Question Bank
Get access to hundreds of Caching Strategies and Cache Aside Pattern interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.