Caching and Performance Optimization Questions

Covers design and implementation of multi layer caching and end to end performance strategies for web and backend systems. Topics include client side techniques such as browser caching, service worker strategies, code splitting, and lazy loading for components images and data; edge and distribution techniques such as content delivery network design and caching of static assets; and server side and data layer caching using in memory stores such as Redis and Memcached, query result caching, and database caching patterns. Includes cache invalidation and coherence strategies such as time to live, least recently used eviction, cache aside, write through and write behind, and prevention of cache stampedes. Covers when to introduce caching and when not to, performance and consistency trade offs, connection pooling, monitoring and metrics, establishing performance budgets, and operational considerations such as cache warm up and invalidation during deploys. Also addresses higher level concerns including search engine optimization implications and server side rendering trade offs, and how performance decisions map to user experience and business metrics at senior levels.

MediumTechnical

38 practiced

Design a monitoring and alerting strategy for a caching tier. Which metrics and logs do you collect (examples: hit ratio, miss rate, avg latency, p95/p99 latency, eviction rate, memory usage, connection errors), what thresholds would trigger alerts, and how would you correlate cache metrics with upstream DB load and end-to-end user latency?

Sample Answer

Approach: define SLOs for availability and latency, instrument cache for observability, set tiered alerts (warning/critical), and correlate cache signals with DB and client-side metrics to diagnose root cause quickly.

Metrics to collect (examples + why):- Hit ratio / miss rate (primary correctness/efficiency signal)- Avg latency, p95/p99 latency (client-facing performance)- Eviction rate / eviction count (memory pressure)- Memory usage vs configured capacity (OOM risk)- Connection errors / refused connections (availability)- Throughput (ops/sec, read vs write) and load per node (heat)- TTL expirations and stale-reads count (data freshness)- Cache fill rate (items/sec) and load-from-origin rate

Logs / traces:- Cache access logs (keys hashed, outcome hit/miss, latency)- Errors with stack/connection details- Sampled distributed traces including client → cache → db to measure end-to-end latency

Alert thresholds (example policy):- Warning: hit ratio drops >15% from baseline for 5m OR miss rate increase x2 for 5m- Critical: hit ratio <50% OR sustained miss rate causing backend QPS > backend capacity (see DB metrics)- Latency: p95 > 100ms warning, p95 > 300ms critical; p99 > 1s critical- Memory: usage > 80% warning, > 95% critical or eviction rate spike- Evictions: > 1000/min or sudden >5x baseline → warning/critical- Connection errors: >1% error rate or absolute >50 errors/min → critical

Correlation and diagnosis:- Ship correlated dashboards showing cache hit ratio, cache miss QPS, DB CPU/threads, DB QPS, DB replication lag, and client p95/p99. If cache miss rate rises and DB QPS increases proportionally with DB CPU/latency, root cause is cache misses causing DB overload.- Use distributed traces: for slow user requests, break down time spent in cache vs DB vs service. If end-to-end latency rises while cache p95 stable but DB p95 rises, problem upstream.- Alerting playbook: when cache alerts fire, automatically query DB metrics and recent traces; trigger on-call with context: recent deploys, config changes (e.g., TTLs, capacity), traffic spikes.

Best practices:- Baseline metrics per workload and use relative alerts (percent change) plus absolute limits.- Use composite alerts (e.g., high miss rate AND increased DB latency) to reduce noise.- Add automated remediation for common issues (scale cache cluster, increase capacity, roll back deploy).

EasyTechnical

34 practiced

Explain the cache-aside (lazy loading) and write-through caching patterns. For each pattern, describe the read and write flows, their consistency guarantees, where you'd use them, and trade-offs with respect to latency, durability, and complexity. Give a concrete example (e.g., product-detail reads and price updates) to illustrate when you'd pick one over the other.

Sample Answer

Cache-aside (lazy loading)- Read flow: App checks cache; on miss, reads from DB, returns result and populates cache (often with TTL).- Write flow: App writes to DB; cache is invalidated/evicted or left stale (then next read repopulates).- Consistency: Eventually consistent. Reads after a write may return stale data until invalidation + repopulate.- Use when: Reads are frequent, writes are less frequent, and occasional staleness is acceptable (e.g., product details, catalog browsing).- Trade-offs: - Latency: Fast on cache hit; cache miss incurs DB latency. - Durability: DB is the source-of-truth; cache is ephemeral. - Complexity: Simple to implement; must handle cache eviction and stampede (use locking or request coalescing).

Write-through- Read flow: App reads cache; cache delegates to DB on miss, populates cache before returning (similar to cache-aside for reads).- Write flow: App writes to cache synchronously which immediately writes through to DB (or write to DB first then update cache atomically).- Consistency: Stronger—cache and DB are updated together so reads see fresh data.- Use when: Low tolerance for stale reads and higher write-read coupling (e.g., inventory counts, session stores where up-to-date state matters).- Trade-offs: - Latency: Writes are higher-latency (synchronous DB write) but subsequent reads are fast. - Durability: Better immediate consistency but must ensure write to DB succeeds; risk if cache write succeeds but DB fails—need transactional or retry handling. - Complexity: More complex to guarantee atomicity across cache and DB; can increase write throughput bottlenecks.

Concrete example: product-detail reads and price updates- Cache-aside: Best for product-detail pages where reads are heavy and price updates are infrequent. On a price change, update DB then evict cache. Users may see old price for a short window but under typical ecommerce this is acceptable with TTLs and fast invalidation.- Write-through: Choose when price changes are critical (e.g., flash sale) and you cannot risk showing old prices. Writes update cache and DB together so subsequent reads immediately reflect new price, but write latency and implementation complexity increase.

When deciding: prefer cache-aside for read-heavy, tolerant-of-staleness cases; choose write-through when you need stronger read-after-write consistency and can accept higher write costs.

HardTechnical

39 practiced

You're given a small code snippet (pseudocode) that implements cache-aside: if cache miss then read DB and write cache. Under concurrent requests, multiple processes perform the DB read and write cache simultaneously, causing unnecessary DB load. Show a concrete concurrency bug example and propose fixes (per-key locking, request coalescing, or early recompute). Explain trade-offs.

Sample Answer

Situation: A cache-aside pattern pseudocode:

value = cache.get(key)
if value is None:
    value = db.read(key)
    cache.set(key, value)
return value

Concurrency bug example:- Two requests R1 and R2 for same key miss cache simultaneously.- Both execute db.read(key) concurrently, causing duplicate DB load (thundering herd).

Concrete race (Python-like):

python

def get(key):
    v = cache.get(key)
    if v is None:
        # race: multiple threads reach here
        v = db.read(key)      # expensive, executed multiple times
        cache.set(key, v)
    return v

Fixes and trade-offs

1) Per-key locking- Use a lock per key (in-process or distributed via Redis SETNX+expire).

python

with acquire_lock(key):   # blocks others for same key
    v = cache.get(key)
    if v is None:
        v = db.read(key)
        cache.set(key, v)
return v

Pros: Simple, prevents duplicate DB reads. Cons: Risk of deadlocks if lock management wrong; distributed locks add complexity and latency; serialized access can increase tail latency.

2) Request coalescing (singleflight)- First requester performs DB read; others wait and receive the same result without hitting DB.- Implement via an inflight map of Futures/Promises.Pros: Low latency for waiters, no blocking other keys. Cons: Requires bookkeeping; memory for inflight entries; if first request fails, propagate errors carefully.

3) Early recompute / probabilistic refresh (read-through refresh)- Serve stale cache while background process refreshes (e.g., use TTL + refresh-ahead or "stale-while-revalidate").Pros: Avoids blocking and DB spikes; good for reads with relaxed freshness. Cons: Stale data risk; background refresh logic complexity; still need to prevent many refreshes (combine with per-key locking).

Recommendation: Use request coalescing (singleflight) for read-heavy hot keys; combine with distributed per-key locks for cross-process safety and add stale-while-revalidate for low-latency tolerance. Always add retries, lock timeouts, and metrics to detect thundering herd.

HardSystem Design

37 practiced

Design a consistent hashing scheme for a distributed cache cluster that supports adding and removing nodes with minimal cache churn. Explain use of virtual nodes, how to rebalance, and how to handle node failures gracefully. Discuss trade-offs in choosing the number of virtual nodes per physical node.

Sample Answer

Requirements:- Even key distribution across cache nodes- Minimal key movement when nodes join/leave- Fast lookup and low-latency routing- Graceful failure handling and high availability

High-level design:- Use a consistent-hashing ring: hash(key) -> point on 0..2^32-1; nodes are placed on the same ring.- Use virtual nodes (vnodes): each physical cache instance owns multiple vnode positions (e.g., 128-1024) to smooth distribution.- Map a key to the first vnode clockwise from hash(key); that vnode’s physical host stores the key.- Add replication factor R > 1 by taking the next R distinct vnodes clockwise (on different physical hosts).

Virtual nodes — why and how:- Each physical node gets M vnodes with independent hash positions (nodeID#i → hash). This evens out skew from uneven hash spacing or heterogeneous node capacities.- When adding/removing a physical node, only the keys in the vnode ranges it gains/loses need moving (proportional to M / total_vnodes), reducing churn.

Rebalancing procedure:- Node add: assign its M vnode positions; for each vnode range it claims, identify predecessor vnode(s) whose keys now move. Pull/migrate those keys lazily or via streaming replication. Option: warm-up cache by prefetching hot keys.- Node remove (planned): mark node draining, stop accepting writes, migrate keys from its vnodes to successor vnodes, then remove vnodes from ring.- Unplanned failure: detect via health checks/heartbeat. Mark node down and treat its vnodes as absent; put keys served by those vnodes onto their next replicas.

Handling failures gracefully:- Replication: store each key on R physical nodes (R distinct hosts via next-vnodes). On primary failure, reads go to next replica; on write, use quorum (e.g., W+R>replicas) depending on consistency needs.- Hinted handoff: when a node is down, a temporary host accepts writes for its vnode ranges and later hands them off when the node returns.- Anti-entropy / background repair: Merkle trees or periodic reconciliation to detect and fix divergence.- Failure detection: multi-layer (TCP health checks, gossip, or membership service like Serf/Consul). Use short detection for read availability but allow short grace to avoid flapping.

Trade-offs for number of vnodes per physical node (M):- Small M (e.g., 1–16): fewer metadata entries, lower routing table size, but higher imbalance and larger churn when nodes change.- Large M (e.g., 128–2048): smoother distribution, smaller per-node churn fraction, easier heterogeneous weighting (assign more vnodes to bigger machines). Cost: larger ring table, slightly more CPU to hash and manage mappings, increased metadata and migration bookkeeping.- Practical choice: start 128–512 vnodes per node; tune by measuring key skew and migration traffic. Use weighted vnode counts for heterogeneous nodes.

Alternatives and trade-offs:- Rendezvous/Highest Random Weight hashing avoids ring maintenance and can be simpler for dynamic node lists; however, it’s O(N) to pick top R hosts unless optimized with caching.- Consistent hashing + vnodes is best when you need predictable small churn and easy weighted capacity control.

Operational considerations:- Monitor key distribution, cache hit/miss, migration traffic, and node health.- Automate graceful drain on maintenance, and use rolling upgrades with small Migrations at a time.- Provide tooling to visualize ring and simulate node changes before applying.

This design minimizes cache churn, balances load via vnodes, supports replication and quick failover, and lets you tune M to meet metadata vs. balance trade-offs.

MediumSystem Design

33 practiced

Design a caching layer for a product detail page expected to handle 100k RPS reads with 1k writes per minute. Requirements: <100ms p99 read latency for cached reads, ability to reflect price or inventory updates within 5 seconds for critical items, multi-region deployment for global users, and minimal origin DB load. Describe architecture (CDN, edge caches, regional caches, central cache), cache key/versioning strategy, invalidation approach, and trade-offs.

Sample Answer

Requirements (clarified)- 100k RPS reads, 1k writes/min (~17 writes/sec)- p99 read <100ms for cached reads- Critical price/inventory updates visible within 5s- Multi-region, minimize origin DB load

High-level architecture- CDN (global edge) for static assets and product HTML fragments (Cache-Control short TTL + stale-while-revalidate)- Edge caches (CDN POPs) serve most reads- Regional read caches (managed Redis clusters or in-region caching tier like ElastiCache / Memorystore) per region for dynamic JSON/product detail- Central cache and write-bus: a primary cache invalidation coordinator (could be Redis Streams or Kafka) that receives write events and publishes invalidations/updates- Origin services (API + DB) are fallback; writes go to origin DB and also publish change events.

Data flow1. Client -> CDN POP: if fragment present return (p99 <100ms).2. Miss -> Regional cache -> if miss -> Origin API (with DB) -> populate regional + CDN fragment.

Cache key & versioning- Key format: product:{productId}:v{dataVersion}:{shardFlags}- dataVersion increments for major schema changes.- Include a short “ephemeral” version for critical attributes: product:{id}:priceVer:{priceSeq},inventoryVer:{invSeq}- Separate keys for frequently-updated critical fields so small updates don't evict whole object.

Invalidation & freshness (meet 5s requirement)- On write (price/inventory change), origin writes DB, then emits an event to the write-bus with type and product id.- Regional caches subscribe to bus (via regional consumers) and: - Update critical-field keys (atomic HSET) or increment priceVer/invSeq - Push a soft-invalidated fragment to CDN via CDN API (PURGE or surrogate-key invalidation), or set short TTLs with stale-while-revalidate.- For critical items, use write-through or near-write-through: write path also updates regional cache synchronously (fast, bounded latency) so reads after commit see updated values within <5s.- Use bloom filters or change-log compacting to avoid thundering herd on hot items.

Consistency model & SLAs- Strong-ish for critical fields: write path updates caches synchronously; eventual for non-critical fields (TTL + async invalidation).- Use conditional GET / ETag between CDN and regional caches to reduce bandwidth.

Scaling & multi-region- Regional caches colocated with region’s app servers to meet latency and reduce cross-region traffic.- Central event bus replicated (Kafka MirrorMaker / MSK) or distributed messaging (SNS+SQS) with regional subscribers.- Autoscale caches and use sharding to handle 100k RPS.

Trade-offs- Synchronous write-through increases write latency and complexity but guarantees sub-5s visibility for critical fields.- Asynchronous invalidation reduces write latency and origin load but may allow brief staleness.- CDN invalidation (PURGE) is eventual and can be slower/costly; surrogate keys + short TTLs are simpler and more reliable.- More granular keys (separate critical fields) increase complexity but minimize cache churn and origin pressure.

Monitoring & safety- Metrics: cache hit ratio, p99 latency at CDN/region, invalidation lag, origin QPS.- Circuit-breakers: if caches stale or event bus failure, fall back to origin with rate-limiting.- Tests: chaos/invalidation drills to ensure <5s propagation for critical updates.

Unlock Full Question Bank

Get access to hundreds of Caching and Performance Optimization interview questions and detailed answers.

Join thousands of developers preparing for their dream job.