Caching and Performance Optimization Questions

Covers design and implementation of multi layer caching and end to end performance strategies for web and backend systems. Topics include client side techniques such as browser caching, service worker strategies, code splitting, and lazy loading for components images and data; edge and distribution techniques such as content delivery network design and caching of static assets; and server side and data layer caching using in memory stores such as Redis and Memcached, query result caching, and database caching patterns. Includes cache invalidation and coherence strategies such as time to live, least recently used eviction, cache aside, write through and write behind, and prevention of cache stampedes. Covers when to introduce caching and when not to, performance and consistency trade offs, connection pooling, monitoring and metrics, establishing performance budgets, and operational considerations such as cache warm up and invalidation during deploys. Also addresses higher level concerns including search engine optimization implications and server side rendering trade offs, and how performance decisions map to user experience and business metrics at senior levels.

EasyTechnical

31 practiced

Explain the difference between cache-aside, write-through, and write-behind (write-back) caching patterns. For each pattern list one ideal use case and one significant downside that a Solutions Architect must communicate to stakeholders.

MediumTechnical

46 practiced

Inventory and price update workflows often require near-real-time consistency. Propose a pragmatic invalidation strategy for prices and stock levels that balances freshness and CDN/edge cache efficiency. Include explicit steps for publishing updates, invalidation, and fallback behavior if the origin is overloaded.

Sample Answer

Situation: Customers require near-real-time correctness for prices and stock, but we must keep CDN/edge efficiency and avoid origin overload. Here’s a pragmatic invalidation strategy that balances freshness, latency, and resilience.

Strategy summary:- Use short soft TTLs + stale-while-revalidate, strong cache keys with versioning, and push invalidation for critical changes. Employ origin-shielding and background refresh to avoid thundering herds. Fall back to serving stale content with circuit-breaker behavior if origin is overloaded.

Publishing & invalidation steps:1. Update event: When inventory/price changes, write to canonical datastore and emit an event to a message bus (Kafka/SQS).2. Compute scope: Event includes SKU, region, price_version (monotonic or hash), and urgency flag (critical price change vs low-priority stock delta).3. Push-to-edge: For critical updates, call CDN invalidation API for affected cache keys (single-object or tag-based purge). For non-critical, publish a “version bump” to an edge-control channel (edge workers subscribe) so they can atomically switch to new keys without full purge.4. Soft TTL policy: Set Cache-Control: max-age=30s, stale-while-revalidate=300s. Edge serves stale for up to 300s while it revalidates in background.5. Conditional revalidation: Edge performs If-None-Match/If-Modified-Since to minimize origin load.

Origin protection & fallback:- Origin shield / regional read-replica reduces global origin hits.- Rate-limit revalidation: restrict concurrent revalidation per key (token bucket) to avoid spikes.- Circuit breaker: if origin 5xx rate > threshold or response latency high, switch to “stale-serve” mode: edges serve stale cached content and respond with X-Cache-Stale: true header and a client-visible TTL warning.- Background refresh worker: async job retries updating edge caches; non-blocking for user requests.- Monitoring & alerting: track cache hit ratio, origin QPS, stale-serve rate; auto-scale origin or relax TTLs as needed.

Key trade-offs & reasons:- Soft TTL + stale-while-revalidate keeps high hit rates and low latency while ensuring eventual freshness; push invalidation for urgent updates ensures near-real-time correctness when business-critical.- Versioned keys avoid edge stampedes and make invalidation idempotent.- Circuit-breaker + stale-serving prioritizes availability and consistent UX under load.

Example cache key pattern:- /product/{sku}?v={price_version}&region={r}- Tag objects by sku and region for efficient tag-based CDN purges.

This approach gives predictable freshness for most updates, near-real-time correctness when required, and robust fallback when the origin is stressed.

HardSystem Design

29 practiced

Design a caching strategy for a faceted product search system (search queries with filters and sorts). Consider full-text search engine caching, query result caching, and caching of facet counts when product index updates are frequent. Describe how to maintain near-real-time relevance while keeping cache effective.

Sample Answer

Requirements & constraints:- Low-latency faceted product search with filters/sorts.- Product index updates are frequent (minutes/seconds).- Need near-real-time relevance (seconds) but effective caching to reduce load.- Support arbitrary combinations of filters/sorts, but some queries are hot.

High-level approach:- Use hybrid caching: (1) full-text engine + query-result caching for hot queries, (2) separate facet-count caches (aggregations) with incremental/upsert updates, (3) schema-level index versioning & event-driven invalidation to keep caches near-real-time.

Architecture & components:1. Search engine: Elasticsearch/Opensearch for full-text, indexing, and aggregation primitives. Index configured with tuned refresh interval (e.g., 1s–5s) to trade throughput vs latency.2. Query router / API: normalizes queries, canonicalizes parameters (sort, filters, pagination) and computes cache keys.3. Result cache: Redis or distributed in-memory cache (clustered Redis or Memcached) that stores rendered query results (IDs + relevance scores + precomputed metadata) for hot queries. Keys include index-version token.4. Facet-count cache: stores counts per dimension for base filter states and for common combinations. Backed by Redis; supports increment/decrement updates using events.5. Event bus: Kafka or Kinesis capturing index changes (create/update/delete) and emitting lightweight events containing product id, changed attributes, and previous/new filter-relevant fields.6. Background updater: consumer services update facet caches incrementally (delta updates) and invalidate/refresh affected result-cache entries by index-version or explicit key invalidation (for truly affected hot queries). Also performs async re-aggregation for expensive combos.7. Cache-warming & popularity tracking: track query frequency and precompute caches for top-N patterns; warm caches after deploys or index rebuilds.

Key strategies to maintain near-real-time relevance:- Index-versioning: include an index-version or sequence-number in cache keys. When a bulk refresh or schema change occurs, bump version to avoid serving inconsistent caches.- Short TTLs for result cache (e.g., 5–30s) for general queries; longer TTLs (minutes) for highly stable pages. Use "stale-while-revalidate": serve slightly stale cached response immediately while asynchronously refreshing cache.- Incremental facet updates: on each product event, determine which facet buckets change (e.g., category, brand, price-bucket) and apply atomic INCR/DECR operations to facet caches instead of recomputing full aggregation. For complex dependent filters, maintain precomputed combinations for top queries and mark others for lazy recompute.- Selective invalidation: instead of broad cache flushes, use a mapping of product -> affected cache keys (maintain a reverse index for hot queries) for targeted invalidation; for cold queries fall back to engine.- Use approximate counts for highly dynamic facets where exactness is less critical (e.g., "results ~ 1200") using counters or probabilistic sketches; mark exact-count requests to trigger on-demand recompute.

Query normalization & cache key design:- Canonicalize filters order, normalize ranges, bucketize numeric filters for facet-level caches.- Cache key = hash(index-name, index-version, normalized-filters, sort, page-size, user-segmentation).- For result cache store: list of product IDs + cursor/token for paging; subsequent page fetches prefer precomputed next-page caches or use cursor to fetch from engine.

Performance & scalability:- Shard facet caches by dimension and hash to scale Redis cluster.- Offload heavy ad-hoc aggregations to worker pool with rate limiting.- Use CDN for static rendered pages or category landing pages.- Monitor cache hit rate, staleness windows, queue lag for event bus; auto-adjust TTLs & refresh intervals.

Trade-offs:- Strong consistency = high invalidation cost; we prefer bounded staleness (seconds) and selective exactness.- Incremental updates reduce recompute but add complexity and eventual consistency edge cases (race conditions); solve with idempotent events and sequence numbers.- Materializing many filter combinations increases memory usage; limit to popular patterns and use on-demand for long-tail.

Example flow (update -> facet counts):1. Product P updated: event published with changed fields.2. Background consumer reads event, computes delta for facets (e.g., brand unchanged, price bucket changed).3. Apply atomic INCR/DECR to Redis facet buckets and update reverse mapping (product -> affected keys) for hot queries.4. For result caches referencing P, either invalidate keys via mapping or bump index-version if broad change occurred.5. If a hot query's cache is invalidated, trigger async refresh and serve stale result with stale-while-revalidate if allowed.

Observability & safety:- Track cache hit ratio, staleness age, event lag, frequency of full re-aggregations, and false-count rate for approximations.- Provide an admin tool to force refresh specific facets/queries and to adjust TTLs per segment.

This design balances near-real-time freshness (index refresh tuning + short TTLs + incremental facet updates) with cache effectiveness (hot-query materialization, selective invalidation, stale-while-revalidate).

HardTechnical

33 practiced

You must deploy a schema change that alters cache key formats across hundreds of microservices. Create a rollout and invalidation plan that avoids long cache inconsistency windows and downtime. Include backward compatibility, migrations, cache warm-up, and rollback steps.

Sample Answer

Situation: We needed to change our canonical ID format used in cache keys across ~300 microservices to support multi-tenant IDs. This change touches many services and cannot cause long cache inconsistency or downtime.

Plan (high level goals)- Maintain backward compatibility so old keys remain valid during rollout- Avoid long inconsistency windows by dual-reading/writing and phased invalidation- Provide automated warm-up and observability- Define clear rollback criteria and fast mitigation steps

Rollout & Migration Steps1. Compatibility layer (code change): - Implement a small library used by all services that supports dual-key logic: - read: try newKey -> fall back to oldKey - write: write both newKey and oldKey (TTL aligned) - flag controlled by config/feature-flag2. Canary rollout (weeks): - Enable library in CI as opt-in for 5% of traffic/services (canary group) - Run for 48–72 hours while monitoring cache hit-rate, latency, error-rate.3. Progressive ramp: - 5% → 25% → 50% → 100% of services over days. For each stage: - Switch service config to write-both keys; continue dual-read. - Run consistency checks comparing values returned by old vs new keys for a sample.4. Cache warm-up: - After a service switches to write-both, run an automated warm-up job to prime new keys: - Use recent request traces to regenerate populated keys (rate-limited). - Mark warm-up complete when hit-rate for newKey > target (e.g., 95% of total hits).5. Invalidation / cutover: - Once 100% services are writing both and newKey hit-rate high: - Flip a global feature flag to make all reads prefer newKey (still fallback allowed). - Monitor errors/latency for 1–2 hours. - Then expire oldKey writes: update library to stop writing oldKey (grace period: keep oldKey TTL for X hours/days).6. Old key GC: - After 2× max-TTL and verification (no clients reading oldKeys), run background job to purge stale old keys.

Monitoring & Verification- Metrics: per-service hit-rate by key type, cache miss rate, request latency, error rate, divergence count (value differences between keys).- Synthetic checks: end-to-end requests validating canonical responses.- Alerts: defined thresholds for hit-rate drop, error spikes, divergence > 0.5% triggers pause/rollback.

Rollback Strategy- Fast rollback: flip feature flags to - Stop preferring newKey (read oldKey first) - Resume writing only oldKey if needed- If widespread issues during ramp: revert canary group to previous config and halt rollout.- Maintain ability to re-enable dual-write quickly to re-sync caches.

Operational & Organizational Steps- Runbook: document commands, flags, monitoring dashboards, owners.- Communication: coordinate release windows with all teams; nightly quiet-window for final GC.- Automation: use deployment orchestration to flip flags in waves; warm-up and verification automated.- Post-mortem: capture metrics, lessons, and update library for future migrations.

Trade-offs & Rationale- Dual-write/dual-read avoids downtime and long inconsistency windows at cost of extra storage and complexity in library.- Phased rollout reduces blast radius and gives measurable checkpoints for rollback.- Warm-up minimizes cache-miss storm during cutover.

This plan balances safety, speed, and operational clarity for a cross-service cache key migration.

MediumTechnical

34 practiced

Hot keys in a cache can become a performance bottleneck. Propose architectural and operational strategies to detect, mitigate, and prevent hot key issues in Redis or Memcached environments serving millions of requests per minute. Include both short-term mitigations and long-term design changes.

Sample Answer

Situation: Serving millions RPS with Redis/Memcached, hot keys (single keys receiving disproportionate traffic) cause CPU, network, and eviction pressure. Below is a structured set of detection, short-term mitigations, and long-term design changes suitable for solution-architect level advice.

Detection / Observability- Instrumentation: emit per-key metrics (top-k), command stats and latency from Redis INFO, slowlog, and LATENCY HISTOGRAMS. Use keyspace notifications, sampling, and client-side telemetry to surface hot keys into Prometheus + Grafana.- Automated alerts: thresholds on ops/sec per key, CPU/spike, increased miss rate or eviction rate, and tail latency.

Immediate (short-term) mitigations- Rate-limit and backpressure: apply token-bucket at API gateway or client library to throttle requests for identified hot keys.- Request coalescing (dedupe): implement “single flight” / request coalescing so concurrent identical misses result in one backend fetch.- Local caching: add per-app in-process cache (LRU) for hottest keys to reduce trips to Redis.- Temporary replication: promote a read-only replica or warm a dedicated Redis instance with just the hot key, updating via pub/sub or CDC.- Increase resource isolation: pin hot-key shard to a larger node (vertical scale) or temporary faster instance class.

Long-term architectural changes- Sharding and partitioning: ensure app-level or client-side consistent hashing with enough slots; migrate to Redis Cluster with hash tags to control collocation and avoid single-slot overload.- Key redesign: split hot keys (fan-out) into multiple sub-keys (time-bucketed, user-sharded) and aggregate at read if needed; use pre-aggregation for counters.- Adaptive caching patterns: use write-through for high-write keys, probabilistic caches (Bloom filters) to prevent cache stampedes, and TTL jitter to avoid synchronized expiries.- CQRS / materialized views: move high-read patterns into dedicated read stores (CDN, edge caches, or specialized in-memory stores) or maintain precomputed views.- Capacity planning & autoscaling: implement horizontal autoscaling of cache nodes and connection pools; ensure client libraries support ASK/MOVED and retry logic.- Circuit breakers & graceful degradation: fall back to degraded responses or stale-read mode when a hot key overwhelms cache.

Operational playbook- Runbook: detect → isolate (route traffic to dedicated shard) → mitigate (throttle/coalesce/local cache) → fix (schema change or shard rebalancing) → revert temporary measures.- Testing: chaos testing on hot-key scenarios, load tests with synthetic hot-key patterns, and rehearsed failover.

Trade-offs & considerations- Splitting keys increases read complexity and potential consistency concerns—choose eventual vs strong consistency per use case.- Local caches add cache-coherency complexity; use short TTLs or invalidation pub/sub.- Client-side sharding requires coordinated rollout and support in client libraries.

This approach balances quick operational measures to stop outages with sustainable design changes—sharding, key redesign, and edge caching—to prevent recurrence while documenting runbooks and metrics for proactive detection.

Unlock Full Question Bank

Get access to hundreds of Caching and Performance Optimization interview questions and detailed answers.

Join thousands of developers preparing for their dream job.