Caching and Asynchronous Processing Questions

Design and operational patterns for reducing latency and decoupling components using caching layers and asynchronous communication. For caching, understand when to introduce caches, cache placement, eviction policies, cache coherence, cache invalidation strategies, read through and write through and write behind patterns, cache warming, and trade offs between consistency and freshness. For asynchronous processing and message driven systems, understand producer consumer and publish subscribe patterns, event streaming architectures, common brokers and systems such as Kafka, RabbitMQ, and Amazon Simple Queue Service, and the difference between queues and streams. Be able to reason about delivery semantics including at most once, at least once, and exactly once delivery, and mitigation techniques such as idempotency, deduplication, acknowledgements, retries, and dead letter queues. Know how to handle ordering, partitioning, consumer groups, batching, and throughput tuning. Cover reliability and operational concerns such as backpressure and flow control, rate limiting, monitoring and alerting, failure modes and retry strategies, eventual consistency and how to design for it, and when to choose synchronous versus asynchronous approaches to meet performance, scalability, and correctness goals.

HardSystem Design

42 practiced

Design a multi-region caching strategy for a user-profile service requiring <50ms reads globally, propagation of updates within ~5s, and tolerance of a single region failure. Describe cache topology (edge caches, regional caches, origin), replication or invalidation mechanisms, consistency guarantees, and failover behavior when regions are down.

Sample Answer

**Clarify constraints & goals**- Global read <50ms, updates visible ≈5s, tolerate single-region failure, user-profile service (read-heavy, occasional writes).

**Topology**- Edge: CDN or client-side local cache (short TTL, 100–500ms) for static profile pieces.- Regional cache: Redis Cluster per region (primary for region) serving app servers in that region.- Global replication: Redis Enterprise (Active-Active / CRDT) or Redis-Replication with async multi-region replication.- Origin DB: Single master-per-region datacenter with cross-region async replication (e.g., primary in one region + read replicas).

**Replication / Invalidation**- Writes go to local regional cache + origin DB. On write: - Write-through to DB (or write-behind with immediate cache write). - Publish invalidation/update event to a global pub/sub (Kafka/Redis Streams). - Regional caches subscribe and apply updates within ~1s; if using Redis Enterprise Active-Active, CRDT handles conflict-free merges.- Short TTL fallback (e.g., 5–30s) to bound staleness.

**Consistency guarantees**- Read-after-write in same region: strong-ish (write updates local cache before ack).- Cross-region: eventual consistency within target ≈5s.- Use versioning (profile version or last-updated timestamp) to resolve conflicts; write wins or merge for fields.

**Failover behavior**- Region fail: traffic routed via global load balancer to nearest healthy region.- Surviving region uses replicated cache/read-replica DB. If cache cold, origin read may increase latency but still try to meet <50ms via optimized read replicas and CDN for static pieces.- On region recovery: resync via replication streams; apply idempotent updates.

**Trade-offs**- Active-Active gives lower latency and resilience but complexity (conflicts). Async invalidation is simpler but allows short staleness.- Use metrics: SLO for read latency and staleness; alert on replication lag.

This design balances low-latency reads, bounded staleness, and single-region tolerance suitable for a full-stack service.

MediumSystem Design

41 practiced

How would you design caching for a multi-tenant SaaS platform where tenant data must be isolated and some tenants are very 'hot'? Discuss cache key design, sharding vs prefixes, per-tenant quotas, eviction policies, and strategies to isolate hot tenants to avoid affecting others or increasing cost unfairly.

Sample Answer

**Clarify goals & constraints**- Tenant isolation (no cross-tenant leaks), fair cost, handle “hot” tenants without impacting others, low latency.

**Cache key design**- Use strong namespacing: tenant:{tenant_id}:type:{object_type}:id:{id}- Include versioning or schema hash to allow safe invalidation.- For aggregated queries, include query fingerprint: tenant:{id}:query:{sha256(params)}.

**Sharding vs prefixes**- Prefer Redis Cluster sharding by key hash to distribute load. Ensure tenant_id is part of key so hot-tenant keys map consistently.- For extreme isolation, allocate dedicated shards/instances to very hot tenants (vertical isolation) to avoid noisy-neighbor effects.

**Per-tenant quotas & throttling**- Enforce cache memory and request quotas per tenant with a token-bucket or leaky-bucket gateway.- Track usage with a lightweight metrics stream (Redis/Prometheus). If a tenant exceeds quota, steer them to a separate instance or degrade to backend reads with rate-limits.

**Eviction policies**- Default: LRU or LFU for shared clusters.- For tenant-aware eviction: implement weighted LRU where tenant weight = 1 / (quota_priority). Or maintain per-tenant LRU lists and perform proportional eviction.- Persist critical objects to a TTL > short-lived caches; use explicit invalidation for writes.

**Isolating hot tenants**- Detect hot tenants via a rolling window of cache ops; automatically migrate to dedicated cache nodes when thresholds hit.- Offer tiered caching plans: standard shared cache vs premium dedicated cache billed to tenant.- Use read-through and write-through strategies to keep cache warm when migrating.

**Operational notes**- Monitor hit/miss, tail latency, memory per-tenant. Automate alerts and autoscaling of Redis clusters.- Design graceful fallback: circuit-breaker to DB and client-side local caches for ultra-low-latency needs.

Example: move tenant 42 to a Dedicated Redis node when 95th-percentile ops > X/sec or memory > Y% of cluster, update key namespace routing, warm keys via background prefetch.

EasyTechnical

46 practiced

Implement an LRU cache in JavaScript with get(key) and put(key, value) operations that run in O(1) time. The constructor accepts a positive integer capacity. Provide the class/function signature, the data structures you will use, and a short code snippet or pseudocode highlighting the critical operations.

Sample Answer

**Approach (brief)**

Use a doubly linked list to track recency (head = most recent, tail = least), and a HashMap (JS Map) to map keys to list nodes. get and put are O(1) by moving nodes in the list and updating the map.

**Signature & data structures**

- Class: LRUCache- Constructor: new LRUCache(capacity)- Data: this.capacity (number), this.map = new Map(), doubly linked nodes { key, val, prev, next }, head and tail sentinels.

**Code (critical operations)**

javascript

class LRUCache {
  constructor(capacity) {
    this.capacity = capacity;
    this.map = new Map();
    // Dummy head & tail to simplify ops
    this.head = { prev: null, next: null };
    this.tail = { prev: null, next: null };
    this.head.next = this.tail;
    this.tail.prev = this.head;
  }

  // remove node from list
  _remove(node) {
    node.prev.next = node.next;
    node.next.prev = node.prev;
  }

  // insert node right after head (most recent)
  _insertToHead(node) {
    node.next = this.head.next;
    node.prev = this.head;
    this.head.next.prev = node;
    this.head.next = node;
  }

  get(key) {
    if (!this.map.has(key)) return -1;
    const node = this.map.get(key);
    this._remove(node);
    this._insertToHead(node);
    return node.val;
  }

  put(key, value) {
    if (this.map.has(key)) {
      const node = this.map.get(key);
      node.val = value;
      this._remove(node);
      this._insertToHead(node);
      return;
    }
    if (this.map.size >= this.capacity) {
      // evict least recently used (tail.prev)
      const lru = this.tail.prev;
      this._remove(lru);
      this.map.delete(lru.key);
    }
    const node = { key, val: value, prev: null, next: null };
    this._insertToHead(node);
    this.map.set(key, node);
  }
}

**Complexity**- Time: O(1) for get and put- Space: O(capacity)

**Edge cases**- capacity = 0 (treat as no-op or always -1)- frequent updates to same key (overwrite and move to head)- ensure sentinel nodes to avoid null checks

This implementation is compact, testable, and suitable for backend caching logic within a full‑stack app.

MediumTechnical

54 practiced

Describe three concrete backpressure and flow-control mechanisms you could implement across microservices communicating asynchronously to prevent downstream overload. Include protocol-level approaches (pause/resume, windowing) and application-level approaches (rate-limiting, circuit-breakers, bulkheads) and discuss trade-offs.

MediumSystem Design

49 practiced

Design a caching strategy for an API gateway that fronts multiple microservices. Requirements: minimize latency for read-heavy endpoints, support per-endpoint TTLs, invalidate caches on writes, and operate across two regions. Describe cache placement (edge vs central), key design, invalidation approach, and trade-offs between consistency and freshness.

Sample Answer

**Clarify goals & constraints**- Read-heavy endpoints: minimize latency.- Per-endpoint TTLs, invalidate on writes.- Two regions: low-latency reads in-region, cross-region coherence.

**High-level placement**- Use a hybrid approach: - Edge caches (local per-region Redis or managed cache like ElastiCache / Memorystore) for lowest read latency. - Central control plane for global metadata (cache key schema, TTLs, invalidation coordination).

**Cache pattern**- Cache-aside at API gateway: - Read: check regional Redis; on miss fetch microservice, populate Redis with endpoint-specific TTL. - Write: update primary datastore, then trigger invalidation.

**Key design**- Deterministic keys: service:resource_type:resource_id:version - Example: orders:summary:12345:v2- Include resource version or last-modified timestamp to allow safe staleness detection.- Store small metadata (ETag/last-modified) alongside value.

**Invalidation approach**- On write, gateway/ service: - Increment resource version in primary DB or write an invalidation event. - Publish invalidation event to a cross-region message bus (Kafka, SNS+SQS, or Redis Streams) and local pub/sub to regional Redis. - Regions subscribe and evict keys or update versions atomically.- For heavy write bursts, use short TTLs or write-through to update caches synchronously for critical endpoints.

**Consistency vs freshness trade-offs**- Strong consistency: synchronous cache-update on write; higher write latency and complexity.- Eventual consistency: async invalidation via pub/sub; very low read latency but small window of staleness.- Suggested compromise: critical endpoints use synchronous invalidation/update (or very short TTL + version checks); less critical use async invalidation.

**Operational considerations**- Monitor cache hit rate, invalidation lag, and cross-region pub/sub latency.- Use metrics and circuit-breakers; fallback to origin on stale/failed cache.- Pre-warm caches for hot items; implement negative caching for 404s.

This design balances low read latency with configurable freshness per endpoint while keeping cross-region coherence manageable for a full-stack engineering context.

Unlock Full Question Bank

Get access to hundreds of Caching and Asynchronous Processing interview questions and detailed answers.

Join thousands of developers preparing for their dream job.