Caching and Asynchronous Processing Questions

Design and operational patterns for reducing latency and decoupling components using caching layers and asynchronous communication. For caching, understand when to introduce caches, cache placement, eviction policies, cache coherence, cache invalidation strategies, read through and write through and write behind patterns, cache warming, and trade offs between consistency and freshness. For asynchronous processing and message driven systems, understand producer consumer and publish subscribe patterns, event streaming architectures, common brokers and systems such as Kafka, RabbitMQ, and Amazon Simple Queue Service, and the difference between queues and streams. Be able to reason about delivery semantics including at most once, at least once, and exactly once delivery, and mitigation techniques such as idempotency, deduplication, acknowledgements, retries, and dead letter queues. Know how to handle ordering, partitioning, consumer groups, batching, and throughput tuning. Cover reliability and operational concerns such as backpressure and flow control, rate limiting, monitoring and alerting, failure modes and retry strategies, eventual consistency and how to design for it, and when to choose synchronous versus asynchronous approaches to meet performance, scalability, and correctness goals.

HardTechnical

41 practiced

Discuss trade-offs for providing transactional guarantees across microservices when local caches exist. Compare two-phase commit, saga/compensating transactions, idempotent APIs, and read-after-write cache invalidation techniques. Provide recommendations for when to use each approach and how to mitigate cache-related anomalies.

Sample Answer

Start with requirements and constraints: are transactions cross-service read/write heavy, latency-sensitive, must be strongly consistent, or is eventual consistency acceptable? Caching introduces stale reads, so designs must balance consistency, availability, and latency.

Two-phase commit (2PC)- What: coordinator asks participants to prepare, then commit/abort.- Pros: strong atomicity across services.- Cons: blocks on failures, high latency, complex in failure recovery, hard with local caches (need cache invalidation or transactions that span cache and DB).- Use when: strict atomicity is mandatory and service count is small (e.g., financial settlement) and you can bear latency and operational complexity.- Mitigation for caches: treat caches as read-only; invalidate or refresh after final commit; implement synchronous write-through cache inside the transactional boundary if supported.

Saga / compensating transactions- What: break global transaction into ordered local transactions with compensating actions on failures.- Pros: non-blocking, fits microservices, lower latency and higher availability.- Cons: eventual consistency, business logic for compensations can be complex, undo may be non-trivial.- Use when: business domain tolerates eventual consistency (orders, inventory), operations are long-running.- Mitigation for caches: publish domain events after each local commit; caches subscribe to events to update/evict; use versioning/causal stamps to detect stale reads.

Idempotent APIs- What: design service operations so retries don’t change semantics.- Pros: simplifies retry and failure handling, critical for networked services.- Use when: combined with sagas or at-least-once message delivery; makes recovery robust.- Cache implications: idempotency alone doesn’t prevent stale reads; combine with cache invalidation/versioning.

Read-after-write cache invalidation techniques- Approaches: synchronous invalidation on write (blocking), write-through (update cache within write path), asynchronous invalidation via events, version/timestamp checks, read-before-write or read-repair.- Trade-offs: synchronous ensures consistency but increases latency; async improves latency but risks stale reads and race conditions.- Best practices to mitigate anomalies: - Use cache-aside with strong ordering: perform write to DB, then publish event, then invalidate cache; ensure subscribers process events in order. - Add version/timestamp metadata to cached entries and API responses; reject reads older than a required version. - Implement read-after-write guarantees at client-side when necessary: after a write, read from primary or bypass cache for a short TTL (stale-while-revalidate). - Leverage distributed locks sparingly for critical sections. - Design APIs to return monotonic tokens (event sequence numbers) so clients can detect stale data.

Recommendations (summary)- If you require strict ACID across services and can accept blocking: 2PC with careful cache write-through/invalidations.- If you prefer availability and scalability and can accept eventual consistency: sagas + idempotent APIs + event-driven cache invalidation.- Always make APIs idempotent and use versioning/causal metadata.- For low-latency reads with strong read-after-write, bypass or refresh cache on writes, or use short-lived write-affinity caches on clients.- Prefer event-driven cache updates with ordered delivery (Kafka, persistent streams) and backpressure/rehydration strategies to reduce anomalies.

This combination — sagas + idempotent operations + event-sourced cache invalidation with versioning — is the pragmatic default for most microservice architectures; escalate to distributed 2PC only for rare, strongly-consistent business-critical flows.

EasyTechnical

45 practiced

Describe the producer-consumer pattern and the bounded-buffer problem. Explain how producers, consumers, and the buffer interact, how backpressure can be applied, and what happens when consumers are slower than producers. Provide practical techniques to prevent unbounded memory growth.

MediumSystem Design

55 practiced

Design a cache invalidation strategy for a user profile service that has 10M users, 100k reads/sec globally, and 5k profile updates/sec. Profiles must be eventually consistent with updates visible within 2 seconds in most regions. Describe cache placement, invalidation propagation across services and regions, TTLs, and how you would handle stale reads and concurrent updates.

Sample Answer

Requirements clarified:- 10M users, 100k reads/s globally, 5k updates/s- Eventually consistent; updates visible within ~2s in most regions- Minimize stale reads, support multi-region scale

Architecture summary:- Origin DB: single primary per region (or multi-master with conflict resolution) for writes; globally replicated read replicas.- Caches: - Edge/region-level: Redis cluster per region for low-latency reads (hot keys sharded by user-id). - Application-level local LRU (in-process) for microsecond-level cache of hottest users.- Invalidation propagation: publish-update pattern using a durable pub/sub (Kafka or cloud pub/sub) with regional brokers and global replication/fanout.

Detailed design:1. Cache placement- Client -> App servers (local in-proc cache) -> Regional Redis -> Read replicas/primary DB.- Redis clusters sized for read QPS per region; partition by user-id for even load.

2. Update flow & invalidation propagation- On profile update: - App writes to DB using optimistic concurrency (version/timestamp). - After commit, app publishes a change event: {user_id, new_version, region, timestamp} to regional topic. - Regional broker replicates or forwards to other regions (geo-replication or global topic). - Each region's invalidation worker consumes events and: - Either (preferred) write-through: update regional Redis with new value (push). - Or publish invalidation: delete key or set version in Redis so next read fetches DB. - In-proc caches receive invalidation via local event bus (or HTTP callback) to evict/update.

3. TTLs- Redis TTL: short (e.g., 30s) to limit long-term staleness; rely on push invalidations for fast visibility.- In-process TTL: very short (200-500ms) or manual eviction on invalidation.- Read-through caching disabled for very fresh reads if client requests consistency.

4. Guaranteeing ~2s visibility- Use immediate push invalidation + optional push-update. With regional propagation + replication, expect sub-second propagation within region and ~1–2s cross-region.- Use partitioned fanout and batching small windows (≤100ms) to keep throughput.

5. Stale reads handling- Each cache entry stores version number and timestamp. On read, if cache.version < client-known-version or cache.age > threshold, serve from DB (or trigger async refresh) depending on API SLAs.- Optionally support read-with-consistency flag: client can request strongly-consistent read (bypass cache & read replica lag-aware) for critical flows.- If cache miss or stale, perform read-through to DB and update caches.

6. Concurrent updates- Use optimistic concurrency control: DB row contains version (monotonic counter / last_updated_ts). Updates include expected_version; on conflict return 409 and retry/merge.- Event ordering: include new_version in events; consumers apply only if event.version > cached_version to handle out-of-order delivery.- Idempotency: events have idempotency keys; consumers ignore replays.

7. Operational considerations & trade-offs- Fanout volume: 5k updates/s -> 5k events/s; with geo-fanout and retries plan capacity accordingly.- Use batching for non-critical updates to reduce churn but ensure latency bound.- Monitoring: metrics for invalidation latency, cache hit ratio, replica lag, version mismatches.- Failure modes: if pub/sub down, fall back to TTL expiry; if DB conflict rate high, consider stricter partitioning or leader per shard.

Why this works:- Push invalidation + short TTL minimizes stale reads and meets 2s visibility.- Versioned events and optimistic concurrency make updates safe and idempotent across regions.- Regional Redis + local caches scale read QPS while keeping per-user invalidation cheap and targeted.

HardTechnical

49 practiced

Design a write-behind cache system with a durable queue that minimizes data loss during crashes. Describe the durable queuing mechanism, ack model between cache and DB writer, checkpointing, and recovery procedure to ensure writes are eventually persisted. Discuss trade-offs around latency, throughput, and durability.

Sample Answer

Requirements & constraints:- Asynchronous write-behind cache: serve reads/writes quickly; persist updates to DB eventually.- Minimize data loss on crashes (strong durability), support high throughput and reasonable latency.

High-level design:- Cache accepts writes and enqueues durable write-ops into an append-only persistent queue (WAL segment files or local RocksDB/LevelDB queue). A separate DB-writer (worker pool) consumes the queue and applies batches to the database.

Durable queuing mechanism:- Use an append-only log file (segmented WAL) or embedded persistent key-value store to store ordered operations. Each enqueued record gets monotonic log-offset and metadata (key, value, op-type, timestamp).- Periodic segment roll and fsync on segment tail; optionally use O_DSYNC for each append depending on durability needs.

Ack model between cache and DB-writer:- Two-layer acks: 1. Cache-ack: On write, cache persists to WAL and returns success to client (write-back acknowledged). This gives low client latency. 2. DB-ack (optional sync): For clients needing stronger durability, support sync flag: return success only after DB-writer confirms commit.- DB-writer reads from WAL, batches contiguous offsets, applies to DB. After DB commit, it writes an acknowledgment record or advances a durable committed-offset watermark (checkpoint file) and fsyncs.

Checkpointing:- DB-writer periodically snapshots the highest committed log-offset into a separate durable checkpoint file (fsynced). Also can delete/compact WAL segments older than last checkpoint to reclaim disk.- Batching threshold by size/time to trade latency vs throughput.

Recovery procedure:- On restart, cache/worker replays WAL starting from checkpointed committed-offset+1.- If partial segment present, use record checksums to discard incomplete tail.- Replay: apply idempotent or deduplicated operations to DB. Use idempotency keys (log-offset + operation ID) or upserts with version/timestamp to avoid double-apply effects.- After replay completes and DB confirms, advance checkpoint and truncate logs.

Durability vs latency vs throughput trade-offs:- Lower client latency: fsync on append deferred (only periodic fsync) => higher risk of lost last-window on crash.- Strong durability: fsync each append => high latency and lower throughput.- Throughput: batching DB writes increases throughput but increases window of unpersisted ops.- Balanced config: fsync WAL on segment roll (e.g., every 100ms or N ops) + small batch DB commits (e.g., 1–5ms or 1–10k ops) gives good throughput with bounded data-loss window.- Use configurable durability modes: async (best-effort), buffered (fsync every X ms), and sync (fsync per write + wait DB ack).

Other considerations:- Ensure operations are idempotent or include per-record version to avoid inconsistencies.- Monitor queue lag, uncommitted bytes, and expose metrics/health to throttle clients when backlog dangerously large.- For distributed caches, replicate WAL to another node (RAFT/consensus) to reduce single-node tail-loss risk.

HardTechnical

44 practiced

Implement a memory-efficient deduplication store using a Bloom filter plus an LRU backing store in Python. Provide functions add(id: str) and seen(id: str) -> bool. Explain how you would handle false positives, limited memory, and periodic cleanup to bound memory usage. Provide code or pseudo-code for the key parts.

Sample Answer

Approach: use a Bloom filter as a compact probabilistic "seen" test and an LRU-backed exact store (OrderedDict) to confirm recent items and to allow false-positive correction when needed. To bound memory, cap LRU size and periodically rotate/rebuild the Bloom filter (time-windowed or count-based) so stale items expire and false-positive rate stays bounded.

python

import math, mmh3, time
from collections import OrderedDict

class BloomFilter:
    def __init__(self, capacity, fp_rate):
        m = - (capacity * math.log(fp_rate)) / (math.log(2)**2)
        k = max(1, int((m / capacity) * math.log(2)))
        self.size = int(m)
        self.k = k
        self.bitarray = bytearray((self.size+7)//8)

    def _hashes(self, item):
        for i in range(self.k):
            yield (mmh3.hash(item, i) % self.size)

    def add(self, item):
        for h in self._hashes(item):
            self.bitarray[h//8] |= 1 << (h%8)

    def __contains__(self, item):
        return all(self.bitarray[h//8] & (1 << (h%8)) for h in self._hashes(item))

class DedupStore:
    def __init__(self, bf_capacity=100000, bf_fp=0.01, lru_capacity=10000, rotate_seconds=300):
        self.bf = BloomFilter(bf_capacity, bf_fp)
        self.lru = OrderedDict()
        self.lru_capacity = lru_capacity
        self.rotate_seconds = rotate_seconds
        self.last_rotate = time.time()

    def _rotate_if_needed(self):
        now = time.time()
        if now - self.last_rotate > self.rotate_seconds:
            # rebuild Bloom from current LRU to forget old items
            self.bf = BloomFilter(self.bf.size or 100000, 0.01)
            for key in self.lru:
                self.bf.add(key)
            self.last_rotate = now

    def add(self, id: str):
        self._rotate_if_needed()
        self.bf.add(id)
        # maintain exact LRU cache (most recent at end)
        if id in self.lru:
            self.lru.move_to_end(id)
        else:
            self.lru[id] = True
        if len(self.lru) > self.lru_capacity:
            self.lru.popitem(last=False)  # evict oldest

    def seen(self, id: str) -> bool:
        self._rotate_if_needed()
        if id in self.lru:
            # exact recent hit
            self.lru.move_to_end(id)
            return True
        if id in self.bf:
            # probable seen: could be false positive
            # Option: verify via external store or treat as seen.
            # Here we conservatively treat as seen but add to LRU to reduce repeated FP cost.
            self.add(id)
            return True
        return False

Key points:- False positives: Bloom can say "seen" when not; mitigations: keep an exact LRU for recent items, use counting or rotating Bloom filters to allow deletions, or verify against persistent store when correctness is critical.- Limited memory: tune Bloom capacity and fp_rate; cap LRU size; use rotation/rebuild to forget old items and keep fp-rate bounded.- Periodic cleanup: rotate/rebuild Bloom from LRU (time-window), or maintain multiple Bloom filters in a ring (sliding-window) to expire old entries without full rebuild.Complexity: add/seen are O(k) hashes; memory is O(m/8 + LRU_size).

Unlock Full Question Bank

Get access to hundreds of Caching and Asynchronous Processing interview questions and detailed answers.

Join thousands of developers preparing for their dream job.