Event Driven and Asynchronous Architecture Questions

Designing and operating systems that decouple components using asynchronous messaging and event driven patterns. Covers message queues and brokered communication models (for example Kafka, RabbitMQ, Amazon SQS), publish subscribe patterns, producer consumer workflows, background job and task queue design, and when to prefer asynchronous versus synchronous request response interactions. Includes higher level architectural patterns such as event sourcing, Command Query Responsibility Segregation, sagas for distributed transactions, and patterns for decoupling services. Operational concerns include delivery semantics (at least once, at most once, exactly once), ordering guarantees and partitioning, dead letter queues, retry strategies, idempotency, error handling, monitoring and alerting (for example message lag, queue depth), scaling consumers, throughput and latency trade offs, consistency implications, and common use cases such as email sending, batch processing, file processing, notification delivery, and distributed work coordination.

EasyTechnical

73 practiced

Compare Apache Kafka, RabbitMQ, and Amazon SQS for backend message handling. For each broker mention typical strengths, common use-cases, delivery semantics they favor, and one operational trade-off (throughput, latency, ordering, persistence).

MediumTechnical

92 practiced

Implement an exponential backoff retry strategy as a middleware for a Node.js message consumer. The middleware should support max retries, jitter, and configurable base/backoff multipliers. Provide code or clear pseudocode showing retry logic, how failures are bubbled to DLQ after max retries, and where to insert idempotency checks.

Sample Answer

**Approach (brief)**

Wrap the consumer handler with an async middleware that retries on transient errors using exponential backoff with optional jitter. After maxRetries the message is routed to the DLQ. Idempotency checks happen before performing the main business logic (and before acknowledging or re-queueing).

**Code (Node.js, async/await)**

javascript

// retryMiddleware.js
function retryMiddleware(handler, opts = {}) {
  const {
    maxRetries = 5,
    baseMs = 100,          // initial delay
    multiplier = 2,        // exponential factor
    jitter = 0.2,          // fractional jitter (0.2 => +/-20%)
    dlq = async (msg, meta) => { /* send to DLQ store */ }
  } = opts;

  function sleep(ms){ return new Promise(r=>setTimeout(r, ms)); }
  function withJitter(delay){
    const rand = (Math.random()*2 - 1) * jitter; // in [-jitter, +jitter]
    return Math.max(0, Math.floor(delay * (1 + rand)));
  }

  return async function wrappedHandler(msg, meta) {
    // Idempotency check: return early if already processed
    // e.g., check messageId in processed table/cache
    if (await opts.isIdempotent && await opts.isIdempotent(msg, meta)) {
      return { status: 'ignored', reason: 'idempotent' };
    }

    let attempt = 0;
    while (true) {
      try {
        attempt++;
        const result = await handler(msg, meta);
        return { status: 'ok', result, attempts: attempt };
      } catch (err) {
        if (attempt > maxRetries) {
          await dlq(msg, { err, attempts: attempt, meta });
          throw Object.assign(err, { dlq: true, attempts: attempt });
        }
        const delay = withJitter(baseMs * Math.pow(multiplier, attempt - 1));
        await sleep(delay);
        // loop to retry
      }
    }
  };
}

module.exports = { retryMiddleware };

**Where to insert idempotency**- Provide an isIdempotent(msg, meta) function (DB/cache lookup) and pass as opts.isIdempotent. Run it before invoking handler and before any ack/nack logic so duplicates are safely ignored.

**Failure flow**- After exceeding maxRetries, send to DLQ and throw so upstream consumer can ack/cleanup.

EasyTechnical

107 practiced

Describe the difference between publish/subscribe and point-to-point queue (work-queue) messaging models. Provide an example backend scenario for each where one model is clearly preferable and explain why.

MediumTechnical

84 practiced

Explain the transactional outbox pattern and how it helps achieve reliable event publication from a service that uses a relational database. Describe steps required to implement it and trade-offs compared to distributing two-phase transactions.

Sample Answer

**What it is (brief):**The transactional outbox pattern ensures events describing state changes are published reliably by storing the event in the same database transaction that updates your domain tables. That guarantees the event is durable iff the domain change commits — avoiding the lost-event problem.

**How it works — implementation steps**1. Schema: add an Outbox table (id, aggregate_id, type, payload, created_at, processed_at, status, attempts).2. Write path: inside the same DB transaction, update domain tables AND insert a row into Outbox with the event payload.3. Delivery component: a separate process (poller or worker) reads unprocessed outbox rows, sends them to the message broker, and marks them processed (or increments attempts / records failure) in the DB.4. Idempotency & dedup: include an event id and make consumers idempotent; the poller should handle retries and idempotent writes to avoid duplicates.5. Ordering and batching: preserve ordering by reading by created_at or sequence id; batch publishes for throughput.6. Observability: add metrics, dead-letter handling for poison messages.

**Why it helps**- Keeps atomicity between state change and event creation without distributed transactions.- Avoids race conditions where the app commits domain change but the publish fails (or vice versa).

**Trade-offs vs distributed two-phase commit (2PC / XA)**- Outbox: eventual consistency, simpler, avoids distributed locks and coordinator complexity, works across heterogeneous systems, higher throughput and availability. Drawbacks: publish latency (poll interval), complexity in maintaining poller and retries, at-least-once delivery requires idempotent consumers, ordering guarantees limited to scope.- 2PC: gives atomic commit across DB and broker (stronger synchronous consistency). Drawbacks: complex to operate, poor scalability, risk of coordinator or resource manager blocking, poor availability under failures, many brokers/databases lack robust XA support.

**Alternatives**- Use Change Data Capture (CDC) streaming (e.g., Debezium) to stream DB binlog -> topic, which automates outbox behavior but requires connector infrastructure and careful schema/evolution handling.

Use outbox for most microservice cases: simple, robust trade-off favoring availability and operational simplicity over global synchronous consistency.

EasyTechnical

77 practiced

You need to design a backend flow for sending account activation emails asynchronously. Describe the high-level architecture: how the API enqueues work, what the consumer does, retry and DLQ behavior, and how you ensure emails are not sent multiple times for the same request.

Unlock Full Question Bank

Get access to hundreds of Event Driven and Asynchronous Architecture interview questions and detailed answers.

Join thousands of developers preparing for their dream job.