This topic focuses on design patterns and infrastructure for real time features. Candidates should compare and justify choices between Websocket based connections, server sent events, and polling, and describe the operational implications of each choice such as connection lifetime, server resource usage, and request patterns. At scale, discuss load balancing strategies, connection routing and state, sticky sessions versus stateless approaches, message brokers, and horizontal scaling. Demonstrate understanding of real time features like presence, live updates, and collaborative editing, and design patterns including broadcast, unicast, and multicast messaging. Address reliability concerns such as message ordering, duplicate delivery versus exactly once semantics, idempotency, persistence, and conflict resolution in collaborative scenarios. Be prepared to mention relevant libraries and protocols and to describe trade offs between flexibility, complexity, and operational cost.
MediumTechnical
45 practiced
A client asks whether to use a managed real-time platform (e.g., AWS API Gateway WebSocket, AppSync, Pusher) versus building and operating their own WebSocket fleet. Create a decision matrix comparing cost, time-to-market, operational risk, customization and feature flexibility, vendor lock-in, and compliance. Provide recommendations for a bootstrapped startup and for a regulated large enterprise.
Sample Answer
Decision matrix — Managed real-time platform (API GW WS, AppSync, Pusher) vs Self‑operated WebSocket fleet- Cost - Managed: Predictable OPEX, pay-per-connection/message; can be cheaper short-term but costs grow with scale and egress. - Self: Higher upfront capital (infra, engineering), lower marginal cost at very large scale but requires headcount.- Time-to-market - Managed: Fast (hours–days). SDKs, auth integrations, scaling handled. - Self: Slow (weeks–months) to build reliable, scalable handlers, HA, autoscaling, backpressure.- Operational risk - Managed: Lower operational burden; SLA-backed availability, provider handles scaling/patches. - Self: Higher risk — you own monitoring, incident response, capacity planning.- Customization / feature flexibility - Managed: Limited to provider APIs/features (message transforms, routing, persistence). Good for standard patterns. - Self: Maximum flexibility — custom protocols, QoS, nonstandard routing, optimized binary protocols.- Vendor lock-in - Managed: Higher lock-in (proprietary APIs, clients, auth). Migration can be nontrivial. - Self: Minimal lock-in; full control over stack and migration paths.- Compliance - Managed: Depends on provider: many offer PCI/HIPAA/GDPR-ready features and certifications, but data residency or audited control may be constrained. - Self: Stronger control for strict regulatory needs (on‑prem, audited controls), but requires own compliance program.Recommendations- Bootstrapped startup: Use a managed platform. Rationale: maximize time‑to‑market, minimize engineering spend and operational risk; iterate product/market fit quickly. Choose a provider with reasonable pricing and exportable client libraries; design abstraction layer in your app to encapsulate provider-specific code so you can migrate later if costs/needs change.- Regulated large enterprise: Prefer self‑operated or hybrid. Rationale: strict compliance, data residency, deep customization, and long-term economics favor owning the stack. Consider a phased approach: start with managed for non-sensitive flows, then implement self‑hosted components for regulated workloads. If using managed, validate provider certifications, contractual SLAs, data residency, and exit/migration arrangements.Operational best practices- Abstract provider interfaces to reduce lock‑in.- Instrument extensively (metrics, tracing, synthetic tests).- Run cost projections at target scale (connections, egress, retention).- Plan migration path and export formats before committing.
MediumSystem Design
47 practiced
Design a cross-platform real-time notification service that consumes events from multiple microservices and delivers them as sub-second high-priority push notifications (mobile push), in-app WebSocket streams, or email for lower priority. Requirements: per-user preferences, per-user throttling, reliable retry for failed delivery, and audit logs. Provide a component diagram and describe event flow, fanout, persistence, transformation, and backpressure handling.
Sample Answer
Requirements (clarify): deliver sub-second high-priority push (mobile push), real-time in-app (WebSocket), and lower-priority email. Support per-user preferences, per-user throttling, reliable retry, and audit logs. Assume millions of users, 10k+ events/sec peak.High-level architecture (ASCII component diagram):Event Producers (microservices) ↓ (events → Event Bus)Event Bus (Kafka / Pulsar) → Validation & Enrichment Service ↓Notification Router / Orchestrator ├─ Preference Service (read-cached, Redis) ├─ Throttle Service (token-bucket per-user, Redis/Redis-Streams) ├─ Transformer (templates, channel-specific payload) ├─ Fanout Worker Pool (Kafka topics / push queues) └─ Persist/Audit Store (WAL + OLAP store)Fanout Workers → Channel Connectors: - Push Gateway (APNs/FCM through gateway + token management) - WebSocket Gateway (scale with HAProxy / nginx + ephemeral session store) - Email Provider (SMTP / SES)Dead-letter / Retry Queue → Retry Worker → Circuit Breaker / BackoffMonitoring / Metrics / Admin UIEvent flow:1. Microservice emits event to Event Bus (lightweight protobuf). Events are immutable.2. Validation & Enrichment reads, attaches user IDs, metadata.3. Router consults Preference Service (cache with TTL; fallback to DB) to determine enabled channels & priority.4. Router asks Throttle Service for allowance (token-bucket keyed by user + priority overrides). If denied, either drop (low priority) or queue with delayed retry (high priority).5. Transformer generates channel-specific payloads (JSON for push, HTML/text for email, JSON frame for WebSocket).6. Fanout Worker publishes per-channel tasks to scalable worker queues (separate partitions per user shard to preserve ordering).7. Channel Connectors perform delivery; success/failure written to Audit Store and metrics emitted. Failures go to Retry Queue with exponential backoff; after N attempts go to DLQ and persisted.Fanout & ordering:- Partition Kafka/queue by user-id to ensure ordering per user. Use worker pools per partition.- For high throughput, group small notifications and coalesce per-user within short window (10–100ms) if acceptable.Persistence & audit:- Immediate write-ahead log of event → Notification DB (append-only), includes status, attempts, timestamps, channel, payload hash.- Audit Store: OLAP (ClickHouse) for queries; raw events archived in object store (S3) for compliance.Throttling:- Token-bucket per user implemented in Redis (Lua ops for atomicity). Different buckets for high/low priority. Admin UI to set SLAs and overrides.- Global rate limits (per-IP, per-app) enforced at Router.Backpressure & reliability:- If downstream connectors are slow, Router will stop consuming by pausing Kafka consumer or use internal backpressure queue; produce to durable queue (retry/holding topic) to avoid data loss.- Circuit breakers for each external provider; when tripped, route deliveries to alternate provider (multi-FCM/third-party) or queue for retry.- Exactly-once-ish: dedupe by event-id in worker (idempotent writes) using Redis/DB TTL keys.- Retry policy: exponential backoff with jitter, configurable per channel and per error type.Transformation & personalization:- Template engine with placeholders, resolving user locale and preferences. For push, payload minimized for speed; for WebSocket full JSON.Monitoring, ops & security:- End-to-end traces (distributed tracing), SLOs for p99 latency, alerting on queue lag, error rates.- Auth between services (mTLS), secrets in vault for push credentials, rate-limited admin APIs.Trade-offs:- Kafka gives durability and replay; Pulsar adds per-message TTL and topic-level delayed delivery—useful for throttling.- Strict ordering per user limits parallelism; partition count must balance throughput vs ordering.- Aggressive coalescing reduces downstream calls but increases delivery latency jitter.This design meets sub-second high-priority paths (direct fast path to Push & WebSocket via in-memory cache + low-latency worker pools), supports per-user prefs/throttling, reliable retries with DLQ/audit, and handles backpressure with durable queues and circuit breakers.
MediumTechnical
45 practiced
You're choosing between CRDTs and Operational Transforms (OT) for a collaborative rich-text editor that must support offline edits, multi-region users, and audit logs. Compare the approaches on correctness guarantees, implementation complexity, bandwidth/metadata overhead, server role (stateful vs stateless), debug/administration, and operational cost. Which would you recommend for an enterprise product and why?
Sample Answer
Situation: Choosing a sync layer for an enterprise rich-text editor that must work offline, across regions, and keep audit logs.Recommendation (short): Use a CRDT-based approach for enterprise product when offline-first, multi-region availability, and reliable auditability are primary. OT can work for simpler centralized deployments but has higher operational fragility at scale.Comparison by dimension:- Correctness guarantees: - CRDT: Strong convergence by design (commutative & idempotent ops); merge semantics are well-defined; deterministic final state even with concurrent edits. - OT: Intention-preserving with transform functions, but correctness requires careful transform algorithms and global ordering; edge cases (complex rich-text transforms) are tricky and historically subtle.- Implementation complexity: - CRDT: Conceptually simpler for distributed clients (local ops apply immediately); complex when mapping rich-text model to practical CRDT (tombstones, position identifiers). - OT: Complex core (compose/transform/undo), especially with rich semantics and multiple concurrent transforms; requires rigorous testing.- Bandwidth / metadata overhead: - CRDT: Higher per-op metadata (identifiers, causal info); can grow (tombstone accumulation) unless compacted. - OT: Typically smaller op payloads; relies on server to maintain transformation context.- Server role (stateful vs stateless): - CRDT: Can operate mostly peer-to-peer; server can be stateless relayer or act as a persistent CRDT replica for sync/backup. - OT: Server must be authoritative and stateful to serialize and transform operations.- Debug / administration: - CRDT: Easier to reason about final state and offline merges; debugging metadata-heavy but deterministic. Garbage collection/tombstone compaction needs tooling. - OT: Harder to replay/debug due to transforms and dependencies; operationally riskier to migrate/restore state.- Operational cost: - CRDT: Higher storage/network cost and tooling for compaction and audits, but lower need for always-on authoritative servers—better multi-region resilience. - OT: Lower per-op cost but requires robust, stateful central servers (higher uptime & scaling ops), careful rollouts, and more engineering to avoid correctness bugs.Audit logs & compliance:- CRDT: Full op logs with causal metadata naturally support immutable audit trails; you must retain or derive a compacted history for compliance.- OT: Server-side authoritative logs are natural; but transforms obscure original intent unless you persist raw client operations.When to pick OT:- Centralized SaaS with low offline use, strict bandwidth limits, and a small engineering team willing to own a high-availability transform server.When to pick CRDT (my recommendation for enterprise):- Offline-first requirements, multi-region low-latency collaboration, and strong auditability. CRDTs reduce single-server availability risk, provide deterministic merges, and simplify client logic. Plan for metadata compaction (garbage collection), rigorous rich-text CRDT selection (e.g., RGA, Logoot, or Yjs-like approaches), and build operational tooling: tombstone compaction jobs, audit exporters, and monitoring for metadata growth.Operational plan highlights:- Start with a proven CRDT library (Yjs or Automerge variants), prototype rich-text UX (formatting, embeds, undo), measure metadata growth and bandwidth, and design server replicas that persist CRDT state and provide read-only audit exports. Allocate engineering effort for GC, backup/restore, and migration tooling.This balances product needs (offline, multi-region, audits) with long-term operational resilience.
MediumSystem Design
40 practiced
Design an observability strategy for a real-time platform. Which metrics, logs, and distributed traces would you collect for connection health (churn, duration), message delivery (latency, success/failure), broker lag, and end-to-end user-perceived latency? Propose SLOs, alerting thresholds, and example dashboards and runbooks for on-call teams.
Sample Answer
Requirements & constraints (clarify first): real-time platform (pub/sub or websocket), target latency SLAs, scale (#connections, msgs/s), multi-region or single-region, persistent brokers vs ephemeral.Observability strategy — goals:- Detect connection health regressions (churn, duration)- Measure message delivery correctness and latency- Track broker lag/backpressure- Correlate end-to-end (E2E) user-perceived latencyWhat to collect- Metrics (time-series, high-cardinality tags: region, cluster, topic, client_version, env): - connection_open_total, connection_close_total, active_connections (gauge) - connection_churn_rate = closes / avg_active_connections per window (1m/1h) - connection_duration_histogram (p50/p95/p99) - messages_produced_total, messages_consumed_total, message_success_total, message_failure_total - publish_latency_ms_histogram (p50/p95/p99), ack_latency_ms_histogram - broker_inflight_msgs, broker_queue_depth, broker_disk_util, broker_cpu, broker_mem - consumer_lag_msgs, consumer_lag_seconds (per partition/consumer group) - e2e_latency_ms_histogram (client publish -> client receive/ack) - error rates by component (auth, protocol, broker, network)- Logs (structured JSON): - Connection lifecycle events with connection_id, user_id, client_version, reason code - Message lifecycle: message_id, topic, producer_id, consumer_id, publish_ts, broker_ts, ack_ts, status - Broker internal errors, GC pauses, rebalances, retention events - Include trace_id / correlation_id in every log- Distributed traces: - Spans: client_connect (TLS/handshake/auth), protocol_handler, publish->broker_enqueue, broker_dispatch, consumer_receive, ack - Capture timings, tags (topic, partition, broker_id), errors, sampling higher for errors + tail latencySLOs & thresholds (examples)- Connection availability: 99.95% of connection attempts succeed (5 min window) — page if <99.9% in 5 min- Churn: churn_rate < 1% per hour per region — page if >3% sustained 5m- Connection duration: median > 30m (if keep-alive expected) — warn if p50 < 5m for many clients- Message delivery success: 99.99% success over 1h — page if success rate < 99.9% in 5m- Publish latency: p95 < 100ms, p99 < 500ms — page if p95 > 200ms sustained 5m- Broker lag: consumer_lag_seconds < 1s or consumer_lag_msgs < 1000 — page if >5s or >5000 msgs- E2E latency: p95 < 200ms, p99 < 1s — page if p95 > 500msAlerting policy (noise-conscious)- Page (P1): SLO breach likely impacting customers (message success <99.9% 5m, broker lag >10s, region-wide connection failure, large error spike)- Page (P2): Infrastructure critical but degraded (p95 latency >200ms, churn >3% 5m)- Ticket (P3): Non-urgent increases, single-broker warnings, minor metric deviations- Use alert grouping by region/topic and suppression for known maintenanceExample dashboards (panels)- Overview: active_connections, churn_rate, connection_duration_p95, connections_by_version- Messaging health: messages_in/sec, messages_out/sec, success_rate, publish_latency_p50/p95/p99- Broker health: inflight_msgs, queue_depth, consumer_lag (heatmap by partition), disk/io, GC/pause- E2E: request flow trace sampler, e2e_latency histograms, top slow clients/topics- Errors & logs: recent error logs with trace links, top error typesRunbooks (short actionable)- Connection storm / mass disconnects: 1. Check active_connections & churn_rate per region. 2. Inspect auth/service errors in logs (reason codes). If auth failure: check certs/identity provider. 3. Check network ACLs and load balancer health. 4. If broker CPU/memory saturated, scale brokers or throttle producers; follow broker scaling playbook. 5. Add temporary traffic routing to healthy region if multi-region.- High publish latency or increased p99: 1. Check publish_latency and broker queue_depth. 2. Examine recent GC/heap pressure on brokers; restart/scale if needed. 3. Correlate traces for slow spans → network vs broker vs client. 4. Roll back recent deploys tagged by client_version if correlated.- Broker lag spike: 1. Identify affected consumer groups/partitions via consumer_lag heatmap. 2. Check consumer throughput, GC pauses, rebalances. 3. If consumers stuck, restart consumer process or trigger rebalance; if broker issue, move partitions or scale horizontally.- Message loss / failure: 1. Query message_success_total vs produced; sample failed message_ids from logs. 2. Use trace_id to follow lifecycle. If persistence failure, check disk space, retention, replication health. 3. Trigger producer retries or reprocess from dead-letter topic.Best practices- Correlate logs, metrics, traces via correlation_id/trace_id.- High-cardinality tags sparingly; sample detailed metrics/traces for errors and tail latency.- Automate synthetic checks: synthetic clients/heartbeat per region to measure E2E latency and connection success.- Regularly review SLOs with business stakeholders; tune alert thresholds to reduce toil.This design gives on-call teams actionable signals, quick triage paths, and measurable SLOs to protect user experience.
MediumTechnical
41 practiced
Estimate and compare the per-connection server resource implications (file descriptors, memory per connection, CPU context switching or event-loop overhead) and request pattern characteristics for: (a) persistent WebSocket connections, (b) Server-Sent Events (SSE), and (c) short polling every 5 seconds. Provide a back-of-envelope calculation for 1M concurrent clients and discuss the main bottlenecks and mitigation strategies.
Sample Answer
Situation: You need to compare per-connection resource implications and request patterns for WebSocket, SSE, and 5s short polling and estimate for 1M concurrent clients, then identify bottlenecks and mitigations.Answer:Baseline per-connection costs (order-of-magnitude):- File descriptors: 1 FD per TCP connection + 1 FD for any TLS socket => ~1 FD.- Memory per connection (TCP + app buffers + runtime): typical ranges: - WebSocket: 8–64 KB (TCP socket ~2–8 KB, lib/event-loop structs + app buffers ~6–56 KB) - SSE: 6–48 KB (HTTP keep-alive headers slightly higher than raw WS) - Short polling: ephemeral connections => server keeps minimal state per request (~1–8 KB) but many concurrent short-lived sockets during bursts.- CPU/event overhead: - WebSocket: low per-message overhead with event-loop callbacks; cost grows with messages/sec. - SSE: similar to WS but text/event-stream framing; marginally lower for simple server push. - Polling: high request-rate overhead: TLS handshake (if HTTPS), HTTP parsing, routing, and app-layer work per poll.Back-of-envelope for 1,000,000 concurrent clients (assume TLS terminated at load balancer or not — give two scenarios). Use non-TLS to simplify:File descriptors:- 1M clients → ~1M FDs just for sockets. Add FDs for listeners, logs => plan for 1.05–1.1M FDs per host cluster capacity.Memory:- WebSocket: assume 32 KB/conn average → 32 GB per million connections.- SSE: assume 24 KB/conn → 24 GB per million.- Short polling: if connections are ephemeral and not concurrently open, memory for concurrency depends on QPS. Polling every 5s from 1M clients = 200k requests/sec. If server keeps ~200k concurrent short-lived sockets (assume 100ms connection lifetime), memory ~200k * 4 KB = 800 MB live; but overall CPU/context cost huge.CPU / request processing:- WebSocket/SSE: steady-state CPU for keep-alive is small (tcp keepalive syscalls off). Main CPU cost = messages/sec * handler cost.- Polling: 200k RPS for 1M clients @5s interval. Each request includes HTTP parse, app logic, serialization. If one request costs 0.5 ms CPU, 200k RPS requires 100 cores (200k * 0.0005s = 100 CPU). TLS increases CPU further.Main bottlenecks:- File descriptor limits and kernel knobs (ulimit, somaxconn).- Memory per-connection — limits total connections per host.- TLS termination CPU (if per-request) — very expensive for polling.- Event-loop starvation if single-threaded process tries to handle many FDs.- Load balancer / reverse proxy scale (Nginx, Envoy tunables).Mitigations / architecture options:- Horizontal scale: shard connections across many servers; use consistent hashing or connection proxies.- Use specialized connection servers (C++/Rust evented servers or NGINX with keepalive) to minimize per-connection memory.- Offload TLS to dedicated terminators or use hardware/accelerators; terminate TLS at LB.- For 1M connections: use many thin instances (e.g., 100 servers each ~10k connections if per-server memory ~320 MB for WS at 32 KB/conn) — adjust by per-host memory.- Use brokers (Kafka, Redis streams) + fan-out services to reduce app-layer work on each connection.- Prefer push (WebSocket/SSE) over polling for scale and latency. If polling unavoidable, increase interval, use long polling or conditional GET (ETag/If-Modified-Since) and CDN caching to reduce backend load.- Tune OS: increase net.core.somaxconn, file-max, tcp_tw_reuse, ephemeral port ranges, and use epoll/kqueue and SO_REUSEPORT; use event-driven runtimes and avoid per-connection threads.Recommendation summary:- For 1M concurrent real-time clients, push technologies (WebSocket or SSE) are far more resource-efficient than 5s polling because polling multiplies RPS and CPU/TLS cost. SSE slightly lighter than WebSocket for simple server-to-client streaming; WebSocket is better for bidirectional traffic. Architect with horizontal sharding, TLS offload, optimized event-loop servers, and autoscaling for message fan-out.
Unlock Full Question Bank
Get access to hundreds of Real Time Communication Architecture interview questions and detailed answers.
