Evaluates a candidate's ability to reason about high level system architecture, component interactions, and integration patterns used to build production services. Candidates should be able to visualize major components and the flow of requests and data between them, and to explain client server models, multi tier layered architecture, routing from ingress through load balancing to auto scaled compute instances, and trade offs between monolithic and microservice approaches. Expect discussion of service boundaries and loose coupling; synchronous application programming interfaces and asynchronous messaging; event driven and publish and subscribe architectures; message queues, retry and backoff patterns; caching strategies; and approaches to data consistency and state management. Integration concerns include application programming interfaces, adapters and connectors, extract transform load processes, data synchronization, data warehousing, and the trade offs between real time streaming and batch processing and single source of truth. Candidates should reason about scalability, reliability, availability, redundancy, failover, fault tolerance, latency and throughput trade offs, security boundaries, and common failure modes and bottlenecks. They should also address operational considerations such as monitoring, logging, observability, deployment implications and run books, and explain how architectural choices influence team boundaries, delivery timelines, dependency complexity, testing strategy, maintainability, and operability. Answers should demonstrate clear explanation of design decisions and trade offs without requiring low level implementation detail, and the ability to communicate architecture to both technical and non technical audiences.
EasyTechnical
41 practiced
Describe the primary use cases for message queues versus synchronous HTTP calls. As an SRE, when would you prefer a message queue and what operational concerns (durability, ordering, delivery semantics, scaling) must you evaluate when introducing a queue between services?
Sample Answer
Message queues are for asynchronous, decoupled communication; synchronous HTTP calls are for immediate request/response needs. As an SRE, prefer a message queue when you need resilience, smoothing of spikes, or decoupling for independent scaling and retries — for example: ingesting telemetry/events, background job processing, fan-out notifications, or buffering bursts to protect downstream services.Operational concerns to evaluate before introducing a queue:- Durability: Do messages need to survive broker restarts? Choose persistence (write-to-disk, replication) for critical data; be aware of increased latency and I/O. Define backup/retention policies and recovery procedures.- Ordering: Do consumers require strict ordering? FIFO queues (or partitioning keys like Kafka partitions) preserve order but limit parallelism. If ordering across all messages isn’t required, partition to scale.- Delivery semantics: Decide between at-most-once, at-least-once, or exactly-once. At-least-once is common but requires idempotent consumers and deduplication; exactly-once (stream transactions) is heavier and supported only by some systems.- Scaling and throughput: Evaluate broker horizontal scalability, partitioning model, and consumer group semantics. Plan capacity (IOPS, network), autoscaling, and backpressure strategies so producers slow or drop gracefully when consumers lag.- Operational runbook items: monitoring (queue depth, consumer lag, broker health), alerting thresholds, throughput/latency SLOs, disaster recovery, and maintenance windows. Test failure modes: broker partition, consumer crashes, message duplication, and restore procedures.- Security and compliance: encryption at rest/in transit, ACLs, and audit logs.Trade-offs: queues add complexity and operational overhead but increase resiliency and elasticity. Choose based on required latency, reliability, and operational readiness to support the queue.
EasyTechnical
41 practiced
Compare monolithic and microservice architectures from an SRE point of view. Discuss operational trade-offs including deployment complexity, fault isolation, observability, testing, dependency management, and how these choices affect on-call burden and runbook complexity.
Sample Answer
From an SRE perspective, monoliths and microservices trade simplicity for granularity.High-level:- Monolith: single deployable, simpler CI/CD, easier local testing, fewer cross-service failures but larger blast radius on failures and harder to scale components independently.- Microservices: independent deploys, fine-grained scaling and fault isolation, but more operational surface area.Operational trade-offs:- Deployment complexity: Monolith: straightforward pipelines and rollbacks. Microservices: many pipelines, coordinated releases, versioning, and orchestration (Kubernetes) complexity.- Fault isolation: Microservices limit impact to a service; monolith failures often affect whole app.- Observability: Monoliths need fewer traces/log correlation; microservices require distributed tracing, standardized metrics, and log aggregation.- Testing: Monolith favors integration tests; microservices demand contract tests, end-to-end testing, and staging for interactions.- Dependency management: Monoliths centralize deps; microservices require careful API versioning and dependency mapping.- On-call burden & runbooks: Microservices increase frequency of alerts and complexity of runbooks (cross-service playbooks, circuit breaker tuning). Monoliths simplify runbooks but incidents are higher-impact.Recommendation: choose based on team maturity and SRE investment—start monolith for early stages, adopt microservices when needs for independent scaling and deploy velocity justify added operational cost; invest in automated CI/CD, service discovery, tracing, and clear SLOs to keep on-call load manageable.
MediumSystem Design
38 practiced
Design an API gateway for a microservices platform that enforces authentication, rate limiting, request routing, TLS termination, and basic request validation. Explain how to make it reliable, horizontally scalable, and observable, and where to implement per-tenant quotas vs global quotas.
Sample Answer
Requirements:- AuthN/AuthZ, rate limits (global + per-tenant), request routing to services, TLS termination, basic request validation (schema, size, headers).- Non‑functional: high availability, horizontal scalability, low latency, strong observability.High-level architecture:- Edge load balancer (L4) → API Gateway fleet (stateless proxies) behind an L7 LB → Auth/Z service + Rate-limiter (fast store) + Config/Policy store → Backend services.Core components and responsibilities:1. API Gateway (envoy/nginx/linkerd or custom): TLS termination, SNI, mutual-TLS if needed, header normalization, path-based routing, request validation (JSON schema, size limits), authentication token extraction and caching, and forwarding.2. Auth service (OAuth/OIDC introspection or JWT verification): validate tokens; for JWT validate locally with public keys to avoid network calls when possible.3. Rate limiter: token-bucket implemented as a distributed service with a low-latency store (Redis with Lua, or consistent-hash sharded in-memory stores). Enforce per-tenant quotas (customer-specific limits, burst and sustained) and global quotas (system-wide emergency limits).4. Policy/config store: central source (etcd/Consul) for routing rules, per-tenant quotas, feature flags; gateways pull via watch.5. Control plane: deploys config, rotates keys, manages certificates (ACME/HashiCorp Vault).6. Observability: structured logs, distributed tracing (trace-id propagation), metrics (request rates, latencies, 429s, auth failures) exported to Prometheus, dashboards and alerts for SLOs.Data flow:- LB → Gateway terminates TLS → validate headers/body → locally verify JWT (or call auth service if opaque token) → consult rate-limiter (local cache+RPC to limiters) → route to backend.Scalability & reliability:- Make gateway stateless so it scales horizontally via autoscaling groups/K8s HPA.- Use L4/L7 load balancers to distribute; health checks and rolling updates.- Shard rate-limiter by tenant id to avoid hot keys; replicate Redis with clustering and use Lua for atomicity.- Cache auth public keys and policy locally; use short TTLs and push updates via pub/sub to avoid stamping effects.- Graceful shutdown to drain connections; circuit breakers and retries with idempotency keys.Per-tenant vs global quotas:- Per-tenant quotas enforced at gateway/limiter close to edge (to reject early and account correctly). Store per-tenant state sharded to limiter nodes.- Global quotas enforced centrally in limiter shard responsible for global counters; used for emergency throttling and capacity protection.Observability & SRE practices:- Emit metrics per tenant, per route; tag with error codes. Create SLOs (p99 latency, availability) and alert on error budget burn.- Trace through gateway into services; sample intelligently.- Chaos testing (rate-limiter failures, high auth latency) and runbooks for mitigation (fallback JWT verification, degrade to cached policies).- Automated certificate rotation, key rollovers, and policy audits.Trade-offs:- Local JWT verification minimizes latency but requires robust key distribution.- Strong consistency for counters vs performance: prefer eventual for non-critical telemetry, but atomic Lua scripts for quota checks.
EasyTechnical
43 practiced
Explain the decision criteria between synchronous and asynchronous communication between services. Include SLO implications, user-facing latency impact, complexity of error handling, and how to design retries and fallbacks for asynchronous flows.
Sample Answer
Decision criteria between synchronous and asynchronous communication boils down to user-visible latency, reliability/SLO goals, and operational complexity.When to choose synchronous:- Use when callers need an immediate, deterministic response (e.g., payment authorization, auth token). - SLO/latency: the end-to-end P99 must meet user-facing SLAs; include downstream latency in budget.- Error handling: simpler in-flow retries and backpressure, but you must bound retries to avoid cascading failures (circuit breakers, timeouts, bulkheads).- Example: HTTP REST call with 200–500ms P99 requirement → sync if downstream can meet that.When to choose asynchronous:- Use when work can be deferred (batch jobs, event processing, notifications) or to decouple availability between services.- SLO/latency: user-facing latency improves because request completes quickly; define separate SLOs for ingestion (ack within tens of ms) and processing (eventual consistency SLA, e.g., process within 30s).- Error handling complexity: higher — you need durable queues, idempotency, visibility into stuck messages, DLQs, and replay mechanisms.- Retries and fallbacks: - Use exponential backoff with jitter for consumer retries; cap max attempts. - Persist failed messages to Dead Letter Queue (DLQ) after attempts; attach metadata for diagnosis. - Design idempotent consumers or store dedupe IDs to allow safe replays. - Provide user-facing fallback: synchronous ack + "processing pending" UI or a compensating action (e.g., optimistic update + eventual reconciliation). - Monitor processing lag and DLQ rate as SLO indicators; alert on growing lag or DLQ spikes.- Example pattern: enqueue order, respond 202 with order-id, process asynchronously with retries up to N, move to DLQ, notify user if final failure.Trade-offs summary:- Sync: simpler semantics, tighter latency guarantees, higher coupling and risk of cascading failures.- Async: better isolation and scalability, more complex operational tooling and eventual-consistency thinking; requires careful SLO decomposition, observability (ingest latency, processing latency, DLQ rate), and robust retry/fallback design.
MediumTechnical
43 practiced
Compare trade-offs between implementing a multi-service profile update as a synchronous fan-out API call versus an asynchronous event-driven flow. Discuss latency, consistency, error handling complexity, user experience, operational tests, and how to revert or correct partial failures.
Sample Answer
High-level summary: Synchronous fan-out (API gateway calls all profile services in request path) favors strong immediate consistency and simpler rollback semantics but increases latency, blast radius, and coupling. Asynchronous event-driven (publish profile-change event; services consume and update) favors lower coupling, better tail-latency and resilience but introduces eventual consistency and more complex corrective workflows.Compare by dimension:- Latency - Sync fan-out: end-to-end latency = sum of downstream latencies + retries; sensitive to slowest service and network—poor tail latency. - Async event: user request returns quickly once event is enqueued; consumers process independently, reducing request latency and improving UX responsiveness.- Consistency - Sync: near‑immediate read-after-write consistency across services if all updates succeed in the same request; easier to reason about. - Async: eventual consistency; reads may see stale data until all consumers process the event. Requires client expectations and read-path design (e.g., read-your-writes caches, versioning).- Error handling complexity - Sync: errors detected immediately; can use two‑phase commit-like patterns or compensating transactions. But implementing distributed transactions is costly; often you do best-effort + rollback which is brittle. - Async: need durable event delivery, retries, dead-letter queues, idempotency, and monitoring for stuck consumers. More components and operational complexity but failures are isolated.- User experience - Sync: users get immediate confirmation that all systems updated; good for workflows that need immediate consistency (billing, security). - Async: faster perceived response but must surface eventual state (e.g., “Changes may take up to X seconds to appear”) and degrade gracefully.- Operational tests and observability - Sync: instrument request traces, per-call latency, and circuit breakers. Test chaos scenarios by injecting latency/failures to downstream services. - Async: require observability for event queues (lag, backlog), consumer processing rates, DLQ metrics, end-to-end reconciliation metrics, and tracing across pub/sub. Add consumer integration tests and chaos tests on event delivery.- Revert / correct partial failures - Sync: if one downstream fails, options are rollback all prior successful updates (compensating calls) or abort and surface error. Rollbacks are synchronous and immediate but can leave transient inconsistencies if rollback fails. - Async: use compensating events or reconciliation jobs. Implement idempotent consumers, versioned profile events, and a reconciliation process that compares authoritative source to consumer state and emits fixes. Provide manual operator playbooks and automation for backfills.Recommendations (SRE perspective):- Choose sync when strict immediate consistency is required and the number of fan-out targets is small and highly reliable.- Choose async for scalability, resilience, and to reduce tail latency—especially when many independent services must react to profile changes.- Regardless of choice: enforce idempotency, observability (traces, SLOs for processing lag), retry policies, and clear UX messaging. Build automated reconciliation and operational runbooks for partial failure recovery; test with chaos/latency injection and end-to-end integration tests.
Unlock Full Question Bank
Get access to hundreds of System Architecture and Integration interview questions and detailed answers.
