System Architecture Principles Questions

Core principles and patterns for designing and evaluating high level and system architectures for distributed and cloud based systems. Candidates should understand high availability and redundancy, fault tolerance and graceful degradation, and how to design stateless and stateful components. They should be able to explain scalability and capacity planning strategies including horizontal and vertical scaling, partitioning and sharding, load balancing, caching and replication, and the trade offs involved. The topic covers consistency models and the trade offs between consistency, availability and partition tolerance, performance and latency optimization, reliability and durability, security for data and access control, and cost efficiency. Candidates should be able to discuss fault domains and why critical components are replicated across availability zones and regions, as well as backup, recovery and disaster recovery approaches. Common architectural patterns such as monolithic and microservice architectures, layered design, event driven and message based systems, and command query responsibility segregation are relevant. Monitoring and observability practices including metrics, logging, distributed tracing and alerting are part of assessments, together with the ability to justify architecture decisions based on functional and nonfunctional requirements, constraints, expected load and operational complexity.

EasyTechnical

30 practiced

Explain the differences between Layer 4 (transport) and Layer 7 (application) load balancing. As a Solutions Architect, describe scenarios where each is more appropriate, including implications for TLS termination, sticky sessions, HTTP routing rules, and observability.

Sample Answer

Layer 4 (Transport) vs Layer 7 (Application) load balancing — quick summary and when to use which:

Definition & capabilities- L4 (TCP/UDP): Routes traffic based on IP and port. Operates at connection/packet level, very fast and low overhead. Doesn’t inspect HTTP headers or payload.- L7 (HTTP/HTTPS): Understands application protocols (HTTP/2, WebSocket). Can inspect URLs, headers, cookies, and JSON bodies to make routing decisions.

TLS termination- L4: Typically passes TLS through to backend (TCP passthrough). Backends manage certificates — good for end-to-end encryption or mutual TLS.- L7: Can terminate TLS at the LB, offloading cert management and enabling HTTP-level features (redirects, header injection). Can also re-encrypt to backends if needed.

Sticky sessions- L4: Achieved via source-IP hashing or consistent hashing — coarse, can break with NAT/proxy.- L7: Use cookies or application session IDs for precise affinity (more reliable for web apps).

HTTP routing rules- L4: No HTTP routing — only port/IP rules.- L7: Route by host, path, headers, query params, method; enables A/B testing, blue/green, canary deployments, API gateway patterns.

Observability- L4: Metrics are connection-level (bytes, active connections, latency). Limited visibility into request semantics.- L7: Rich telemetry — request/response codes, URL-level latency, per-route metrics, distributed tracing headers, easier logging and security inspection (WAF integration).

When to choose- L4: High-performance TCP/UDP services (databases, SMTP, gaming, low-latency proxies), simple pass-through TLS, or when you must preserve end-to-end TLS/mTLS.- L7: Web APIs, microservices, multi-tenant host/path routing, security inspection, and advanced traffic control (rate limiting, header-based auth).

Trade-offs- L4: Lower CPU, lower latency, simpler scaling; less control and observability.- L7: More features and visibility, but higher CPU cost and potential latency from parsing/termination.

As a Solutions Architect, map customer needs: prioritize L4 for raw performance and strict end-to-end encryption; choose L7 when you need application-aware routing, observability, or operational simplicity via TLS offload.

MediumSystem Design

29 practiced

Design a backup and recovery strategy for a multi-region system that must meet an RTO of 1 hour and RPO of 15 minutes for core transactional data. Describe backup cadence (full vs incremental), continuous replication options, cross-region snapshot mechanics, operational runbooks for restore, and how you'd test and validate recovery time objectives.

Sample Answer

Requirements & constraints:- RTO = 1 hour, RPO = 15 minutes for core transactional data, multi-region active-passive (or active-active) deployment, cost and compliance constraints.

High-level approach:- Combine near‑real‑time replication for RPO with periodic cross‑region snapshots and immutable backups for retention/forensics. Use automated runbooks and frequent DR drills to guarantee RTO.

Backup cadence & mechanisms:- Continuous replication (primary mechanism) — use asynchronous logical replication / WAL shipping / Change Data Capture (Debezium/Kafka CDC) from primary region to standby region(s). Configure replication lag alerts; aim for <1 minute typical lag so RPO 15m is easily met.- Incremental (near‑continuous) backups — point-in-time logs retained in object storage (WAL segments) every minute.- Full snapshots — daily full snapshots stored cross‑region (immutable, versioned) in S3/Blob with lifecycle rules; weekly archival to lower‑cost cold storage.- Incremental snapshots — every 4 hours for large datasets to speed restores and reduce storage.

Cross-region snapshot mechanics:- Take application-consistent snapshots by quiescing or using DB-specific snapshot APIs (e.g., RDS snapshot + export, ZFS/BRPO, LVM freeze) or coordinate via distributed transaction markers. Upload to region‑agnostic object store with encryption, checksum, and manifest containing metadata (LSN/timestamp, schema version).- Use object-store cross‑region replication (CRR) for near‑instant availability of snapshots and WAL segments.

Restore & operational runbooks (automated + manual):- Runbook A — Failover to warm standby (automated): 1. Detect outage via health checks + SLO alarms. 2. Promote standby DB (apply pending WAL/CDC up to latest), update DNS/traffic (TTL low), bring app autoscaling groups in target region. 3. Validate schema and run smoke transactions. 4. Post‑failover: reconfigure replication from promoted node to other regions. Estimated steps and automation should restore within RTO 1 hour.- Runbook B — Full restore (from snapshot): 1. Provision compute and storage in target region. 2. Retrieve latest snapshot + incremental WALs, perform PITR to target timestamp. 3. Run integrity checks and replay queues. 4. Cutover and validate.- Maintain runbooks as code (Terraform/Ansible) and one‑click playbooks via orchestration (Step Functions, Runbooks in PagerDuty).

Validation & testing:- Execute quarterly full failover drills and monthly tabletop walkthroughs.- Measure MTR (mean time to recover) and actual data loss during drills; ensure measured RTO ≤1h and data lag ≤15m.- Use synthetic workloads and data-consistency checks (checksums, reconciliation jobs) post-failover.- Chaos testing: simulate region failures, network partitions, and storage corruption in staging.- Continuous monitoring: replication lag, snapshot health, restore time metrics, audit logs. Automate alerts when SLAs at risk.

Trade-offs & notes:- Active-active lowers failover time but increases complexity for conflict resolution.- Asynchronous replication meets latency/RPO with careful monitoring; if stricter RPO needed, consider synchronous or quorum writes with higher latency.- Cost: frequent WAL retention and cross-region storage increase cost—apply tiered retention and lifecycle rules.

This design maps automation + continuous replication + cross-region immutable snapshots to meet RPO 15 minutes and RTO 1 hour while ensuring testability and operational readiness.

MediumTechnical

23 practiced

Design a public REST API for a product catalog service used by third-party sellers and mobile apps. Define endpoints, query parameters (pagination, filtering, sorting), authentication, rate limiting strategy, versioning plan, and error response format. Provide 2-3 sample HTTP endpoints and a short JSON response example for a product search.

Sample Answer

Requirements & goals: public REST API for product catalog that supports third‑party sellers and mobile apps: search, read product details, list seller products. Needs pagination, filtering, sorting, secure auth, rate limiting, clear versioning, and consistent error format.

API surface (base): https://api.example.com/v1

Endpoints (examples)- GET /v1/products - Query params: q (full-text), category, brand, price_min, price_max, availability (in_stock/any), seller_id - Pagination: page (default 1), per_page (max 100) - Sorting: sort (relevance|price_asc|price_desc|newest)- GET /v1/products/{product_id}- GET /v1/sellers/{seller_id}/products - Same filters/pagination as /products but scoped to seller

Authentication- OAuth2 Bearer tokens (client_credentials for server-to-server; authorization_code for third‑party UIs)- Optional API keys for lightweight clients (read-only) with limited scope- All endpoints require TLS

Rate limiting strategy- Per-client token buckets: default 600 requests/min for authenticated clients; 60 requests/min for API-key clients- Header-based feedback: - X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset (unix epoch)- 429 returned when exceeded, with Retry-After header (seconds)

Versioning plan- URL versioning (/v1). Backwards-compatible additive changes allowed; breaking changes via new major version (/v2).- Deprecation: publish deprecation header X-API-Deprecated: true and a Deprecation-Date in response; 90-day minimum for major breaks.

Error response format (JSON)- Consistent problem object:

json

{
  "error": {
    "code": "INVALID_PARAMETER",
    "message": "per_page must be <= 100",
    "details": [
      { "field": "per_page", "reason": "exceeds_max", "info": "max=100" }
    ],
    "request_id": "c6f0b2a1"
  }
}

Sample product search response (200 OK)

json

{
  "meta": {
    "page": 1,
    "per_page": 20,
    "total": 342,
    "returned": 20
  },
  "products": [
    {
      "id": "sku-12345",
      "title": "Wireless Headphones X100",
      "brand": "Acme",
      "price": 79.99,
      "currency": "USD",
      "in_stock": true,
      "thumbnail": "https://cdn.example.com/sku-12345/thumb.jpg",
      "rating": 4.5,
      "seller_id": "seller-77"
    }
  ],
  "links": {
    "self": "/v1/products?q=headphones&page=1&per_page=20",
    "next": "/v1/products?q=headphones&page=2&per_page=20"
  }
}

Additional notes / trade-offs- Use search engine (Elasticsearch) for full-text + filters; single source of truth in primary DB with asynchronous indexing for eventual consistency.- Prefer stateless servers, CDN for assets, and edge caching for public product reads; use short TTLs for dynamic fields (price/availability).- Monitoring: expose metrics for latency, 5xx rate, auth failures; log request_id for tracing.

MediumSystem Design

28 practiced

Design a distributed rate-limiting system that enforces per-user and per-API-key limits across a fleet of stateless API servers deployed in multiple regions. Discuss algorithms (token-bucket, leaky-bucket), handling of bursts, accuracy vs performance trade-offs, coordination across regions (global vs local limits), and how to expose limit state to clients.

Sample Answer

Requirements & constraints:- Enforce per-user and per-API-key limits across stateless API servers in multiple regions.- Allow bursts, low-latency checks, high availability, and reasonable global accuracy.- Prefer operational simplicity for sales/clients.

High-level approach:- Use token-bucket for rate-limiting (supports bursts naturally) implemented in two complementary tiers: 1. Local fast path: each API server maintains an in-memory token-bucket per key/user with a conservative local quota shard. Checks are O(1), zero network hop, ideal for low latency and most requests. 2. Global reconciliation: a centralized quota service (multi-region, strongly-available via leaderless replication like Redis Cluster or a cloud-managed quota-store) maintains authoritative global counters and periodically rebalances local shards.

Algorithms and burst handling:- Token-bucket: tokens regenerated at rate R; bucket capacity B defines allowed burst. Local shards get capacity Blocal and refill rate Rlocal = R * fraction assigned.- Leaky-bucket could be used where smoothing is desired, but token-bucket better supports controlled bursts.- To support bursts above local shard, implement “borrow” mechanism: when local tokens exhausted, server async/fast-synchronous calls global quota service to request additional tokens. Use short timeouts/fail-open/closed policy depending on SLAs.

Accuracy vs performance trade-offs:- Strong global consistency (exact enforcement) requires central checks on every request -> high latency and single-point scaling pressure.- Sharded local quota favors performance and availability but yields eventual consistency: short-term overage across regions possible. Mitigate by conservative shard sizing, global reconciliation, and audit/retroactive throttling or billing adjustments.- Offer two modes to customers: strict mode (global sync, higher latency) and best-effort mode (local shards, low latency).

Coordination across regions:- Partition quotas deterministically (consistent hashing) across regions to minimize cross-region coordination.- Use periodic lease-based shard reallocation to adapt to traffic shifts (e.g., every 1–10s).- Use CRDT-like counters or commutative reconciliation (additive increments) to tolerate network partitions for approximate counts.- For critical global limits (e.g., per-tenant spending cap), route checks to central service via regional read replicas with synchronous commit (higher latency but accurate).

Exposing limit state to clients:- Return standard HTTP headers (e.g., RateLimit-Limit, RateLimit-Remaining, RateLimit-Reset, X-RateLimit-Type) computed from local state for low latency. Add header flag indicating “approximate” vs “guaranteed”.- For strict mode, headers reflect authoritative global state.- Provide a monitoring API/webhook to query real-time and historical quota usage, and a dashboard for sales/clients showing limits, bursts, and consumption.- Document semantics clearly: eventual vs strict consistency, failure behaviors (fail-open vs fail-closed).

Operational considerations:- Auto-scale global store and use caching (local bloom/fingerprint) to reduce load.- Instrument metrics (rejects, borrow rate, reconciliation mismatches) and alert on shard exhaustion or high borrow rates.- Test with chaos scenarios and simulate bursts across regions.

Recommendation:- Default to token-bucket with conservative local shards + global reconciliation (best latency/availability balance). Offer a strict global-check option for customers needing absolute enforcement.

MediumSystem Design

24 practiced

A customer asks for an observability strategy for a Kubernetes-based microservices platform to improve incident detection and root cause analysis. As a Solutions Architect, propose a monitoring and observability architecture detailing which metrics, structured logs, and distributed traces to collect; instrumentation approaches; alerting strategy; SLOs; and how dashboards and runbooks would be organized.

Sample Answer

Requirements & constraints:- Real-time incident detection, fast RCA across services on Kubernetes, multi-tenant namespaces, <1s metric ingest for alerts, retention: metrics 90d, traces 30d, logs 365d. Support SLOs and compliance.

High-level architecture:- Metrics: Prometheus (k8s + node exporters) → long-term store (Thanos/Cortex) for global view.- Logs: Structured JSON logs shipped via Fluentd/Vector → central ELK/Opensearch or managed Cloud logging.- Traces: OpenTelemetry SDKs/OTLP collectors → Jaeger/Posgra-Backed or managed tracing backend (Tempo/Lightstep).- Correlation: Inject trace_id and span_id into logs and metrics (via Otel resource attributes). Use service mesh (Istio/Linkerd) or sidecar for automatic tracing/metrics.

What to collect:- Metrics (high-cardinality but sampled): - Infra: node_cpu_usage, node_memory_used_bytes, node_disk_io - K8s: kube_pod_status_phase, kube_deployment_replicas, kube_node_conditions - App: request_rate (rps), latency_histogram (p50/p95/p99), error_rate (4xx/5xx), queue_depth, goroutine_count/heap_usage - Platform: kube_api_server_latency, etcd_op_latency, control-plane errors- Structured logs: - JSON with fields: timestamp, service, namespace, pod, container, level, message, trace_id, span_id, request_id, user_id (if allowed), feature_flag, error_code, stack - Include rich context for business keys to speed RCA- Distributed traces: - Capture end-to-end spans for inbound requests, DB calls, RPCs, message queue operations, external HTTP calls. Instrument critical paths and high-latency components. Sample at adaptive rates: 100% for errors, 1-5% for successful traces; increase sampling for SLO breach windows.

Instrumentation approaches:- Use OpenTelemetry in apps (auto-instrumentation where available). Export metrics via Prometheus exporters + Otel -> Prometheus remote write to Thanos.- Service mesh for automatic telemetry of network calls, with enrichment (mTLS metadata).- Libraries enforce structured logging and automatic trace_id injection (middleware/filters).- CI gating: linting to ensure new services emit required metrics and follow log schema.

Alerting strategy:- Define alert tiers: - P0 (on-call immediate): total service error_rate > X% for 5m, request latency p99 > SLO threshold for 5m, control-plane down, etc. - P1 (high): increase in error_rate by factor >3 vs baseline, sustained SLO burn rate > critical. - P2 (notice): resource saturation nearing capacity (node disk >85%), pod restart spike.- Use Prometheus Alertmanager (or cloud equivalent) with routing to on-call, Slack, pager, runbook link. Implement alert deduping, inhibition, and escalation policies.- Alerting on symptom, not cause. Pair symptom alerts with contextual diagnostic alerts (e.g., DB connection pool exhausted).

SLOs & SLIs:- Service SLI examples: availability (successful requests / total requests), latency (p95 < 300ms).- SLO targets: Availability 99.95% monthly for public APIs, lower-tier 99.9% for internal services. Error budget policy: map releases to remaining error budget—block risky deploys if budget exhausted.- Define SLO windows (rolling 28d and 90d) and automated burn-rate alerts (e.g., when 50% of budget burned in 1/4 window).

Dashboards & runbooks:- Dashboard organization: - Global cluster dashboard: cluster health, control-plane, infra metrics. - Service-level dashboards: traffic, latency (histograms), errors, traces sample, logs (top errors), resource usage, SLO status and error budget. - Dependency dashboards: DB/queue external latency and error rates. - On-call landing page: current active alerts, affected services, top traces, recent deploys.- Runbooks: - For each high-severity alert create a concise runbook: Symptoms, quick checks (commands, dashboards), likely root causes, mitigation steps, escalation path, postmortem checklist. Link runbooks in alerts. - Maintain playbooks for common RCA flows (network partition, DB outage, control-plane failure, memory leaks).

Operational practices & trade-offs:- Sampling reduces cost; prioritize full traces on errors. Retention trade-offs: move cold data to cheaper object store.- Secure telemetry: redact PII, use mTLS for collectors, RBAC for dashboards/log access, encryption at rest.- Test restore & incident drills; runbook telemetry exercises; periodic SLO reviews with product owners.

Outcome:- Faster detection via combined symptom alerts and adaptive tracing, sub-minute RCA through correlated traces+logs+metrics, controlled releases via error-budget gating, and clear on-call/runbook workflows to reduce MTTD/MTTR.

Unlock Full Question Bank

Get access to hundreds of System Architecture Principles interview questions and detailed answers.

Join thousands of developers preparing for their dream job.