Network Architecture and Communication Patterns Questions
Design and analysis of network architectures and service communication patterns for reliable, performant, and secure distributed systems. Topics include network topology and capacity planning, load balancing strategies, content delivery networks, caching and edge delivery, application programming interface gateway design, service to service communication patterns including synchronous and asynchronous messaging, message queues, publish subscribe, request routing, retries and backoff, timeouts, idempotency, circuit breakers, bulkheads, and service mesh considerations. Also covers latency optimization, failure modes and resilience, observability and monitoring, network security principles such as encryption and segmentation, and how architectural choices affect scalability and operational complexity.
HardTechnical
37 practiced
A service mesh sidecar adds an average 5ms latency per hop and your critical service spans three hops. Evaluate options to reduce latency overhead while preserving key features like mTLS and observability. Consider bypassing the mesh for hot paths, lightweight proxies, or native in-app libraries and discuss operational consequences.
Sample Answer
Clarify objective & constraints:- Goal: reduce added latency (current +15ms for 3 hops) while preserving mTLS and observability for critical “hot path” transactions.- Constraints: minimal security regression, maintain SLOs, feasible operational overhead.Options and evaluation:1) Bypass mesh for hot paths (split traffic)- Approach: route hot-path traffic directly (sidecar-bypass) using egress rules / traffic-annotations or separate service port.- Pros: eliminates sidecar hop latency entirely.- Cons: loses uniform mTLS and automatic tracing/metrics unless reimplemented; increases attack surface; requires strict ACLs, host-level auth and auditing; complicates policy rollout and testing.- Operational impact: need careful rollout (canary), audit logging, additional network policy enforcement, and emergency rollback plan.2) Lightweight proxy sidecars- Approach: replace full-featured sidecar with minimal proxy providing only mTLS + pass-through headers + basic metrics.- Pros: preserves mTLS/observability while reducing per-hop latency (depends on implementation; e.g., smaller Envoy build or specialized proxy).- Cons: engineering cost to maintain fork; feature gaps (policies, complex routing); compatibility testing across services.- Operational impact: added build/test pipelines, lifecycle management, and documentation; easier to reason about than bypass.3) Native in-app libraries (mTLS + telemetry)- Approach: embed mutual TLS and tracing libs into the application, bypass sidecar for hot RPCs.- Pros: lowest latency, full control, can instrument only hot paths.- Cons: significant dev effort, duplication, security maintenance burden (cert rotation, crypto ops), potential for inconsistent telemetry.- Operational impact: requires dev investment, secure key distribution (SPIFFE/SDS integration), CI changes, cross-team coordination.Recommendation:- Short term: implement selective bypass for strictly-defined hot RPCs behind feature flags and network policies, with compensating controls (service-level mTLS where feasible, strict ACLs, enhanced logging). Canary and measure impact.- Medium term: evaluate lightweight proxy prototype to balance features vs latency; measure end-to-end latency and CPU cost.- Long term: where latency is critical and change-control allows, standardize an in-app auth/telemetry SDK integrated with the identity provider (SPIFFE) and automated cert management.Safety nets:- Test plans, canaries, automated rollbacks, extensive synthetic and real-user latency monitoring, SLO adjustments, and threat modeling before enabling bypass or in-app changes.
HardTechnical
32 practiced
From an SRE perspective, evaluate managed message brokers (e.g., hosted Kafka) versus serverless event buses (e.g., cloud pub/sub) for high-throughput service-to-service communication. Discuss operational costs, operational complexity, SLA guarantees, scaling, latency, and tooling for monitoring and recovery.
Sample Answer
Approach: I’ll compare managed Kafka (e.g., Confluent Cloud, MSK) vs serverless event buses (e.g., GCP Pub/Sub, AWS EventBridge/SNS) across the requested SRE dimensions, highlight trade-offs, and give pragmatic recommendations for high-throughput service-to-service communication.Operational cost- Managed Kafka: higher baseline cost (provisioned brokers, storage, network egress). Predictable if capacity planned; additional ops-costs reduced vs self-hosting. Costs grow with retention, replicas, and throughput.- Serverless event bus: lower operational overhead and often pay-per-use (messages, ingest/egress). Cost is usage-driven — economical for variable workloads but can be expensive at sustained extreme throughput if per-request charges aggregate.Operational complexity- Managed Kafka: more operational surface even when hosted—topic partitions, ISR, broker metrics, consumer lag, compaction, retention tuning, schema management. Requires SRE knowledge and runbooks for broker issues.- Serverless: minimal infra ops; focus shifts to application retries, idempotency, and shaping messages. Fewer knobs but less control over internals.SLA guarantees- Managed Kafka: providers often offer broker-level SLAs; however SLAs may exclude data loss during consumer lag or extended broker maintenance windows. Durable storage and explicit replication give strong durability semantics if configured correctly.- Serverless: strong availability guarantees and regional/global replication from cloud vendors; SLA coverage typically includes message durability and delivery; check exactly-once vs at-least-once semantics.Scaling- Managed Kafka: scaling requires partition rebalancing; high throughput achievable with partitioning but operationally complex—repartitioning is heavyweight.- Serverless: autoscaling is seamless; supports bursty traffic without manual partitioning. Less control but faster horizontal scaling.Latency- Managed Kafka: very low end-to-end latency possible (sub-10ms internal) when colocated; network and disk config affect latency.- Serverless: typically low but slightly higher and more variable due to multi-tenant environments and regional routing. For ultra-low predictable latency, managed Kafka often wins.Tooling for monitoring and recovery- Managed Kafka: rich metrics (ISR, under-replicated partitions, broker CPU/io, consumer lag). SREs must run alerting (lag SLOs), runbooks for leader imbalances, reassignments, partition recovery, and retention-related data-loss scenarios. Supports message replay, compaction, and DLQs if built.- Serverless: cloud-native metrics (publish/ack rates, undelivered messages, retry metrics). Built-in DLQs and easier replay via retained logs (varies by provider). Simpler incident playbooks focused on consumer/backpressure handling.Operational patterns and recommendations- If you need strict ordering, high throughput with low deterministic latency, and fine-grained control (consumer groups, retention, compaction, exactly-once-ish patterns), prefer managed Kafka and invest in automation (partition management, alerting, runbooks), capacity planning, and SLOs for consumer lag and end-to-end latency.- If you prioritize low ops overhead, rapid scaling, multi-region availability, and variable workloads, prefer serverless event buses. Compensate by building idempotent consumers, robust retry/DLQ flows, and observability dashboards for publish/ack/backlog metrics.SRE playbook items (both)- Define SLOs: availability, end-to-end latency p50/p99, consumer lag thresholds; error budgets for message loss.- Monitoring: ingest broker/event metrics, consumer lag, processing success rate, DLQ growth, and end-to-end traces.- Recovery: automated alert-to-runbook links; runbooks for lag remediation, rebalance, throttling; automated replay tooling; chaos tests for failover and scaling.- Cost controls: alert on cost anomalies (ingress/egress spikes), use quotas, and retention policies.Trade-offs summary- Managed Kafka = operational power + control + predictable low latency at higher ops and cost.- Serverless = minimal ops and elastic scaling but less control, slightly higher/variable latency, and different cost profile.Recommendation: For sustained, latency-sensitive, ordered high-throughput internal pipelines where you need replay/compaction, choose managed Kafka with heavy SRE automation. For variable workloads, cross-region pub/sub, or when you want to shift ops burden to cloud provider, choose serverless and invest in robust consumer-side resilience and observability.
HardTechnical
36 practiced
Frequent deployments cause DNS caching to route users to deprecated IPs during rollbacks, creating availability issues. Design deployment and routing strategies (weighted DNS, short TTLs, sticky connections, load balancer traffic-shifting, and health checks) to minimize disruption when rolling forward or back.
Sample Answer
Clarify goal: minimize user impact when rolling forward or back despite DNS caching and sticky connections, while keeping deployments frequent and safe. Proposal combines DNS controls, traffic-shifting at load balancers, health checks, session handling, and automation.1) DNS & TTL- Use low TTLs (30–60s) for service records used in active deployments; reserve longer TTLs for stable infra.- Publish weighted A/AAAA records only as a last-mile signal; don’t rely on DNS for precise, rapid cutovers because caches/ISPs may ignore TTLs.2) Primary traffic-shifting: load balancer (LB) / global load balancing- Use LB-level weighted routing (GCLB/ALB/NGINX/Envoy) for precise control. Shift traffic in small increments (canary: 1→5→25→50→100%) with automated pauses and metrics gating.- Configure connection draining (graceful termination) with timeout slightly longer than typical request durations (e.g., 30–120s) to avoid cutting in-flight requests.- For rollbacks, reverse weights on the LB rather than relying on DNS update.3) Health checks & automated rollback- Fast, frequent active health checks (10–15s) + passive failure sensing (consecutive 5xx or latency spikes).- Auto-rollback policy: if error-rate or latency breach thresholds during a weight step, automatically revert weights to previous stable group and alert.4) Session affinity & sticky connections- Avoid long-lived sticky sessions during deploys. If needed, use sticky cookies with short TTLs (e.g., 60–300s) and store session state centrally (Redis / DB) so users aren’t tied to instance IPs.- For TCP workloads where stickiness is unavoidable, use LB that supports connection-level draining and limit client keepalive durations.5) DNS rollback mitigation- When rolling back to an older cluster/IP set, avoid immediate DNS pointing at deprecated IPs. Instead: - Reintroduce old version behind LB and shift traffic at LB level. - If DNS must change, pre-warm old IPs in upstream caches (announce via BGP/Anycast or keep A records alive during rollout window) and use short TTL earlier so caches expire.6) Observability and runbook- Gate each weight step by SLO-oriented metrics: error rate, p95 latency, traffic volume, saturation.- Predefine runbook: thresholds, rollback step sizes, communication channels.- Simulate rollbacks in staging; test client behavior with cached DNS and long-lived connections.Trade-offs:- Low TTLs increase DNS query volume and cost but improve responsiveness.- LB-based shifting is more reliable for fast rollbacks than DNS, but requires proper global LB capacity and health-check coverage.- Short sticky TTLs require centralized session stores.Concrete parameters to start with:- DNS TTL: 30–60s- LB shift increments: 1% → 5% → 25% → 50% → 100% (pause 1–5 min between)- Health-check interval: 10s, unhealthy after 3 failures- Draining timeout: 60–120s- Auto-rollback threshold: error-rate > 1% absolute increase OR p95 latency +30% vs baselineThis approach uses LB-weighted traffic-shifting as the control plane for precise rollouts and rollbacks, keeps DNS as a coarse, resilient fallback, and minimizes user impact via short affinities, graceful draining, and automated health-gated rollbacks.
MediumTechnical
44 practiced
Discuss pros and cons of client-side retries versus server-side retries (or brokered retries). Consider operational visibility, control over retry policies, amplification risk, idempotency needs, and when the SRE should prefer one over the other.
Sample Answer
Definition: Client-side retries = the caller re-attempts requests after failures. Server-side (brokered) retries = an intermediary (API gateway, message broker, or service mesh) retries on behalf of client or queues the request for later processing.Pros of client-side retries:- Low latency control: client decides backoff, jitter, and max attempts tuned to use case.- Simple operational model: logic lives with caller; no extra infra required.- Avoids central bottleneck; scales with clients.- Easier to implement per-client SLAs and adaptive throttling.Cons of client-side retries:- Visibility is fragmented: retries are distributed across callers, making observability and centralized metrics harder.- Amplification risk: many clients retry simultaneously (thundering herd) unless well-coordinated.- Harder to enforce consistent policies across heterogeneous clients.- Idempotency burden on service and callers — server must handle duplicate requests.Pros of server/brokered retries:- Centralized control and observability: uniform retry policy, aggregated metrics, and dashboards.- Can implement safety measures (rate limiting, circuit breaking) to prevent overload.- Better for asynchronous workloads: durable queues and delayed retries avoid client pressure.- Simplifies clients: clients can be thin and not responsible for complex retry logic.Cons of server/brokered retries:- Can hide client behavior and increase end-to-end latency; harder to attribute failures.- Introduces additional infrastructure and operational complexity.- Amplification still possible if broker retries aggressively without backpressure.- Still requires idempotency at the server if retries result in duplicate processing.When SRE should prefer one over the other:- Prefer client-side when low latency, simple topology, few client types, and clients can be updated/controlled. Use when you need per-call quick failover and when you can enforce client backoff and jitter.- Prefer server/brokered when many heterogeneous clients exist, you need centralized observability/control, or the workload is asynchronous (background jobs) and durability is important. Also prefer brokered retries to protect downstream from bursty retry storms and to decouple retry timing from client lifecycles.Operational recommendations:- Always design for idempotency (or safe deduplication) when retries are possible.- Use exponential backoff + jitter to reduce amplification; enforce at least one side (client or broker).- Centralize metrics: instrument retry counts, latencies, error types, and circuit-breaker state.- Combine approaches: lightweight client retries for transient network glitches (1–2 attempts), and brokered retries for persistent/backoffable failures.- SREs should codify retry policies, provide libraries, and enforce them via gateways/proxies to balance control and performance.
HardSystem Design
36 practiced
Architect a global network topology to support 100k RPS with 99.99% availability across three regions. Include load distribution, capacity planning for network devices, redundancy, cross-region failover, health checks, and how you'd test and validate capacity and failover behavior.
Sample Answer
Requirements & constraints:- 100k RPS global sustained, 99.99% availability (~4.38 min downtime/year), three regions (active-active), tolerate single-region failure, regional latency targets <150ms for users in-region.High-level approach:- Active-active across three regions with traffic steered by global DNS+GSLB (DNS + Anycast + regional load balancers). Traffic splits based on geo-proximity and real-time region capacity. Each region sized to handle ~50% of normal load (N), with cross-region overflow capacity to survive one region failing.Load distribution:- Global layer: Anycasted edge (Cloud CDN/edge or BGP Anycast) + GSLB with health-aware latency-based routing.- Regional layer: Public regional LBs (N LB nodes in AZs) → regional ingress (K8s ingress / API GW autoscaled) → stateless app pools → backing state tiers (sharded DB/cache).Capacity planning (numbers & buffers):- Target steady-state per-region: 40k RPS each (even split 100k/3 ≈ 33k, add 20% buffer → ~40k). If one region fails, remaining two must absorb 100k → 50k each.- LB sizing: choose LBs that support >100k concurrent RPS aggregate; per-region deploy at least 3 LB instances each rated ≥20k RPS (N+1). Network throughput: estimate request size; e.g., avg response 50KB → 100k RPS ≈ 5 GB/s global. Plan NICs, routers, and firewall throughput accordingly.- Compute: container pods sized to serve 200 RPS each → need 200 pods per region for 40k RPS; autoscale with headroom to 250 pods to absorb surge.- State tiers: caches sized for miss-rate to keep DB QPS under limits. DB replicas per region sized for peak QPS; cross-region read replicas for failover.Redundancy & fault domains:- Multi-AZ per region (3 AZs). LB, ingress, pods, cache and DB replicas across AZs.- Devices: routers/firewalls in HA pairs, use ECMP and BGP sessions, N+1 redundancy.- Graceful degradation: circuit-breakers, rate-limits, feature flags to reduce load on backend systems.Cross-region failover:- GSLB monitors regional health (latency + app-level). On region degraded, advertise withdrawal via Anycast/BGP and shift traffic by updating DNS + traffic steering; TTL low (30–60s) for faster convergence.- Warm standby capacity: regions keep autoscaling margins and pre-warmed caches; use traffic-scaling policies to ramp up compute quickly.- Data consistency: use async replication with conflict resolution; for critical writes, route to primary region or use quorum-based multi-region writes where required.Health checks and observability:- Multi-layer health checks: network-level (ping/BGP), LB-level (TCP/HTTP), app-level (business-health endpoints checking DB/cache), synthetic user journeys from multiple locations.- Metrics & alerts: RPS, latency P50/P95/P99, error rates, capacity utilization, instance start times. SLIs mapped to SLOs (99.99% availability).- Distributed tracing and logs for root-cause.Testing & validation:- Capacity tests: ramp tests using distributed load generators from multiple regions to simulate 100k+ RPS with realistic traffic mix; run soak tests for hours/days.- Failover tests: scheduled game-day exercises: - Region kill: withdraw BGP or shut down a region’s LBs to validate global failover and capacity absorption. - AZ failure: simulate AZ loss in region and verify intra-region redundancy. - Partial degradations: inject latency, DB failures, cache outages to validate graceful degradation.- Chaos engineering: automated, controlled chaos runs with rollback playbooks.- Observability validation: ensure alarms trigger and runbooks produce required run times.- Post-test: capture metrics, validate SLOs, tune autoscale thresholds, and adjust capacity buffers.Trade-offs:- Active-active increases complexity (data consistency) but reduces failover time. Warm standby reduces cost vs full duplication but requires careful warm-up. DNS TTL impacts failover speed; combine DNS with Anycast for faster convergence.This design gives capacity headroom, multi-layer redundancy, health-aware routing, and a test-driven validation plan to meet 100k RPS with 99.99% availability.
Unlock Full Question Bank
Get access to hundreds of Network Architecture and Communication Patterns interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.