Practical scaling techniques and patterns for application and data layers. Topics include horizontal and vertical scaling strategies and the trade offs of each; caching topologies and strategies such as cache aside write through and write behind and approaches to cache invalidation and consistency; database scaling techniques including read replicas partitioning and sharding and rebalancing strategies; load balancing algorithms including round robin least connections consistent hashing and strategies for sticky sessions and service discovery; message queue and event streaming patterns for decoupling backpressure and asynchronous processing; content distribution using content delivery networks; connection pooling and resource management; rate limiting throttling retry strategies and approaches to avoid thundering herd problems; and how to combine patterns effectively given workload characteristics and operational constraints. Interviewers expect candidates to explain interactions between patterns and the operational pitfalls of each technique.
HardTechnical
28 practiced
Define a safe migration plan for applying schema changes across a sharded relational database with hundreds of shards and live traffic. Describe expand-contract patterns, rolling migrations, dual-write strategies, feature flags, online schema change tools, backfill and validation strategies, and rollback plans. Include automation and monitoring considerations.
Sample Answer
**Situation & goals**Define a zero-downtime, reversible plan to apply schema changes across hundreds of sharded RDBMS instances while preserving correctness and performance for live traffic.**High-level approach**1. Expand-Contract pattern - Expand: Add new nullable columns / new tables / new indexes and deploy code that reads from old and writes to both (dual-write). - Migrate/backfill: Backfill existing rows asynchronously per-shard. - Contract: Switch reads to the new schema and remove old columns after validation.2. Rolling migrations - Orchestrate per-shard rolling updates in waves (canaries → regional → global) using automation (Terraform/Ansible + orchestration service). - Limit concurrency to control load. Use shard affinity and traffic-splitting.3. Dual-write & feature flags - Dual-write at app layer with idempotent logic; use feature flags to enable reads from new schema gradually (canary users → percentage rollout). - Ensure writes are retried and reconciled.4. Online schema-change tools - Use tools like gh-ost/pt-osc/online DDL for MySQL or native online ALTER for cloud RDBMS to avoid table locks. - Monitor DDL progress, IO and replication lag.5. Backfill & validation - Backfill jobs per-shard with rate limits and checkpointing. - Use consistency-checks: row counts, checksums (per-shard), sampling queries, and end-to-end tests. - Automate strong validation: compare tombstone and null handling.6. Rollback plans - Roll forward preferred: if validation fails, stop rollout, fix backfill, then resume. - If immediate rollback needed: revert feature flags and switch reads/writes to old schema; avoid expensive schema DROP—leave compatible columns for safe reversion. - Maintain backups and PITR snapshots per-shard.7. Automation & monitoring - Pipeline: CI/CD tasks to run DDL, enqueue backfill, flip flags, and run validation. - Observability: dashboards for per-shard DDL status, replication lag, error rates, backfill throughput, checksum diffs. - Alerts for thresholds and automatic throttling or pause of rollout. - Audit logs and runbooks for operators.**Trade-offs**- Dual-write increases complexity; strong validation and idempotency mitigate risks.- Keeping backward-compatible columns increases storage but enables safe rollback.This plan emphasizes safety, observability, and automation suitable for large-scale cloud architectures.
HardTechnical
30 practiced
As the lead cloud architect, draft a high-level migration strategy to move a monolithic application to microservices to improve scalability and team autonomy. Include criteria for service boundaries, the strangler pattern, chosen scaling patterns (CQRS, async messaging, caching), CI/CD and observability requirements, SLOs, rollback plans, and how you will coach and mentor teams through the change.
Sample Answer
**Approach (high-level goal)** I would phase the migration to maximize safety, deliver incremental business value, and decentralize ownership so teams gain autonomy while the platform remains reliable.**Service-boundary criteria** - Align to business capabilities (BCD) and bounded contexts from domain-driven design - Low coupling, high cohesion (measure by call frequency, data ownership) - Independent deployability and failure isolation**Strangler pattern** - Implement an API gateway + feature flags. - Iteratively replace monolith endpoints: route selected traffic to new services, expand when metrics validate behavior. - Keep a fall-back path to monolith during rollout.**Scaling patterns** - CQRS for complex read/write separation where read scale differs from write; event-sourced change log for audits. - Async messaging (Kafka/RabbitMQ) for resiliency and decoupling; use idempotent consumers. - Distributed caching (Redis/Elasticache) for hot reads; cache invalidation via events.**CI/CD & deployment** - Per-service pipelines: lint, unit/integration tests, contract tests, security scans, canary deployments, automated rollback on health/regression. - Immutable artifacts and GitOps for infra.**Observability & SLOs** - Traces (OpenTelemetry), metrics (Prometheus), logs (ELK/Cloud-native). - Define SLOs: latency p50/p95, error rate (<1%), availability (e.g., 99.95%). Alert on SLI/SLO burn rates and error budget exhaustion.**Rollback & safety** - Blue/green or canary with automated health checks, circuit breakers, and feature flags. - Automatic rollback on degraded SLOs; replayable messages for async recoveries.**Coaching & change management** - Create migration playbooks, runbooks, and templates. - Pair-programming, brown-bag sessions, and mentoring rotations embedding platform engineers in teams. - Establish platform APIs, guardrails, and a migration squad to unblock teams. - Track progress with measurable milestones and celebrate early wins.This plan balances risk, scalability, and team autonomy while enforcing platform-level reliability and observability.
EasyTechnical
33 practiced
Describe common load balancing algorithms (round-robin, least-connections, consistent hashing, IP-hash) and scenarios where each is appropriate. Explain the differences between L4 and L7 load balancing and operational implications for long-lived connections (WebSockets), sticky sessions, and connection draining during deployments.
Sample Answer
**Overview — common algorithms**- **Round‑Robin**: cycles requests evenly across healthy backends. Best for homogeneous stateless pools (web servers, microservices) where capacity is similar. Simple, low overhead.- **Least‑Connections**: forwards to the server with fewest active connections. Good for uneven request durations or heterogeneous capacity (e.g., some instances handle heavier work).- **Consistent Hashing**: maps keys (user id, session id) to backend via a hash ring; minimizes remapping when nodes change. Ideal for cache clusters (Redis/memcached) or sticky routing without centralized state.- **IP‑Hash**: uses client IP to pick backend. Useful when sticky affinity is needed and you cannot set cookies (legacy clients), but poor for clients behind NAT or mobile clients with changing IPs.**L4 vs L7**- **L4 (Transport)**: routes by IP/port; very fast, transparent TCP/UDP proxying. Can't inspect HTTP semantics (cookies/paths). Lower CPU, good for TLS passthrough (preserve client certificate).- **L7 (Application)**: understands HTTP(S); can route by URL, headers, cookies, and implement rewrites, auth, and fine-grained health checks. Higher CPU cost and terminates TLS.**Operational implications**- **Long‑lived connections (WebSockets)**: prefer L4 or L7 with connection persistence and health-aware draining. L7 must support proxying/upgrading to WebSocket and keep idle timeouts long enough; otherwise connections drop.- **Sticky sessions**: implement via L7 cookies or consistent hashing. L7 cookies are robust (application-controlled); IP‑hash is fragile. For scale, prefer stateless designs or sticky plus session replication.- **Connection draining during deploys**: mark instance unhealthy, stop sending new connections, allow existing connections to finish within a timeout, and forcibly close after grace period. For WebSockets/long‑lived flows use longer drain windows or route through a message broker or sticky handoff to avoid client disruption.Choice depends on latency, inspectability, session needs, and failure domain. As an architect I favor stateless services and L7 routing with consistent hashing for caches and careful draining policies for long‑lived connections.
HardSystem Design
29 practiced
Design a global rate-limiting service that enforces per-user quotas across regions while preserving fairness and low latency. Discuss exact counters versus approximate data structures (CRDTs, sketches), sliding-window vs token-bucket semantics, partitioning to avoid hot shards, and behavior under network partitions. Describe how you'd measure fairness and resilience to abuse.
Sample Answer
**Clarify goals & constraints**- Global, multi-region, low-latency enforcement of per-user quotas (requests/sec, bursts/day), fairness (no region/user advantage), resilience to partitions and abuse.**High-level architecture**- Regional edge proxies enforce local fast-path limits; a global control plane maintains authoritative quotas and syncs state.- Per-user token-buckets stored in region-local caches (Redis cluster or in-process) for sub-10ms checks; background sync with global stores.**Exact counters vs approximate structures**- Exact counters (strongly-consistent DB): precise enforcement but high latency and global coordination cost — use for high-value users or long-window billing.- CRDTs/sketches (PN-Counters, Count-Min Sketch, HyParView CRDTs): low-latency, eventual-consistency; accept bounded error for high-cardinality, low-value users. Use sketches for long-window aggregated quotas and analytics.- Hybrid: token-bucket exact locally, aggregated approximate global view with CRDTs for reconciliation and quota reporting.**Semantics: sliding-window vs token-bucket**- Token-bucket (leaky-bucket): supports burstiness; cheap local checks (consume tokens). Good for per-second fairness and low-latency.- Sliding-window counters: more accurate spike detection and strict rate limiting; requires more coordination to maintain exact windows.- Choice: token-bucket at edge + short sliding-window sampled centrally for abuse detection and reconciliation.**Partitioning & hot-shard avoidance**- Partition by user-id using consistent hashing with virtual nodes or rendezvous hashing to spread load.- Client affinity: route a user's requests to the same regional cache when possible.- Hot-key mitigation: detect heavy hitters, route to dedicated shard pool, apply adaptive throttling or escalate to global coordinator.**Behavior under network partitions**- Define policy per SLA: - Fail-open (availability prioritized): region uses local tokens and local quotas; periodically reconcile; enforce conservative local caps to limit drift. - Fail-closed (consistency prioritized): deny requests requiring global check — higher latency/availability loss.- Use CRDTs for monotonic counters and causal metadata to reconcile post-partition; apply reconciliation rules (min, max, or capped) to prevent quota overspend.**Measuring fairness & resilience**- Fairness metrics: Jain’s fairness index across users/regions for granted rates; variance of per-user granted RPS normalized by quota.- Accuracy: false-allow / false-deny rates vs ground truth.- Latency: p95/p99 request-check latency.- Resilience/abuse: detection rate of anomaly detectors (precision/recall), time-to-mitigate for DoS events.- Instrumentation: per-user telemetry, global sampler, alerts for hot-key, reconciliation drift dashboards.**Abuse mitigation**- Behavioral detectors (rate-of-change, burst patterns), adaptive per-user downgrade, CAPTCHA/challenge, progressive backoff, black/grey lists; escalate to exact-accounting for suspicious users.**Trade-offs summary**- Fast local token-bucket + eventual CRDT reconciliation = low latency, reasonable fairness, bounded error.- Exact global counters = perfect fairness but high latency and cost.- Pick hybrid per user class: precise for premium, approximate for bulk.This design balances low latency, fairness, and operational resilience while providing clear knobs (fail-open/closed, exact vs approximate) for SLAs.
HardSystem Design
33 practiced
Design an end-to-end, highly scalable architecture for a global social feed (timeline) service that supports 50M daily active users, high fan-out writes (some users have millions of followers), and low read latency. Explain data models, sharding, caching, fan-out strategies (push, pull, hybrid), real-time updates, ordering guarantees, backlog handling, and operational concerns like rebalancing and monitoring.
Sample Answer
**Clarify goals & constraints**- 50M DAU, some creators have millions of followers, sub-100ms read latency for cold/warm reads, durable history, global distribution, cost-conscious.**High-level architecture**- Ingest API (edge LB & API GW) → Write service → Fan-out pipeline (streaming) → Per-user feed store + cache → Read service → Clients.- Components: Kafka (or Cloud Pub/Sub) for event bus, Cassandra/Scylla for timeline storage (wide-row), Redis/Memcached for per-user cache, CDN for static assets, WebSocket/Push service for real-time push.**Data model**- Feed item: { item_id, author_id, created_ts, content_ref, priority, seq_no }- Per-author timeline: append-only event stream.- Per-user timeline (fanout): sorted set keyed by user_id with entries (created_ts, item_id, author_id, priority); stored in Cassandra wide-rows and top-N in Redis sorted sets.**Sharding**- Partition both author streams and user timelines by consistent-hashed user_id ranges; use token-aware routing to send writes to responsible replica set.- Hot authors: detect via counters; route their fan-out through special “hot” partitions to avoid hotspots (shard followers list across buckets).**Fan-out strategies**- Push (precompute): For normal authors, on write, stream handler fans out writes into follower timeline partitions in parallel (batch followers, async, idempotent writes).- Pull (on-read): For high-fanout authors (millions), do not write to all followers. Keep author-post index and on read merge top-K from subscriptions (lazy merge, cache results).- Hybrid: Precompute for users with manageable follower counts; pull for celebrity accounts; use cached merged feed for heavy readers.**Real-time updates & ordering**- Use monotonic sequence numbers per author and created_ts to provide causal ordering per-author. For global ordering, define ranking (time + relevance score) and use stable tie-breaker (seq_no).- Push real-time notifications via WebSocket/Push; send item metadata and cache key invalidation; clients fetch full content lazily.**Backlog & replay**- Kafka retains full stream; a replay worker can reconstruct timelines (rebuild user timelines into Cassandra) using batched idempotent writes, with rate-limiting and priority queues to avoid overload.- For backfill after schema change, run fanout rebuild in partitioned parallel jobs with checkpoints.**Caching & read optimization**- Redis per-user top-N sorted set (TTL ~ minutes) for sub-100ms reads; cold reads pull from Cassandra and assemble merged feed from author-index plus cached hot-author summaries.- Use Bloom filters to avoid unnecessary lookups for deleted or pruned items.**Consistency & guarantees**- Eventual consistency is acceptable for global rank; per-author ordering is strong (seq_no). Writes are idempotent; use dedup keys.- Provide “read-after-own-write” via session-affinity or writing to both author stream and per-user cache synchronously for the author.**Operational concerns**- Rebalancing: consistent hashing + virtual nodes; automatic re-sharding by moving token ranges with traffic steering and phased drain. Hot-split follower lists into buckets transparently.- Monitoring: Prometheus + Grafana for throughput, lag (consumer group lag), tail latency (P95/P99), cache hit ratio, per-partition latencies, spike detectors. Use distributed tracing (Jaeger) for fanout write paths.- Alerts & runbooks: consumer lag, storage saturation, high retry/error rates, skewed partition load.- Capacity & cost: autoscaling for stateless services; fixed nodes for storage with planned scaling; cold storage for old items; TTL and compaction for wide rows.- Safety: rate limit writes per author, backpressure on ingestion, circuit breakers for failing downstream stores, canary deploys for pipeline changes.This design balances low-latency reads for most users via push+cache, prevents hot-spot write storms with pull for mega-followers, supports replay/backfill, and provides operational mechanisms for rebalancing and observability.
Unlock Full Question Bank
Get access to hundreds of Scalability Patterns and Techniques interview questions and detailed answers.
