Scalability Fundamentals Questions

Core concepts and back of the envelope estimation techniques for junior to intermediate engineers. This includes converting business requirements into technical metrics such as requests per second, data volume, and bandwidth; understanding when a single machine is insufficient and when to move to distributed systems; basic vertical versus horizontal scaling trade offs; basic sharding, replication, and caching patterns; monitoring signals to track capacity such as CPU trends and disk usage growth; and considerations for backup and recovery times and maintenance windows. Emphasis is on foundational calculations and practical guidelines for when and how to scale.

MediumTechnical

33 practiced

Given a product with 200k MAU growing 10% month-over-month, show how to project traffic over 12 months and calculate when capacity (in RPS) will double. Explain how seasonality (e.g., +50% peak in November) and a marketing campaign that doubles traffic for 2 weeks would alter your planning.

Sample Answer

Approach — derive month-by-month MAU using exponential growth, convert to traffic/RPS scale, then find when traffic hits 2× baseline. Show how additive multiplicative events (seasonality, campaign) change peak planning and required headroom.

Projection formula:- MAU(t) = 200,000 * (1.10)^t, where t = months from now (t=0..12).- After 12 months: MAU(12) = 200,000*(1.10)^12 ≈ 200,000*3.138 ≈ 627,600 MAU.

When will capacity (traffic) double?- Solve (1.10)^t = 2 → t = log(2)/log(1.10) ≈ 7.27 months.- So baseline traffic (and required RPS capacity) doubles in roughly 7.3 months — plan capacity growth accordingly (round up to 8 months for safety).

If you have a baseline RPS example:- Baseline average RPS = 100 → doubled RPS target = 200. At month 8 expected RPS ≈ 200.

Seasonality and marketing effects:- Seasonality (Nov +50% peak): treat as multiplicative spike in that month. If November corresponds to month k, peak MAU(k) = MAU(k) * 1.5. That increases peak RPS transiently; ensure capacity meets expected peak RPS for that month.- Marketing campaign: doubles traffic for 2 weeks. For monthly planning, approximate average multiplier = (2 weeks * 2x + 2 weeks * 1x)/4 weeks = 1.5× for that month. So both seasonality and campaign can act together (e.g., if campaign occurs in Nov, combined multiplier = 1.5 * 1.5 = 2.25× for that month).

Operational planning recommendations:- Use autoscaling with target CPU/RPS thresholds and fast ramp policies; preload canaries and warm pools before expected peaks.- Run load tests at predicted peak multipliers (e.g., 1.5–2.25×) plus safety buffer (20–30%).- Cost/perf trade-off: use burstable instances, CDN and caching to shave RPS, and throttling/backpressure for noncritical paths.- Monitor real-time metrics and have playbooks for rapid scale-up (cloud capacity requests, queueing levels).

Edge considerations:- Campaigns concentrated within a day require finer-grained capacity planning than monthly averages.- Account for increased write-heavy vs read-heavy patterns (different scaling knobs).- If SLAs are strict, provision reserved capacity for critical services to avoid transient failures.

EasyTechnical

35 practiced

A client expects 1,000,000 monthly active users (MAU). Average session length is 5 minutes, with 10 page requests per session. 60% of traffic occurs in the peak hour. Average response payload is 200 KB. Assume a 30-day month and that each page view generates one request to your service.

Calculate:1) average requests per second (RPS)2) peak RPS during the busiest hour3) required outbound bandwidth (Mbps) during peak4) approximate daily log ingestion (GB) if each request logs 800 bytes (compressed 0.6x)

Show assumptions and steps.

HardSystem Design

60 practiced

Design a multi-region architecture for an application that must be available globally with 99.99% uptime and must keep personal data in the user's home region for compliance. Discuss active-active vs active-passive approaches, data replication patterns, routing, and consistency trade-offs. Include network and operational considerations.

Sample Answer

Requirements & constraints:- Global availability 99.99% (≈52.6 min/year downtime) and low user latency.- Personal data must reside only in the user’s home region for compliance (data residency).- System must tolerate region failure and support operational observability and failover runbooks.

High-level architecture:- Regions = independent deployment units containing: - Regional API gateway / ingress - Regional application fleet (stateless services) - Regional data plane: primary regional database(s) that store resident personal data - Global control plane for routing, config, monitoring (no resident personal data)

Active-active vs active-passive (trade-offs):- Active-active: - Each region actively serves traffic; good for latency, capacity, and failover. - Challenge: must avoid violating residency — personal-data writes must land in home region. Achieve by routing policy (see below). - Data conflicts if user can update from multiple regions → need conflict resolution or strong coordination (synchronous cross-region transactions increase latency).- Active-passive: - One home region is primary for a user; other regions forward requests or act as hot standbys. - Simpler consistency and residency enforcement; easier to guarantee data locality. - Higher failover complexity and potential cold-latency on failover.

Recommended hybrid for this use case:- Active-active for stateless services and non-personal data (global caches, feature flags, public content).- Per-user active-primary model for personal data: each user assigned a “home region” where their personal data is primary and writable. Other regions proxy or route writes to that home region; read-only caches can be placed locally (see replication patterns).

Data replication patterns:- Personal data: region-local primary (single-writer) with asynchronous read-replicas in other regions if allowed by compliance; otherwise, disallow replication outside home region. Use logical sharding by user-region.- Global shared data (catalogs, public content): multi-master with CRDTs or per-record leader election, or strong-consistency global DB (e.g., Spanner) if latency/SLAs allow and compliance permits.- Change Data Capture (CDC) + message bus for eventual consistency propagation; include per-change metadata and TTLs to honor residency.- For metadata that can cross regions, encrypt and pseudonymize; keep keys per region.

Routing and traffic management:- Edge: global Anycast or CDN for static assets. Use geolocation-based DNS (Route53 latency/geoproximity) + health checks to direct users to closest region.- Application layer: regional API gateways validate user residency and route write requests to the user’s home-region endpoint. Use a small, highly-available mapping service (regional caches + global control plane) that resolves user→home-region.- For reads: prefer local reads from cached or replicated slices; enforce policy that sensitive fields are only returned from home-region stores.

Consistency and trade-offs:- Strong consistency: synchronous cross-region consensus (Paxos/Raft or Spanner) ensures correctness but adds regional RTT to critical ops → higher latency and risk to availability (if consensus quorum crosses failed region).- Eventual consistency: async replication reduces latency and improves availability but requires conflict resolution and complexity for read-your-writes guarantees. Use local session caches or request routing to home region for writes to provide read-your-writes.- Choose per-data-class consistency: personal profile = strong (single-region primary), activity streams = eventual.

Network considerations:- Optimize WAN links: private backbone (MPLS/Direct Connect) or cloud inter-region low-latency links. Use TLS, mutual auth, and IP ACLs.- Monitor latency and packet loss; implement congestion-aware replication throttling.- Use retries with idempotency tokens and backoff to handle transient failures.

Operational considerations:- Automated failover runbooks, chaos testing, and region-failure drills. For user failover, reassign home-region (with consent/controls) only after full data sync and verification.- Observability: distributed tracing, per-region SLO dashboards, synthetic checks, and alerting on residency policy violations.- Deployment: CI/CD with region-aware feature toggles and canary rollouts per region.- Compliance controls: regional key management (KMS per region), audit logs stored in-region, data access policies, and periodic audits.- Cost & capacity planning: over-provisioning for regional peaks, cross-region egress cost analysis.

Final summary:- Use hybrid: active-active for stateless/global, single-writer home-region for personal data. Enforce routing so writes land in home region; use async replication only where compliance permits. Balance consistency per data class: keep critical personal data strong-consistent within home region, accept eventual consistency for global material. Invest in resilient networking, rigorous operational playbooks, observability, and compliance controls to meet 99.99% global availability while honoring data-residency requirements.

MediumSystem Design

43 practiced

Design a scalable approach for a write-heavy leaderboard (millions of updates per minute) that must serve top-N queries with low latency. Discuss data model, caching, eventual vs strong consistency, sharding, and how to handle leaderboard recalculation and windowed leaderboards (daily/weekly).

Sample Answer

Requirements clarification:- Write-heavy: millions of score updates/minute- Read pattern: low-latency top-N queries (global + windowed: daily/weekly)- Consistency: mostly reads can tolerate slight staleness; some actions (prize award) require stronger guarantees

High-level approach:- Ingest updates via a high-throughput queue (Kafka/Kinesis). Producers publish user score deltas/events.- Use a stateless fleet of workers consuming the stream to apply updates to distributed in-memory sorted indices and to a durable store.

Data model:- Use partitioned sorted-sets per leaderboard (Redis Cluster / key-sharded). Key naming: leaderboard:{window}:{shardId} (e.g., global:daily:shard42).- Persistent backing store (Cassandra/Scylla) for full user aggregates and audit trail: userscores(user_id, leaderboard_id, score, ts).

Sharding & scaling:- Shard by user_id hash across N Redis shard nodes to spread write load. Each worker routes updates to the appropriate shard.- For top-N queries, each shard maintains its local top-K (K >= requested N). To serve global top-N, query all shard top-Ks (parallel) and merge using a min-heap to produce final top-N. To avoid cross-shard fanout for every read, maintain an aggregated top-N cache (next section).

Caching:- Keep two caches: - Per-shard top-K in Redis (in-memory sorted set) for quick local merges. - A hot global top-N cache updated asynchronously (every 1–5s or on significant change) stored in a read-optimized tier (Redis or CDN edge cache) to serve low-latency reads with minimal compute.- Read path: try global top-N cache -> if miss, perform parallel shard top-K merge.

Consistency:- Use eventual consistency for the main leaderboard: updates applied to shards are propagated via Kafka and applied; reads from cached global top-N may be slightly stale (seconds).- For operations requiring strong consistency (prize awarding), implement a transactional read from persistent store + optimistic locking or a single-writer canonical process that takes a snapshot and locks the user’s eligibility for the medal.

Leaderboard recalculation & correctness:- Periodic full-recompute jobs (batch) run during off-peak or as streaming aggregations: - Recompute per-shard aggregates from durable store and repopulate sorted sets to fix drift. - Use watermarking in stream processing to avoid double-counting.- Use idempotent updates: workers apply deltas or set-to-max logic so retries are safe.- Keep an audit log of events in durable store for forensic recalculation.

Windowed leaderboards (daily/weekly):- Maintain time-partitioned sorted sets: leaderboard:daily:YYYYMMDD:shardId, leaderboard:weekly:YYYY-WW:shardId.- On window rollover: - Materialize a new empty set for the new window. - Optionally snapshot previous window to cold storage and set TTL.- For sliding windows, use tumbling/stream aggregations with tumbling window operators in stream processor; write final aggregates to partitioned keys.

Operational considerations & trade-offs:- Write durability: write to Kafka first (durable) then to in-memory indices; if Redis node fails, rebuild from Kafka + persistent store.- Latency vs freshness: tuning aggregator interval (1s-5s) balances freshness and CPU/IO.- Hotspots: heavy users can be rate-limited or have per-user sharding logic adjusted; use local caches for very hot reads.- Cost: Redis Cluster for high-throughput in-memory operations; persistent store sized for write throughput (Cassandra/Scylla).

Failure modes & mitigation:- Redis shard failure: promote standby, rebuild from Kafka/persistent store.- Consumer lag: autoscale consumers; monitor lag metrics.- Inconsistent merges: use monotonic scores and idempotent apply semantics; periodically reconcile.

Why this fits role needs:- Scales linearly by adding shards and consumers.- Provides low-latency reads via cached global top-N and per-shard top-K merge.- Balances cost and consistency; supports audits and strong consistency where required.

EasyTechnical

34 practiced

Your single-node web service runs on a VM with 8 vCPUs and 32GB RAM. Monitoring shows CPU trending from 30%→60% over 6 months and disk usage 55% with weekly growth of 40GB. Latency p95 has increased from 80ms to 180ms. Describe the indicators you would use to decide whether to scale vertically (bigger VM) or horizontally (more instances). Include thresholds, risk factors, and non-technical constraints (licenses, operations).

Sample Answer

Situation: I’m evaluating a single-node web service (8 vCPU, 32GB) with rising CPU (30%→60% over 6 months), disk at 55% and +40GB/week growth, and p95 latency up from 80ms→180ms.

Decision indicators I’d use (and thresholds):- CPU utilization: sustained >70% average or frequent spikes to >85% → favors horizontal scaling if CPU-bound across requests; vertical only if short-term bursts.- Memory pressure: high swap, OOM events, or >75% sustained memory → vertical scaling (larger RAM) or optimize app.- Disk capacity growth: projected full in <8–12 weeks (given +40GB/week, full in ~10 weeks) → must add storage immediately (scale vertically with larger disk or add separate volume); long-term, horizontal only helps if instances can share storage or use object store.- Latency root cause: profile request CPU vs I/O. If p95 correlates with CPU/memory contention → horizontal (add instances, load balance). If latency tied to disk I/O (high iowait) → vertical (better disks, provisioned IOPS) or move to networked storage/caching.- Throughput and concurrency: if RPS is growing linearly and adding instances reduces p95 → horizontal.- Failure domain: single-node risk → horizontal improves availability.

Risk factors:- Statefulness: if app is stateful or uses local disk for sessions, horizontal is harder (requires refactor or sticky sessions).- Operational complexity: more instances → orchestration, load balancer, deployment pipeline overhead.- Cost: larger VM vs multiple smaller VMs — compare hourly and licensing.- Data consistency and shared storage latency.

Non-technical constraints:- Licensing: per-core or per-instance licenses may make vertical cheaper or prohibit many instances.- Operational capacity: team ability to manage clustering, CI/CD, monitoring.- Compliance: data residency, encryption requirements affecting storage choices.- Budget and procurement cycles.

Action plan:1. Run profiling (CPU, mem, iowait, RPS) and capacity projection.2. Short-term: add storage volume/IOPS and tune caching if disk is immediate limiter.3. Medium-term: if CPU-bound and stateless, deploy autoscaling group + LB (horizontal); if memory/disk-bound and licensing prohibits many instances, choose vertical upgrade and plan refactor.4. Add alerts: CPU>70% sustained, disk fill >80%, p95>150ms.

Unlock Full Question Bank

Get access to hundreds of Scalability Fundamentals interview questions and detailed answers.

Join thousands of developers preparing for their dream job.