System Design and Scalability Questions

Covers architectural thinking and design tradeoffs for building reliable, high performance systems. Topics include design decision reasoning given constraints such as cost, latency and availability; scaling strategies including horizontal and vertical scaling, load balancing, caching patterns, database partitioning and sharding, read replicas, and asynchronous processing; capacity planning and observability; spotting and explaining bottlenecks such as hot partitions, single points of failure, database locks and network limits; and communicating technical impact in business terms. Candidates should be able to justify choices, compare alternatives, and articulate metrics and monitoring approaches to validate design decisions.

EasyTechnical

22 practiced

You inherit a large monolith and need to create a high-level plan to decompose it into microservices. Describe techniques to identify service boundaries, how to manage data ownership and API contracts, and strategies to minimize customer impact while incrementally moving functionality out of the monolith.

Sample Answer

**Situation & Goals (brief)** As a Cloud Architect I’d aim to reduce coupling, improve deployability and enable independent scaling while preserving customer SLAs and low-risk migration.

**Identify service boundaries** - Use Domain-Driven Design: map business capabilities into Bounded Contexts. - Run event-storming and stakeholder workshops to reveal seams and transactional boundaries. - Analyze runtime traces, DB access patterns, and change-coupling (git history) to find high-cohesion, low-coupling candidates. - Prioritize domains with clear interfaces and high value (e.g., billing, auth).

**Data ownership & consistency** - Adopt “database per service” where each microservice owns its canonical data. - Use asynchronous integration (event sourcing / change-data-capture -> events) to replicate state and avoid distributed transactions. - For cross-service queries, use materialized read models or API composition rather than synchronous DB joins. - Define ownership and retention policies in a data catalog.

**API contracts & governance** - Design consumer-driven contracts; publish OpenAPI specs and run contract tests in CI. - Enforce semantic versioning; prefer additive changes and provide vN endpoints during transitions. - Use an API Gateway for routing, auth, rate-limiting and gradual traffic steering.

**Minimize customer impact / migration strategy** - Apply the Strangler Fig pattern: incrementally route functionality from monolith to services. - Start with non-critical or high-change domains. - Use feature flags, dark launches, and canary/blue-green deployments to validate behavior. - Automate end-to-end tests, monitoring, and alerting (SLOs/SLIs) to detect regressions. - Maintain rollback paths and runbooks; communicate timelines to stakeholders.

**Phased plan (example)** 1. Discovery: domain mapping, telemetry/CI baseline. 2. Pilot: extract one bounded context, implement events + API, run in parallel. 3. Expand: repeat with prioritized domains, enforce contracts. 4. Harden: centralize governance, cost/operational monitoring, decommission monolith parts.

This approach balances architectural rigor with pragmatic, low-risk migration suitable for cloud-native operations.

HardTechnical

16 practiced

Evaluate multi-tenant database partitioning strategies for a SaaS product: shared schema for all tenants, separate schema per tenant, and fully isolated databases per tenant. Discuss tenant isolation, operational overhead, migration complexity, hot-tenant mitigation and how each approach affects backup/restore and compliance obligations.

Sample Answer

**Clarify trade-offs (one-sentence)** Choosing between shared schema, per-tenant schema, and isolated DBs is a trade-off among isolation, cost/ops, scalability, and compliance — pick based on risk profile, scale, and SLA.

**Tenant isolation**- Shared schema: weakest (row-level isolation); requires robust application-layer RBAC, row-level security (RLS) and careful query hygiene.- Per-tenant schema: strong logical isolation within same DB instance; reduces blast radius for schema changes.- Isolated DBs: strongest isolation (OS/process/db instance boundary); preferred for high-risk or regulated tenants.

**Operational overhead**- Shared schema: lowest infra cost, easiest provisioning, simplest schema changes, but complex access control and testing matrix.- Per-tenant schema: moderate overhead — schema migrations must run per schema; tooling needed to manage many schemas.- Isolated DBs: highest overhead — provisioning, connection pooling, monitoring, and patching multiply with tenant count; automation required.

**Migration complexity**- Shared → per-tenant/isolation: moderate to high; need to extract tenant data and transform schemas, careful downtime planning.- Per-tenant → isolated DBs: lower effort than shared → isolated, mostly data copy and endpoint reconfiguration.- Isolated → consolidate: most complex due to reversing many-to-one merges.

**Hot-tenant mitigation**- Shared schema: noisy neighbor risk; mitigate with query throttling, per-tenant rate limits, resource governor, and workload-aware indexing.- Per-tenant schema: can prioritize heavy schemas, but still contend for instance-level resources.- Isolated DBs: easiest to throttle/scale individually — move hot tenant to larger instance or dedicated pools.

**Backup/restore & compliance**- Shared schema: backups are at DB level; tenant-level restore requires complex export/import; challenging for per-tenant retention and legal hold.- Per-tenant schema: easier to target schema-level dumps if DB supports it; still single-instance failure domain.- Isolated DBs: straightforward per-tenant backups/restores, retention policies, and easier to meet data residency/SOC/GDPR requirements.

**Recommendation (Cloud Architect lens)**- For early-stage SaaS with many small tenants: start shared schema with RLS and strong monitoring; plan migration path.- For mixed risk/scale: per-tenant schema as compromise; automate migrations, backups, and tenants lifecycle.- For enterprise/regulatory/high-SLA tenants: isolated DBs or dedicated instances; use automation (Terraform, IaC), DB pools, and cost allocation.

Design for eventual migration: include tenant identifier abstraction, feature flags, and automated data-export/import playbooks.

MediumTechnical

17 practiced

Define a 'hot partition' and outline techniques to detect and mitigate hot partitions or shard hotspots in a distributed service. Include examples such as adaptive hashing, request routing, caching, throttling, and re-sharding, and explain the operational tradeoffs for each solution.

EasyTechnical

21 practiced

Design API versioning and idempotency strategies for a public RESTful API. Explain how you would roll out breaking changes, support multiple client versions simultaneously, and ensure idempotent behavior for create and update endpoints to enable safe retries by clients and proxies.

Sample Answer

**Approach summary**As a Cloud Architect I design API versioning and idempotency to minimize customer impact, support parallel client fleets, and allow safe network retries by clients/proxies.

**Versioning strategy**- Prefer explicit major versioning in the URL for breaking changes: /v1/...; use minor/patch via headers (Accept-Version) for non-breaking evolution.- Adopt Semantic Versioning for API contract changes: only bump major for breaking changes.- Rollout for breaking changes: - Introduce /vN alongside existing versions; run canary traffic and feature flags. - Provide migration guides, automated tests, SDK updates, and a deprecation schedule (e.g., 6–12 months) with telemetry-based adoption targets. - Support multiple versions concurrently via routing rules and API Gateway stage mapping; automate sunset once client usage is low.- Backward-compatible changes: add fields (nullable/optional), new endpoints or query parameters; never remove fields without a major bump.

**Idempotency strategy**- Use HTTP semantics: - PUT for full-resource updates (idempotent by design). - PATCH for partial updates (make idempotent via idempotency keys or versioned PATCH with preconditions). - POST for create: support client-provided id (UUID) or idempotency-key header.- Idempotency-key pattern: - Clients send Idempotency-Key header (UUID) for POST/PATCH; server stores request hash -> response mapping for a bounded TTL (e.g., 24h), returning same response for duplicate keys. - Ensure idempotency records are durable, partitioned by key, and expire to limit storage.- Concurrency control: - Use ETags and conditional requests (If-Match) for optimistic concurrency on updates. - Return 409 on version conflicts with guidance for client retry.- Safe retries and proxies: - Document which methods are safe to retry; ensure gateway/lb preserves Idempotency-Key and does not coalesce requests incorrectly. - Provide Retry-After and idempotent response codes (200/201 for duplicate create, 409 for conflict, 422 for invalid).

**Operational considerations**- Monitor idempotency key collisions and TTL expirations.- Log and expose migration telemetry per API version to drive deprecation.- Automate SDK generation per major version and CI tests guaranteeing backward compatibility for minor releases.

This approach balances compatibility, safety for retries, and operational manageability across large-scale cloud deployments.

MediumTechnical

18 practiced

You need to decide between higher-cost managed cloud services (low-latency, SLA-backed) and cheaper self-managed solutions requiring in-house operational capacity. Describe a framework to make this decision: include business KPIs, operational capacity, failure costs, TCO, and migration/exit costs.

Unlock Full Question Bank

Get access to hundreds of System Design and Scalability interview questions and detailed answers.

Join thousands of developers preparing for their dream job.