Scalability and Code Organization Questions

Focuses on designing software and codebases that remain maintainable and performant as features and user load grow. Areas include modularity and separation of concerns, component and API boundaries, when and how to refactor, trade offs between monolith and service oriented architectures, data partitioning and caching strategies, performance optimization, testing strategies, dependency management, code review practices, and patterns for maintainability and evolvability. Interview questions may ask candidates to reason about design choices, identify coupling and cohesion issues, and propose practical steps to evolve an existing codebase safely.

HardTechnical

41 practiced

You are evaluating whether to migrate a product from a modular monolith to microservices for expected scale growth to 1B monthly users. Present a decision framework: operational costs, team topology, data ownership, deployment complexity, latency/throughput considerations, and a migration timeline with safety checks. Include scenarios where staying monolithic is better.

Sample Answer

Start by clarifying requirements and constraints: estimated peak QPS, data consistency needs, SLA (latency), team size and org growth, regulatory/data residency, and expected feature velocity.

Decision framework- Operational costs - Estimate infra cost: more services → more JVMs/containers, cross-service traffic, service mesh. Add SRE/on-call headcount and monitoring/logging costs. - Compare with monolith costs: larger instances, simpler ops, cheaper networks.- Team topology - Microservices suit multiple small, autonomous teams (Conway’s Law). If you can form product-aligned teams owning full service lifecycle, microservices scale. If teams are few or tightly coupled, monolith avoids coordination overhead.- Data ownership - Prefer microservices when bounded contexts and independent data ownership are clear. Plan for data duplication, change-data-capture, and anti-corruption layers. If strong single-transaction consistency is required across domains, monolith simplifies ACID guarantees.- Deployment complexity - Microservices require CI/CD per service, versioning, backward compatibility, contract testing, and rollout strategies (canary, blue/green). Monolith simplifies deployment pipeline but increases blast radius.- Latency / throughput - Model network hops: convert to tail-latency budget. If critical paths require sub-ms or single-hop access, monolith or colocated components may be better. Microservices help horizontal scale hot paths (stateless tiers) but add RPC overhead.- Safety & observability - Invest in distributed tracing, centralized metrics, structured logs, circuit breakers, and rate limiting before migrating.

Migration timeline with safety checks (phased, ~6–18 months depending scale)1. Discovery (1–2 months): map bounded contexts, critical paths, data flows, SLAs, and dependent teams. - Safety: performance baseline, runbooks.2. Modular hardening (2–4 months): turn monolith into a clean modular architecture with explicit interfaces and feature flags. - Safety: automated tests, contract tests, feature flags.3. Extract read-only services (2–3 months): move non-critical read-heavy modules to separate services with replicated data or APIs. - Safety: compare metrics, fallbacks to monolith.4. Extract stateless services and caches (3–6 months): separate stateless compute and edge services; add API gateway and tracing. - Safety: canary deploys, circuit breakers, throttles.5. Extract stateful domains (3–6 months): migrate ownership of databases using CDC or strangler pattern; ensure eventual consistency patterns. - Safety: dual-write validations, shadow reads, rollback plans.6. Operate & iterate (ongoing): optimize costs, refine team boundaries, retire monolith components. - Safety: SLOs, alerting, capacity tests, chaos engineering.

When staying monolithic is better- Team small (<10 engineers) with high feature coupling and need for rapid coordination.- Strong transactional, cross-domain consistency requirements that are expensive to redesign.- Latency budgets that disallow additional RPC hops.- When operational maturity (SRE, observability) is missing and the cost of building it outweighs benefits.- If projected growth can be handled by vertical scaling and sharding within a modular monolith.

Final decision rule: migrate only when organizational readiness (teams + SRE), clear bounded contexts, measurable cost/latency benefits, and a staged rollback-safe plan exist. Otherwise invest in modularization, performance tuning, and CI/CD improvements within the monolith first.

EasyTechnical

35 practiced

Explain what 'modularity' and 'separation of concerns' mean in a software codebase. For a software engineer working on a growing web application, give concrete examples of how you would structure modules/packages (e.g., API layer, business logic, persistence), who owns each boundary, and how these patterns improve scalability, testability and team autonomy. When would you introduce additional module boundaries?

Sample Answer

Modularity: splitting a codebase into well-defined, loosely-coupled pieces (modules/packages) with clear responsibilities and interfaces. Separation of concerns: each module addresses one concern (e.g., HTTP, validation, business rules, persistence) so changes in one area don’t ripple across others.

Concrete structure for a growing web app:- API layer (controllers/routes): handles HTTP, authentication, request/response mapping. Owns input validation, auth hooks. Owned by frontend-backend integration or API team.- Application / Business Logic (services/use-cases): orchestrates domain rules, transactions, validations, and coordinates lower layers. Owned by backend/domain engineers.- Domain models/entities: pure domain objects, invariants and domain-specific logic. Owned by domain experts/backend engineers.- Persistence layer (repositories/DAOs): translates domain operations to DB queries, handles transactions, migrations. Owned by infra/backend engineers or a data-access team.- Infrastructure / cross-cutting (logging, metrics, cache, queues): owned by SRE/platform team.- Adapters (third-party APIs, payment gateways): encapsulate external integrations; owned by engineers responsible for that integration.

How this helps:- Scalability: clear boundaries let teams optimize independently (e.g., scale only API pods or caching layer). Replace implementations (switch DB) behind repository interfaces without changing services.- Testability: unit-test pure business logic without network/DB; mock repositories/adapters at boundaries for fast tests; write integration tests for adapters.- Team autonomy: teams own modules/edges and APIs; bounded contexts reduce coordination, allow parallel work and independent deploys.

When to add boundaries:- When a module grows > ~300–500 LOC or has multiple responsibilities.- When release cadence diverges (need independent deploys).- When coupling causes frequent merge conflicts or cross-team coordination.- When performance or scaling needs demand separate services (move to microservice or separate packages).Introduce boundaries incrementally: start with packages and explicit interfaces; extract services when costs justify added complexity.

HardTechnical

31 practiced

Compare API gateways and service meshes. For each: explain responsibilities (routing, auth, observability), performance characteristics, operational overhead, and scenarios where one is more appropriate than the other. Provide an example architecture that uses both and justify the division of responsibilities.

Sample Answer

Definition & primary intent:- API gateway: a north‑south edge component that exposes backend services to clients (mobile/web/third parties). Acts as a single ingress, handling routing, protocol translation, request aggregation, auth at boundary, rate limiting, and coarse observability.- Service mesh: an east‑west infrastructure layer inside the cluster made of sidecar proxies that handle inter‑service networking: mTLS, fine‑grained routing/retries/timeout, per‑service telemetry, and policy enforcement.

Responsibilities:- Routing: Gateway does external routing, host/path-based routing, and can do API composition. Mesh provides service-to-service routing (service discovery, canary, traffic shifting).- Auth: Gateway handles authentication/authorization for external clients (OAuth, API keys, JWT validation). Mesh enforces mutual TLS, service identity, RBAC between services and can check end-to-end policies.- Observability: Gateway provides request-level logs, metrics and traces for incoming traffic. Mesh produces dense telemetry (per-call latency, retries, success rates) and distributed tracing for internal calls.

Performance characteristics:- Gateway: single or scaled gateway instances; adds one hop for external requests. Optimized for throughput but can become bottleneck if not scaled.- Mesh: sidecar introduces per‑hop proxy overhead (CPU/memory + latency ~0.5–5ms depending on proxy). Overhead multiplies with call graph depth.

Operational overhead:- Gateway: relatively low; manage fewer instances, certificates, external routing rules.- Mesh: higher; deploy sidecars for every service, manage control plane (policy, config), more complex observability data volume and RBAC rules.

When to use which:- API gateway preferred when exposing APIs to clients, performing auth at edge, TLS termination, request aggregation, and protecting backend from abusive clients.- Service mesh preferred when you need fine‑grained service identity, progressive delivery (traffic shifting, mirroring), observability between services, and uniform resilience patterns.

Combined architecture example:- Internet → CDN → API Gateway (Kong/Envoy as edge) → Ingress → Kubernetes cluster with sidecar service mesh (Istio/Linkerd)Division of responsibilities:- API Gateway: authenticate external users (OAuth/JWT), rate limit, WAF, protocol translation, coarse metrics, request routing to appropriate internal service.- Service Mesh: enforce mTLS, per‑service RBAC, circuit breakers, retries, canary traffic splitting, per‑call metrics/traces, fine‑grained policy.Justification: Edge handles client concerns (auth, aggregation, exposure) and reduces attack surface; mesh handles internal reliability/security and observability where you need per-service control. This separation keeps external policy centralized and internal policies decoupled, reducing blast radius and enabling independent scaling and evolution.

HardSystem Design

35 practiced

Design a hybrid consistency strategy for a payments platform where some operations (balance updates) require strong consistency while others (transaction history display) can be eventually consistent. Explain data modeling, API semantics, and how you'd implement and test guarantees across services.

Sample Answer

Requirements & constraints:- Strong consistency for balance-affecting ops (debits/credits, fraud checks). No lost or double-applied money.- Eventually consistent transaction history feed (UI can show slight delay).- High throughput, low latency, auditability, idempotency.

High-level architecture:- Core ledger service (strong-consistency datastore + transactional API).- Event stream (Kafka) for asynchronous propagation.- Read-model service / history DB (event consumers maintain transaction history, searchable by user).- API gateway + idempotency layer + monitoring.

Data modeling:- Ledger (primary, strongly consistent): - accounts(account_id PK, balance, reserved_amount, version) - ledger_entries(entry_id PK, account_id, amount, type, status, created_at) - Use optimistic concurrency (version) or DB transactions to enforce serializability.- Event envelope: - event_id, type, payload (ledger_entry_id, account_id, amount, prev_balance, new_balance), sequence, timestamp- Read model (eventual): - transactions(account_id, entry_id, amount, status, created_at, posted_at)

API semantics:- POST /payments/debit (body: account_id, amount, idempotency_key) - Synchronous: perform transactional write to ledger (check sufficient funds, update balance, insert ledger_entry with status=posted), return 200 + new_balance. - Guarantee: linearizable update for that account (use single-row transactions or leader for partition). - Idempotency: idempotency_key stored with result to prevent duplicates.- POST /payments/credit - similar.- GET /accounts/{id}/balance - Strongly consistent read from ledger primary (read-after-write guarantees).- GET /accounts/{id}/transactions?cursor=... - Read from transactions read-model (eventual). Include metadata: last_event_seq and "as_of" timestamp to indicate staleness.

Implementation details:- Ledger DB: use relational DB with strong transactions (e.g., PostgreSQL with SERIALIZABLE or row-level locking) or a single-sharded leader for each account to keep operations serializable. Partition accounts by consistent hash; each shard elects leader for writes.- Use optimistic concurrency: UPDATE accounts SET balance = balance + :delta, version = version + 1 WHERE account_id=:id AND version=:v. If no rows affected, retry.- Write path: within same DB txn: insert ledger_entry, update accounts, persist idempotency record, then publish event to Kafka (transactional outbox pattern). Outbox ensures atomicity between DB and Kafka.- Event consumers build transactions read-model; they apply events idempotently (use event_id dedupe).

Guarantees explained:- Strong consistency for balance: transactional updates + idempotency + per-account serialization ensure invariant: sum(balances) consistent and no double-spend.- Eventual consistency for history: outbox → Kafka → consumers may lag. UI can show "pending" or include sequence numbers to explain staleness.

Testing strategy:- Unit tests for business logic (sufficient funds, idempotency).- Integration tests with DB and Kafka (use testcontainers): simulate multi-threaded concurrent debits to same account; assert final balance deterministic and ledger entries unique.- Chaos tests: kill consumer, simulate partition, ensure no lost events (outbox replay), validate idempotent reprocessing.- Contract tests: consumers validate event schema evolution.- End-to-end: create workload generator that performs random credits/debits with idempotency keys and then: - Assert ledger sums equal applied operations. - Compare balance read from ledger vs. computed from transactions read-model; measure replication lag; set SLOs.- Monitoring & alerts: consumer lag, outbox queue depth, failed idempotency lookups, divergence counters. Periodic consistency checker job that recomputes balances from ledger entries and raises alerts on mismatch.

Trade-offs:- Using strong DB transactions simplifies correctness but limits horizontal write scalability; mitigate with sharding by account.- Eventual history gives scalability and UI responsiveness, acceptable for non-critical reads.

This hybrid keeps money operations safe (strong consistency) while enabling scalable, responsive history and analytics (eventual consistency) with clear API semantics, idempotency, and robust testing/monitoring.

HardTechnical

37 practiced

You're rolling out a cross-service feature that must reach only 10% of users initially and be easy to rollback. Design a rollout plan using feature flags, targeting, telemetry, and emergency rollback measures. Explain how to avoid flag debt and ensure flags are removed once stable.

Sample Answer

Situation: We need to release a cross-service feature to 10% of users, with low risk and quick rollback ability.

Plan (high-level):- Flag strategy: Implement a centralized, typed feature-flag ("new_checkout_v1") in a feature-flag service accessible by all services. Make it boolean + staged rollout metadata (cohort id, start time, reason, owner, TTL).- Targeting: Use deterministic bucketing (hash(user_id) % 100 < 10) for the 10% cohort to ensure consistent treatment across services. Allow overrides for QA, internal canaries, and specific segments (e.g., region, device).- Deployment: Deploy code paths behind the flag in a dark-launch fashion (feature off by default). Run integration tests and rollout to internal canaries (1% engineers) first, then to 10% production cohort.

Telemetry & observability:- Instrument key business metrics (conversion, latency, error rate, API success rate) and feature-specific events (feature_evaluated, flow_entered, flow_completed) with the cohort tag.- Create dashboards and alerts for statistical regressions (e.g., >3σ increase in error rate or 5% drop in conversion) and service health (p95 latency, DB errors).- Run A/B analysis during rollout window with pre-defined SLOs and significance thresholds.

Emergency rollback:- Ensure flag evaluation is fast and centralized so switching off is immediate.- Pre-script rollback runbook: toggle flag off in feature-flag service (global and service caches invalidated), rollback deployment only if flag toggle fails, notify pager, and run mitigation steps.- Test rollback path during staging and chaos exercises.

Avoiding flag debt:- Tag flags with metadata: owner, creation date, purpose, removal date candidate.- Enforce lifecycle: every flag must have a removal plan in PR and code review. Create automated linting that warns about long-lived flags (>30 days) and reports to owners.- Post-launch: After metrics show stability for the agreed period (e.g., 2 weeks, statistically insignificant differences), convert flagged code into default-on behavior and remove guard rails. Run a cleanup sprint to delete flag checks, associated config, and telemetry hooks.- Use code search and CI checks to ensure no residual flag references before merge.

This plan gives deterministic targeting, rapid rollback, measurable safety guards, and a clear process to eliminate flag debt.

Unlock Full Question Bank

Get access to hundreds of Scalability and Code Organization interview questions and detailed answers.

Join thousands of developers preparing for their dream job.