Clarifying Scope and System Constraints Questions

Ability to ask targeted questions to understand system requirements: user base, traffic volume (requests per second), latency targets, data consistency requirements, compliance/regulatory constraints. Understanding that different systems have different requirements and that constraints shape architecture decisions.

EasyTechnical

78 practiced

Clients often use 'low latency' as a requirement. Describe the targeted questions you would ask to turn 'low latency' into measurable engineering targets. Include how you would capture p50/p95/p99 goals, allowed tail latency for specific flows (login, search, checkout), payload size expectations, and latency SLIs for both user-facing and B2B API calls.

Sample Answer

First, translate “low latency” into measurable requirements by asking targeted, contextual questions:

- Scope & users: Which flows are latency-sensitive (login, search, checkout, reporting)? How many concurrent users, percent interactive vs. batch, and peak load patterns?- UX tolerance: What is acceptable perceived wait per flow? (e.g., <100ms for keystroke-autocomplete, <500ms for page load)- SLAs & penalties: Do you need contractual SLAs? What % of requests can breach targets?- Traffic & payloads: Typical and max payload sizes (request/response), number of calls per user action, and acceptable amplification from retries/headers.- Dependencies: Which downstream systems affect latency? Are they in-path synchronous or can be async/queued?- Monitoring & remediation: Required observability, alert thresholds, and acceptable MTTR.

Turning answers into targets:- Define percentiles per flow: p50/p95/p99 with concrete numbers. Example defaults: - Login: p50 80ms, p95 200ms, p99 500ms - Search (query): p50 40ms, p95 150ms, p99 400ms (autocomplete p50 <30ms) - Checkout (payment commit): p50 120ms, p95 400ms, p99 800ms- Tail SLAs: specify allowed breach budget (e.g., p99 must be met 99.9% of business hours; or error budget 0.1%).- Payloads: state expected sizes (e.g., search response median 2KB, max 100KB); enforce limits and streaming for large responses.- SLIs for APIs: - User-facing: measure end-to-end client-observed latency (browser RTT + backend). SLI = fraction of requests with end-to-end latency < target. - B2B APIs: measure server-side request processing time and 95th/99th percentiles; also include SLA for availability and throughput.Instrumentation:- Capture metrics at ingress e.g., trace spans, histograms for p50/p95/p99, tagging by flow, payload size, and caller.- Add synthetic transactions for critical flows and dashboards/alerts on p95/p99 breaches and error budgets.

This converts vague “low latency” into concrete, testable engineering targets tied to business impact and monitoring.

MediumSystem Design

97 practiced

A client wants per-user rate-limits of 100 requests/minute for 10M users. What clarifying questions will you ask (burst allowance, global vs per-region limits, shared tenants, penalties), and how would you scope a distributed rate-limiter? Discuss options: centralized store, token bucket with local caches, or probabilistic approaches, and the trade-offs for correctness and latency.

Sample Answer

Clarifying questions- Is the 100 req/min strict per individual user ID, or can related identities share a bucket (e.g., household, org)? Any anonymous/unauthenticated traffic?- Burst policy: allowed burst size and refill behavior? Do you want smoothing (leaky bucket) or token-bucket bursts?- Scope: are limits global, per-region, or both? Is enforcement required at edge (APIGW) before routing?- Penalties: hard reject (HTTP 429), soft-throttle, or delayed retries/backoff + reporting/analytics?- Consistency SLAs: is occasional overage acceptable (eventual correctness) or must be strict? Acceptable request latency budgets?- Scale/ops: expected QPS distribution, geo distribution, and retention window for usage logs/metrics.

Scoping a distributed rate-limiter- Requirement: enforce 100/min for 10M users → steady-state tokens = 10M * 100/60 ≈ 16.7k TPS aggregate. Need horizontally-scalable, low-latency checks at edge.- Architecture: enforce at the API gateway edge for latency; make decision local when possible; fallback to a consistent global store only when necessary.

Options and trade-offs1) Centralized store (Redis Cluster with Lua)- How: single authoritative counter per user in Redis, atomic increments with TTL or token-bucket Lua script.- Pros: correctness (strong), simple policy changes, easy metrics.- Cons: network latency to central store, large throughput and memory footprint (10M keys), single class of failures; needs heavy sharding and HA; higher tail latency.

2) Token bucket with local caches (hybrid)- How: distribute small local token caches at each gateway instance populated from a central authoritative store. Local node decrements tokens until cache exhausted, then sync/refill.- Pros: low latency, reduces central load, supports bursts, good for geo-distributed enforcement.- Cons: risk of transient over-allocation (inconsistency) proportional to cache size; complexity in refill logic and reconciliation. Choose conservative per-node allotments to bound overage.

3) Probabilistic/sketch-based approaches (e.g., Count-min + sampling, Bloom filters, or probabilistic token buckets)- How: use compact sketches for many users and probabilistic rejection for low-value clients to save state.- Pros: much lower memory, high throughput.- Cons: false positives/negatives → possible incorrect throttling; not suitable where strict guarantees required.

Recommendations- If strict correctness required: Redis Cluster per-region with consistent hashing + Lua token-bucket; mitigate latency with edge co-located Redis replicas and pipelining; autoscale shards; accept higher cost.- If low-latency and eventual correctness acceptable: hybrid token-cache at gateways with small conservative allotments (e.g., N tokens per minute per node), periodic reconciliation and central logging for enforcement/analytics.- For cost-sensitive ultra-large scales with relaxed SLAs: probabilistic sketches for best-effort limits, combined with premium users on strict paths.

Other considerations- Key lifecycle: TTLs to avoid 10M hot keys when inactive.- Monitoring/alerts: per-key hotspot detection, refill errors, and eviction metrics.- Rate-limit config management: feature flags, dynamic policies, quota tiers.- Testing: chaos/failure injection, clock skew handling, burst replay tests.

Trade-off summary: centralized → strong correctness, higher latency/cost; local caches → low latency, bounded overage; probabilistic → minimal resources, reduced correctness. Choose based on strictness of guarantees, latency SLA, and cost.

HardTechnical

81 practiced

A stakeholder asks you to choose between CDN-only caching, edge compute, or regional microservices to meet global performance goals. What clarifying questions do you ask about cacheability, personalization, SLA, and operational complexity? Provide a comparative analysis of the three approaches focusing on latency, cost, developer experience, and consistency.

Sample Answer

Clarifying questions

Cacheability- Which responses are fully cacheable (public, immutable) vs cacheable with short TTL vs uncacheable?- Are there cache-control headers or ETag semantics available? Frequency of updates/invalidations?- What percentage of payloads are static assets vs rendered HTML/API responses?

Personalization- How much per-user personalization (auth-dependent content, A/B tests, geo/locale)?- Can personalization be done client-side (JS) or must it be server-generated?- Is personalization deterministic (cookie, header) or requires session/state lookup?

SLA & performance goals- Target p95/p99 latencies by region and global traffic distribution?- Required availability and allowable error budget (RTO/RPO for failures)?- Peak vs steady-state traffic and expected growth.

Operational complexity & constraints- Existing infra (cloud vendor, CDNs supported), deployment pipeline maturity?- Team skillset for edge JS/Wasmtime vs running regional services?- Monitoring/observability requirements, debugging needs, and compliance (data residency)?

Comparative analysis (CDN-only caching | Edge compute | Regional microservices)

Latency- CDN-only: Lowest read latency for cache hits (~<20ms global) since served from PoPs; cache misses go to origin.- Edge compute: Very low latency for logic executed at PoP; avoids origin roundtrips for lightweight personalization.- Regional services: Low latency within region; cross-region users experience higher latency and potential hops.

Cost- CDN-only: Lowest operational cost for high cache hit ratio; pay per GB and requests. Invalidation/origin egress can add cost.- Edge compute: Higher costs per request than raw CDN due to compute charges; cheaper than many origin hits if it avoids origin traffic.- Regional services: Higher infra and networking costs (many replicas, inter-region data transfer); greater ongoing ops cost.

Developer experience- CDN-only: Simplest model (cache headers) — easy to reason about; limited control for dynamic behavior.- Edge compute: Moderate complexity — needs smaller runtime-compatible code, new debugging/deploy patterns, testing for cold starts and state handling.- Regional services: Familiar server-side dev model; full frameworks supported, easier local debugging but requires CI/CD for many regions.

Consistency & correctness- CDN-only: Eventual consistency depending on TTL/invalidation; risk of stale reads unless aggressive invalidation or cache-bypassing used.- Edge compute: Can be consistent for read-mostly cases if it consults origin/state; if caching at edge, same staleness trade-offs apply.- Regional services: Stronger consistency possible (region-local) but global consistency requires replication strategy — trade-offs between latency and consistency (CRDTs, leader/follower, quorum).

When to choose- CDN-only: Choose when high cacheability, low personalization, and strict cost/latency for reads are priority.- Edge compute: Choose when you need low-latency personalization or A/B logic at PoP without full regional deployment.- Regional microservices: Choose when requests require heavy server-side processing, strong consistency, or access to regional resources/state.

Recommendation approach- Quantify % cacheable and latency SLA, run cost model (cache hit vs edge exec vs origin), and prototype a critical path on the chosen CDN/edge provider. Combine: prefer CDN-first, add edge compute for light personalization, and regional services where stateful/consistent processing is required.

MediumTechnical

83 practiced

You receive an RFP that says 'must handle high traffic.' As a Solutions Architect doing intake, what specific, measurable questions would you ask to convert this into engineering requirements? Include metrics like average vs peak RPS, concurrent connections, request size, tolerance for dropped requests, caching expectations, SLAs, and acceptable cost model. Explain why each metric matters to design choices.

Sample Answer

Situation: The RFP says only "must handle high traffic." To convert that into engineering requirements I would run a focused intake with the buyer and ask the following specific, measurable questions — each drives architecture, capacity, cost, and SLO choices.

1) What do you mean by "high traffic"? Give average RPS and peak RPS (e.g., avg 2k RPS, peak 50k RPS for 10 minutes). - Why: capacity planning, autoscale thresholds, load balancer and network sizing.

2) What are expected concurrent connections/sessions (web sockets/long-polling)? - Why: impacts connection-oriented infrastructure (TCP limits, proxy choice, connection pooling, memory).

3) Typical and max request payload size and response size (KB/MB) and % of requests that are heavy (file upload/download)? - Why: influences bandwidth, CDN use, node memory, and timeout settings.

4) Traffic profile: steady vs spiky (burst frequency, duration, and seasonal patterns)? - Why: determines autoscaling policy, buffer queues, and burst capacity.

5) Acceptable latency targets (p95/p99) and SLAs for availability (e.g., 99.95%)? - Why: affects replication, caching strategy, multi-AZ or multi-region design.

6) Tolerance for dropped/failed requests and retry behavior (max acceptable error rate, e.g., <0.1%)? - Why: drives backpressure, circuit breakers, queuing, and SLOs.

7) Caching expectations: cache hit ratio targets, TTLs, cache invalidation windows, and if edge CDN is acceptable? - Why: reduces origin load, shapes cache tiering and consistency trade-offs.

8) Data consistency requirements (eventual vs strong), and durability needs (lossless vs best-effort)? - Why: choice of databases, replication lag tolerances, and replication topology.

9) Authentication/authorization cost per request (complex JWT verification, external IAM calls)? - Why: adds CPU/latency per request and may need token caching.

10) Failure scenarios and RTO/RPO targets for outages? - Why: dictates cross-region failover and backup strategies.

11) Compliance/security constraints affecting design (e.g., crytpo offload, VPC-only access)? - Why: may prevent CDN/CDN edge or public cloud-managed services.

12) Budget/cost model preferences: predictable monthly vs pay-per-use, and max acceptable monthly cost per peak RPS? - Why: influences reserved instances, serverless vs provisioned, use of managed services.

13) Monitoring/observability requirements: required metrics, retention, alert thresholds, and demo dashboards? - Why: necessary for capacity planning, SLO monitoring, and incident response.

14) Integration constraints (third-party rate limits, upstream/downstream systems)? - Why: end-to-end throughput may be limited by external systems, requiring buffering or throttling.

Result: With concrete answers to these, I can produce engineering requirements: provisioned RPS capacity, autoscale rules, connection limits, caching layers, CDN plan, SLOs/SLA language, cost estimates, and a resilience design.

HardTechnical

97 practiced

Client traffic can spike 100x unpredictably but their budget is constrained. What clarifying questions would you ask about acceptable latency and failure modes, business-critical transactions vs best-effort, and acceptable cost vs availability trade-offs? Propose a cost-aware architecture combining serverless, queueing, and reserved instances to handle spikes while containing base costs.

Sample Answer

Clarifying questions (grouped so design matches business constraints):- Latency & SLAs: Which operations require sub-100ms/500ms/—or can be async? Are there hard deadlines (payment auth, user checkout) vs background work (analytics, email)?- Failure modes: On overload, should requests be rejected, queued, or downgraded? Is eventual consistency acceptable? Any regulatory/risk constraints (no loss of payments)?- Priority/criticality: Define business-critical transactions (billable, revenue-impacting) vs best-effort. What percent of traffic must be preserved during spikes?- Cost vs availability: What’s acceptable monthly spend variance? Max pay-per-use during spikes? Is predictable baseline cost preferred even if it increases base spend?- Recovery & visibility: RTO/RPO, metrics for success, required audit/logging retention.

Proposed cost-aware architecture (cloud-agnostic, examples reference AWS services):1. Tiering by criticality - Synchronous-critical: small reserved fleet (Reserved Instances / Committed Use) behind API Gateway + autoscaling group sized for baseline load. Use health checks and sticky sessions if needed. - Elastic-burst: Serverless functions (Lambda/Cloud Run) fronted by API Gateway with reserved concurrency for critical flows. Serverless covers 100x spikes without provisioning additional VMs. - Asynchronous/best-effort: Front-end accepts requests and immediately enqueues to durable queue (SQS/Kafka/Cloud PubSub). Worker pool processes at controlled rate.

2. Buffering & backpressure - Durable queue to absorb spikes; producers get immediate accept/ack for async operations. - Implement token-bucket rate limits and client-facing throttling for sync paths; expose retry headers (Retry-After). - Circuit breaker to fail fast for non-critical features.

3. Cost containment - Reserved instances (or committed use) sized for predictable baseline traffic → low steady cost. - Serverless for unpredictable spikes — pay-per-execution only during bursts. - Worker autoscaling: use spot instances for best-effort workers to lower cost; scale-in protection and fallback to on-demand for critical jobs. - Cold-start mitigation: provisioned concurrency for a small fraction of serverless for latency-sensitive paths.

4. Reliability & data durability - DLQs, retry policies with exponential backoff, idempotency keys for at-least-once processing. - Persistent storage (S3/GCS) for large uploads; pre-signed URLs to avoid compute costs.

5. Observability & guardrails - Cost alerts, usage-based quotas, throttling policies per API key/client. - Fine-grained metrics (latency P95/P99, queue depth, cost per request). Automated policy to shift lower-priority jobs to delayed processing when cost thresholds hit.

Trade-offs and reasoning:- Reserved instances reduce baseline cost but are inflexible — acceptable if baseline is stable.- Serverless offers massive elasticity with higher per-op cost; ideal for unpredictable spikes without overprovisioning.- Queueing decouples ingestion and processing, smoothing compute usage and reducing peak instance needs.- Using spot for best-effort reduces cost but may increase preemption risk; isolate critical work on reserved/on-demand.

Example flow (payment vs analytics):- Payment: API Gateway → auth/validation → routed to synchronous pool (reserved instances + provisioned serverless concurrency) → immediate response or graceful reject.- Analytics: API Gateway → enqueue → spot/worker pool processes with backpressure and DLQ.

This approach preserves low steady-state cost, supports 100x spikes via serverless+queue buffering, and lets you declare clear failure modes and SLA tiers for business alignment.

Unlock Full Question Bank

Get access to hundreds of Clarifying Scope and System Constraints interview questions and detailed answers.

Join thousands of developers preparing for their dream job.