DoorDash Solutions Architect (Junior Level) Interview Preparation Guide

Start by clarifying scope: which API path, whether latency is client-side or server-side, and if the issue is per-tenant. Then recommend changes in two tracks: observability to diagnose, and architecture/controls to reduce per-customer latency and enforce tiered SLAs.

Observability / diagnosis

• Add distributed tracing (e.g., OpenTelemetry + Jaeger/Tempo) with tenant-id propagated through headers to see per-request span breakdown and hotspots.
• Per-tenant metrics: instrument service latency, p95/p99, request rate, error rate, resource usage (CPU, memory, DB calls) labeled by tenant. Export to Prometheus/Grafana and create per-tenant dashboards.
• Synthetic and real-user monitoring: synthetic tests per key tenant during peak hours; RUM for client-side latency.
• Logging correlation: include trace-id and tenant-id; sample high-latency traces for deeper profiling (cpu/memory/heap).
• Alerting and SLOs: define per-tier SLOs (p95 < X ms for Platinum, p95 < Y ms for Gold) and error-budget alerting per tenant.

Architecture & QoS controls

• Tenant isolation: consider logical (sharded DB, per-tenant caches) or physical isolation (dedicated pods/namespaces) for high-tier tenants.
• Rate limiting and priority queues: implement per-tenant rate limits and priority queuing in ingress (API Gateway like Kong/Envoy) to protect shared downstreams.
• Circuit breakers & backpressure: add circuit breakers to degrade nonessential features for low-tier tenants when downstreams are saturated.
• Caching: edge and per-tenant caches (CDN or Redis with tenant keys) for hot-read paths.
• Autoscaling with headroom: configure HPA/HVPA using request latency and per-tenant load insights; use node/pod affinity to reserve capacity for premium tenants.
• DB tuning: read replicas, connection pool per tenant, query caching, and tenant-aware throttles.
• QoS enforcement: token-bucket per-tenant tokens representing priority; when exhausted, apply soft throttling or return 429 with Retry-After.

Operationalize

• Create playbooks: steps to triage per-tenant incidents using trace+metrics+logs.
• SLA contracts: map Platinum/Gold/Silver to concrete SLOs, cost/isolation options, and support/penalties.
• Rollout: start with observability (low risk), identify offending tenants/paths, then iterate architecture changes for most impacted tenants.

This combination surfaces root causes quickly (trace+metrics), lets you enforce isolation/prioritization, and ties technical measures to commercial SLA tiers.

Event-sourcing

• Operational complexity: High. Requires designing aggregates, append-only event store (partitioning, retention, snapshots), transactional guarantees, and schema/versioning strategy. Needs tooling for projections and event migrations.
• Replayability: Native — the full history is the source of truth and can be replayed to rebuild any projection or to audit state.
• How consumers build views: Consumers (projections) subscribe to event stream and apply domain events idempotently to build read models; typically run as separate services that maintain materialized views or caches.
• Typical use-cases: Complex domain logic, strong auditability, business workflows where intent/history matters (order lifecycle, dispute resolution, compensation flows).

Change Data Capture (CDC)

• Operational complexity: Moderate. Depends on reliable capture from primary DB (log parsing, connectors like Debezium), schema evolution handling, and exactly-once semantics across systems. Less domain modeling than event-sourcing.
• Replayability: Partial — you can replay captured DB changes if retention exists, but semantics are DB-centric (row-level) and may lack semantic event intent; replaying requires restoring DB or storing change-log.
• How consumers build views: Consumers transform CDC records into denormalized views or events; often used to populate caches/analytics or sync microservices. Consumers must map row images to domain events.
• Typical use-cases: Legacy system integration, replicating authoritative relational state to data warehouses, keeping read models in sync without rearchitecting the write path.

Simple Event Streaming (e.g., Kafka events produced by services)

• Operational complexity: Low-to-moderate. Producers emit events (can be compacted topics), brokers handle retention/partitioning; requires schema registry and producer/consumer discipline for idempotency.
• Replayability: Good if topics are retained/compacted; replay is straightforward per partition/offset, though meaningfulness depends on event design (semantic events vs. telemetry).
• How consumers build views: Consumers subscribe and materialize views from semantic events; multiple consumers can build independent projections; consumer group coordination for parallelism.
• Typical use-cases: Real-time notifications, routing (assignments), telemetry, streaming analytics, integration between microservices where full event history isn’t the authoritative source.

Trade-offs & recommendation

• If you need legal/audit-grade history and business intent is critical -> event-sourcing.
• If migrating or integrating existing relational stores with minimal rewrite -> CDC.
• If you want decoupled real-time integrations with moderate effort -> simple event streaming.
  A common pragmatic architecture for a large delivery platform is hybrid: use event-sourcing or rich domain events for core order lifecycle, CDC to keep analytics/warehouse in sync, and event streaming for integrations, routing, and fan-out — ensure schema registry, idempotency, and replay/runbook practices across all.

A CRDT-based counter (CRDT: Conflict-free Replicated Data Type, a data structure whose replicas can be updated independently and merged with a deterministic rule that always converges to the same value) lets every region increment locally with no coordination, so writes never block and never fail because of a remote outage. A consensus-backed counter (built on Raft or Paxos, where a majority of replicas must durably agree on each state transition before it counts as committed) gives you a single, linearizable value where every acknowledged read reflects every previously acknowledged write, at the cost of needing a live majority and at least one round trip per operation. The concrete thing the CRDT approach gives up is that read guarantee: a client can observe a stale, undercounted value during the window before a remote increment has propagated, with no signal that the value is incomplete, whereas consensus makes that undercounting structurally impossible.

What each approach actually guarantees

Worked example: where the freshness guarantee actually breaks

Take a G-Counter (a grow-only counter CRDT) tracking a global increment total across three regions, A, B, and C, each holding its own local counter; a read sums the counters a replica currently knows about.

1. Start: A = 0, B = 0, C = 0.
2. Region A processes one local increment: A = 1.
3. Region C processes two local increments: C = 2.
4. Before anti-entropy has propagated A's and C's updates to B, a client reads the counter at region B. B's local view is still A = 0, B = 0, C = 0, so the read returns 0, even though three increments are already durably accepted somewhere in the system.
5. An anti-entropy round runs: B receives the vectors {A: 1} and {C: 2}, merges by taking the elementwise maximum, and B's state becomes A = 1, B = 0, C = 2. A read at B now returns 3, the correct eventual total.

No increment was lost between steps 2 and 5, which is the CRDT's core promise. But the read at step 4 was not just a little stale, it was materially undercounted with no indication of that to the caller. A consensus-backed counter cannot produce this outcome, because an increment is not considered committed until a majority has durably logged it, and any successful read after that point is defined to include it.

A related but distinct choice: automatic CRDT merge vs. application-level conflict resolution

The same coordination-free idea shows up one level up in shared-document collaboration, where two users edit the same paragraph while offline. There a third option exists beyond CRDT and consensus: application-level conflict resolution, where conflicting edits are detected and handled by explicit business logic (show both versions to the user, prefer the longer edit, run a custom three-way merge) instead of a mathematically guaranteed merge function. A CRDT gives the same coordination-free availability as the counter case, with a merge that is provably correct for that specific data type; application-level resolution can encode arbitrary rules a generic CRDT cannot express, such as preferring the document owner's edit, but only for data types someone is willing to hand-write a merge function for, and it carries no formal convergence guarantee if that logic has a bug.

Trade-offs and pitfalls

This is not a universal ranking of CRDT versus consensus, it is a question of which correctness property a specific use case cannot live without. A monitoring signal or a rolling feature aggregate feeding a dashboard, or a non-blocking model feature, can absorb a bounded, self-correcting undercount, so a CRDT is the right default: no leader, no quorum loss during a regional outage, cheaper per write. A signal that gates an irreversible action, such as a billing or quota threshold, a fraud rule, or an inventory decrement, cannot absorb that undercount, because the moment a threshold decision fires is exactly the moment staleness turns from cosmetic delay into a correctness bug, so it belongs on a linearizable path even though every write costs more there. A common mistake is defaulting to CRDTs everywhere for their operational simplicity and only discovering the freshness gap when a threshold check fires on stale data; the fix is rarely to abandon the CRDT for everything, but to route the one decision that needs freshness through a linearizable read, or a periodic consensus-backed reconciliation snapshot, while leaving the bulk of the aggregation coordination-free.

Direct answer

Owning a migration that many people depend on is fundamentally a sequencing and risk-management problem, not a technology problem. You inventory what is actually affected before touching anything, prove the new platform can do what the old one does on a small slice before committing the whole system, cut over in a way you can reverse, and you do not call it done until you have actively verified nothing regressed, rather than waiting to see if anyone complains.

Structured elaboration

A migration plan for shared infrastructure has six parts:

1. Inventory. Enumerate every consumer, integration point, and edge-case usage of the current system, not just the documented ones. The single biggest risk in a migration like this is the undocumented dependency nobody remembers until it breaks.
2. Compatibility and parallel-run strategy. Before migrating anything real, prove the new platform matches the old one's behavior on a representative slice, ideally by running old and new side by side on the same input and diffing the output (comparing the two outputs field-by-field to spot any mismatch), so you find divergence before anyone actually depends on the new path.
3. Sequencing and cutover checklist. Migrate the lowest-risk, most self-contained consumer first, not the biggest or most visible one, to build confidence and catch problems cheaply. Each cutover step needs an explicit go or no-go checkpoint tied to a measurable signal, not a calendar date.
4. Rollback plan. For every cutover step, know in advance, before you need it, exactly how you would reverse it and how long that reversal takes. A rollback plan you have not thought through until the moment you need it does not really exist.
5. Communication plan. Dependent teams need to know what is changing, when, what (if anything) they need to do on their end, and how to reach you if something breaks. Give enough lead time that nobody is surprised, and confirm the message landed rather than just broadcasting it.
6. Parity verification. After cutover, actively check that outputs, behavior, and performance match the old system's, on real traffic where possible, rather than waiting for a complaint. Absence of complaints is not evidence of parity.

Worked example

Take migrating a shared internal library used by a dozen services to a new major version with breaking changes.

Inventory turned up 12 documented consumers plus two undocumented ones found by searching for actual usage, one of which called a function everyone assumed was already retired. The parallel-run phase put the new version behind a flag in the three lowest-risk consumers first, running both code paths against the same inputs for two weeks and comparing outputs before trusting the new path alone. The cutover checklist then moved the remaining nine consumers in two more batches, each batch gated on the prior batch showing no divergence for a full week, rather than moving everyone at once. The rollback plan kept the old version available at each step for a window matching that same soak period (the week-long stretch of watching a batch for divergence before trusting it and moving to the next one), so any batch could revert without a redeploy of every consumer. Communication gave each consuming team two weeks of notice, a migration guide, a named point of contact, and one specific action to take, a config flag to flip, and the plan tracked an explicit acknowledgment from each team lead rather than assuming an email had been read. After full cutover, parity verification compared error rate and latency for each consumer against its own pre-migration baseline for two more weeks before the project was closed, since a subtle regression is more likely to look like a slow drift than an outage.

Trade-offs and pitfalls

The most common failure is starting with the most important consumer to "prove it fast," which puts your first, least-proven attempt on the path with the largest blast radius (the widest set of dependent consumers or systems it could break if the attempt goes wrong). A second is treating communication as a one-time announcement instead of a confirmed, two-way acknowledgment, since silence is not the same as consent. A third is skipping parity verification because nothing broke immediately, when many real regressions show up as gradually rising latency or a rare edge case rather than a clean outage. And a rollback plan that exists only on paper, never rehearsed, is close to having no rollback plan at all, because its first real use will be under the worst possible time pressure.

Situation: A Director of Sales repeatedly promises features engineering can't deliver, creating friction, missed commitments, and risk to customer trust.

Coaching script (step-by-step, 1:1 and follow-ups):

1. Prepare: gather facts — specific promises, timelines, engineering capacity, customer impact.
2. First 1:1 (60 mins) — Empathize and align on shared goal.
   • "I value your drive to close deals. I want to ensure we deliver on those wins without risking churn. Can we walk through the specific commitments you made and the customer's must-haves?"
   • Listen, restate commitments, and surface technical constraints.
3. Diagnose: explain technical feasibility plainly — trade-offs, dependencies, delivery timelines.
   • "To deliver X, we need Y engineering cycles or a phased approach; otherwise quality/scale will suffer."
4. Co-create options: present three buyer-facing alternatives (off-the-shelf, phased MVP + roadmap, professional services workaround) with timelines and risks.
   • Ask: "Which option preserves the sale while keeping delivery realistic?"
5. Commit: agree on language for customer conversations and escalation path. Role-play the customer call and craft a “safe” promise script.
   • e.g., "We can commit to delivering core capability A within 90 days, with B on our roadmap for Q3 and regular demos in between."
6. Follow-up: weekly sync for first 8 weeks, review wins, blocked items, and calibration with engineering.

Alignment tactics (team-level):

• Create a predefined “commitment matrix” tying feature categories to approval levels (Director-level can promise configs; anything requiring >2 engineers or >8 weeks needs VP/product sign-off).
• Introduce a lightweight Deal Review (Solutions Architect + Product + Engineering) for high-risk deals before verbal commitments.
• Provide sales with templated, customer-facing commitment language and solution diagrams showing what’s included vs. roadmap.
• Use shared success metrics: win-rate weighted by delivery accuracy and customer satisfaction.

Organizational changes to prevent recurrence:

• Formalize a pre-sales governance process: threshold-based Deal Review board and SLA for technical turnaround (e.g., 48–72 hours for feasibility).
• Empower Solutions Architects as mandatory approvers on Statement of Work / proposal templates for technical deliverables.
• Introduce a Product-Sales playbook mapping common asks to delivery patterns (supported, professional services, roadmap) and estimated time-to-market.
• Create incentives: tie a portion of sales compensation or recognition to “delivery alignment” KPIs (e.g., percentage of deals delivered as promised).
• Invest in a rapid prototype/POC pool (small engineering reserve) to de-risk commitments quickly.

Why this works:

• It preserves sales momentum by offering practical, customer-friendly alternatives.
• It builds trust between Sales and Engineering through structured collaboration and clear approval paths.
• It protects customers and the company from over-commitment while enabling controlled, accountable exceptions when strategically necessary.

Calculation:

• One model = 100 GPU-hours. Five models weekly = 5 × 100 = 500 GPU-hours/week.
• Monthly (4 weeks) ≈ 500 × 4 = 2,000 GPU-hours/month.

Baseline cloud cost (examples):

• If using an NVIDIA A100 at ~$3.00/hr → 2,000 × $3.00 = $6,000/month.
• If using an NVIDIA V100 at ~$2.00/hr → 2,000 × $2.00 = $4,000/month.
  (Use your provider’s SKU prices; these are illustrative.)

Cost‑saving tactics (quantified estimates and tradeoffs)

1. Spot / preemptible GPUs

   • Savings: 60–80% lower hourly price vs on‑demand.
   • Example: A100 spot at ~$0.9/hr → monthly ≈ 2,000 × $0.9 = $1,800 (70% savings).
   • Trade-offs: risk of interruption; require checkpointing and retry logic / orchestration (Kubernetes + KubeFlow, Spot APIs).

2. Mixed precision (FP16 / AMP)

   • Savings: 25–50% reduction in wall‑clock training time (depends on model/IO).
   • Example: 40% speedup → GPU-hours drop from 2,000 to 1,200 → cost falls proportionally.
   • Trade-offs: minor numerical stability checks; possible small changes in convergence.

3. Multi‑tenant / shared training clusters & job packing

   • Savings: 10–25% through higher utilization and fewer idle GPUs.
   • Example: with cluster autoscaler and bin-packing reduce effective GPU-hours by 15%.

4. Efficient training techniques

   • Lower precision optimizers, gradient accumulation, smaller batch early stopping, curriculum training, or transfer learning.
   • Savings: additional 10–30% depending on how much from scratch training you can avoid.

5. Hyperparameter tuning optimization

   • Use multi‑fidelity (Successive Halving, ASHA) or Bayesian search to reduce wasted trials.
   • Savings: can cut tuning cost by 50–80% vs naive grid/random search.

Combined example (realistic stack): Spot (70% off) + mixed precision (40% time reduction) + utilization improvements (15%):

• Effective GPU-hours = 2,000 × 0.6 (mixed precision) × 0.85 (utilization) ≈ 1,020 GPU-hours.
• Cost on spot A100 at $0.9/hr ≈ 1,020 × $0.9 ≈ $918/month vs baseline $6,000 → ~85% total saving.
  (Combination is multiplicative; actual savings depend on failure rate, engineering overhead.)

Recommendation (Solutions Architect view)

• Short term: enable mixed precision and robust checkpointing; switch noncritical runs to spot/preemptible with retry logic.
• Mid term: central shared cluster with autoscaling, job queueing, and tuned HPO pipelines (ASHA/Bandit).
• Measure: instrument per‑job GPU utilization, preemption rate, cost per converged model; iterate.

This approach balances engineering cost, reliability, and the largest levers: pricing (spot) and runtime reduction (mixed precision / smarter experiments).

Direct answer
The choice between synchronous and asynchronous service communication is really a choice about whether the calling service can afford to wait for an answer right now, and getting this wrong in either direction shows up as either a slow, fragile user experience or a system that's needlessly hard to reason about.

Structured elaboration
Trade-offs across the key dimensions:

• Latency: synchronous calls add the full latency of the downstream service directly to the caller's response time; asynchronous decouples the caller's response time from the downstream work, which can complete later.
• Reliability: synchronous calls fail together (if the downstream service is down, the caller's request fails immediately); asynchronous calls (via a durable message queue) can be retried and processed once the downstream service recovers, without the original caller needing to know or retry itself.
• Complexity: synchronous is simpler to reason about (a request either succeeds or fails, immediately, in one place); asynchronous introduces real complexity, including message ordering, duplicate delivery, and the need to communicate eventual results back to whoever's waiting for them.
• Error handling: synchronous errors surface immediately to the caller, who can decide how to respond in the moment; asynchronous errors require an explicit mechanism (a dead-letter queue, a status callback) or they risk failing silently, invisible until someone notices the expected downstream effect never happened.
• Debugging: synchronous call chains are easier to trace (a single request's path through the system is linear); asynchronous flows require distributed tracing across a queue and multiple independent consumers, which is a real operational investment to build well.
• User experience: synchronous is appropriate when the user is actively waiting for the result and needs it to proceed (checking out an order); asynchronous is appropriate when the user doesn't need to wait (an email confirmation, a background report generation), and forcing them to wait synchronously for non-blocking work degrades their experience for no real benefit.

Examples: a payment authorization during checkout should be synchronous, since the user needs an immediate answer to proceed or be told why they can't; a post-purchase email receipt and a recommendation-model retraining triggered by that purchase should be asynchronous, since neither needs to complete before the user's checkout flow finishes.

Validating these choices under load: load-test the synchronous path specifically for how it behaves when the downstream dependency itself slows down under load (does the caller's own capacity get consumed waiting, causing a cascading slowdown), and load-test the asynchronous path for queue depth and processing lag under a sustained high-volume burst, confirming the eventual-completion time stays within an acceptable bound even during peak load, not just under average conditions.

Trade-offs and pitfalls
The most common mistake is defaulting to synchronous calls throughout a system because it's simpler to build initially, creating a fragile chain where a slowdown in any downstream service cascades directly into user-facing latency everywhere upstream. The second common mistake is making something asynchronous that the user is actually waiting for, which just adds indirection and complexity without improving the actual experience, since the user still can't proceed until the async work completes anyway.

Approach: I’d evaluate both options across five dimensions (quantified in a 3–5 year TCO model) then recommend the option that meets regulatory, availability and risk constraints with practical mitigations.

Evaluation checklist

• TCO (3–5 yrs): include license fees, paid support/subscriptions, cloud/infra (instances, storage, networking), backup/DR, migration/transformation, staffing (salaries, training, contractor), high-availability and DR run costs, monitoring, and incident costs (estimated MTTD/MTTR impact). Produce best/likely/worst scenarios.
• Supportability & SLAs: compare vendor SLA (uptime %, credits, escalation, response times, dedicated TAM) vs community support responsiveness; include third-party support options for OSS. Validate on-call rotation impact and runbook maturity.
• Security patching: measure average CVE-to-patch time, vendor patch cadence, backport policy, zero-day response, availability of signed binaries, and automated patch pipelines.
• Licensing & legal risk: review licenses (GPL/AGPL, Apache, commercial), indemnity/no-indemnity, export controls, data residency, supplier risk (single-vendor lock-in), and audit/royalty exposure.
• Staff skills: current team proficiency, hiring market, training time/cost, availability of certified engineers or partners.

Example findings (typical)

• Enterprise DB: higher capex/op-ex for licenses (~30–50% of TCO), strong SLAs (99.95–99.99), vendor indemnity, predictable patches, single-vendor dependency.
• Open-source with commercial support: lower license cost, higher ops effort, potential faster innovation, licensing/legal review needed; commercial support add-on reduces operational risk.

Recommendation (for a regulated financial client)

• Prefer enterprise DB offering if regulatory requirements demand vendor indemnity, strict SLAs, certified compliance attestation (SOC2/ISO), and rapid security support — especially where financial risk from downtime or data exposure is high.
• If cost pressure is primary and requirements allow, adopt OSS with a commercial support contract from a reputable vendor and a hardened architecture.

Principal risks & mitigations

• Risk: Vendor lock-in / high renewal costs -> Mitigate: negotiate exit terms, data export formats, and phased portability exercises; maintain abstraction layer (e.g., data access service).
• Risk: Delayed security patches (OSS ops burden) -> Mitigate: buy commercial support with guaranteed patch SLAs; implement automated patch testing and canary rollout.
• Risk: Skill gaps -> Mitigate: budget for certified training, hire 1–2 senior DB engineers or an external managed service for initial 12–18 months.
• Risk: Compliance gaps -> Mitigate: require vendor attestation, include audit rights in contract, run annual penetration tests and compliance scans.
• Risk: Availability/DR failure -> Mitigate: define RTO/RPO in contract, run quarterly DR drills, multi-AZ/multi-region deployment.

Next steps

1. Build a 3–5 year TCO spreadsheet with scenarios and sensitivities.
2. Run a security/legal license review and vendor reference checks.
3. Pilot the shortlisted option (proof-of-concept) with failover and patching exercise.
4. Negotiate contract terms (SLA, indemnity, exit, support hours) and finalize.

Direct answer

SLI, SLO, and SLA are three layers of the same idea, stated with increasing weight. An SLI (service level indicator) is what you actually measure. An SLO (service level objective) is the internal target you set for that measurement. An SLA (service level agreement) is the external promise, usually contractual, built on top of that target, with consequences if you miss it. In plain terms: the SLI is the speedometer, the SLO is the speed limit you've set for yourself, and the SLA is the speed limit you've promised a customer you won't exceed, with a penalty if you do.

Picking the example and the threshold

• Choose one measurable thing the executive already cares about, not an internal engineering metric they have no context for. "Percent of requests that succeed" beats a raw latency percentile for this audience, because success or failure needs no further explanation.
• State the SLO as a number deliberately below what looks achievable. This is the part executives most often misread: an SLO of 99.9% isn't "we're at 100% and slipping a little," it's a chosen buffer that leaves room to ship changes and absorb normal failures without over-investing in reliability nobody needs.
• The SLA number sits below the SLO, with a consequence attached, and that gap is itself worth explaining: it exists so that missing the internal target doesn't automatically mean breaking a customer promise.
• The same three-layer structure holds outside web services too, whether you're onboarding a new product manager on a team's SLOs for the first time or defining an SLI/SLO for a streaming data pipeline (there the SLI might be how stale the data is, instead of whether a request succeeded), the relationship between the three layers doesn't change, only what's being measured does.

Worked example

Say the team sets it up this way for a checkout API. SLI: percent of checkout requests that return successfully within two seconds. SLO: 99.9% of checkout requests meet that bar, measured over a rolling 30 days, the number engineering is held to internally. SLA: 99.5% of checkout requests meet that bar, measured monthly, written into the enterprise customer contract; falling below it triggers a service credit specified in the contract.

Business consequence of missing the SLA (say the month comes in at 99.3%): this is a contractual breach. The customer is owed the agreed credit, and depending on the contract, may have grounds to escalate or walk away. It's a direct, quantifiable cost and a trust hit that shows up outside engineering entirely.

Business consequence of exceeding the SLO (say the month comes in at 99.97% against a 99.9% target): this isn't a "consequence" in the SLA sense, it's a signal. Consistently beating the SLO by a wide margin means either the target is stale and could absorb more risk (ship faster, take on more ambitious changes), or the team is over-investing effort in reliability the product doesn't need. Either way it's a prompt to revisit the number, not something to report as a win on its own.

Trade-offs and pitfalls

The most common executive misunderstanding is treating the SLO as the promise, when the SLA is the promise and the SLO is the internal cushion above it. Say that gap out loud every time, or the SLO number will get quoted externally by mistake. The second pitfall is picking a metric that's technically correct but means nothing to the audience, an uptime percentage without saying what "down" costs the business, always translate the metric into what the customer actually experiences before attaching a number to it.

Approach: start with clear objectives (sustain 100k RPS bursts up to X minutes, <200ms p95 latency, 99.95% availability) and an architecture diagram mapping bottlenecks (LB, app instances, DB, caches, network). Document assumptions (payload size, read/write mix, session stickiness, TLS overhead) and safety margins (design for 1.5x expected peak).

Load-testing strategy:

• Traffic profile: synthetic bursts (100k RPS sustained for 1, 5, 15 minutes), ramp-up scenarios, steady-state baseline, and mixed real-user replay for request variety.
• Tools & environment: k6/JMeter upstream traffic generators in a pre-prod environment that mirrors prod (same instance types, autoscaling configs, networking); use service virtualization for external dependencies.
• Test phases: capacity discovery (find breaking points), tuning (optimize), validation (confirm with 1.5x burst), chaos tests (node failures, AZ loss).
• Observability: correlate load generators with end-to-end tracing, metrics, and logs.

Key metrics & thresholds to present:

• RPS and concurrency (target 100k RPS; test to 150k)
• Latency p50/p95/p99 (<100ms / <200ms / <500ms)
• Error rate (<0.1% business errors, <1% 5xx)
• CPU/memory utilization per tier (keep <70% at peak), GC pause times
• Throughput (MB/s), network packet drops, connection saturation
• Autoscaler behavior: scale-up time <60s, scale-down with cooldown
• Cost per peak-hour and cost per 1M requests

Capacity vs cost justification to finance:

• Present three options: Over-provisioned (hot spare capacity), Autoscaling (elastic), Hybrid (baseline reserved + burstable on-demand).
• Quantify: monthly cost, expected percent utilization, and risk (chance of SLA breach). Show cost per request and cost of downtime (revenue impact, SLA penalties).
• Recommend baseline reserved capacity to cover typical traffic + autoscale to cover bursts; include throttling/backpressure and queueing policies to protect critical paths.
• Provide sensitivity analysis: cost vs. SLA curves and break-even points; show how caching, CDNs, and read-replicas reduce backend cost dramatically.
• Deliverables: architecture doc, test reports, runbooks (scaling thresholds, failover), and an executive one-pager: recommended plan, costs, risks, and contingency.