System Design Methodology and Trade-off Analysis Questions

Direct answer

Convert the calls whose result the client-facing response does not actually need into async, queue-backed steps, and keep only the calls that determine what you tell the client (an authorization decision, a price, a reservation outcome) on the synchronous path. What you give up is immediacy for the deferred steps (the caller no longer knows they succeeded before the response returns) and simple error handling (you now need retries, idempotency, and a plan for a step that fails after you already told the client it succeeded).

Structured elaboration

How to pick decoupling candidates

For each hop in the chain, ask, in order:

1. Does the client's response body or status depend on this call's result? If no, it is a decoupling candidate.
2. Is it only on the critical path because of implementation order, not because it is logically required before responding (sending a confirmation email after an order is placed is the classic case)? Decouple it.
3. Can the caller tolerate this step failing and retrying later without the user noticing? If yes, move it behind a queue with at-least-once delivery and an idempotency key so a retry cannot double-apply the effect.
4. If it must run after you've already told the client the request succeeded, what compensates if it fails? Sagas and compensating-transaction patterns are the standard answer here (a Saga: a sequence of local transactions where each step has a paired undo action that runs if a later step fails, so you get a rollback without a distributed transaction); treat them as a named sibling mechanism rather than re-deriving them.

What you give up, named explicitly

• Immediacy: the client no longer gets confirmation that the deferred step (email sent, loyalty points applied, analytics recorded) actually happened; if the product needs that confirmation, either keep the step synchronous or change the experience to a pending state.
• Simple error handling: a synchronous chain fails loudly and immediately; an async step fails quietly somewhere else, later, and needs monitoring (consumer lag, dead-letter queue depth) to even notice.
• Ordering: once two steps are decoupled, the free ordering guarantee a sequential call chain gave you for nothing is gone; if two async steps can race, an explicit ordering key or a saga is needed to keep them coherent.

The one thing not to decouple just to hit the number

Do not move a correctness-critical write (a payment capture, an inventory decrement, a seat reservation) to async purely to shave latency. That trades correctness for speed: the client sees a fast success response for something that has not actually been secured yet, and overselling or double-charging becomes an incident instead of a design decision.

Worked example: latency-budget arithmetic

Assume today's chain and its 95th-percentile (P95) latencies, all sequential: 10 ms gateway, 50 ms auth check, 120 ms inventory check, 80 ms pricing calculation, 300 ms fulfillment-order creation, 150 ms confirmation-email send, 90 ms audit-log write.

$$\text{current P95} = 10+50+120+80+300+150+90 = 800 \text{ ms}$$

If the service-level objective (SLO) is P95 at or under 700 ms, that is 100 ms over budget. The email send and the audit-log write are both decoupling candidates by the test above, since the client's response does not need either to have completed:

$$\text{after decoupling} = 10+50+120+80+300 = 560 \text{ ms}$$

That clears the 700 ms budget with 140 ms of headroom, without touching the correctness-critical inventory check or the fulfillment write.

Worked example: the absorbed booking-system angle

A synchronous seat-booking monolith migrating to event-driven has the same one thing it must not decouple: the seat reservation itself. Keep "reserve the seat" synchronous, using an atomic decrement or a compare-and-swap style check so two concurrent bookings cannot both win the same seat, which is exactly the double-booking risk the migration has to guard against. Move "send the confirmation email," "credit loyalty points," and "sync to the analytics warehouse" behind a queue. If payment fails after the seat was reserved, that is a compensating action (release the hold), not a reason to make the reservation itself asynchronous.

Trade-offs & pitfalls

• New failure mode: a message that fails repeatedly needs a dead-letter queue (DLQ) and an owner who actually looks at it, or side effects silently disappear.
• New monitoring surface: queue and consumer lag become a latency input in their own right; if the queue backs up, "async" steps can end up more stale than the SLO tolerates even though they are off the synchronous critical path.
• Pitfall: decoupling a call because it is slow rather than because its result is unneeded. If the client genuinely needs the answer, moving it to async just hides the latency problem behind a pending-state experience instead of solving it.

Direct answer

A high-level architecture shows the system's scope: the major building blocks (client, API layer, service tier, datastore, cache, external dependencies), how they relate, and the non-functional constraints (scale, availability) that shaped them. A component-level design zooms into one of those blocks and specifies its interfaces, request/response schemas, data flows, and sequencing. You show the high-level view to stakeholders who need to understand what the system is and what it costs or risks; you show component-level design to the people who have to build, test, or integrate against one specific piece.

Structured elaboration

The reason both layers matter: the high-level view sets the strategy and the constraints everyone else has to work inside; the component-level view is what actually gets implemented, tested, and integrated. A good design doc keeps an explicit mapping from each high-level block down to its component-level detail, so a reviewer can move between the two without re-deriving context.

Worked example

Say you're designing a subscription billing feature. At the high level you'd draw: client apps, an API gateway, a billing service, a payments component, a database, and a message queue for async notifications, with an arrow showing the billing service calls out to a third-party payment processor. The one decision that belongs only at this level: whether billing lives inside the existing monolith or is split into its own service, because that choice affects deployment, on-call ownership, and the blast radius of an incident, not just this one feature.

At the component level, you'd zoom into just the billing service and produce: a sequence diagram for "create subscription" (client → billing service → payments component → processor → database write → event published), the exact request/response schema for the POST /subscriptions endpoint, and an entity-relationship diagram for the subscription and invoice tables. None of that detail belongs on the high-level diagram; it would bury the one decision (monolith vs separate service) that the high-level view exists to surface.

Trade-offs & pitfalls

• Showing component-level detail (full schemas, every retry path) to an executive or product stakeholder buries the one decision they actually need to weigh in on.
• Skipping the high-level view and jumping straight to component design risks locking in a boundary (a shared database, a synchronous call where an event would do) that is expensive to undo later, because it was never surfaced as a decision.
• A common weak answer just says "high-level is the big picture, low-level is the details" without naming a decision that is exclusive to one level; naming that decision is the signal an interviewer is listening for.
• Keep a living link between the two artifacts (a component-level design should reference which high-level block it belongs to) so the documentation doesn't drift apart as the system evolves.

Direct answer

Choose a relational database with a flexible-attribute column, for example PostgreSQL with a JSONB (binary JSON) column, rather than a pure document store. The workload's defining features, frequent reads, moderate writes, and occasional complex queries with joins across related entities, are exactly what a relational engine with atomicity-consistency-isolation-durability (ACID) transactions and a real query planner are built for. A document store would force those occasional joins to be rebuilt in application code, which is a worse trade than tolerating a bit more schema rigidity for the flexible fields.

Structured elaboration

Decision criteria to walk through: how often do "occasional" joins actually occur in practice (if frequent, this favors relational strongly); how correctness-sensitive is the data (account state favors strong transactional guarantees); how much of the schema is genuinely unpredictable versus a fixed set of optional fields (JSONB handles the latter well without needing a schema-less engine).

Worked example

A concrete schema: core relational columns, user_id (primary key), email, status, created_at, with foreign-key relationships to organizations and permissions tables to support the join-heavy queries (for example, "list all users in an organization with a given permission"). A JSONB attributes column holds optional or evolving profile fields, indexed with a generalized inverted index (GIN) for filtering on specific attribute keys without requiring a migration every time a new optional field is added.

Decision branch: if the write volume for this profile service later grows to a level one primary node can no longer sustain, that crosses into write-heavy datastore territory (partitioned, write-optimized storage engines) and would call for revisiting this choice; "moderate writes" as stated in this scenario doesn't cross that line, so the relational-plus-JSONB design holds.

Trade-offs & pitfalls

• Choosing a document database by default because "user profile" sounds document-shaped, then discovering the "occasional" joins aren't so occasional in practice, and rebuilding relational logic in application code, is a common wrong turn.
• Over-normalizing the flexible attributes into their own relational tables, when a JSONB column with a targeted index would have been simpler and equally queryable for the actual filter patterns, adds unnecessary schema churn.
• If writes were instead high-concurrency across many independent keys with no cross-record transactions needed, a document or wide-column store would flip this recommendation, this decision is shaped by the workload, not a permanent rule.
• A field that becomes a genuine business invariant (something the system must enforce, not just store) should graduate from the flexible column into a real, constrained relational column, leaving it in JSONB indefinitely trades away the very guarantees the relational choice was made for.

Direct answer

Break the estimate into three buckets, compute, storage, and network egress, state a small number of explicit assumptions for each (traffic shape, cache hit ratio, unit prices), and multiply through. For this workload, 1,000,000 requests/day and 10 TB of outbound data/month, the arithmetic below lands around 1,246 dollars/month using illustrative unit rates, with network egress the dominant line item. The one sanity check worth doing at the end is dividing total outbound data by total requests: it implies each request carries roughly 333 KB on average, which is large for a "simple web app" and should prompt asking whether the two given numbers actually describe the same traffic.

Structured elaboration

Why three buckets, always kept separate
Compute, storage, and network egress scale with different things (request rate, data volume at rest, data volume transferred), so lumping them together hides which one actually drives the bill. For a workload described mainly by a request count and an outbound-data figure, network egress is very often the surprise line item, since it scales with bytes moved, not with request count.

Stated assumptions (illustrative unit rates, not any specific vendor's current list price, so the arithmetic below is fully reproducible from these inputs alone):

• 1 TB = 1,000 GB for this estimate (a decimal convention, kept simple and stated once; real billing sometimes uses the binary definition instead).
• A 5x peak-to-average traffic ratio, a common assumption absent a stated diurnal profile.
• Compute: $0.10 per instance-hour; a minimum of 3 instances regardless of load, for basic redundancy and zero-downtime deploys.
• Storage: a flat $20/month (small, since a "simple web app" is not primarily a database- and storage-heavy workload).
• Network: 60% of requests served from a content delivery network (CDN) cache; origin egress (cache misses only) at $0.09/GB; CDN edge egress (all bytes delivered to users) at $0.06/GB.
• Miscellaneous (load balancer, monitoring, DNS): $50/month.

Worked example

Compute.

$\text{avg RPS (requests per second)} = \frac{1{,}000{,}000}{86{,}400} \approx 11.6, \qquad \text{peak RPS} \approx 11.6 \times 5 \approx 58$

Even one modest instance clears 58 req/s comfortably for a typical stateless web app, so the instance count here is driven by redundancy, not raw capacity: 3 instances.

$\text{compute} = 3 \times \$0.10/\text{hr} \times 24 \times 30 = \$216/\text{month}$

Network egress, the dominant cost for this workload. Two separate legs, both real: the cache-miss traffic the CDN pulls from the origin (pricier), and the full volume the CDN delivers to end users regardless of hit or miss (cheaper, volume-discounted).

$\text{miss traffic} = 10{,}000 \text{ GB} \times (1 - 0.6) = 4{,}000 \text{ GB}$

$\text{origin cost} = 4{,}000 \text{ GB} \times \$0.09/\text{GB} = \$360$

$\text{CDN cost (all delivered bytes)} = 10{,}000 \text{ GB} \times \$0.06/\text{GB} = \$600$

Total.

$\text{total} \approx \$216 + \$20 + \$360 + \$600 + \$50 = \$1{,}246/\text{month}$

Sanity check, the part the question explicitly asks for. Cross-check the two given numbers against each other, not just against the chosen unit prices:

$\text{requests/month} = 1{,}000{,}000 \times 30 = 30{,}000{,}000$

$\text{avg payload} = \frac{10{,}000 \text{ GB} \times 1000 \text{ MB/GB}}{30{,}000{,}000} \approx 0.33 \text{ MB} \approx 333 \text{ KB per request}$

A typical JSON API response is a few KB, not a third of a megabyte. A 333 KB average suggests this "simple web app" is actually serving images, downloads, or media, not just API calls, or that the two input numbers don't describe the same traffic, for instance if the 10 TB includes a batch export job outside the 1,000,000 daily request count. That is the real value of this sanity check: it isn't re-verifying the arithmetic, it's confronting whether the two numbers handed to you are internally consistent with the story you were told, before handing a stakeholder a dollar figure built on an unstated contradiction.

Trade-offs & pitfalls

• Pricing only one leg of egress (origin-to-CDN or CDN-to-user) and treating the network line item as done.
• Skipping the sanity check and presenting the total as precise when the underlying assumptions (cache hit ratio, peak ratio) were guesses; state which input the total is most sensitive to.
• Sizing compute purely off average load; for small-to-medium workloads, redundancy and deploy safety often set the compute line, not raw throughput math.
• What separates a senior answer: showing the arithmetic and stating which two given numbers were cross-checked at the end, and why, rather than presenting a total as if it fell out of a spreadsheet with no further scrutiny.

Direct answer

Vertical scaling means giving one machine more resources, more vCPU, more RAM; horizontal scaling means adding more machines and spreading load across them. Vertical scaling is simpler because the application doesn't have to change at all, but it has a hard ceiling, the biggest machine available, and concentrates failure in one box. Horizontal scaling has effectively no ceiling and shrinks the blast radius of any single machine failing, but it requires the application to tolerate running as multiple, coordinated instances rather than one.

Structured elaboration

Concrete examples

• Vertical: moving a relational database from a 4-vCPU/16 GB box to a 16-vCPU/128 GB box, with no code change required.
• Horizontal: adding more stateless web server instances behind a load balancer, whose internal balancing algorithm is its own topic and not re-derived here.

Failure domain
Vertical scaling keeps a single point of failure regardless of how big the box gets. Horizontal scaling spreads risk, but only if the application is actually stateless or its state is externalized to a shared cache or session store; otherwise "horizontal" is cosmetic, since any one instance still holds state nothing else can serve.

Cost is not just a resilience argument
Cloud pricing for a single bigger machine is usually not linear with its capacity; the largest instance sizes often carry a premium. That is a computable, not just intuitive, reason horizontal scaling is often cheaper at a given capacity level, not only more resilient (worked example below).

A practical default
Default to horizontal for anything stateless and customer-facing. Reach for vertical only for components that are genuinely hard to distribute, a single-writer relational database primary, some legacy single-threaded systems, and even then pair it with a replication and failover plan, since vertical scaling alone still leaves a single point of failure.

Worked example

Say a 4-vCPU instance costs $0.20/hour and a 32-vCPU instance, 8 times the vCPUs, costs $2.40/hour, 12 times the price, a common cloud pattern where the largest instance sizes carry a premium rather than a linear price-per-vCPU:

$\frac{\$0.20/\text{hr}}{4 \text{ vCPU}} = \$0.05 \text{ per vCPU-hour (small instance)}, \qquad \frac{\$2.40/\text{hr}}{32 \text{ vCPU}} = \$0.075 \text{ per vCPU-hour (large instance)}$

To cover 100 vCPU-equivalents of load, the same figure from the instance-sizing question above:

$\left\lceil \frac{100}{4} \right\rceil = 25 \text{ small instances} \times \$0.20/\text{hr} = \$5.00/\text{hr (horizontal)}$

$\left\lceil \frac{100}{32} \right\rceil = 4 \text{ large instances} \times \$2.40/\text{hr} = \$9.60/\text{hr (vertical, chunkier boxes)}$

Scaling out with many small instances here is essentially half the cost of the same capacity in a few large boxes, purely from the assumed per-vCPU pricing premium at the top end, and it spreads that 100 vCPU-equivalents of load across 25 failure domains instead of 4. This is illustrative pricing, not any specific vendor's current rate, but the shape, larger instances costing more per unit of capacity, is common enough to be worth checking for any real vendor before assuming vertical scaling is free of a cost penalty beyond its availability risk.

Trade-offs & pitfalls

• Assuming vertical scaling is always simpler and cheaper because it "requires no code changes"; it can cost more per unit of capacity and still leave a single point of failure.
• Scaling horizontally without externalizing state first, sessions or in-memory caches tied to one instance, producing a fleet of instances that isn't actually interchangeable and defeats the point.
• Not having a plan for vertical scaling's eventual ceiling; if a component structurally can't be distributed, that is a standing operational risk worth naming explicitly, not a decision to defer indefinitely.
• What separates a senior answer: bringing in the cost-per-unit-of-capacity angle alongside the resilience angle, since stakeholders outside engineering often hear only the failure-domain argument and miss that horizontal can also be the cheaper option.