System Design Methodology and Trade-off Analysis Questions

Direct answer

Before sketching anything, I separate three questions: who is this for and what must it do (functional scope), what quality bar does it have to hit (non-functional requirements like scale, latency, and compliance), and what am I explicitly choosing to leave out for this iteration. I get there by asking a short list of targeted questions, writing down the assumptions I have to make when answers aren't available yet, and drawing an explicit line between what ships now and what's deferred, instead of letting scope grow implicitly as the conversation continues.

Structured elaboration

A repeatable order of operations

1. Clarify the primary user and the one core job the service must do for them.
2. Ask about scale and growth (expected load today, expected growth rate, read-versus-write ratio), because these numbers, not taste, determine how much architecture is actually warranted.
3. Ask about non-negotiable constraints: compliance obligations, systems it must integrate with, budget, deadline.
4. Ask what's allowed to degrade: is a few seconds of staleness acceptable, is brief downtime during a deploy acceptable, does every read need to be exact.
5. State assumptions explicitly wherever a real answer isn't available yet, and mark them as assumptions to validate, not facts to build on silently.
6. Draw the scope line: list primary use cases that must ship, and secondary or deferred use cases that are explicitly out of scope for this iteration, written down so nobody discovers the gap later.

The judgment underneath the checklist
A senior candidate treats every "yes, and also" as a scope decision with a cost, not a free addition, and pushes back on a vague ask like "make it fast" by translating it into a testable target before designing a single component, which is the same move a strong answer makes when a client says a product must "feel fast" for users worldwide.

Worked example

Take the one-line prompt "design a URL shortener." Before sketching components, I'd ask: how many new links are created per day, and what's the read (redirect) to write (creation) ratio? Suppose the answer is 10,000 new links/day with a 100:1 read-to-write ratio, typical of a link-sharing product:

$\text{redirects/day} = 10{,}000 \times 100 = 1{,}000{,}000$

$\text{avg redirect RPS (requests per second)} = \frac{1{,}000{,}000}{86{,}400} \approx 11.6 \text{ req/s}$

That single clarifying question, the read-to-write ratio, turned a vague prompt into a concrete, low-single-digit-RPS system, which tells me this is a read-heavy, cache-friendly problem, not a write-scaling problem, before a single box has been drawn. If the interviewer instead says the product is a bulk-import tool with a roughly 1:1 read-to-write ratio, the answer to nearly every later design question changes, which is the point: the clarifying question, not the diagram, is where the real design decision happens.

Scope line for this example: in scope for a first version is create-and-redirect with a randomly generated short code. Explicitly out of scope for the first version, stated to the interviewer rather than silently dropped, are custom vanity aliases, click analytics, and link expiration, each a real feature with its own cost that can be added once the core path is validated.

Trade-offs & pitfalls

• Designing before scoping: sketching a box diagram before knowing the read-to-write ratio, scale, or constraints wastes limited interview time on a shape that may not fit the real problem.
• Silently assuming numbers instead of stating them, so a listener can't tell you're reasoning from an assumption rather than a fact.
• Treating scope-cutting as a failure rather than a design decision; a strong candidate narrates what they are choosing not to build and why, instead of trying to design everything at once.
• Requirements-gathering theater: asking a long, generic checklist of questions instead of the two or three that would actually change the design.

Direct answer

Turn "handle spikes" into numbers by asking for the spike multiplier over baseline, its duration and arrival shape, the peak concurrency it implies, and what is allowed to degrade versus what must stay within the service-level agreement (SLA) during it. Those four answers are what actually let you size autoscaling, connection pools, and a degradation plan; without them, "handle spikes" is a feeling, not a requirement.

Structured elaboration

The four questions that make it measurable

The general skill, applied to a different vague ask

The same discipline works on any vague requirement, not just traffic spikes. "Handle a fifteen-year-old legacy system with no APIs" is exactly as unmeasurable until you ask the analogous questions: what data-access surfaces actually exist (direct database reads, nightly file exports, screen automation), who owns changes to that system, what staleness is tolerable in whatever gets extracted, and what happens to your system if that legacy system goes down for a day. "No APIs" becomes a concrete integration contract the same way "handle spikes" becomes a concrete capacity contract, by naming the constraint that changes the design instead of accepting the vague label.

Worked example: turning "5x for ten minutes" into a server count

Assume measured baseline steady-state traffic of 1,000 requests per second (RPS), and product says the spike is "5x for about ten minutes." Assume each server instance safely handles 200 RPS at target latency:

$$\text{baseline servers} = \frac{1{,}000}{200} = 5$$ $$\text{spike RPS} = 5 \times 1{,}000 = 5{,}000$$ $$\text{spike servers needed} = \frac{5{,}000}{200} = 25$$

Now check whether autoscaling can even react in time. Assume it takes 3 minutes from scale-out trigger to a new instance serving traffic:

$$\text{spike duration (10 min)} > \text{scale-out reaction time (3 min)}$$

Autoscaling alone is workable here, with roughly 3 minutes of degraded capacity at the start of the spike. If the same 5x spike instead lasted 60 seconds (a flash-crowd shape rather than a sustained one), the 3-minute scale-out reaction time would exceed the entire spike duration, and the only real fix is pre-warmed standby capacity, not faster autoscaling. That is why duration and arrival shape change the design, not just the multiplier.

Trade-offs & pitfalls

• Pitfall: designing for "handle any spike" instead of a bounded one. Every system has a ceiling; the point of these questions is choosing it deliberately instead of discovering it during an incident.
• Pitfall: assuming autoscaling reaction time is negligible. If it is not faster than the spike itself, pre-provisioned headroom is needed, which costs money sitting idle.
• Graceful degradation (returning cached or partial results, shedding low-priority requests) is usually cheaper than provisioning for the absolute peak, but only if product has said which features are allowed to degrade.

Direct answer

Score each non-functional requirement (NFR, a quality attribute like security, maintainability, or observability rather than a feature) by the risk of skipping it, not by how important it sounds in the abstract, then fund the highest-scoring ones first and consciously document what you are deferring. In this scenario that usually means security and enough observability to see when something breaks get funded first, while maintainability work (broad refactors, exhaustive test coverage) is the one to accept debt on, because a small team can still move fast without it in the short term, while an invisible security or reliability gap can end the project.

Structured elaboration

A repeatable scoring rule

Score each candidate NFR on impact, likelihood, and effort:

$$\text{risk score} = \frac{\text{impact} \times \text{likelihood}}{\text{effort}}$$

where impact and likelihood are rated on a small scale, say 1 to 5 (illustrative severity ratings calibrated with the team) and effort is the cost to address it now. Rank by score, fund top-down until the budget runs out, and document what falls below the line and why.

Worked example (the three from the question)

Assume illustrative ratings for a client project on a tight timeline:

By this scoring, observability actually ranks first here, cheap and high odds you'll need it fast when something breaks. Security ranks second, highest impact and worth the extra effort. Maintainability ranks last, which is the one to consciously under-invest in: ship with a thinner test suite and postpone larger refactors, but only after writing down that decision so it is a choice, not an accident.

Defending the deferred one

Under-investing in maintainability is defensible specifically because its failure mode is slow (code gets harder to change over months) rather than sudden (unlike a security breach or a blind outage), and because a small team on a tight timeline has not yet hit the coordination cost that makes poor maintainability expensive. Conway's Law (a system's structure tends to mirror the communication structure of the team that built it) means that cost shows up later, once more people touch the same code, which is exactly when the decision should be revisited.

Extension (absorbed angle): the same rubric on six NFRs under a revenue constraint

Given six candidate NFRs for a new API (availability, latency, security, observability, maintainability, scalability) and a fixed budget, weight impact by revenue at risk instead of a generic scale, then rank the same way:

The mechanics are identical to the three-NFR case: rank by risk per unit of effort, fund down the list, write down what was deferred and why.

Trade-offs & pitfalls

• Pitfall: treating this as "pick two of three" instead of a continuous funding line; you can partially fund all three (a minimal security baseline plus basic dashboards plus a lighter test suite) rather than fully skipping one.
• Pitfall: scoring by gut feeling instead of writing the numbers down; the value of the rubric is that it survives being questioned by a stakeholder later.
• What changes the ranking: a prior incident (raises likelihood), a compliance requirement (raises impact on security specifically), or a known team-scaling event on the horizon (raises maintainability's score because the Conway's Law cost is about to arrive).
• Under-investing is not the same as ignoring: document the gap, set a revisit trigger (a metric or a milestone), and make sure whoever inherits the debt knows it exists.

Direct answer

Translate "feels fast" into measurable, percentile-based service-level objectives (SLOs, the internal targets a team designs to) broken out by user geography and device class, because a single global average latency number hides the users who are actually having a bad experience. Concretely: pick a small set of user-perceived timing metrics, set targets for the 95th and 99th percentile (P95/P99), not just the median, and set different targets per region, since physics, not engineering effort, sets a latency floor for users far from the servers.

Structured elaboration

Why percentiles, not averages
The median (P50) reflects the typical user; P95 and P99 reflect the users who are actually complaining, and those are the ones a business should worry about losing.

Candidate user-perceived metrics (standard web-performance terms, named here without inventing a universal target for each, since the right target is a product decision):

• Time to First Byte (TTFB): how long until the server starts responding.
• First Contentful Paint (FCP): how long until something appears on screen.
• Time to Interactive (TTI): how long until the page actually responds to input.

Segmentation

• By region: a request served from a single origin has a very different latency floor depending on how far the user is from that origin (worked example below).
• By device and network class: a phone on a mobile network experiences different bandwidth and queuing behavior than a laptop on a wired connection; the specifics of that are their own topic, but the targets should differ, not share one number.

From target to commitment
An SLO is the internal target a team designs to; a service-level agreement (SLA) is the external, often contractual, promise made to a customer. The SLA should sit inside the SLO with room to spare (an error budget: the amount of time the SLO is allowed to be missed before it counts as a real problem), otherwise there is no margin for a bad day.

Worked example

Physics sets a hard floor before any engineering happens. Light in fiber travels at roughly 200,000 km/s (about two-thirds the speed of light in vacuum, due to the refractive index of glass). If a user in Mumbai is served from a single origin server in Virginia, the one-way great-circle distance is roughly 12,000 km:

$t_{one\text{-}way} = \frac{d}{v} = \frac{12{,}000 \text{ km}}{200{,}000 \text{ km/s}} = 0.06 \text{ s} = 60 \text{ ms}$

$\text{RTT}_{min} = 2 \times t_{one\text{-}way} = 120 \text{ ms}$

That is the theoretical best case for one round trip before the server does any work at all, and a real page load needs several round trips (DNS lookup, then a TCP/TLS handshake, then the actual request), so a single-origin design cannot hit an aggressive global P95 no matter how fast the backend code is. This is the concrete argument for a content delivery network (CDN, a network of edge servers that cache content closer to users) or a multi-region deployment: it is not a nice-to-have, it is the only way to shrink the distance term in the equation above for users far from wherever the service is deployed.

Trade-offs & pitfalls

• Setting one global latency target and being surprised it's missed for distant regions; the fix is a region-aware target, not "optimize the backend more."
• Optimizing for the average and declaring victory while P95/P99, and the users behind them, stay slow.
• Promising an SLA as tight as the internal SLO, leaving no error budget for a bad day.
• The cost trade-off worth naming explicitly: hitting a tight worldwide P95 costs real money (CDN, edge compute, multi-region infrastructure and replication). "How fast" is really "how much are we willing to spend to move the physical floor closer to zero," and that should be a deliberate decision, not an assumed one.