System Design Methodology and Trade-off Analysis Questions

Direct answer

Decide per field with a simple test: what does a user or the business lose if this field is read stale for a few seconds, and does that loss involve authorization, money, or identity? Email, password, and account status gate who can act as whom, so they get a linearizable (single, globally agreed order) read/write path even at a latency cost. Display name and avatar are cosmetic: a stale value for a few seconds costs nothing but a visual blip, so they get eventual, region-local, low-latency writes and reads. Defending the split means showing what making everything strong actually costs on the read path, not just asserting that it is safer.

Structured elaboration

Per-field decision table

Mechanism

This paragraph is implementation detail, useful to know by name but not required to follow the field-by-field argument made above it. Two logical stores per user: a small, strongly-consistent store (consensus-replicated, for example a Raft-based database, where Raft is an algorithm that gets a cluster of replicas to agree on the same order of writes, or a globally-consistent database) for the identity-critical fields, and a multi-region, eventually-consistent store (Dynamo-style or similar) for everything else. Reads compose a single user object from both stores, so only the strong-store portion pays the cross-region latency cost. Read-after-write for the strong fields comes from routing that specific read to the writer's region or the current leader; monotonic reads (once a client has seen a value, a later read never shows it an older one) for the weak fields come from a session token, not from the strong store.

Extending the framework: billing correctness without going fully strong

Billing state is the case that tempts people into "just make everything strong." Resist it: instead of a synchronous global commit for every billing event, use compensating transactions, an idempotent charge (safe to run the same charge request twice, say after a retry, without actually billing the customer twice) plus a defined reversal or refund path if a downstream step (fraud check, inventory hold) fails after the charge already happened. This gets you correctness (the ledger is right once reconciliation finishes) without paying the linearizable-everything latency tax on a field written far less often than it is read.

Defending the split with a number, not an opinion

The strongest defense against "why not just make it all strong" is quantifying what "all strong" costs on the read path, since profile reads vastly outnumber profile writes.

Worked example

Assume a region-local cache read costs 5 ms, and a linearizable read from the strong store (contacting a majority of replicas across 3 regions, with an illustrative one-way inter-region round-trip time (RTT) of 100 ms) costs roughly two one-way trips:

$$\text{strong-store read latency} \approx 2 \times 100 \text{ ms} = 200 \text{ ms}$$ $$\text{latency multiplier if every read used the strong path} = \frac{200 \text{ ms}}{5 \text{ ms}} = 40\times$$

If, say, 95% of profile reads only ever touch display-name or avatar fields (illustrative traffic mix, would come from real access logs), forcing all of them through the strong store means 95% of read traffic pays a 40x latency tax for a guarantee only the remaining 5% of fields ever needed. That is the number to put in front of someone asking why you didn't make everything strongly consistent.

Now the revenue-risk quantification (the second absorbed angle): the case for still investing in correctness on the billing fields, even though they don't get the fully linearizable treatment either.

$$\text{assumed error rate on a race-prone billing path} = 0.1\% = 0.001$$ $$\text{assumed volume} = 200{,}000 \text{ billing transactions/day at average value } \$50$$ $$\text{expected daily exposure} = 200{,}000 \times 0.001 \times 50 = \$10{,}000/\text{day}$$

Ten thousand dollars a day of exposure (illustrative; in practice pulled from real incident and error-rate data) is what justifies spending engineering time on compensating transactions for billing.

Trade-offs & pitfalls

• The strong store becomes a small, high-value target: shard it narrowly (identity fields only) so its lower throughput ceiling never becomes the bottleneck.
• Session tokens that carry the last-seen strong-store commit are what give read-your-own-writes on the critical fields without every read hitting the leader; skipping this is a common miss that reintroduces stale-password bugs.
• Pitfall: treating "eventual consistency" as a synonym for "no correctness work needed." The weak store still needs a conflict-resolution rule (last-writer-wins or a merge function), or two concurrent display-name edits silently lose one.
• Pitfall: treating billing as either fully strong or fully eventual instead of reaching for the third option, compensating transactions, which is usually the right cost and correctness balance for money-adjacent but not identity-adjacent fields.

Direct answer

Structure the mapping as a matrix: one row per requirement, columns for the target metric, the component(s) that satisfy it, and what you gave up to get there. Walking three rows for this payment subsystem: 99.99% availability drives multi-availability-zone (multi-AZ) redundancy at the cost of doubled infrastructure and failover complexity; sub-200ms p95 (95th-percentile) authorize latency drives a token cache and dedicated crypto hardware at the cost of extra compute spend; and PCI-DSS (Payment Card Industry Data Security Standard) plus 7-year retention drives tokenization and immutable long-term storage at the cost of losing raw-card analytics fidelity and paying for years of storage.

Structured elaboration

Use a table with these columns for every requirement in the list:

This format forces every requirement to land on a concrete component and a concrete cost, rather than staying as an aspiration in a requirements document. It also makes conflicts visible: if two rows both compete for the same fixed budget, that surfaces in the table instead of being discovered mid-build.

Worked example

Three rows from the matrix, with the underlying arithmetic shown:

Row 1: 99.99% availability. A 99.99% target permits:
$\text{allowed downtime/year} = (1 - 0.9999) \times 365 \times 24 \times 60 \text{ min} = 52.56 \text{ min/year}$
Component: the authorize API runs multi-AZ with automated failover rather than a single instance. Gave up: roughly double the compute footprint (active-active or hot-standby) plus the operational cost of regularly testing failover, in exchange for that 52.56-minute annual downtime budget instead of the far larger downtime a single-AZ deployment would risk.

Row 2: sub-200ms p95 authorize latency. An illustrative latency budget that sums to the target:
$20\text{ms (network)} + 30\text{ms (tokenize/HSM)} + 50\text{ms (fraud rules)} + 20\text{ms (cache read)} + 60\text{ms (network to processor)} + 20\text{ms (buffer)} = 200\text{ms}$
Component: an in-memory cache for token lookups and a hardware security module (HSM) colocated with the authorize path, rather than a network round trip to a shared crypto service. Gave up: dedicated cache and HSM capacity that sits idle outside peak hours, which is more expensive per request than a shared pool would be.

Row 3: PCI-DSS plus 7-year retention. Assume, as illustrative pinned inputs, 1 million transactions/day and a 2 KB (kilobyte) retained metadata record per transaction (tokenized, not raw card data):
$\text{bytes/day} = 1{,}000{,}000 \times 2\text{KB} = 2{,}048{,}000{,}000 \text{ bytes} \approx 2.05\text{ GB/day}$
$\text{total (7yr)} = 2.05\text{ GB/day} \times 365.25 \times 7 \text{ days} \approx 5{,}236\text{ GB} \approx 5.2\text{ TB}$
Component: a tokenization service so raw card numbers never enter long-term storage, plus write-once immutable object storage for the 5.2 TB of retained metadata. Gave up: the ability to run ad hoc analytics on raw card attributes, since only tokens and derived fields are retained.

Trade-offs & pitfalls

• The fixed monthly budget row is where the other three collide: if multi-AZ plus dedicated cache/HSM plus 7 years of immutable storage exceeds the budget, something has to re-scope, not silently degrade in production.
• A weak answer lists components without naming what was given up; the "what you gave up" column is the actual trade-off-analysis signal, not the component list itself.
• Treat compliance requirements (PCI-DSS, retention) as filters applied before cost optimization, not something to negotiate down after the architecture is built.
• Revisit the matrix at each design review; a requirement's target or its owning component can shift as the system evolves, and a stale matrix gives false confidence.

Direct answer

A genuinely non-negotiable constraint (a legal, regulatory, or security requirement with no waiver path) changes trade-off analysis from optimizing across all options to first pruning the option set down to only what's compliant, and only then optimizing cost, performance, or time-to-market among what's left. It doesn't get a weight in a scoring matrix alongside other factors; it eliminates options before scoring starts.

Structured elaboration

Treat a hard constraint as a filter applied in a distinct first pass, before any cost or performance comparison: list every candidate architecture, remove any that violate the constraint outright (not "weight them lower", remove them), and only run the normal trade-off analysis (cost, latency, time-to-market) across what survives. This ordering matters because scoring an already-infeasible option wastes analysis effort and can create a false sense that it was seriously considered.

Two realistic examples of constraints that eliminate options outright, not just penalize them:

PCI-DSS (Payment Card Industry Data Security Standard) card-data scope. If a design stores raw card numbers to power broader analytics, that option is gone the moment PCI-DSS applies, regardless of how much better the analytics would be; the only surviving options tokenize card data (replace the real card number with a random, non-sensitive placeholder token that maps back to it only inside the certified payment vault) or route it through an already-certified payment gateway.

Regulatory data residency. A requirement that a jurisdiction's data (for example, European Union customer data under data-protection law) must remain within that jurisdiction's borders eliminates any single-region deployment outside it outright, even if that region is meaningfully cheaper or already has spare capacity; there's no scoring adjustment that makes a non-compliant region viable.

Worked example

An illustrative scenario: a new payments feature needs to store transaction detail for both fraud analytics and customer support. Three candidate designs exist: (A) store full raw card data plus transaction detail for maximum analytics flexibility, (B) tokenize card data and store only tokens plus transaction metadata, (C) tokenize card data and additionally keep only aggregated, non-identifying analytics rather than per-transaction detail. Once PCI-DSS scope is applied as a hard filter, option A is eliminated outright, not down-weighted, because storing raw card data outside a certified, PCI-scoped environment isn't a slower or costlier version of the same design, it's a design that isn't legally available. The remaining trade-off analysis, cost and analytics fidelity, runs only between B and C: B keeps more per-transaction detail at a higher tokenization and storage cost (illustratively, storing a token plus full transaction metadata for 10 million transactions/month at roughly $0.0004/record runs about $4,000/month), C is cheaper (aggregating to per-customer monthly summaries cuts that record volume by roughly 95%, to around $200/month) but sacrifices per-transaction granularity for fraud analysis. That second-stage comparison is where a normal cost-vs-capability trade-off analysis applies; the first stage had none, only elimination.

Trade-offs & pitfalls

• The most common mistake is treating a hard constraint as one more weighted factor in a scoring matrix; that understates it and risks a stakeholder pushing back with "can we just accept a bit more risk here," when the honest answer is there's no risk-acceptance path available.
• Document what was eliminated and why, not just what was chosen; a stakeholder who wasn't in the room needs to see that the more attractive option was never actually on the table, not that it lost a close call.
• Distinguish a genuinely non-negotiable constraint from a strongly-preferred one; treating a soft preference as a hard filter needlessly shrinks the option set and can be walked back once challenged, which undermines trust in the rest of the analysis.
• Residual risk still needs to be documented and mitigated even after the hard filter is applied; "compliant" doesn't mean "risk-free," it means the specific eliminated risk is off the table.

Direct answer

Shared infrastructure with strong logical isolation (separate namespaces, per-tenant auth tokens, tenant tagging, resource quotas) is usually the right default, since it pools GPU utilization across customers whose peaks rarely align, cutting cost significantly. Dedicated infrastructure per tenant is worth the extra cost when a customer's contractual or regulatory requirements demand a hard blast-radius boundary, where their data or model weights must never be reachable from another tenant's compute, even in a bug scenario.

Structured elaboration

Shared, logical isolation: pooled GPU utilization means one customer's idle hours cover another's peak, most of the cost saving in multi-tenant serving; the security posture depends entirely on the isolation layer (auth, routing, process isolation) being bug-free, since one flaw there is a cross-tenant leak.

Dedicated per-tenant: no pooling benefit, meaningfully more expensive at the same load; a compromise of the isolation layer cannot cross tenant boundaries since there's no shared compute to cross into; more fleets to patch, but any incident is contained to one tenant.

Worked example

20 customers, each needing a peak of 4 GPUs for 2 hours a day, spread through the day. Dedicated:

$\text{dedicated GPUs} = 20 \times 4 = 80$

Shared, sized to the busiest overlap window (at most 6 customers overlapping at once):

$\text{shared GPUs} = 6 \times 4 = 24$

a little over 3x fewer GPUs. That gap is exactly what a customer with hard isolation requirements, like a bank, is asking you to give up when it demands dedicated infrastructure.

Trade-offs and pitfalls

"Logical isolation" is a spectrum: container-level is weaker than VM-level, which is weaker than physically separate hardware. The mistake is treating isolation as binary instead of naming exactly which layer, network, compute, storage, or model weights, needs separation, since compliance often only demands one specific layer.

What the interviewer probes next

Tenant-scoped quotas to stop a noisy tenant from starving others, whether you'd offer a middle tier of dedicated compute with a shared control plane, and how incident response differs between the two designs.

Direct answer

Mutual TLS (mTLS, where both client and server present and verify certificates, not just the server) everywhere assumes the internal network is not trustworthy, so a compromised service or a misconfigured firewall rule cannot be used to eavesdrop on or impersonate another service. Perimeter-only encryption assumes the internal network is a trusted zone once past the edge, cheaper to run, but means one breached internal host has broad access to plaintext traffic between every other internal service.

Structured elaboration

mTLS everywhere: contains lateral movement, since a compromised pod cannot silently sniff or spoof traffic between two other services; costs certificate issuance and rotation infrastructure (usually a service mesh sidecar, a small helper process deployed alongside each service instance that handles the mTLS handshake and certificate rotation for it so the application code doesn't have to), added CPU for handshakes and encryption on every hop, and new per-hop latency; certificate expiry becomes a new outage class if rotation automation breaks.

Perimeter-only: no security gain past the edge, everything inside the perimeter is implicitly trusted; much lower CPU and latency overhead internally, no per-service certificate management; a single compromised internal host has plaintext access to everything else inside the perimeter.

Worked example

A request chain touches 5 internal services, each handshake plus encryption overhead adding 2ms per hop, a stated assumption for this exercise:

$\text{added latency} = 5 \times 2\text{ms} = 10\text{ms}$

Against a 200ms end-to-end SLA:

$\frac{10\text{ms}}{200\text{ms}} = 5\%$

a cost worth paying for a payments or healthcare system handling regulated data, and possibly not worth paying for an internal analytics dashboard with no sensitive data in the path.

Trade-offs and pitfalls

The most common failure is not the crypto overhead, it is operational: certificate rotation automation breaking silently until certificates expire and take down the whole mesh at once. Teams that adopt mTLS everywhere without investing in automated rotation and monitoring often experience their first real outage from the mTLS layer itself, not from an attacker.

What the interviewer probes next

Expect questions on rolling this out incrementally without a big-bang cutover, monitoring that catches certificate rotation failure before it becomes an incident, and whether you would carve out exceptions for latency-critical hot paths.