System Design Methodology and Trade-off Analysis Questions

Direct answer

Use two different datastores, because the two workloads sit at opposite ends of nearly every axis that matters: the session store needs low-latency point lookups by key with automatic expiry, while the audit log needs durable, ordered, append-only writes with long retention and replay. Forcing one engine to serve both means either paying the audit log's durability and retention cost on every fast session read, or giving up the audit log's append-only, replayable guarantees.

Structured elaboration

Why this generalizes beyond sessions and audit logs: the absorbed framing of a machine-learning (ML) metadata store shows the same underlying shape. Continuous integration (CI) pipelines generate a high write rate logging experiment runs, while dashboards and monitoring run read-heavy, scan-heavy queries over historical runs. That's structurally the identical pattern, a write-optimized append path pulling against a read-optimized query path, which is why the "one store or two" question should be re-asked per component rather than defaulting to whatever store is already in the stack.

Worked example

Primary scenario (session + audit). Use Redis for sessions, with native per-key TTL handling expiry automatically, no cleanup job required. Use an append-only log or a partitioned table for the audit trail.

Storage growth check on the audit side: assume, illustratively, 500 audit events per second at 1 KB each.

$500 \text{ events/s} \times 1\text{KB} = 500 \text{KB/s}$
$500 \text{KB/s} \times 86{,}400 \text{s/day} = 43{,}200{,}000 \text{KB/day} = 43.2 \text{GB/day (decimal GB, 1 GB = 1{,}000{,}000 KB)}$
$43.2 \text{GB/day} \times 730 \text{days (2 years)} = 31{,}536 \text{GB} \approx 31.5 \text{TB}$

Thirty-one and a half terabytes of append-only history is exactly the kind of cheap, sequential, long-retention storage a log-oriented system is built for, and exactly the kind of volume that would be prohibitively expensive to hold in an in-memory session store's provisioned capacity.

Second scenario (ML metadata, the absorbed framing). The same reasoning applies with a relational or document store for structured experiment metadata (read-heavy, queried by dashboards across many attributes) paired with an append-only log or object storage for the raw high-volume run logs written continuously by CI, again two engines, chosen for two genuinely different access patterns, not one engine stretched across both.

Trade-offs & pitfalls

• Storing audit events in the session store "for convenience" either evicts old sessions to make room or silently drops the durability guarantee the audit log actually needs, neither is acceptable for an audit trail.
• Using an append-only log system for session storage means reimplementing per-key expiry logic that a key-value store like Redis provides natively, most log/stream systems have topic-level retention, not per-key TTL.
• Two datastores are two systems to operate, monitor, and back up, that overhead is worth paying here because the access patterns are genuinely incompatible; it would not be worth it if the two workloads were similar enough to share one engine.
• Deletion requirements, for example a regulatory "right to be forgotten" request, are harder to satisfy in an append-only log than in a key-value store, this needs a tombstone (a marker record saying "treat this key as deleted") plus a compaction strategy (the background process that rewrites and shrinks the log over time, actually purging tombstoned data) decided up front, not retrofitted after the first request arrives.

flowchart LR
Client --> SessionAPI[Session Read/Write]
Client --> AuditWriter[Audit Event Writer]
SessionAPI --> Redis[(Redis Session Store)]
AuditWriter --> Log[(Append-only Log)]
Log --> ColdStorage[(Long-term Storage)]

Direct answer

Turn "handle spikes" into numbers by asking for the spike multiplier over baseline, its duration and arrival shape, the peak concurrency it implies, and what is allowed to degrade versus what must stay within the service-level agreement (SLA) during it. Those four answers are what actually let you size autoscaling, connection pools, and a degradation plan; without them, "handle spikes" is a feeling, not a requirement.

Structured elaboration

The four questions that make it measurable

The general skill, applied to a different vague ask

The same discipline works on any vague requirement, not just traffic spikes. "Handle a fifteen-year-old legacy system with no APIs" is exactly as unmeasurable until you ask the analogous questions: what data-access surfaces actually exist (direct database reads, nightly file exports, screen automation), who owns changes to that system, what staleness is tolerable in whatever gets extracted, and what happens to your system if that legacy system goes down for a day. "No APIs" becomes a concrete integration contract the same way "handle spikes" becomes a concrete capacity contract, by naming the constraint that changes the design instead of accepting the vague label.

Worked example: turning "5x for ten minutes" into a server count

Assume measured baseline steady-state traffic of 1,000 requests per second (RPS), and product says the spike is "5x for about ten minutes." Assume each server instance safely handles 200 RPS at target latency:

Now check whether autoscaling can even react in time. Assume it takes 3 minutes from scale-out trigger to a new instance serving traffic:

Autoscaling alone is workable here, with roughly 3 minutes of degraded capacity at the start of the spike. If the same 5x spike instead lasted 60 seconds (a flash-crowd shape rather than a sustained one), the 3-minute scale-out reaction time would exceed the entire spike duration, and the only real fix is pre-warmed standby capacity, not faster autoscaling. That is why duration and arrival shape change the design, not just the multiplier.

Trade-offs & pitfalls

• Pitfall: designing for "handle any spike" instead of a bounded one. Every system has a ceiling; the point of these questions is choosing it deliberately instead of discovering it during an incident.
• Pitfall: assuming autoscaling reaction time is negligible. If it is not faster than the spike itself, pre-provisioned headroom is needed, which costs money sitting idle.
• Graceful degradation (returning cached or partial results, shedding low-priority requests) is usually cheaper than provisioning for the absolute peak, but only if product has said which features are allowed to degrade.

Direct answer

A high-level architecture shows the system's scope: the major building blocks (client, API layer, service tier, datastore, cache, external dependencies), how they relate, and the non-functional constraints (scale, availability) that shaped them. A component-level design zooms into one of those blocks and specifies its interfaces, request/response schemas, data flows, and sequencing. You show the high-level view to stakeholders who need to understand what the system is and what it costs or risks; you show component-level design to the people who have to build, test, or integrate against one specific piece.

Structured elaboration

The reason both layers matter: the high-level view sets the strategy and the constraints everyone else has to work inside; the component-level view is what actually gets implemented, tested, and integrated. A good design doc keeps an explicit mapping from each high-level block down to its component-level detail, so a reviewer can move between the two without re-deriving context.

Worked example

Say you're designing a subscription billing feature. At the high level you'd draw: client apps, an API gateway, a billing service, a payments component, a database, and a message queue for async notifications, with an arrow showing the billing service calls out to a third-party payment processor. The one decision that belongs only at this level: whether billing lives inside the existing monolith or is split into its own service, because that choice affects deployment, on-call ownership, and the blast radius of an incident, not just this one feature.

At the component level, you'd zoom into just the billing service and produce: a sequence diagram for "create subscription" (client → billing service → payments component → processor → database write → event published), the exact request/response schema for the POST /subscriptions endpoint, and an entity-relationship diagram for the subscription and invoice tables. None of that detail belongs on the high-level diagram; it would bury the one decision (monolith vs separate service) that the high-level view exists to surface.

Trade-offs & pitfalls

• Showing component-level detail (full schemas, every retry path) to an executive or product stakeholder buries the one decision they actually need to weigh in on.
• Skipping the high-level view and jumping straight to component design risks locking in a boundary (a shared database, a synchronous call where an event would do) that is expensive to undo later, because it was never surfaced as a decision.
• A common weak answer just says "high-level is the big picture, low-level is the details" without naming a decision that is exclusive to one level; naming that decision is the signal an interviewer is listening for.
• Keep a living link between the two artifacts (a component-level design should reference which high-level block it belongs to) so the documentation doesn't drift apart as the system evolves.

Direct answer

Treat it as an investment decision: translate the latency improvement into a dollar figure, usually via conversion or engagement lift, compare it to the extra cost plus the risk the added complexity introduces, and ship it only if the net benefit is positive with a margin that survives your uncertainty about the conversion assumption. What changes the answer is the size of that margin: a break-even case should prompt more evidence (a real experiment) before committing, not a coin flip.

Structured elaboration

The decision framework

1. Translate tail latency into revenue using your own historical relationship between latency and conversion, not an assumption invented for this decision alone.
2. Compare that revenue gain to the extra infrastructure cost plus the risk-adjusted cost of the added complexity (a higher chance and cost of an incident, a longer time to diagnose an outage).
3. Validate with a phased rollout (a small canary first) before committing the full 3x spend, so the confirmed number is what's being paid for, not the projected one.
4. Set a rollback trigger up front: if the reliability cost shows up (more incidents, longer mean time to recover) before the revenue gain is confirmed, pull back.

Worked example (illustrative assumptions; would come from your own A/B data in practice)

Assume current infrastructure costs $50,000/month and the proposed change triples it to $150,000/month:

Assume 99th-percentile (P99) tail latency drops from 2,000 ms to 1,000 ms, a 1,000 ms reduction, and assume (illustrative, pulled from historical experiments in a real decision) a 0.5% relative conversion lift per 100 ms of P99 reduction:

Against baseline monthly revenue of $2,000,000:

That is a break-even case at these assumptions, exactly the situation that should prompt a real experiment (canary the change to a fraction of traffic, measure the actual conversion delta) rather than a decision made purely on the spreadsheet. If the elasticity assumption were even slightly optimistic, this ships negative.

What would change the answer

• A larger baseline revenue (the same 5% lift is worth more on a bigger base) tips it positive without changing anything else.
• A confirmed, measured elasticity from a canary experiment replaces the illustrative 0.5% per 100 ms figure with a real one.
• A materially lower or higher risk-adjusted reliability cost (the 3x infrastructure is also more than 3x more complex to operate) shifts the true cost side of the equation.
• A non-revenue reason, such as a contractual service-level agreement (SLA) or a strategic customer who explicitly asked for this, can justify shipping even at a break-even or slightly negative revenue case.

Trade-offs & pitfalls

• Pitfall: treating the latency-to-revenue elasticity as a known constant instead of an assumption to validate; shipping a 3x cost change on an invented number is the actual failure mode this question is testing for.
• Pitfall: ignoring the complexity side of the cost. Three times the infrastructure is usually more than three times the operational surface area (more failure modes, longer incident diagnosis), and that risk has a cost even when nothing has broken yet.
• A break-even or narrowly-positive case is a signal to run a smaller, reversible experiment, not to commit fully in either direction.
• Don't ignore alternatives: a targeted optimization for only the highest-value request paths, or a hybrid where only certain traffic gets the expensive treatment, sometimes captures most of the benefit at a fraction of the cost.

Direct answer

A sustained 10,000 writes-per-second, low-latency, stateful workload pushes you away from a design tuned for reads (a single write primary, heavy indexing, read replicas) and toward one built for write scaling: a storage engine optimized for sequential writes, a partitioning scheme that spreads writes across many nodes, and a replication model with an explicit, tunable durability-versus-latency trade-off rather than a single write bottleneck.

Structured elaboration

Why a read-optimized design breaks down here. Traditional B-tree storage engines perform random-access writes and update every index on every insert, each additional index roughly adds another write per record. A single-writer relational primary caps total write throughput at whatever one node's disk and CPU can sustain, and read replicas do nothing for write capacity, they only copy the primary's write stream.

What changes for write-heavy:

• Storage engine: log-structured merge (LSM) tree engines (used by databases like Cassandra, HBase, and the storage layer behind DynamoDB-style stores) append writes sequentially and merge them in the background, trading some read amplification (a single logical read may have to check several separate on-disk files before it can answer, since recent and older writes land in different segments) for much higher sustained write throughput than a B-tree.
• Partitioning: writes are sharded across many nodes by a partition key. The key must be chosen for even cardinality, a monotonically increasing key (like a timestamp or auto-increment ID) concentrates all new writes on one shard regardless of how many nodes exist.
• Replication and durability: instead of one primary with no built-in fan-out, use a quorum-based replication scheme, writes are acknowledged once a majority of replicas confirm, giving a tunable point between "acknowledge on one node" (fast, risks data loss) and "acknowledge on all nodes" (safest, slowest).
• Indexing discipline: keep secondary indexes to the minimum the write path can afford, every index is a write, this is the opposite instinct from a read-heavy design where more indexes are usually free wins.

Worked example

Assume, illustratively, that a single write-optimized node sustains 2,000 writes per second at the target latency.

Nodes needed for raw throughput: $10{,}000 / 2{,}000 = 5$ shards.

For durability, replicate each shard three ways (tolerate one node failure without data loss): $5 \times 3 = 15$ total storage nodes.

A quorum write with $N=3$ replicas and a write quorum of $W=2$ means the client waits only for the second-fastest replica to acknowledge, not the slowest, bounding tail write latency while still guaranteeing the write survives a single node failure.

Cost contrast, provisioned versus per-operation pricing. At an illustrative $0.00001 per write operation under a consumption-priced managed service:

$\text{ops/day} = 10{,}000 \times 86{,}400 = 864{,}000{,}000 \text{ writes/day}$

$\text{daily cost} = 864{,}000{,}000 \times \$0.00001 = \$8{,}640/\text{day} \approx \$259{,}200/\text{month}$

Against 15 provisioned nodes at an illustrative $400/node/month: $15 \times \$400 = \$6{,}000/\text{month}$. At this sustained write rate the per-operation model costs roughly 40 times more, which is why sustained high-volume writes usually favor provisioned or self-managed clusters, and why consumption pricing fits bursty, low-average workloads instead.

Trade-offs & pitfalls

• Carrying over every index from a read-heavy design roughly multiplies write cost by the number of indexes, audit which indexes the write path can actually afford.
• A low-cardinality or monotonically increasing partition key creates a hot shard that caps total throughput no matter how many nodes you add, this is the single most common write-scaling mistake.
• Waiting for all replicas ($W=N$) is the safest durability setting but the slowest; a majority quorum balances safety and latency, the exact quorum size is itself a trade-off decision, not a default.
• High write concurrency needs connection pooling and write batching, naive one-connection-per-request patterns hit connection limits long before they hit the storage engine's real capacity.

flowchart LR
Client --> Router[Write Router]
Router --> ShardA[Shard A Leader]
Router --> ShardB[Shard B Leader]
Router --> ShardC[Shard C Leader]
ShardA --> ShardARep[Shard A Replicas x2]
ShardB --> ShardBRep[Shard B Replicas x2]
ShardC --> ShardCRep[Shard C Replicas x2]