Staff-Level Systems Engineer Interview Preparation Guide - FAANG Standards

Brief definition

• Encryption at rest: protects stored data (disks, object stores, databases).
• Encryption in transit: protects data while moving across networks (client→server, service→service).

Common cloud implementation patterns

• At rest
  • EBS / managed block storage: cloud-provider SSE (e.g., AWS EBS encryption) or LUKS on VMs.
  • S3 / object stores: server-side encryption (SSE-S3, SSE-KMS) or client-side envelope encryption.
  • Databases: Transparent Data Encryption (TDE) or application-level field encryption.
• In transit
  • TLS for HTTP/GRPC (public and internal services) — use mTLS where possible.
  • VPN / IPsec for network-level protection (site-to-site, hybrid cloud).
  • Service mesh (e.g., Istio) to enforce mTLS between pods.

Where key management fits

• Centralized KMS/HSM (cloud KMS, CloudHSM) stores root keys; used to wrap data keys.
• Envelope encryption: data encrypted with symmetric data key; data key encrypted by KMS key.
• Key lifecycle: rotation, access policies (IAM), audit logs, and secure backup of keys (BYOK considerations).

Pitfalls that lead to noncompliance

• At rest
  • Misconfigured snapshots/shared AMIs containing unencrypted data or keys—snapshots inherit encryption only if created correctly.
  • Relying on “provider-managed” encryption but leaving object metadata or application secrets in plaintext (logs, tags).
• In transit
  • Using deprecated TLS versions or weak cipher suites; failing to enforce TLS for internal traffic.
  • Assuming private networks are secure and not enabling mTLS or VPNs — lateral movement risk.
• Cross-cutting
  • Poor KMS IAM policies or lack of key rotation/rotation tracking; compromised or over-permissioned principals can decrypt data.
  • Storing keys alongside data (in same VM or repo) or exposing keys in logs/configs.

Practical recommendations

• Use envelope encryption with KMS + least-privilege IAM, enable automatic rotation and audit logging.
• Enforce minimum TLS versions and cipher suites; use mTLS for service-to-service, and scan for plaintext secrets in backups/logs.

Design summary (goal): low read latency worldwide using local read replicas, eventual consistency for product-catalog data, single writable primary region for writes, fast failover and minimized write latency.

High-level architecture

• One writable Primary region (A) holding the authoritative DB.
• Multiple Read Replica regions (B, C, D) hosting read-only replicas and a caching layer (CDN + regional cache).
• Asynchronous geo-replication pipeline from Primary -> replicas (CDC / log shipping).
• Global traffic routing via DNS latency-aware policy + Anycast / regional edge proxies.

Replication strategy

• Use async physical or logical replication (WAL shipping or CDC like Debezium) to stream changes from Primary to regional replicas.
• Apply changes in near-real-time (sub-second to low seconds) — tune batch/window for throughput vs freshness.
• For non-critical catalog data, favor async replication to avoid write latency hit.

Traffic routing

• Read path: clients resolve to nearest region via geo-DNS or Global Load Balancer; edge proxies serve from regional cache first, then read replica.
• Write path: clients send writes to nearest regional edge which forwards to Primary via:
  • Persistent optimized connection (GRPC, HTTP/2) with connection pooling and regional egress peering to reduce RTT.
  • Optionally use a write-proxy fleet (stateless) in each region to batch/route writes to Primary, reducing handshake overhead.
• Health checks + latency-based DNS fallback to route around unavailable regions.

Conflict resolution / consistency

• Since writes go only to Primary, conflicts are minimized. For the small window during failover or simultaneous local writes (e.g., cached optimistic updates), use:
  • Monotonic timestamps (NTP-synced or hybrid logical clocks) + Last-Write-Wins for simple fields.
  • For richer merges (inventory, attributes), use application-level merge rules or CRDTs where appropriate.
• Periodic background reconciliation jobs (read-compare-fix) to detect replication lag or divergence and repair.

Minimizing write latency

• Keep Primary in a region with highest write volume and good network connectivity.
• Use:
  • Persistent multiplexed connections and HTTP/2 or gRPC from edge to Primary.
  • Batching small updates where acceptable.
  • Fast path acks: Primary can acknowledge receipt once committed locally (not after replicas apply) for speed; use monitoring to track replication lag.
• For UX improvements, allow local optimistic responses (UI shows change) with eventual correction if write fails.

Failover & promotion

• Automated leader election using a consensus mechanism (external coordination via etcd/consul or cloud-managed regional failover) to promote a new Primary if current Primary becomes unreachable.
• Steps on failover:
  • Quiesce writes, ensure last WAL segments are shipped/applied, cut-over to promoted region after ensuring no split-brain (use majority / lease-based locks).
  • Reconfigure read routing and update DNS/global LB.
• Mitigations: cross-region quorum for safe promotion, manual intervention window for risky promotions, and automated rollback mechanisms.

Observability & operations

• Monitor replication lag, error rates, traffic patterns.
• Alert on >X seconds replication lag, write queue growth, or proxy failures.
• Run regular DR drills to validate promotion and rollback procedures.

This design trades immediate strong consistency for low-latency global reads and operational simplicity; it minimizes write latency by optimizing network paths and using async replication, while safe failover relies on coordinated promotion and reconciliation.

Preparation summary (one-week window)

Documentation to request

• Network diagrams, system architecture docs, runbooks, on-call rota, incident postmortems for last 6 months, security/compliance checklist, environment inventories (prod/preprod), CI/CD pipeline docs, SSO/IDP and VPN setup guides, Terraform/CloudFormation repos and state info.

System access to pre-arrange

• Corporate email and SSO account, VPN, jumpbox/ bastion access, read access to monitoring (Grafana, Prometheus), logging (ELK/CloudWatch), ticketing (Jira/ServiceNow), source control (gitops repos), cloud console IAM role with least-privilege, build and deployment pipeline view permissions, pager/alert routing.

People to contact

• Hiring manager (clarify 30/60/90 expectations), onboarding buddy, primary ops lead, security engineer, CI/CD owner, SRE on-call for overlap week, key application owners.

Measurable 30-day goals (propose to manager)

• Gain read access and verify with 1–2 smoke tests on staging within 5 days.
• Complete onboarding checklist and two runbook walkthroughs by day 10.
• Resolve one low-priority operational ticket and triage two incidents under mentorship by day 20.
• Deliver a documented improvement proposal (rollback test or minor automation) and get manager sign-off by day 30.

These steps reduce time-to-productivity and show early impact while maintaining safety and least-privilege.

Approach summary
Brief, repeatable framework to evaluate SaaS vs. in-house across functional, non‑functional, cost, ops, risk, migration — produce a recommendation with decision drivers and sensitivity analysis.

Technical requirements

• Ingestion: peak events/sec, burst handling, backpressure, agent protocols (HTTP, syslog, fluentd), guaranteed delivery.
• Retention: hot/warm/cold tiers, retention policies, archival to S3, restore times.
• Query patterns: ad-hoc full‑text, aggregation, dashboards, alerting latency, ML/anomaly support.
• Integrations: IAM, K8s, cloud logs, tracing, metrics correlation.

Non-functional

• SLA: availability %, RTO/RPO for queries/ingest.
• Compliance: GDPR, HIPAA, SOC2, encryption-at-rest/in-transit, data residency.
• Security: tenant isolation, RBAC, audit logs.

Cost model (3yr TCO)

• SaaS: subscription, data ingress/egress, storage, premium features.
• In‑house: infra (compute, storage), licenses, SRE salaries, monitoring, backups, network egress, depreciation.
• Model: annualize CAPEX, project growth, run 3 scenarios (base, +50% load, +100%).

Operational staffing

• Headcount for build (design, dev, infra), ongoing SRE, on-call.
• Training, runbooks, incident response.

Failure modes

• Ingest overload, index corruption, query performance degradation, costly egress, vendor lock-in, misconfigurations causing data loss.

Migration complexity

• Data migration plan, dual-write period, schema mapping, dashboard rewrite, cutover strategy, rollback plan.

Recommendation structure for CTO

1. Executive summary (1–2 lines) with recommended option and primary rationale.
2. Key drivers (cost delta, time-to-value, compliance gaps, ops burden).
3. Sensitivity analysis (at what load/cost build becomes favorable).
4. Risks & mitigations.
5. Recommended next steps (pilot vendor X for 3 months / MVP build + metrics).

Direct answer

A tabletop exercise is a facilitated, discussion-based drill: participants talk through how they'd respond to a scenario in real time, in their actual roles and using the real plan, without touching production systems, distinct from a functional exercise (partial live actions) or a full-interruption test (an actual failover or live activation). For a primary-data-center-loss scenario, the room needs the same roles that would activate in a real event, a sequence of timed injects that escalate realistically, and a facilitator whose job is to probe decision-making and plan gaps, not to check whether anyone remembers the right terminology.

Who's in the room

Mirror the real activation roster, not a subset: the continuity commander or their designated backup (which also tests the succession structure), function leads for the services most exposed by the scenario, the communications lead, and, since data-center loss carries real business and legal weight, a legal or compliance representative and a finance representative who can speak to emergency-spend authorization. An executive observer is useful for buy-in but should stay observing, not steering.

Structuring the injects

Injects are short, timed pieces of new information dropped into the scenario to force decisions, not a single static scenario dumped up front. Good injects escalate:

1. T+0: "Monitoring shows the primary data center has lost power; status unknown." Tests whether anyone moves to declare or the group waits for certainty.
2. T+20 min: "Power confirmed out, no restoration ETA from the facility. Two Tier 1 services are now inaccessible." Tests whether the group applies the criticality tiers to prioritize, and whether declaration actually happens.
3. T+45 min: "Customer support is fielding a spike in complaints and asking what to tell customers." Tests whether the pre-agreed communications cadence and templates get used.
4. T+90 min: "The facility now says restoration could take 6-8 hours, not the 1-2 originally estimated." Tests whether the group re-evaluates its recovery sequencing and communications, or stays anchored to the first estimate.

A genuinely useful alternate scenario, testing different plan assumptions, is a regional outage that takes down authentication and payments specifically rather than a single data center: because those two services sit upstream of almost everything else, this scenario is better at exposing dependency-ordering gaps, which service has to come back first because everything else depends on it, than a straightforward single-site loss.

What the facilitator is actually probing for

Not whether participants can recite the runbook, but whether the plan itself holds up under realistic pressure: does the right person actually step up to declare, or does the room wait for permission that was supposed to be pre-granted; do function leads know their own recovery sequencing without being told; does the communications lead use the pre-built templates or improvise, under time pressure, exactly what the templates exist to prevent; and when an inject invalidates an earlier assumption, does the group visibly adapt or keep executing a plan that no longer fits the facts. The gaps surfaced here are the actual output of the exercise, more than the scenario itself.

Capturing readiness afterward

The exercise should produce artifacts, not just a shared feeling that it went fine:

• A timestamped log of decisions made and by whom, using the same decision-log discipline as a real event, which doubles as practice for that skill.
• A list of plan gaps or ambiguities surfaced by each inject.
• Explicit action items with owners and due dates.
• A short facilitator's readiness note scoring how close the group's real-time behavior tracked the documented plan, and whether any divergence revealed a plan flaw or a training gap.

These artifacts are what make the exercise auditable and feed the next plan revision, rather than the exercise being a one-off team-building event.

Worked example

Continuing the T+90 inject above: told the outage will now run 6-8 hours instead of 1-2, the finance representative in the exercise realizes the pre-approved emergency-spend threshold only covers a 2-hour activation of the backup facility contract, and the group has to work out, live, who can authorize the additional spend. That gap, an approval threshold that never anticipated a longer event, is exactly the kind of finding a tabletop is meant to surface cheaply, before it's discovered for real during an actual multi-hour outage.

Trade-offs and pitfalls

• Making the scenario too easy, a clean, fast resolution, teaches nothing. The value concentrates in injects that force genuine judgment calls and expose where the plan is silent or wrong.
• Running the exercise with only technical responders, leaving out legal, finance, or comms, validates only part of the plan and gives false confidence about the rest.
• A tabletop that never produces a documented action item is a team-building exercise, not a continuity exercise. The artifacts are the point, not just the conversation.
• Reusing the same scenario every time tests memorized responses, not real plan quality. Rotating scenario type, single-site loss, service-specific regional outage, third-party or supplier failure, surfaces different weaknesses each round.

Direct Answer

A saga coordinates a business transaction that spans multiple services by breaking it into a sequence of local transactions. Each service commits its own step immediately with no cross-service lock held, and if a later step fails, the saga undoes the steps that already succeeded by running a compensating action for each one, in reverse order. This trades strict, all-or-nothing atomicity for eventual, recoverable consistency and loose coupling between services.

Choreography vs. Orchestration

• Orchestration: a central coordinator issues each step as a command to the relevant service and decides, based on that service's response, what to do next, including which compensations to trigger if something fails. The whole workflow lives in one place, which makes it easier to see, test, and reason about end to end.
• Choreography: there is no central coordinator; each service publishes an event when it finishes its local step, and whichever service is subscribed to that event reacts by doing its own step and publishing its own event in turn. This avoids coupling every service to a central coordinator's command contract, but it scatters the workflow logic across services, so understanding or changing the whole sequence means tracing through several services' event subscriptions instead of reading one place.

Worked Trace: Order Fulfillment When Shipment Fails (Orchestration Style)

Order O123, three steps: reserve inventory, charge payment, schedule shipment.

1. Orchestrator sends ReserveInventory(O123, sku=42, qty=1) to Inventory. Inventory reserves the unit and replies Reserved.
2. Orchestrator sends ChargePayment(O123, $50) to Payment. Payment captures the charge and replies Charged.
3. Orchestrator sends ScheduleShipment(O123) to Shipping. Shipping tries to allocate a carrier slot and replies Failed: no carrier capacity.
4. The orchestrator now runs compensations in reverse order. It sends RefundPayment(O123, $50) to Payment, undoing step 2. Payment replies Refunded.
5. It sends ReleaseReservation(O123, sku=42, qty=1) to Inventory, undoing step 1. Inventory replies Released.
6. The orchestrator marks order O123 as Failed and notifies the customer.

The same sequence in choreography looks like this instead: Inventory reserves and emits InventoryReserved(O123). Payment, subscribed to that event, charges and emits PaymentCharged(O123). Shipping, subscribed to PaymentCharged, tries to schedule and, on failure, emits ShipmentFailed(O123). Both Payment and Inventory are subscribed to ShipmentFailed: Payment independently issues its own refund and emits PaymentRefunded(O123), and Inventory independently releases its reservation and emits ReservationReleased(O123). No single component ever holds the full picture of the workflow; each service only knows what to do when it sees an event it's subscribed to.

Trade-offs and Pitfalls

• Every forward step and every compensating action has to tolerate being retried, since at-least-once delivery means ChargePayment could be delivered twice; this is a system-property requirement on the saga's steps, a separate concern from how an external API exposes idempotency to its own callers.
• The saga's state, meaning which steps have completed and which compensations are pending, needs to be durably persisted, whether by a central orchestrator or by each participant in a choreography, so that a crash and restart can resume the saga correctly instead of leaving it stuck partway.
• Not every action has a true inverse. Compensating a shipment step after the package has physically left the warehouse can't undo the physical fact, only correct the system's record and possibly trigger a real-world return process; a senior design puts the hardest-to-compensate steps as late as possible in the sequence.
• Choose a saga when the steps naturally live in separate services or databases and each one can be given a real, working compensating action. Reach for a real distributed transaction only when an intermediate, partially-applied state genuinely cannot be tolerated and you can afford a synchronous locking protocol across every participant, which a saga specifically avoids.

Direct answer

Don't force one team's cadence onto the other. Design a policy that separates what must be shared (the guardrails that protect everyone) from what can stay team-specific (how fast a given team is allowed to move within those guardrails), then negotiate the guardrails, not the cadence itself. That reframing turns "fast team vs. cautious team" into a joint design problem both sides can own.

Structured elaboration

1. Split invariant from flexible. List what truly must be uniform across teams (a working rollback path, a minimum test bar, an incident-response process) versus what can legitimately vary (deploy frequency, staging gate count, review depth). Most conflicts collapse once you see that only a small slice actually needs to be shared.
2. Reframe cadence as risk exposure. Ask each side what they're protecting (customer trust, an SLA, a compliance obligation) versus what they want (velocity). Convert both into measurable guardrails: blast radius limits (how much of the system or traffic a change could affect if it goes wrong), an automated rollback trigger (a rule that reverts the change automatically once a threshold is crossed, without waiting for a human to notice), a minimum observation window before a change is considered "safe."
3. Build a tiered policy, not a single rule. Changes that touch a small blast radius and have a fast, automatic rollback can move on the fast-moving team's cadence. Changes that touch shared, hard-to-reverse surfaces get the slower team's gates, regardless of which team wrote the change. The tiering criteria, not the team identity, decides the process.
4. Add an explicit exception path. Either side can request a deviation (ship something in a higher tier faster, or hold something in a lower tier longer) with a documented reason and a named approver, so departures from the policy are visible instead of quiet workarounds.
5. Time-box a trial and revisit with real data. Don't debate the policy hypothetically forever. Run it for a fixed period, then bring incident counts and delivery-time data back to the table instead of re-litigating the original positions.

The same negotiation pattern applies beyond deploy-frequency disputes: whenever two functions have structurally different operating rhythms, the fix is a shared cadence at the boundary, not a winner. As a concrete cross-team cadence clash from the machine-learning world: a feature store (the shared system that stores and serves the data used to train and run machine-learning models) team can only refresh labels every two weeks, while the product team needs weekly model retraining (rerunning the training process on newer data so the model's predictions stay current). That isn't a risk-tolerance disagreement at all. It's a hard technical constraint on one side meeting a business cadence need on the other, and it gets negotiated the same way: agree what must move on the constrained cadence (the underlying label refresh) versus what can be decoupled (the product team retrains weekly on the two most recent completed label batches, accepting known staleness, rather than blocking on a refresh that can't happen faster).

Worked example

Team A ships to production many times a day behind feature flags. Team B owns a regulated, customer-facing billing surface and wants a weekly release train. Instead of debating "how often should we deploy," the negotiated policy ties process to blast radius: any change gated behind a flag to less than 1% of traffic can auto-promote if the error rate stays under 2x the pre-change baseline for a 30-minute observation window (a policy parameter both sides agreed to, not a claimed result). Changes that touch the billing ledger directly, regardless of author, require the slower manual review and a scheduled release window. Team A keeps most of its velocity because most of its changes are low blast radius; Team B keeps its protection because the surface it cares about is gated the same way no matter who wrote the change.

For the cadence-mismatch variant: the feature store team commits to publishing a refreshed label snapshot every two weeks, on a fixed schedule the product team can plan around. The product team's weekly retraining job consumes the most recent snapshot plus a lightweight, clearly-labeled interim signal for the intervening week, rather than either side pretending the refresh can happen weekly or the product team silently retraining on stale labels without acknowledging it.

Trade-offs & pitfalls

• Pitfall: writing a single global policy. It's either too loose for the regulated team or too strict for the fast-moving one, and both sides end up circumventing it.
• Pitfall: treating this as a one-time meeting. Without a scheduled revisit, the policy calcifies around the political balance of the original conversation instead of actual incident/velocity data.
• Pitfall: hiding exceptions. If deviations aren't logged and visible, the "shared" part of the policy erodes silently and trust breaks down the next time there's an incident.
• Senior differentiator: designing the guardrail so it's parameterized by risk (or, in the cadence case, by the actual constraint) rather than by team identity. That's what lets both sides keep their operating model instead of one side losing the negotiation.

Unix/Linux permission bits & umask — short answer

• Permissions: owner (u), group (g), others (o) with read (r=4), write (w=2), execute (x=1). Shorthand: rwxr-xr--.
• umask: bitmask subtracted from a process’s default creation mode; applied as new_mode = default_mode & ~umask.

Calculate with umask 022

• New file default 0666: 0666 & ~0022 = 0666 & 0755 = 0644 → rw-r--r--
• New directory default 0777: 0777 & ~0022 = 0777 & 0755 = 0755 → rwxr-xr-x

setuid, setgid, sticky — what & example

• setuid (u+s): executable runs with file owner's UID. Example: /usr/bin/passwd has setuid root so it can update /etc/shadow.
• setgid on executable (g+s): executable runs with file's group ID. Example for collaboration tools that need group privileges.
• setgid on directory (g+s): files/dirs created inside inherit directory’s group — useful for team-shared folders: chgrp dev /srv/shared; chmod g+s /srv/shared.
• sticky bit (t): on directories prevents users from deleting/renaming files they don’t own. Example: /tmp is drwxrwxrwt.

POSIX ACL to grant single user write

• Use setfacl to add a user ACL without changing group:

setfacl -m u:alice:rw- /path/to/file
getfacl /path/to/file   # verify

• To make it default for a directory (new files): chmod g+s dir; setfacl -d -m u:alice:rw- dir

These are standard sysadmin practices for managing shared access and least privilege.

Direct answer
The bandwidth-delay product (BDP) is the amount of data that can be "in flight" on a link at any instant, bandwidth multiplied by round-trip time, and it's the minimum window size TCP needs to keep the link fully utilized. On a 10 Gbps link with 150ms round-trip time, the BDP is large enough that the original 16-bit TCP window field (max 65,535 bytes) can represent only a tiny fraction of it, which is exactly why window scaling exists.

Structured elaboration
The BDP calculation is:
$\text{BDP (bits)} = \text{bandwidth (bits/s)} \times \text{RTT (s)}$
For a 10 Gbps link ($10 \times 10^9$ bits/s) with a 150ms ($0.150$ s) round-trip time:
$\text{BDP} = 10 \times 10^9 \times 0.150 = 1.5 \times 10^9 \text{ bits} = 187{,}500{,}000 \text{ bytes} \approx 187.5\text{ MB}$
The plain TCP window field is 16 bits, so the largest window it can express without scaling is $2^{16}-1 = 65{,}535$ bytes, about 0.035% of the 187.5 MB needed. Without more window than that, the sender would have to stop and wait for an ACK every 64KB, and at this RTT that caps throughput far below the link's actual 10 Gbps capacity, no matter how fast the link itself is.

The window scale option (negotiated ONLY during the handshake, in the SYN and SYN-ACK) adds a scale factor $S$ (0 to 14) that the receiver applies to its advertised window: the real window becomes $\text{advertised value} \times 2^S$. To cover a 187.5 MB requirement, we need the smallest $S$ such that $65{,}535 \times 2^S \geq 187{,}500{,}000$, which comes out to $S=12$ (giving a maximum representable window of $65{,}535 \times 4096 = 268{,}431{,}360$ bytes, comfortably above the 187.5 MB requirement).

Worked example
For a smaller, more common case, a 100 Mbps link with 100ms RTT, the BDP is:
$100 \times 10^6 \times 0.100 = 1 \times 10^7 \text{ bits} = 1{,}250{,}000 \text{ bytes} \approx 1.25\text{ MB}$
Here the minimum scale factor needed is $S=5$ (giving a max window of $65{,}535 \times 32 = 2{,}097{,}120$ bytes), a much smaller ask than the 10 Gbps case, illustrating that window scaling matters more the higher the bandwidth-delay product climbs, not RTT or bandwidth alone. Beyond window sizing, actually reaching close to line rate on a link like this also typically needs the OS socket buffers (net.ipv4.tcp_rmem/tcp_wmem on Linux) raised to match the negotiated window (a window the OS hasn't allocated buffer space for is wasted), and Selective Acknowledgment (SACK) enabled so a single lost segment somewhere in a large in-flight window doesn't force retransmission of everything after it.

Trade-offs & pitfalls
Window scaling is negotiated ONLY at connection setup; if either endpoint doesn't advertise the option in its SYN, the connection falls back to the un-scaled 64KB ceiling for its entire lifetime, a frequent, hard-to-spot cause of "high-bandwidth link, mysteriously capped throughput" that shows up when an old middlebox strips the option or a misconfigured host has scaling disabled.

Direct answer

With no historical traffic, you do not guess a single number: you build a workload estimate from comparable analogs and top-down business inputs, wrap it in an explicit safety margin, put it behind elastic capacity so the estimate does not have to be exact, and then replace the estimate with real data as fast as possible after launch through staged rollout and monitored experiments.

Structured elaboration

1. Build an initial estimate from two independent angles and reconcile them.

• Top-down: start from a business number you do have (invited users, marketing reach, sales pipeline) and multiply down to requests. This is the only lever available with zero history.
• Analog: find the closest comparable system you or the industry already operates (a similar feature, a similar-sized customer base, a similar product category) and scale its known request-per-user rate to your expected user count.
• Reconcile the two. If they disagree by more than roughly 2-3x, that gap itself is useful information: it tells you where your uncertainty is concentrated and what to instrument first.

2. Convert the estimate into a load shape, not just a total.
A daily total hides the number that actually threatens the system: peak requests per second (RPS, requests per second). Apply a peak-to-average ratio to account for daily cycles and, for a launch specifically, a possible synchronized spike (a launch email, a push notification, a press mention) that behaves nothing like organic steady traffic.

3. Set headroom deliberately, and say why.
Headroom on a zero-history estimate covers two different kinds of error: normal variance (traffic is noisier than a smooth average implies) and estimate error (the whole model could be wrong). Treat these as multiplicative: a peak-shape multiplier for the first, then a separate safety-margin multiplier for the second. Document both numbers as assumptions, not facts, so whoever revisits capacity later knows which parts were guessed.

4. Plan for elastic capacity so the estimate does not have to be right.
Because pre-launch numbers are inherently soft, favor a design where compute scales out automatically (for example an Auto Scaling group, ASG, sized with a low minimum and a generous maximum) over one where you provision a fixed fleet sized to the estimate. Stateless request handlers are what make this possible: any instance can pick up any request, so the ASG can add or remove capacity without session-affinity constraints. Identify the one component that will NOT scale elastically as fast as the rest (usually the database or a rate-limited third-party dependency) and size or protect that one deliberately, since it becomes the real ceiling regardless of how large the compute fleet grows.

5. Define what you will measure and how you will validate the assumption after launch.
Before launch, decide: the metrics that reveal reality (RPS, P95/P99 latency [95th-percentile/99th-percentile], error rate, queue depth, database connection saturation), the rollout mechanism that limits blast radius while those metrics come in (percentage-based ramp or canary release to a small traffic slice first), and the trigger for pausing the ramp (an explicit threshold on any of the above, decided in advance rather than improvised under pressure).

Worked example

Assume, as planning inputs rather than measured facts:

• 10,000 users are active on day one (from a marketing pre-registration count, discounted for expected activation rate).
• Each active user generates 15 requests over the day (from an analog product's per-user request rate).
• A peak-to-average ratio of 4x, reflecting a synchronized launch announcement rather than smooth organic arrival.
• A safety margin of 2x on top of the peak, to absorb estimate error since there is no history to validate the inputs against.

That "14 RPS" is not a forecast you defend, it is a starting point for the ASG's scaling policy and a number you replace with observed data within the first days of traffic.

Trade-offs & pitfalls

Over-provisioning a fixed fleet to the safety-margin number wastes money for a launch that may undershoot; under-provisioning without elastic headroom risks a visible outage on the day traffic is most scrutinized. The middle path (a small guaranteed baseline plus autoscaling) is usually right, but it only works if the service is stateless and the true bottleneck (often the database, not the request tier) is identified and protected separately, since databases scale far less elastically than compute. The most common senior-vs-junior tell is whether the candidate treats the initial number as a fact to defend or as an assumption to instrument and correct quickly after launch.