Load Balancing and Traffic Management Questions

Direct answer

Use STAR: set up the situation and your role in one or two sentences, describe concretely what you did during the incident (not what "the team" did), state the outcome and what changed afterward, and end with the lesson. The strongest signal an interviewer is listening for is not the outage itself, it is whether your actions were deliberate and whether the fix addressed the root cause rather than just the symptom.

Structured elaboration

Situation: Name the system, the trigger, and the specific way the load balancer or failover mechanism misbehaved (routed to a dead node, failed to detect an unhealthy backend, split-brained across regions, flapped between healthy/unhealthy). Vague ("a server went down") is weak; specific ("health checks marked degraded instances as healthy because the probe timeout was shorter than the instance's real response time under load") is strong.

Task: State your role and scope of ownership plainly, on-call engineer, service owner, incident commander, so the interviewer knows what you controlled versus what you coordinated.

Action: This is the section to spend the most time on. Structure it in the order things actually happened:

• Immediate containment: what you changed to stop user impact (pulled a node/region from rotation, rolled back a config, manually failed over).
• Diagnosis: how you found the actual cause, not just the symptom (log correlation, comparing healthy vs unhealthy instance behavior, checking recent changes).
• Communication: who you told, how often, and why that mattered for a coordinated response.

Result: State the outcome honestly and in terms you can actually defend if asked to go deeper: service impact was contained, the specific fix that resolved it, and what changed in the system or process afterward (a corrected health-check timeout, an added synthetic probe, a canary requirement for load-balancer config changes). If you do not have precise numbers from memory, describe the outcome qualitatively (restored to normal operation, contained to a subset of traffic) rather than inventing statistics. An interviewer who probes "how many users were affected" and gets a fabricated number will trust the rest of the story less, not more.

Worked example

A representative version of this story, deliberately written without invented precision so it illustrates structure rather than fake data:

Situation: A payment service ran active-active across two regions behind a global load balancer. During a regional network blip, the load balancer's health checks did not fail the degraded region out of rotation, so a meaningful share of traffic kept landing on instances that were timing out.

Task: As the on-call engineer, I owned containing user impact and driving the fix.

Action: I manually pulled the degraded region from the load balancer's pool through the control plane rather than waiting for automatic health-check convergence, which was clearly not going to trigger in time. In parallel, I checked what changed recently and found that a prior change had shortened the health-check probe timeout below the instance's real p99 response time under load, so probes were failing even on instances that were still serving most requests correctly, the health check itself was miscalibrated, not just unlucky timing. I posted incident updates on a fixed cadence so stakeholders were not pinging for status mid-fix.

Result: Traffic normalized once the region was pulled and the timeout was corrected; the underlying fix was reverting the probe timeout to its previous value and adding a synthetic cross-region probe that exercises the failover path continuously (rather than only during real incidents) so a miscalibrated health check would be caught before the next config change shipped, not during the next real outage.

What I learned: A load balancer's failover is only as good as its health check's ability to distinguish "genuinely unhealthy" from "temporarily slow"; changes to health-check parameters deserve the same review rigor as changes to the service itself, because they directly control blast radius during a real failure.

Trade-offs and pitfalls

• Do not narrate what "we" did when the question asks what you did. Interviewers are listening for your specific decisions and reasoning, not a team summary.
• Avoid ending at "it got fixed." The strongest answers name the systemic change that makes the same failure mode structurally harder to repeat.
• Do not manufacture precision you do not have. A specific, remembered detail (which config changed, what the health check was checking) is more credible than an invented percentage or timestamp, and it holds up better under a skeptical follow-up.
• Watch for blame framing. A root cause that reads as "someone else misconfigured it" without acknowledging what allowed that misconfiguration to ship (no canary, no review gate) reads as less senior than one that owns the systemic gap.

Direct answer

Active checks are probes the load balancer or service discovery system initiates on a schedule; passive checks are inferences drawn from real request outcomes, such as a 5xx response, a timeout, or a connection reset, as traffic naturally flows. Active checks can catch a problem before a real user hits it, but can false-positive on transient blips; passive checks add no extra probe traffic and reflect real user impact directly, but by definition only detect a failure after at least one real request has already suffered it. Production systems combine both.

Structured elaboration

Worked example

A production strategy that combines both without over-reacting to a single blip: mark an instance "suspect" after 2 consecutive active probe failures, but do not remove it from rotation yet. Corroborate with a passive signal over a rolling window of the last 20 real requests: if 50% or more of those requests failed, that is a threshold of $20 \times 0.5 = 10$ failed requests out of 20, remove the instance from rotation. Requiring both signals means a single flaky probe or a single unlucky real request cannot remove a healthy instance on its own, while a genuinely failing instance still gets caught quickly because the two signals reinforce each other.

Trade-offs & pitfalls

• Active-only checks can pass while real traffic fails: a backend can answer a generic TCP or HTTP probe correctly while returning errors to requests carrying real auth headers or payloads the probe never sends.
• Passive-only checks mean, by definition, that some real users experience the failure before it is detected; that is not acceptable on its own for canary or rollout gating, where the goal is to catch a bad release before it reaches most users.
• Avoid routing probe traffic through the same accounting a load-balancing algorithm like least connections uses for real capacity; counting probes as connections can skew the balance away from actual user traffic.

Direct answer

This is a deterministic-cause outage: the fix is to restore traffic through a path that has a valid certificate (failover to a standby LB, hot-load a backup cert, or temporarily terminate TLS elsewhere) as fast as possible, communicate on a fixed, predictable cadence while that happens, and then treat the root cause as a monitoring and automation gap rather than a one-time human mistake, since a cert that reached expiry without anyone acting means the process that was supposed to catch it failed silently well before the outage.

Immediate triage (first ~15 minutes)

• Confirm scope and cause: check LB logs for TLS handshake failures and the certificate's expiry timestamp to distinguish a cert issue from an application or network issue reporting similar symptoms (global 503s can also come from a bad deploy or DNS problem).
• Freeze churn: pause any in-flight deploys or automated config pushes so the incident isn't complicated by unrelated changes landing mid-triage.
• Pull in the right people: notify infra/network/security on-call and declare an incident commander if this is genuinely global impact.

Short-term mitigation

• Fastest path: if a hot-standby LB or path with a valid certificate exists, shift traffic to it (DNS weight change or traffic steering) rather than trying to fix the primary path under pressure.
• If the LB supports multiple certs or hot-reload, load a previously-issued backup/rollover certificate if one exists.
• If neither is available, use automated ACME (Automatic Certificate Management Environment, the protocol that lets software request and renew certificates without a human) / CA (certificate authority, the trusted party that issues certificates) tooling to issue a short-lived emergency certificate and install it, prioritizing the automated path over manual cert generation to reduce the chance of a second mistake under time pressure.
• As a last resort only, and only if policy allows, terminate TLS at a CDN or edge proxy that already has a valid cert and forward over a private/trusted link to the backend.
• Validate before declaring resolved: run synthetic TLS handshakes and smoke tests against key endpoints, not just "the dashboard looks green."

Stakeholder and customer communication

• Acknowledge within the first ~10 minutes: post to the status page and internal channels with scope, known impact, and a time for the next update, even if the next update is "still investigating."
• Update on a fixed cadence (every 15-30 minutes) regardless of whether there's new information, since silence reads worse than "no change yet" during a global outage.
• Close the loop once restored with an accurate timeline and a plain-language root cause, then follow with a fuller postmortem summary to execs, product, and customer-facing teams so they can answer customer questions consistently.

Long-term remediation and process changes

• Automate the full certificate lifecycle (issuance and renewal) end to end so a human is never the trigger for routine renewal; the emergency manual path should exist only as a fallback, not the primary mechanism.
• Alert on the action, not just the deadline: a single "certificate expires in N days" alert is not enough, because it doesn't distinguish "renewal will happen automatically before then" from "renewal has already been silently failing." A stronger design pages a human if both conditions hold: fewer than a threshold of days remain, and there has been no successful renewal within the automation's normal cycle. For example, with a 90-day certificate and automation that renews at the 30-days-remaining mark, an escalation alert at 14 days remaining plus "no successful renewal event logged in the last 16 days" catches a silently-failing automation with real margin before it becomes a repeat of this incident, rather than firing on the same schedule the (broken) automation was supposed to act on.
• Rehearse the failure: run a game day where the cert automation is deliberately disabled and confirm the escalation alert actually fires and the manual fallback actually works, rather than assuming it does.
• Track the postmortem to closure with named owners and deadlines for the automation and alerting changes, not just the incident write-up.

Trade-offs and pitfalls

• A pure "days until expiry" alert is the common mistake: it looks reasonable until you realize it fires on the same cadence the automation was already supposed to satisfy, so if the automation silently breaks right after a renewal, the alert doesn't add meaningfully more warning time than the automation's own schedule did. Tying the alert to "no successful renewal observed" closes that gap.
• Emergency manual certificate issuance under time pressure is itself a risk (wrong domain, wrong chain, wrong key usage); preferring automated emergency issuance over a fully manual process reduces that risk even during an incident.
• Communicating on a fixed cadence even with "no update" costs credibility less than going silent, but it does require someone dedicated to comms so the person fixing the issue isn't also the one writing status updates.

Direct answer

Treat the cascade as a causal chain to reconstruct, not a single event: build a timeline correlating the failover trigger (DNS/LB cutover) against the target region's saturation signals, then look specifically for the amplifying mechanism that turned a redirect into a second outage, whether that's insufficient headroom, retries piling on top of already-degraded capacity, or slow autoscaling. The single highest-value question in the investigation is usually: was the surviving region ever provisioned to absorb the failed region's full traffic, or only a fraction of it?

Investigation approach

Build the timeline first, then work backward from the second region's failure point to find what was different about the moment it tipped over: a specific traffic level, a specific downstream dependency saturating, or a specific automation event (autoscaler decision, config push).

Common contributing factors

• No global rate limiting, so the surviving region absorbed 100% of demand instead of shedding the excess.
• Autoscaling too slow to react within the failover window, so capacity lagged the traffic shift.
• DNS TTL or connection-draining overlap meaning the "instant" failover wasn't actually instant, briefly inflating total concurrent demand across both regions.
• Retries from clients and internal services amplifying load on top of an already-degraded region rather than backing off.
• Regional headroom sized for local (within-region) redundancy, not for absorbing a peer region's entire traffic.

Worked example

Suppose pre-incident traffic is 100,000 RPS total, split 60/40 between Region A and Region B, and each region carries standard N+1-style headroom (provisioning one extra unit of redundant capacity beyond what's needed for steady-state load) of 1.3x its own steady share:

$$\begin{aligned} D_A &= 0.6D = 60{,}000\ \text{RPS}, \quad D_B = 0.4D = 40{,}000\ \text{RPS} \\ C_A &= 1.3 D_A = 78{,}000\ \text{RPS}, \quad C_B = 1.3 D_B = 52{,}000\ \text{RPS} \\ L_B^{failover} &= D_A + D_B = 100{,}000\ \text{RPS} \\ deficit &= \frac{L_B^{failover} - C_B}{C_B} = \frac{100{,}000 - 52{,}000}{52{,}000} \approx 0.923\ (92.3\%\text{ over capacity}) \end{aligned}$$

Region B was provisioned with a comfortable 30% margin over its own normal share, but a full regional failover requires it to absorb Region A's traffic on top of its own, roughly double what its headroom was ever sized for. This is the analytical version of "the surviving region became overloaded and failed too": the headroom math was correct for local redundancy and simply never accounted for full peer-region failover, which is a sizing decision, not a runtime bug.

Short-term response

• Shed or throttle low-priority traffic in the surviving region immediately to bring load back under its safe ceiling.
• Apply global (not per-region) rate limits so no single region can be driven past capacity by a redirect.
• If a third region or emergency capacity exists, spread the failed-over traffic further rather than concentrating it on one target.

Long-term changes

• Provision each region's headroom against "can absorb a full peer failover," not just local N+1, or explicitly accept and design for graceful degradation (shed low-priority traffic) as the intended behavior under full failover instead of assuming full capacity will always be there.
• Add global rate limiting/quotas so a cutover can't silently push one region past its ceiling.
• Rehearse failover under load (game days) so the deficit above is discovered in a drill, not during a real incident.
• Tighten the cutover mechanism (shorter TTLs, coordinated draining) to avoid the transition period itself briefly inflating total demand.

Trade-offs and pitfalls

• Provisioning every region to absorb 100% of peer traffic (true N+1 at the global level) is expensive; the alternative is explicit, tested graceful degradation, which is cheaper but requires deciding in advance what gets shed and validating that decision under real load, not assuming it during the incident.
• A postmortem that stops at "Region B got overloaded" without asking whether B's headroom was ever meant to cover 100% of A's traffic will produce the wrong fix (e.g. "scale B faster") instead of the real one (headroom policy was undersized for the failure mode that actually occurred).
• Retries and fan-out amplification can make the root cause look like "the region couldn't handle X RPS" when the real offered load, once amplified, was meaningfully higher than X; traces are what separate offered load from amplified load.

Direct answer

Layer 4 load balancers make routing decisions using only transport-layer metadata (source and destination IP, port, protocol) and forward or proxy TCP/UDP connections without looking at the payload. Layer 7 load balancers terminate the application protocol, usually HTTP or HTTPS, and route on request content: host header, URL path, cookies, or other headers. L4 is faster and protocol-agnostic because it never parses the payload; L7 costs more CPU per request but can make far smarter routing, security, and traffic-shaping decisions. Pick L4 when you need raw throughput or must preserve end-to-end encryption; pick L7 when routing needs to understand HTTP semantics.

Structured elaboration

Decision guidance: choose L4 when the balancer must not (or need not) understand the payload, or when throughput at minimal overhead is the priority. Choose L7 when the routing decision itself depends on request content. Many production systems run both: an L4 tier absorbing raw connections at the edge, with an L7 tier immediately behind it for content-aware routing.

Worked example

Consider two systems that need a load balancer. First, a Postgres connection pooler in front of a cluster: clients authenticate to the database itself over TLS, connections are long-lived, and there is no HTTP semantics to route on. An L4 balancer is the right fit here: it forwards TCP connections without touching the encrypted stream, so end-to-end TLS stays intact and per-connection overhead stays minimal.

Second, a public REST API that must send /v1/users and /v1/orders to two different backend services, apply a WAF rule to block known bad user agents, and terminate TLS once at the edge. None of that is possible without reading the request line and headers, so this requires an L7 balancer, even though it costs more CPU per request than the L4 case.

Trade-offs & pitfalls

• L7 termination breaks end-to-end encryption unless the balancer re-encrypts to the backend (TLS bridging); this is a common follow-up in security-conscious interviews.
• L4 cannot do content-based routing or cookie affinity. Bolting host/path logic onto an L4 balancer just pushes the work down a layer where it is harder to operate.
• Hybrid designs (L4 at the outer edge, L7 immediately behind it) are standard practice, not a compromise: the L4 tier absorbs raw connection volume and the L7 tier handles content-aware decisions.
• Common wrong turn: treating L7 as strictly better. It adds a per-request parsing and termination cost and a larger attack surface (header injection, request smuggling) that an L4 balancer never has to reason about.