Netflix Solutions Architect Interview Preparation Guide - Senior Level

Direct answer

Reactive autoscaling scales based on what's happening right now (a metric crosses a threshold); predictive autoscaling scales based on a forecast of what's about to happen. Neither is inherently better; they answer different questions, and the metric you feed either one matters more than which mode you pick. The strongest signals are the ones closest to actual user-facing backlog or experience (request latency, queue depth) rather than machine-level resource stats (CPU, memory), which correlate with load but can lag or miss it entirely for I/O-bound services.

Structured elaboration

Reactive vs. predictive, and why the metric choice compounds. A reactive policy watching CPU will always be a step behind, because CPU rises only after load has already increased. A reactive policy watching queue depth is closer to real time, because a growing backlog is itself the leading indicator of "we're falling behind," not a downstream symptom of it. Predictive scaling (using historical traffic patterns or short-term forecasting to act before load arrives) reduces this lag structurally, but it only works well when it's built on a metric with a real predictable pattern (daily/weekly seasonality in request rate, for instance); it doesn't fix the fact that CPU is a noisy, lagging signal to forecast against in the first place.

Target-tracking math. A common reactive scaling pattern (used by cloud provider target-tracking policies and by Kubernetes' Horizontal Pod Autoscaler alike) scales replica count proportionally to how far the current metric is from target:

$$\text{desired replicas} = \left\lceil \text{current replicas} \times \frac{\text{current metric}}{\text{target metric}} \right\rceil$$

If 10 replicas are running at 80% average CPU against a 50% target:

$$\left\lceil 10 \times \frac{80}{50} \right\rceil = 16$$

The policy adds 6 replicas in this step. This same formula works for any metric with a sensible linear relationship to load (request rate, queue depth per worker), which is why the choice of which metric feeds it matters more than the formula itself: feed it a noisy or lagging metric and it will confidently compute the wrong target.

Worked example

A web-facing auto scaling group (ASG) targets 50% CPU with a 60-second scale-out cooldown and a 300-second scale-in cooldown, intentionally asymmetric: react to a spike quickly, but wait much longer before removing capacity. Without that asymmetry, consider what happens if CPU oscillates naturally between 45% and 65% around a single 50% threshold with equal cooldowns on both directions: the group scales out when CPU ticks above 50%, the added capacity immediately drags average CPU back below 50%, the group scales back in, CPU climbs again, and the fleet thrashes (adds and removes instances repeatedly) without ever settling. The fix has two parts: a hysteresis band (scale out above, say, 70%; scale in below, say, 30%, rather than a single shared threshold) so normal noise doesn't cross both boundaries, and a longer scale-in cooldown than scale-out cooldown, so the group is quick to protect against real load but slow and conservative about giving capacity back. That second part is the "safe scale-down" half of the policy: it's just as important as reacting fast to a spike, because a policy that scales in as aggressively as it scales out will flap on exactly the kind of noise a real production CPU curve always has.

Trade-offs & pitfalls

• Relying on a single metric is brittle: a service can be CPU-healthy while its queue backs up (I/O-bound work) or CPU-saturated while user latency is fine (CPU-bound but well within budget); combining two or three signals catches what any one alone would miss.
• Custom metrics add real value but add a real dependency: if the metric pipeline itself lags or goes stale, the autoscaler is making decisions on old data and can either overreact to a stale spike or fail to react to a live one.
• Symmetric cooldowns (same wait time for scale-out and scale-in) are a common oscillation trap; scale-in should almost always be more conservative than scale-out.
• Predictive scaling built on a metric with no real seasonal pattern (or one whose pattern just changed, e.g., after a product launch) will confidently mis-forecast; it needs the reactive layer as a backstop, not as a replacement.

Direct answer

Cold standby (infrastructure defined as code, nothing running until failover) costs very little to hold in reserve but takes longer to bring up because you're booting instances, restoring data, and warming caches from a stopped state. Warm standby (a smaller version of production running continuously, scaled up on failover) costs meaningfully more every month because you're paying for idle-but-running capacity, in exchange for a much faster recovery because the instances, connections, and caches are already live. The decision comes down to comparing the dollar cost of that continuous capacity against the dollar cost of the extra downtime cold standby would cost you during an actual failover.

Worked example

Assume (illustrative inputs for this worked example, not live pricing): a production tier of 10 instances, an on-demand rate of $0.504 per instance-hour, cold-standby storage-only cost of $50 per month, and a warm standby running 3 of the 10 instances continuously, scaled to 10 on failover.

Warm standby monthly cost:

$$\text{warm monthly} = 3 \times \$0.504/\text{hr} \times 720\text{ hr} = \$1{,}088.64$$

Cost delta versus cold standby (storage-only, no running compute):

$$\Delta\text{cost} = \$1{,}088.64 - \$50 = \$1{,}038.64 \text{ per month}$$

Recovery time for each:

Cold standby has to boot instances from scratch, restore data, and validate before serving:

$$RTO_{cold} = 8 + 45 + 15 = 68 \text{ min}$$

(8 minutes to boot instances, 45 minutes to restore data, 15 minutes to validate and run health checks.)

Warm standby just scales the already-running, already-warmed instances up and cuts traffic over:

$$RTO_{warm} = 5 + 1 + 1 = 7 \text{ min}$$

(5 minutes for the autoscaler to add the remaining 7 instances, 1 minute for DNS or load-balancer cutover, 1 minute for health checks, since caches and connections are already primed.)

Putting a price on the recovery-speed difference:

$$\Delta RTO = 68 - 7 = 61 \text{ min}$$ $$\frac{\$1{,}038.64}{61} \approx \$17.03/\text{min of RTO saved}$$

That $17.03-per-minute figure is the number to compare against the business's actual cost of downtime. If an hour of full outage costs the business more than the $1,038.64 monthly premium calculated above (the same delta, since it's exactly the 61-minute RTO gap priced at $17.03/minute), warm standby is straightforwardly worth paying for every month whether or not a failover ever happens; if the realistic outage cost is lower than that, cold standby's much smaller ongoing bill is the better trade for the same eventual RTO.

Trade-offs & pitfalls

Warm standby's cost is fixed and recurring whether or not a disaster ever occurs, which makes it easy to defend during a postmortem but easy to question during a routine budget review, since it's a real monthly line item paying for something that (hopefully) is never used. Cold standby's failure mode is subtler: the 68-minute RTO in the example assumes the restore and boot process actually works cleanly, which is exactly what the restore-verification practice from backup testing is meant to confirm; an untested cold-standby runbook routinely takes much longer than its assumed number the first time it's actually exercised for real. The most common mistake is picking one mode uniformly for an entire system instead of tiering it: keeping the truly critical, revenue-blocking tier warm while leaving lower-priority or easily-rebuilt tiers cold captures most of the recovery-speed benefit without paying the full warm-standby bill across everything.

Situation: Client needs sub-second ingest and query across billions of series (high-cardinality), strict SLAs, and predictable TCO. Below I compare two choices: specialized TSDBs (Timescale/InfluxDB) vs. custom pipeline (Kafka ingest + ClickHouse queries), across the requested dimensions and conclude with recommendation.

Ingestion throughput

• TSDBs: Designed for time-series writes; can achieve very high ingest per node (millions/sec) using batched line protocol/Copy APIs and partitioning. Throughput often limited by disk I/O and write amplification from retention/compaction.
• Kafka+ClickHouse: Kafka handles virtually unlimited ingest scale by scaling brokers/partitions; ClickHouse can sustain very high insert-throughput with batched/columnar bulk inserts (native formats). Overall more horizontally scalable for bursty workloads because Kafka decouples producers from consumers.

Query latency

• TSDBs: Optimized for typical TS queries (recent-window, aggregates). With hypertables/continuous aggregates (Timescale) or TSM indices (Influx) you get low-latency sub-second queries on recent data. High-cardinality ad-hoc queries can degrade if indexes/series cardinality explode.
• ClickHouse: Extremely fast OLAP execution for large aggregates over many series, often sub-second for pre-aggregated or well-partitioned data. Ad-hoc point queries across many series may be heavier but mitigated via materialized views and using Kafka → CH streaming.

Index / metadata costs

• TSDBs: Maintain per-series metadata (schema, tag indices). High-cardinality means memory-resident index growth (open series, per-series state) — can consume large RAM and complicate cluster sizing.
• ClickHouse: Columnar storage avoids per-series in-memory index explosion; partitions and primary key (sorting key) control read efficiency. Metadata is lighter, but you may need secondary structures (materialized views, projected columns) to emulate TSDB tag-indexed queries.

Retention / compaction

• TSDBs: Native retention policies and background compaction; efficient downsampling (continuous aggregates) and TTLs. Simpler to manage retention per hypertable.
• ClickHouse: Supports TTLs, merge tree compaction tunables, and materialized views for downsampling. Requires tuning merges to avoid CPU/disk spikes; retention enforcement can be more manual across partitions.

Operational complexity

• TSDBs: Simpler operational model—single product stack with built-in ingestion, retention, and query semantics. Easier for teams with limited SRE resources.
• Kafka+ClickHouse: Higher operational burden: manage Kafka clusters, connectors (Kafka → ClickHouse), ClickHouse cluster, monitoring, backpressure, schema evolution, and end-to-end exactly-once/at-least-once semantics. But offers composable scaling and fault isolation.

Total cost

• TSDBs: Lower upfront operational cost for moderate scale. At extreme cardinality and scale, license (enterprise) and larger instances for memory-heavy indices raise costs.
• Kafka+ClickHouse: Higher initial op and infra cost (more components) but better cost-efficiency at very large scale due to commodity scaling and columnar compression; storage-to-query cost ratio often lower for historical queries.

Recommendation (Solutions-Architect view)

• If workload is dominated by recent-time, low-to-moderate cardinality, and you want lower operational overhead: choose a TSDB (Timescale or Influx) with continuous aggregates and hypertables.
• If you expect billions of series, highly bursty ingest, heavy historical analytical queries, or need independent scaling of ingest vs query layers: choose Kafka + ClickHouse. Invest in stream processing (ksql/Flafka/Materialized views) and an operational playbook (monitoring, automated schema migrations, backpressure handling).
• Hybrid option: Use Kafka as ingestion buffer feeding a TSDB for hot data (sub-second queries) and ClickHouse for cold/historical analytics—gives best-of-both at cost of extra complexity.

Key trade-offs: TSDBs simplify operations but struggle at extreme cardinality due to in-memory indices; Kafka+ClickHouse scales more predictably for massive cardinality and analytics but requires stronger SRE and higher integration effort.

Direct answer

Without formal authority, the lever is technical credibility built through artifacts other people can independently check: a written proposal grounded in real data, a working prototype, and a track record of small delivered wins, not persuasion technique. Leading through influence differs from direct management in exactly this: you cannot assign the work, so every step has to make it easier for someone else to say yes than to say no.

Structured elaboration

• Diagnose before proposing. Collect the evidence (incident data, latency trends, where teams keep colliding) before writing anything. An undiagnosed proposal reads as an opinion; an evidence-backed one reads as a finding.
• Write it down concretely. A short design document with a specific problem statement, two or three named milestones, and a measurable success criterion for each (a target latency or error-rate range, not a vague goal) lets someone evaluate the idea without trusting your judgment on faith.
• Build the smallest thing that proves the idea, not the whole thing. A scoped prototype against a single team's workload is cheap to say yes to and gives you a concrete result to point at instead of a projection.
• Pull in the people who would implement or be affected, deliberately. A proposal with co-authors from outside your own team is harder to dismiss as one person's pet project. This is also the mechanism that keeps direction from becoming siloed inside your own team's worldview: without deliberately involving adjacent teams, "technical direction" quietly becomes "what my team already wanted to build."
• Keep it visible. Regular short updates and a shared tracker mean momentum does not depend on you personally chasing people down.

Worked example

A platform initiative is expected to eventually support on the order of a million users, and teams currently ship changes ad hoc with no shared plan. As an individual contributor, you spend several weeks pulling incident and latency data into a few named failure themes, then write a short design proposal with milestones for an observability baseline, a prototype for the highest-risk theme, and a backward-compatible rollout, each with an explicit success measure. You pilot the riskiest piece with one team first, because a single team's result is concrete evidence rather than a projection, then bring that data back to the wider group before asking anyone else to adopt it. The honest result of this kind of effort is usually partial: some teams adopt the pattern quickly because the pilot removed their specific pain, others wait for a second team to prove it first, and the plan itself gets revised once a stakeholder objects to a milestone you had not stress-tested. That is expected, not a failure of the approach; the goal was to make the direction adoptable, not to force it.

Trade-offs and pitfalls

The dependency on artifacts cuts both ways: a proposal or prototype that turns out to be wrong is now visible and attributable to you in a way a vague opinion never was, which is uncomfortable but is also what makes the influence real. The bigger failure mode is over-investing in the write-up and under-investing in the pilot: a well-argued document with no working proof is easy to admire and easy to ignore. Influence exercised entirely within your own team's technical culture is the other common trap: it produces direction that only makes sense to your team, which is exactly the siloing this approach is meant to avoid.

Direct answer

In the moment, the sequence is acknowledge, then a tight piece of evidence, then a small concrete next step, not a bigger version of the argument that just got dismissed. The goal in the room is not to win the disagreement outright, it's to convert "I'm not convinced" into "let's find out," while leaving the executive's standing in the room intact.

Structured elaboration

The three-step in-room conversation plan. When a VP publicly disagrees and is skeptical of your evidence, the structure that reliably works is: (1) opening, an acknowledgment that takes the concern seriously rather than defending immediately; (2) presenting evidence and reframing, concise, specific, and aimed at exactly what they said was unconvincing, not a restatement of everything you already said; (3) reaching agreement in the room, a small, low-risk next step both people can say yes to right now, rather than asking them to simply change their mind.

1. Opening: acknowledge specifically, not generically. "I hear you, that concern about X is fair" lands better than a vague "I understand your point."
2. Evidence and reframing: two or three sharp points, tied directly to what they said was weak, not a restatement of the original pitch.
3. Reaching agreement: propose a bounded next step (a short pilot, a follow-up with specifics, a narrow test) with an explicit exit condition if it doesn't hold up, so agreeing costs them little.

Setting changes the moves, not just the tone. A pushback that happens privately in a small meeting is a different problem from the same pushback happening publicly, in an all-hands, in front of the exec's own reports and peers. In public, add one more move: correct the record without undermining the executive's credibility in front of the room. That usually means agreeing publicly to follow up with the specifics rather than contradicting them point by point in front of others, then having the detailed evidence conversation offline. Winning the technical point in public while making the executive look uninformed in front of their own team is a worse outcome than losing the point that day.

When it's not disagreement, it's an in-the-moment directive. Sometimes the executive isn't just skeptical, they're asking for an immediate change you believe is unsupported. The move here is to script the pushback so it protects the relationship while still registering the concern: something like "I want to make sure we don't regret this. Can I take ten minutes with you right after this to pressure-test it before we commit?" This buys the review you need without refusing the executive in the room, and it signals that the concern is about the decision, not about resisting them.

Worked example

Situation: a recommendation to move a workload to a new vendor gets pushed back on live in a small planning meeting: "this feels like vendor marketing, I'm not convinced."

Opening: "I hear you, vendor marketing can definitely overpromise, that's a fair thing to be skeptical of. Can I ask specifically what feels like hype, the performance claims, the security posture, or the cost savings? That tells me exactly where to focus."

Evidence: once the concern narrows to performance claims specifically, the response stays tight: independent benchmark results against the current baseline, the relevant compliance attestations already on file, and two reference customers in a similar industry who can be contacted directly.

Reaching agreement: "Rather than deciding on the vendor's word or mine, I'd propose a six-week pilot on a non-production workload with four explicit pass criteria: latency, integration effort, operational fit, and cost at scale. If it misses any of them, we stop and reassess. Does that address the concern, and are you comfortable greenlighting a bounded pilot?"

The public variant of the same scenario: the same pushback happens in an all-hands instead. The opening and evidence stay nearly identical, but the close changes: "That's worth getting right, let me pull the benchmark data and the reference contacts together and walk you through it right after this," rather than debating the specifics live in front of the room. The detailed evidence conversation and the pilot proposal then happen in the follow-up, not in public.

Trade-offs & pitfalls

• Escalating the argument in the moment (more data, louder) rarely works on a skeptical senior audience; it reads as not having heard the objection.
• In public settings, correcting the record too bluntly can win the specific point and cost the relationship; the better move is usually to take the detailed disagreement offline while still registering, briefly and respectfully, that the record needs a follow-up.
• Conceding immediately to an unsupported in-the-moment directive avoids conflict but can commit the team to a bad call; the scripted "let's pressure-test it first" move protects both the relationship and the decision.
• Not every pushback deserves the full three-step treatment. If the executive is right, the senior move is to say so plainly and move on, not to manufacture a pilot to save face.

Decide per feature, not per product: for each piece of state, ask what a stale or lost read or write actually costs, in both directions. Features whose operations are naturally commutative or idempotent, like a like count or a view count, tolerate eventual consistency cheaply, because being briefly wrong self-heals and nobody's safety depends on the exact number. Features where a stale or lost update directly causes an incorrect, hard-to-reverse outcome, money, a limited resource, or an invariant like at least one thing must remain true, need strong consistency or a convergent structure specifically engineered not to lose updates, even though that costs latency and availability during a partition.

The three named features

• Like counter: pure eventual consistency is fine. It is a simple, non-negative, additive count; a grow-only-counter-style commutative merge, or even just an approximate cache, means a brief undercount self-corrects on the next sync, and no one's correctness depends on the exact number at any instant.
• Timeline feed: needs causal consistency, not full linearizability. A reply must never be visible before the post it replies to, but unrelated posts from different authors can be shown in different orders to different viewers without breaking anything.
• A user's own settings: needs read-your-writes, or session consistency, for that user, not global linearizability. If a user just changed a setting, their own very next read must reflect it, or the product looks broken to them, but there is no requirement that every other user's session see that change instantly.

Feature store: per-user causal consistency, not global

In a machine learning feature store, a user's own online feature update, say their most recent click, must be visible to their own next inference request; that is the same read-your-writes requirement as the settings example, scoped per user. It does not need to be globally linearizable across all users' sessions, since one user's features have no bearing on another user's inference.

One service, different conflict-resolution policy per preference type

Within a single settings service, the right conflict-resolution policy varies by preference type, not just by feature:

• A boolean toggle preference, say dark mode on or off, is naturally last-write-wins-safe: whichever value wins is still a valid state, and there is nothing to lose except which of two valid values stuck.
• A set-valued preference, a list of blocked users, is not last-write-wins-safe. Concretely: a user's phone, offline with edits queued, sets blocked_users to {X}; concurrently, the same user's laptop sets blocked_users to {Y}, unaware of the phone's change. If the laptop's clock happens to run a few minutes fast, a naive last-write-wins merge picks the laptop's write purely because its timestamp looks later, giving blocked_users = {Y} and silently unblocking X, an actual correctness bug the user never asked for. An observed-remove-set merge, unioning the adds while respecting only the removes each device actually observed, instead gives blocked_users = {X, Y}, preserving both edits.

Billing and metering aggregation: under-counting vs over-counting, both cost money

A usage counter feeding billing must never silently under-count, that is straightforward revenue leakage, and ideally should not over-count either, since that produces customer complaints and refund credits. Both are direct cost consequences of picking the wrong merge strategy, not just an abstract correctness concern.

Worked example: plain overwrite vs a G-Counter, same events, different outcomes

Two shards independently record usage events for the same customer and need to combine into one total.

Plain mutable counter, naive approach:

• Shared counter starts at 0.
• Shard 1 reads the counter (0), adds 3 new usage events, writes 3.
• Shard 2, concurrently, also reads the counter before shard 1's write lands (0), adds 5 new usage events, writes 5.
• Final stored value: 5. Shard 1's update was overwritten and lost.

True total is 3 + 5 = 8, but the stored value is 5: 3 units of usage vanished, a direct case of under-counting and revenue leakage.

G-Counter approach, same events:

• Each shard keeps its own slot, both starting at 0.
• Shard 1 increments its own slot by 3.
• Shard 2 increments its own slot by 5.
• Read = sum of slots.

$\text{total} = c_1 + c_2 = 3 + 5 = 8$

No event is lost, because each shard only ever writes to its own slot; there is no shared mutable field for a concurrent write to overwrite.

This is the concrete cost of getting the direction wrong: assuming a plain field is fine because writes are rare quietly loses exactly the increments that happen to race, and the fix is not more locking, it is picking a data structure whose merge cannot lose an update in the first place.

Trade-offs & pitfalls

• Getting it too eventual: silent lost updates, as in the plain-counter example above, and user-visible correctness bugs, as in the blocked-users example, both compounded by a debugging nightmare, since the bug is nondeterministic and only shows up when two writes race.
• Getting it too strong: unnecessary coordination latency and reduced availability during a network partition for state that never needed it. A like counter does not need to block its write path on a quorum round trip.
• Common wrong turn: picking one consistency model for the whole product instead of reasoning feature by feature. A senior answer explicitly separates what needs strong or linearizable behavior, what needs causal or session guarantees, and what tolerates pure eventual consistency, rather than defaulting the entire system to one setting.

As a Solutions Architect, explicitly surface these primary constraint categories during discovery — each with a probing question and one concrete example of how a hidden constraint can change the solution approach.

1. Budget/Cost

• Probing question: “What is the total lifecycle budget (initial + 3–5 year Opex) for this project?”
• Hidden constraint impact: Strict Opex cap forces favoring managed PaaS over custom on‑prem build to reduce staffing and maintenance costs.

2. Timeline/Delivery

• Probing question: “Which calendar dates are immovable and what are penalties for missing them?”
• Hidden constraint impact: Unmovable launch pushes us to an incremental MVP with limited features and later phased rollouts rather than a big‑bang rewrite.

3. Legacy technical debt / Existing architecture

• Probing question: “Which current systems cannot be changed or taken offline during migration?”
• Hidden constraint impact: In-place constraints mean designing a strangler pattern with adapters instead of a full replatform.

4. Regulatory / Compliance

• Probing question: “Are there specific regulations (GDPR, HIPAA, PCI, local data laws) that restrict data flows or storage?”
• Hidden constraint impact: Data residency rules require regional encryption and separate regional deployments instead of a single global cluster.

5. Third‑party vendors / Integrations

• Probing question: “Which external systems/APIs must be used and what SLAs or licensing limit usage?”
• Hidden constraint impact: A vendor’s API rate limit necessitates caching and batching layers or alternative vendor selection.

6. Security & Access Controls

• Probing question: “What minimum security certifications, auth methods, and audit capabilities are required?”
• Hidden constraint impact: Need for MFA, SSO, and audit trails may rule out certain lightweight SaaS and require enterprise‑grade identity integration.

7. Performance / Scalability

• Probing question: “What peak loads and latency SLAs must we meet, and are they expected to grow?”
• Hidden constraint impact: High concurrency SLA pushes scalable stateless services and autoscaling design rather than a single DB instance.

8. Operational & Skillset Constraints

• Probing question: “What skills does your operations/dev team currently have, and what level of handover is acceptable?”
• Hidden constraint impact: Lack of DevOps expertise favors managed services or including an operations runbook and training in scope.

9. Data & Migration

• Probing question: “Are there data quality, archival, or migration windows we must honor?”
• Hidden constraint impact: Large legacy datasets with limited migration windows require bulk transfer tools, offline ETL, or a pilot phased migration.

Asking these explicitly — and following up with concrete examples and non‑functional targets — reveals hidden constraints early so solutions are feasible, cost‑effective, and deliverable within real business limits.

Approach: with only diagrams and two stakeholder interviews I’d do a focused risk-first feasibility assessment: identify the highest-impact unknowns, capture testable assumptions, and prepare a rapid validation plan to resolve them when I get access.

Top five risks

1. Integration risk — unknown APIs, protocols, adapters, and data formats between systems.
2. Data quality/volume risk — unclear data size, growth, latency, and transformation needs.
3. Security & compliance risk — unverified auth flows, encryption, network zones, and regulatory constraints.
4. Operational/Runbook risk — gaps in monitoring, backups, DR, and support model.
5. Timeline/cost/dependency risk — hidden third-party dependencies or licensing that block delivery or inflate cost.

Key assumptions to document (explicit, testable)

• Source systems expose REST/gRPC or require middleware (integration assumption).
• Peak data throughput and retention (e.g., X TPS, Y TB/month).
• Authentication method and trust boundary (OAuth2/SAML, client certs).
• SLAs for RTO/RPO and acceptable latency.
• Ownership of infrastructure and access timelines (who provides accounts/config).

Quick validation plan once access is granted (first 1–7 days)

1. Sanity walkthrough: run a 1-hour architecture review with stakeholders to confirm diagrams and record divergences.
2. Access checklist: request sample API specs, schema dumps, a representative data sample, network diagrams, and credentials in a vault.
3. Smoke tests: call a sample API, transfer a small data batch, authenticate, and verify basic telemetry to confirm protocols and auth.
4. Metrics check: run simple load probe on representative data to validate throughput and storage sizing assumptions.
5. Security check: confirm TLS, auth flows, and any compliance docs; run a quick threat checklist.
6. Dependency map & reforecast: enumerate third parties, lead times, and revise timeline/cost and residual risk heatmap.
7. Risk mitigation plan: for each residual risk propose short-term mitigations (stubs/queues/feature flags), and recommend a focused PoC if critical (2–3 day spike).

Why this works: it prioritizes high-impact unknowns, makes assumptions explicit and measurable, and uses minimal-touch validation (walkthroughs + smoke tests + metrics) to quickly convert unknowns into decisions or mitigations for a reliable feasibility answer.

Direct answer. HTTP/2's multiplexing removes the old "avoid too many small requests, batch them into one big one" advice almost entirely for requests to the SAME origin, since many requests can now share one connection with no head-of-line blocking at the HTTP layer; HTTP/3's move to QUIC (over UDP instead of TCP) additionally removes TCP-level head-of-line blocking and speeds up connection setup, which matters most on flaky mobile networks, and both change what your load balancer and CDN need to actually do.

Multiplexing changes API design guidance that predates it. Under HTTP/1.1, browsers opened a limited number of parallel TCP connections per origin, so an API client benefited from batching many small requests into fewer, larger ones (a classic piece of REST API advice: avoid chatty, many-small-calls API shapes). Under HTTP/2, many logical requests share ONE connection via multiplexed streams, so issuing several smaller, well-scoped requests to the same origin is no longer the performance problem it used to be; the old advice does not fully disappear (a request still has real per-call overhead: headers, server-side routing, business logic), but the SPECIFIC "too many TCP connections" penalty it was originally guarding against is gone. Concretely: under a common browser limit of about 6 concurrent connections per origin under HTTP/1.1, a page issuing 30 small API calls to the same origin would need to queue them into 5 sequential batches of 6; under HTTP/2's multiplexing, all 30 can be in flight at once over the single connection, with no batching required at all.

Header compression (HPACK/QPACK). HPACK is HTTP/2's header-compression format, and QPACK is its HTTP/3 counterpart, redesigned so header compression still works correctly when QUIC delivers streams out of order, which plain HPACK cannot tolerate. Both work by having each side maintain a shared table of previously-seen header name/value pairs: once a header (an auth token, a standard Accept or User-Agent value) has been sent once on a connection, later requests on that SAME connection can send a short reference into that table instead of the full header text again. Repeated headers across many requests to the same connection get compressed incrementally this way, which meaningfully reduces overhead for an API client making many small, frequent calls, exactly the pattern multiplexing now makes more attractive in the first place.

HTTP/3 and QUIC specifically. Because QUIC runs over UDP and manages its own stream multiplexing independently, one lost packet on one logical stream no longer blocks delivery of data on OTHER streams sharing the same connection, unlike TCP where a single lost packet stalls the entire connection until it is retransmitted; this specifically helps a mobile client on a lossy network, which is exactly the scenario where TCP-level head-of-line blocking used to hurt the most. QUIC's connection setup also folds the transport and TLS handshakes together, cutting the round trips needed before the first real request can even be sent, which matters disproportionately on high-latency mobile connections where each round trip is expensive. Concretely: a fresh HTTPS connection over TCP + TLS 1.3 needs 2 round trips before the client can send its first request byte (1 to establish the TCP connection, 1 for the TLS 1.3 handshake); QUIC combines both into one combined handshake, needing just 1 round trip for a new connection (and its 0-RTT mode can send request data on the very first flight when resuming a connection to a host the client has already talked to). On a mobile connection, where a single round trip commonly costs on the order of 50-100ms, that one saved round trip is a real, directly-felt latency difference before any actual request-response work even starts.

What changes in load-balancing and CDN configuration. Load balancers and CDNs need to actually terminate and understand HTTP/2 or HTTP/3 themselves (not simply pass bytes through) to preserve these benefits end-to-end, and per-request load-balancing algorithms that assumed "one connection roughly equals one in-flight request" need rethinking, since one HTTP/2 or HTTP/3 connection can now carry many concurrent logical requests, changing what "an overloaded backend" or "a slow connection" even looks like from the load balancer's point of view. Many CDNs and cloud load balancers already handle this transparently, but a self-managed edge layer built assuming HTTP/1.1 semantics needs deliberate upgrading, not an assumption that the protocol change is invisible to it.

Trade-offs and pitfalls. The most common mistake is treating this purely as an infrastructure upgrade with no API-design implications; an API whose CLIENTS were specifically designed around HTTP/1.1's connection limits (aggressively batching everything into one mega-request "to save connections") may actually see LESS benefit from multiplexing than a more naturally-shaped API would, since it already paid the batching cost the new protocols were meant to make unnecessary.

Requirements & goals:

• Maximize throughput while keeping per-item latency below SLA (e.g., 95th percentile < L_sla).
• Adapt to varying load and processing cost.

Observed metrics (per worker / per shard, windowed e.g., 10s):

• backlog_size: queued items awaiting processing
• batch_size_current: current batch size
• avg_proc_time_per_item (μ): exponential moving average of processing time per item
• avg_batch_overhead (o): fixed overhead per batch (serialize/deserialize, network)
• p95_latency: measured 95th percentile end-to-end latency
• arrival_rate λ: items/sec (optional; estimate from backlog delta)

Control model (closed-loop PID-like with safety):

1. Compute target batch size B_target that optimizes throughput vs. latency:
   • Estimated per-batch latency = o + B * μ
   • Want p95_latency ≈ o + Bμ ≤ L_sla_target (use margin α, e.g., 0.9L_sla)
   • So B_latency_limit = floor((α*L_sla - o)/μ)
2. Use backlog to boost throughput when safe:
   • If backlog_size > high_watermark H_high and B_latency_limit> B_current → increase
   • If backlog_size < low_watermark H_low → decrease to reduce latency
3. Smooth adjustments:
   • ΔB = clamp(Kp*(B_target - B_current) + Kiintegral_error + Kdderivative, -max_step, +max_step)
   • Apply integer steps and ratelimit changes (e.g., once per control_interval = 1s–5s)

Safe bounds:

• B_min = 1 (or small lower bound for efficiency)
• B_max = min(system_limit, B_latency_limit * safety_margin e.g., 0.8) to avoid hitting SLA
• Absolute cap based on memory / processing constraints.

Anti-oscillation & stability:

• Hysteresis between increase/decrease thresholds (separate H_high/H_low).
• Exponential moving averages for μ and p95 to avoid reacting to spikes.
• Min dwell time between opposite direction changes (cooldown).
• Use conservative integral gain to avoid wind-up; reset integral on large load changes.
• When p95 crosses SLA, immediately shrink batch aggressively (emergency decrement) and backoff growth rate.

Starvation & variable load handling:

• If arrival_rate drops to near-zero but backlog persists, allow B to decay slowly to B_min to reduce latency.
• If arrival_rate spikes, opportunistically increase B up to B_latency_limit and B_max; if workers idle frequently, increase B to amortize overhead.
• Add deadline-aware priority: if oldest item wait_time > wait_threshold, force immediate dispatch even if batch small.

Operational notes:

• Per-shard tuning preferred (heterogeneous processing).
• Expose metrics and knobs (α, H_high/H_low, control gains) for ops and SRE.
• Simulate with representative traces before production rollout; implement canary rollout with telemetry and automatic rollback if SLA breached.