Infrastructure Strategy and Technology Selection Questions
Setting technical direction for infrastructure and deciding what to build on. Covers infrastructure vision and long-term roadmap, modernization and technical-debt strategy, platform decisions, and organizational and governance considerations, alongside the decision framework for adopting or retiring technology: build-versus-buy-versus-cloud-versus-on-premises trade-offs, vendor and platform evaluation, technology-portfolio rationalization, requirements-driven service selection, and weighing total cost of ownership and risk. It also covers managing an existing vendor relationship: quantifying and mitigating lock-in, negotiating contract terms, and planning an exit from a service the organization depends on. The leadership-altitude discipline of directing an infrastructure estate over time, building consensus among stakeholders with conflicting priorities, and persuading leadership or a team to accept a major technology change.
Design a proof-of-concept to compare a managed platform service against a self-managed equivalent before committing to one. Specify the experiments and metrics you'd collect (throughput, latency, operational overhead, mean time to recover), the failure-injection scenarios you'd run, and how you'd reach a go/no-go decision.
Sample Answer
A proof of concept (PoC) that actually settles a managed-versus-self-managed decision has its success and failure thresholds written down before any test runs, deliberately tests failure modes and not just peak throughput, and ends in a documented go/no-go against those pre-committed numbers rather than a subjective impression of "it felt fine."
Designing the PoC
- Metrics to collect: sustained throughput at a defined load, tail latency (p95/p99, the 95th and 99th percentile response time, which shows what your slowest real users experience, not just the average), operational overhead (hours of hands-on toil per week to run and tune it during the pilot), and mean time to recovery (MTTR) from an injected failure.
- Failure-injection scenarios, not just load tests: kill a node or process mid-traffic, saturate the CPU or network on a dependency, simulate an upstream dependency outage, and simulate a bad deploy that has to be rolled back. A platform that only gets load-tested on the happy path will pass a PoC and then fail in its first real incident, because the PoC never asked the question that actually matters operationally.
- Latency measured from the vantage point that matters: measure from the client's perspective (the caller actually waiting on the response), not just server-side processing time, since network hops and queuing between the two can dominate the difference between "the backend is fast" and "the user experienced it as fast." A benchmark that only reports server-side numbers is measuring the wrong thing for a latency-sensitive decision.
- Pre-committed go/no-go thresholds: set numeric thresholds for each metric before the test begins, plus at least one "kill criterion," a failure so serious it overrides every other metric (for example, any data loss during failover is an automatic no-go regardless of how good throughput looked).
- A lighter-weight variant for lower-stakes decisions: when the decision's blast radius doesn't justify a multi-week experiment-and-metrics exercise, a lightweight PoC template, objectives, scope, timeline, and deliverables, with a shorter and less rigorous set of go/no-go criteria, is a legitimate and honest alternative. The rigor of the PoC should match the cost of getting the decision wrong, not be applied uniformly to every choice.
Worked example (illustrative)
Consider comparing a managed message queue against a self-hosted equivalent for an order-processing pipeline that needs to sustain 2,000 messages per second at peak with a 99.9% availability target. That target implies roughly 8.8 hours of allowed downtime per year (0.001 x 365.25 x 24 ≈ 8.8), which is a useful number to have in hand before setting the PoC's failure-recovery threshold.
A three-week PoC plan: week one runs a sustained load test at 2,000 messages/second for at least an hour, recording p99 latency measured client-side; week two runs failure injection (kill a broker node, saturate disk I/O) and measures message loss and recovery time; week three tracks operational load, how many alerts fired and how many required a human to act. A team running this exercise might set thresholds along these lines before starting: p99 latency under 250ms at the target load, recovery from a node failure in under 2 minutes with zero message loss, and less than 2 hours per week of hands-on operational toil during the pilot. Those specific numbers are illustrative of how a team sets its bar, not a report of an actual measured run; the discipline that matters is committing to numbers like these in writing before the test, not the particular values chosen.
Trade-offs and pitfalls
- Testing only the happy path is the single most common way a PoC gives false confidence; the PoC's job is specifically to find the failure modes production would otherwise find for you, at a worse time.
- Moving the goalposts after seeing results, quietly relaxing a threshold because the preferred option narrowly missed it, defeats the entire purpose of pre-committing them. If a threshold needs to change, that change should be justified and documented separately from the result it happens to rescue.
- A PoC environment that doesn't match production's real data shape (message size distribution, traffic burstiness, request skew) can pass cleanly and still mislead, because throughput and failure behavior both depend heavily on that shape, not just on the average load.
You need to establish measurable infrastructure KPIs for a platform or organization. Propose 6-8 KPIs spanning availability, latency, cost and delivery velocity, with clear owners and data sources, and explain how each ties back to a business outcome leadership actually cares about.
Sample Answer
Eight infrastructure KPIs (key performance indicators), grouped into reliability, delivery velocity, cost, and adoption, each with a named owner and data source, give leadership a small enough set to actually review and a large enough set to catch the trade-offs that a single number would hide.
The KPIs
| KPI | Group | Data source | Owner | Business outcome it ties to |
|---|---|---|---|---|
| Availability (uptime %) | Reliability | Synthetic monitoring / uptime checks | SRE (site reliability engineering) lead | Customer trust, churn, and any SLA (service-level agreement) credits owed |
| p95 latency (the 95th-percentile response time: the threshold under which 95% of requests complete) on the top user-facing endpoint | Reliability | Application performance monitoring (APM) | Service team lead | Conversion rate and perceived product quality |
| Change/deployment lead time | Velocity | CI/CD (continuous integration and delivery) system | Engineering manager | Time-to-market for revenue-generating features |
| Change failure rate | Velocity | Incident tracker and deploy log | SRE lead | Reliability reputation and downstream support cost |
| Mean time to recovery (MTTR) | Velocity/Reliability | Incident tracker | On-call/SRE lead | How much a real outage actually costs when one does happen |
| Cost per unit of business value (e.g., spend per 1,000 transactions) | Cost | Billing / cost-management tooling | Platform or FinOps (financial operations) lead | Gross margin, directly |
| Unplanned-work ratio (% of capacity spent firefighting vs. planned roadmap) | Efficiency | Project tracker | Engineering manager | Whether roadmap commitments to the business are credible |
| Capacity headroom (% of peak capacity used at normal load) | Adoption/Risk | Infrastructure monitoring | Capacity planning / SRE | Risk of an outage during a demand spike, and cost of over-provisioning |
Two of these are worth pairing deliberately: change/deployment lead time and change failure rate, because optimizing speed alone (shipping faster by skipping review) shows up immediately as a worse failure rate, so tracking both together prevents either one from being gamed in isolation.
Setting targets that progress over time
A KPI needs a target and a trend, not just a current value. For example, an availability target might start at 99.9% in year one (about 8.8 hours of allowed downtime per year: 0.001 x 365.25 x 24 ≈ 8.8) and tighten to 99.95% (about 4.4 hours/year) by year three as the platform matures and the cost of achieving the tighter target becomes justified by the business's growing dependence on uptime.
Worked example
Suppose monthly cloud spend is $40,000 against 2,000,000 transactions a month: cost per 1,000 transactions is 40,000 / 2,000,000 x 1,000 = $20. A rightsizing project targeting a 15% spend reduction with flat transaction volume would move that metric to (40,000 x 0.85) / 2,000,000 x 1,000 = $17 per 1,000 transactions. That's the kind of before-and-after a cost KPI is meant to make visible: not just "we spent less," but a normalized figure leadership can compare quarter to quarter regardless of how much the business itself grew in the meantime. (These particular figures are illustrative of how the metric would move under a stated assumption, not a report of a specific measured result.)
Trade-offs and pitfalls
- Too many KPIs is its own failure mode: a dashboard nobody owns end to end decays into noise nobody acts on. Eight is close to the practical ceiling for a metric set leadership will actually review on a cadence.
- Any KPI without a counter-metric can be gamed; the fix is pairing metrics that check each other, as with deployment lead time and change failure rate above, rather than trusting a single number in isolation.
- A KPI that's collected but never appears in an actual review meeting has no behavioral effect at all; the review cadence is part of the KPI design, not an afterthought.
Engineering leadership worries about vendor lock-in as the organization standardizes on one cloud provider. Propose a framework to quantify lock-in risk in business-relevant terms (not just technical ones) - covering egress fees, proprietary managed-service dependence, and estimated cost/effort to migrate later - and the architectural mitigations (abstraction layers, data-format standards, periodic migration drills) you'd recommend.
Sample Answer
Direct answer
Convert lock-in from a vague worry into a weighted numeric score built from a handful of business-relevant dimensions, then convert the score into an actual migration-cost-and-effort estimate in dollars and months. A number a finance stakeholder can compare against the value of staying is far more useful than a qualitative "high" or "low" risk label, and the scoring dimensions and mitigation catalog both need to flex by technology category, because a serverless function and an analytics platform fail differently.
Structured elaboration
Weighted scoring dimensions
Score each candidate service 1 (low risk) to 5 (high risk) on each dimension, multiply by its weight, and sum:
LockInScore=20×∑i=16wi⋅si
| Dimension | Weight | What it captures |
|---|---|---|
| Data egress cost exposure | 0.25 | Egress is the data that leaves the vendor's network; this dimension is the cost to move your data volume out at current pricing |
| Proprietary managed-service dependence | 0.25 | How much business logic lives inside vendor-specific managed features, not portable code |
| API/interface uniqueness | 0.15 | Whether an open standard or common interface exists as an alternative |
| Contractual constraints | 0.15 | Commitment length, termination penalties, minimum spend |
| Skills/tooling specialization | 0.10 | How much team expertise is tied to vendor-specific tooling |
| Ecosystem interdependency | 0.10 | How many other services from the same vendor this one is wired into |
The weights sum to 1, so the weighted average itself never leaves the 1-5 scoring range; the 20x multiplier rescales that 1-5 average into a 20-100 business-facing score, 20 if every dimension scores the safest possible 1, 100 if every dimension scores the riskiest possible 5, not a 0-100 score, since a weighted average of numbers that are never below 1 can never reach 0.
Technology-specific indicator sets
- Serverless/FaaS (function-as-a-service, a model where you deploy individual functions rather than servers): a proprietary event-trigger schema, an orchestration layer (a step-function-style workflow engine) that embeds business logic in vendor-specific configuration, and a per-invocation billing model that is hard to replicate elsewhere for cost comparison.
- Cloud analytics platform: a query dialect with non-standard SQL extensions, a storage format tied to the vendor's own compute engine, and data gravity (the sheer volume and the time and egress cost required to move it) as its own lock-in force independent of the API surface.
Cost and effort to migrate
MigrationCost=M⋅Ceng+E+D
where M is estimated engineer-months to rebuild the integration, Ceng is fully loaded monthly cost per engineer, E is one-time data egress cost, and D is the cost of running both systems in parallel during a dual-run validation window.
PaaS-specific scoring rubric
For a platform-as-a-service (PaaS) dependency such as a proprietary workflow-orchestration platform, score these four dimensions the same 1-5 way: API compatibility (can an equivalent platform run the same interface), data exportability (can workflow definitions and history be exported intact), custom runtime dependence (does your code rely on a runtime only this platform provides), and vendor-specific tooling dependence (deployment, monitoring, or debugging tools with no equivalent elsewhere).
Mitigations as a 24-month roadmap
| Phase | Action | Checkpoint |
|---|---|---|
| Months 0-6 | Introduce an abstraction layer around the highest-risk dependency; land canonical data in an open format | Abstraction layer covers the top-scoring service |
| Months 6-12 | Stand up a multi-cloud orchestration layer (for example, infrastructure-as-code and a container runtime that both major clouds support) for new workloads | New workloads default to the portable pattern |
| Months 12-24 | Run a live migration drill on the highest-risk existing dependency; refactor it if the drill reveals gaps | Drill completes within the estimated M months from the cost model |
Worked example
Scoring a proprietary FaaS platform: egress exposure 4, managed-service dependence 5, API uniqueness 4, contractual constraints 2, skills specialization 3, ecosystem interdependency 4. Weighted sum: 0.25(4)+0.25(5)+0.15(4)+0.15(2)+0.10(3)+0.10(4)=1.0+1.25+0.6+0.3+0.3+0.4=3.85, times 20 gives a LockInScore of 77 out of 100, solidly high risk. Migration estimate: rebuilding the function layer on a portable container runtime is estimated at 5 engineer-months at a fully loaded $18,000 per engineer-month ($90,000), plus $12,000 in one-time data egress, plus a 6-week dual-run window costing roughly $15,000 in duplicated infrastructure, for a total estimated migration cost of $117,000. That number, not the abstract "77," is what goes in front of a budget owner.
Trade-offs and pitfalls
The most common failure is scoring the dimension that is easiest to measure (egress cost, which is a line item) while under-scoring the one that matters more (managed-service dependence, which requires actually reading how deeply the code is coupled). A second failure is treating the score as permanent: usage and dollar exposure grow with adoption, so a service scored low at pilot scale should be re-scored as it becomes load-bearing. Finally, do not spend more on the scoring exercise than the position justifies; a low-weighted, low-volume service does not need the full rubric.
A small, cost-constrained startup is deciding between cloud hosting and on-premises infrastructure. Outline a decision framework covering speed to market, cost (near-term vs long-term), compliance, vendor lock-in, operational burden and hiring constraints, and give a recommendation.
Sample Answer
For a small, cost-constrained startup, cloud is the recommendation almost by default, because nearly every criterion in this framework, speed to market, near-term cash cost, hiring pool, and operational burden, points the same direction at this scale; the case for on-premises being cheaper only holds at a scale most startups haven't reached and may never reach.
The framework
- Speed to market: cloud wins decisively. Infrastructure is provisioned in minutes, not the weeks it takes to procure and rack physical hardware.
- Cost, near-term versus long-term: near-term, cloud avoids capital expense entirely, which matters most when runway, not long-run unit economics, is the binding constraint. Long-run, at very large steady-state scale, on-premises can become cheaper, but a startup evaluating this today is optimizing for surviving to that future, not for the economics of a scale it may never reach.
- Compliance: cloud providers already hold most of the relevant certifications (SOC 2, an independent audit of a vendor's security controls; ISO 27001, an international information-security management standard; PCI-DSS attestation, certification against the payment-card industry's data-security standard), which is almost always faster for a startup to inherit than to build the equivalent controls on-premises from nothing.
- Vendor lock-in: real, but the right mitigation is designing for portability at the edges (containerized workloads, avoiding unnecessarily deep use of a single vendor's most proprietary managed services for the core differentiator), not avoiding cloud altogether.
- Operational burden: a 5-20 person engineering organization cannot also run a reliable data center; cloud shifts that burden to a provider whose entire job is doing it well.
- Hiring constraints: nearly every engineering candidate already has cloud experience; on-premises and data-center operations is a shrinking, harder-to-hire specialty, which matters directly to how fast a small team can staff up.
Before recommending, ask the discovery questions that actually determine whether the default applies: what's the expected growth curve and time horizon for this decision; is there a genuinely non-negotiable regulatory or data-residency constraint; what's the team's existing skill set; is there already sunk investment in specific hardware or a platform; and how sensitive is the burn rate to a few thousand dollars a month in infrastructure spend.
Worked example
A 20-person startup building a customer-analytics product needs a small ETL (extract, transform, load: the process of pulling data from its sources, reshaping it, and loading it somewhere it can be analyzed) pipeline. Building it on cloud-managed components, a managed queue, a serverless transform step, and a managed warehouse, costs roughly $800/month at current volume and can be running within a week with one engineer. Standing up the equivalent on self-hosted, on-premises infrastructure would cost tens of thousands of dollars upfront and consume a meaningful fraction of a scarce engineer's time for weeks just to reach the same starting point, before the startup has even validated that the product needs this pipeline at all. At this stage, the near-term cash and speed cost of getting the infrastructure choice wrong dwarfs the long-term efficiency that on-premises might theoretically offer at a scale this company hasn't reached.
Trade-offs and pitfalls
- The most common mistake is a startup spending precious runway and engineering time building the infrastructure footprint of a company ten times its size, "to save money later," when later may never arrive, or may arrive with a completely different usage pattern than assumed today.
- Ignoring a genuine, non-negotiable compliance or data-residency constraint because "cloud is the default answer for a startup" skips the one check that would have overridden the default; always run the discovery questions before assuming cloud is the right call.
- Recommending cloud without a plan for the portability edges (containerization, avoiding excessive lock-in on the differentiating parts of the product) sets up a harder, more expensive migration later if the vendor relationship sours or pricing changes unfavorably.
Two engineering organizations being merged run different tooling stacks for the same function (for example one team's observability stack vs. the other's). As the person responsible for the combined estate, produce a consolidation plan that unifies the tooling with minimal data loss and minimal double-instrumentation.
Sample Answer
Consolidating two merged organizations' overlapping tooling stacks (for example one team's observability stack against the other's) succeeds or fails on sequencing: run both stacks in parallel first, migrate consumers of the data before touching producers, and only decommission the losing stack once every downstream user has verifiably switched, not once you believe they have.
Consolidation plan
1. Inventory and classify, not just the tools but their consumers. List every tool in both stacks, and for each one, list who and what depends on it: dashboards, alert rules, on-call runbooks, compliance retention requirements, and any downstream system that ingests its data. The tool inventory is easy; the consumer inventory is what actually determines migration risk, and it is the step most consolidation plans skip.
2. Pick the surviving stack on evaluation criteria decided before you know which team "wins." Score both stacks on data retention already accumulated, cost at combined scale, feature parity against each team's current use cases, and operational maturity of the team that would own it going forward. Deciding this before comparing removes the organizational-politics failure mode where the larger or more senior team's tool wins by default regardless of fit.
3. Run both stacks in parallel with dual-write or dual-instrumentation for a defined overlap window. Instrument new and migrated services to send data to both the surviving and the retiring stack for a fixed period (commonly one to two full business cycles, so the overlap window covers at least one peak-load event both teams need visibility into). This is the one point in the plan where "minimal double-instrumentation" is intentionally violated temporarily, because it is the only way to validate the surviving stack against real production behavior before anyone loses their safety net.
4. Migrate consumers before producers, in dependency order. Move dashboards and alert rules to point at the surviving stack first, validated against the same incidents both stacks are capturing during the overlap window, before removing instrumentation from any producer. Migrating the other order (cutting a producer over and hoping consumers catch up) is the direct cause of the coverage gaps this plan exists to prevent.
5. Preserve historical data with an explicit decision, not a default deletion. For each retiring tool, decide per data category whether to bulk-import history into the surviving stack, export and archive it cold for compliance retention only, or accept a bounded loss window, and document that decision, since silently losing six months of incident history because "the migration script only handled the last 30 days" is a common, quiet failure.
6. Decommission only after a verification gate, not a calendar date. Require every consumer team to confirm in writing (or via a tracked checklist) that they have validated the surviving stack against their own use cases before turning off the retiring tool, and keep the retiring stack in a read-only, low-cost state for a final buffer period rather than deleting it outright at cutover.
Worked example
Two merged engineering organizations each run an observability stack: Team A on a commercial application performance monitoring (APM) platform with 18 months of trace history, Team B on a self-hosted open-source stack with custom dashboards tuned to their specific service-level objectives (SLOs, the internally defined reliability targets a team commits to). Scoring criteria (cost at combined scale, retention, feature parity, team operational maturity) put the commercial platform ahead on operational maturity and cost predictability but behind on customization depth. The plan: adopt the commercial platform as the one to keep, run Team B's self-hosted stack in parallel for two full sprint cycles while porting their custom dashboards and SLO alert rules, archive Team B's pre-merger trace history to cold storage for compliance rather than importing it (since the two systems' trace formats are not compatible enough to migrate cheaply), and decommission the self-hosted stack only after both teams jointly confirm, on a tracked checklist, that every alert rule fired correctly during at least one real incident on the new platform.
Trade-offs and pitfalls
Running two stacks in parallel costs real money and, briefly, real double-instrumentation effort, which is the opposite of the "minimal double-instrumentation" goal; the trade-off is accepting a bounded, time-boxed violation of that goal in exchange for avoiding the much larger risk of a silent coverage gap during a genuine incident. The most common pitfall is treating consolidation as a technical migration with the organizational and political dimension bolted on afterward: the team whose tool loses will resist unless the evaluation criteria were visibly decided before the comparison and the overlap window gives them time to validate rather than just accept the outcome. A second pitfall is setting the decommission date on a calendar rather than a verification gate, which is exactly how coverage gaps get discovered during an incident instead of during the migration.
Unlock Full Question Bank
Get access to all 19 Infrastructure Strategy and Technology Selection interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.