Infrastructure Strategy and Platform Decisions Questions

Focuses on making technical infrastructure and platform choices with consideration for business impact and organizational factors. Topics include build versus buy trade offs, vendor and platform evaluation, scalability and reliability considerations, migration and deprecation planning for legacy systems, total cost of ownership, developer productivity impact, organizational readiness, and stakeholder involvement. Candidates should show how to structure these decisions, evaluate technical and non technical risks, and communicate clear rationale and implementation plans.

MediumTechnical

61 practiced

You are asked to deprecate an internal API used by multiple teams. Draft a deprecation lifecycle: announcement timing, migration windows, compatibility shims, enforcement, and how to handle teams that do not comply. Include communication channels and metrics to track progress.

Sample Answer

Situation: Our SRE team must retire an internal API used by multiple teams without impacting reliability or SLAs.

Plan (deprecation lifecycle):

1. Announcement timing- T0 (Week 0): Public notice to all consumers: purpose, rationale, replacement API, backward-compatibility risks, timeline.- Follow-ups at T0+2w and monthly thereafter until enforcement.

2. Migration windows- Advisory window (0–60 days): guidance, docs, migration samples, automated compatibility tests.- Supported window (60–180 days): both APIs operate; compatibility shim enabled; helpdesk/office hours.- Sunset window (180–240 days): read-only / rate-limited access, alerts to remaining consumers.- Enforcement (after 240 days): throttle or remove endpoint.

3. Compatibility shims & tooling- Provide a proxy shim that translates old calls to new API for 90 days; expose perf/latency delta.- Offer client libraries, migration scripts, CI checks, and sample IaC changes.- Provide automated smoke tests and a pre-release environment to validate.

4. Enforcement & non-compliance- Before enforcement: escalate via team leads, product owners, and execs.- If teams still non-compliant: apply progressive throttling, then block; document exceptions with a formal SLA-based extension (max 30 days) approved by SRE manager.- Offer targeted engineering support for stuck teams.

5. Communication channels- Primary: service catalog entry + detailed deprecation doc (Confluence)- Alerts: email + team Slack channel (#api-deprecations)- Direct: weekly migration office hours, 1:1 support via PagerDuty on-call rota for critical blockers- Governance: monthly deprecation review with architecture board

6. Metrics to track progress- Adoption %: fraction of traffic hitting new API vs old- Consumer count remaining- Error rate / SLO impact per API- Migration velocity: number of teams moved per week- Time-to-resolution for migration blockers- Latency and error deltas for shim vs native

Actions I would take as SRE:- Instrument endpoints to emit the metrics above and dashboards/alerts.- Automate discovery of callers (logs/traces) and notify owners.- Run pre-enforcement dry-run to measure impact and validate rollback paths.- Ensure runbooks for rollback and incident response are ready.

Result expectation: predictable, low-risk retirement with clear accountability, measurable progress, and minimal customer-facing disruption.

HardTechnical

49 practiced

You must decide whether to build an internal PaaS (developer platform) or adopt a managed platform. Create an assessment covering architecture, run cost, developer velocity, long-term maintenance, and organizational readiness. Define decision gates and KPIs that would help you abandon or continue the project.

Sample Answer

Situation: We need to choose between building an internal PaaS (developer platform) or adopting a managed platform. Below is an SRE-focused assessment, decision gates, and KPIs to stop or continue.

Assessment (by dimension)- Architecture - Build: Full control—custom auth, networking, service mesh, CRDs, multi-tenant isolation. Requires design for HA, upgrades, and observability. - Buy: Opinionated architecture, fewer integration points, vendor SLAs. Limits deep customization but accelerates standardization.- Run cost (TCO) - Build: High upfront engineering + ongoing ops headcount; infra costs can be optimized but require automation investment. - Buy: Higher steady-state licensing/cloud spend, lower personnel cost. Predictable billing.- Developer velocity - Build: Can tailor APIs/CLI for internal workflows (high long-term velocity) but slow initial rollout. - Buy: Immediate win for velocity via ready-made workflows; may hit friction for non-standard needs.- Long-term maintenance - Build: Continuous patches, security, upgrades—burden on SRE team. Risk of bit-rot if not funded. - Buy: Vendor handles upgrades; integration and migration risk remain.- Organizational readiness - Build: Need dedicated platform product team (PM, SREs, infra engineers), clear governance, budget for 12–24 months. - Buy: Need change management, internal training, and integration work.

Decision gates (milestones to evaluate)1. MVP deliverable (3–6 months): deploy a minimal self-service flow (deploy+monitor+rollback). Gate: >80% of target devs can deploy via platform in <=30min.2. SRE operational readiness (6–9 months): automated upgrades, canary rollouts, alerting/SLOs in place. Gate: MTTR < target and error budget consumption <= threshold for 30 days.3. Cost & adoption review (9–12 months): measure TCO vs vendor quote and adoption growth. Gate: projected 24-month TCO < managed vendor + training OR developer satisfaction > managed alternative by NPS delta.4. Scale & sustainability (12–18 months): stable roadmap, documented SLAs, dedicated 2-3 FTEs sustaining platform. Gate: committed budget + org support for long-term ops.

KPIs (continue if positive, abandon if negative)- Adoption: % of services using platform (target 60% by month 12). If <25% => consider abandon.- Developer lead time: median deploy-to-prod time (target reduction 50%). If no improvement in 6 months => reconsider.- SRE burden: platform ops FTE-hours/week and incident count/MRTR (target: ops hours decrease per service). If ops hours increase >20% and incidents rise => fail gate.- Cost: 24-month TCO vs managed vendor delta (target ROI within 24 months). If build TCO exceeds managed by >20% without commensurate velocity gains => abandon.- Reliability: SLO compliance and error budget burn (target 95%+ SLO adherence). If persistent misses for 2 consecutive quarters => pause/abandon.

Recommendation (SRE lens)- If organization can staff a platform product team, has complex internal requirements, and long-term scale (hundreds of services), build. Require strict milestones and funding.- Otherwise adopt managed platform to buy time, then evolve a small internal paved-path for custom needs.

EasyTechnical

46 practiced

You must evaluate three managed SQL database vendors for production use. Describe a vendor-evaluation checklist that covers performance, reliability, data portability, security/compliance, operational model, and commercial terms. What minimal PoC would you run and what success criteria would you set?

Sample Answer

Vendor-evaluation checklist (SRE-focused)

1) Performance- Baseline and peak throughput: QPS/transactions, latency p50/p95/p99 for read/write.- Scaling behavior: vertical/horizontal auto-scaling, failover switchover times.- Resource isolation: noisy-neighbor protections, IOPS/CPU limits.

2) Reliability / HA- SLA history, multi-AZ/multi-region replication, RPO/RTO guarantees.- Automatic failover mechanics and time-to-recovery tests.- Backup frequency, point-in-time recovery, backup integrity checks.

3) Data portability- Export/import formats (logical/physical), compatibility with open engines (Postgres/MySQL).- CDC support (Debezium/CDC connectors), replication to external consumers.- Lock-in risks (proprietary features, extensions).

4) Security & Compliance- Encryption in transit & at rest, KMS integration, key rotation.- IAM, least-privilege roles, audit logging, VPC/private endpoints.- Compliance certifications (SOC2, ISO27001, PCI, HIPAA) and pen-test history.

5) Operational model- Monitoring integrations (Prometheus, CloudWatch), metric granularity, alerting hooks.- Maintenance windows, patching model, schema-change tooling, read replicas.- Support SLAs, runbooks, incident escalation, observability of internals.

6) Commercial terms- Pricing model (storage/IO/compute), cost predictability, discounts, exit costs.- Data egress fees, snapshot retention costs, contractual SLAs and credits.

Minimal PoC (2–4 weeks)- Deploy identical schema + representative dataset (~10–20% prod).- Run workload generator: mixed read/write transactions, background batch jobs, and peak burst tests.- Simulate failure modes: AZ failure, primary failover, network partition, and restore from backup.- Validate operational workflows: scale-up/down, schema migration, restore, cross-region replication.

Success criteria (quantitative + qualitative)- Latency: p95 < target (e.g., <200ms) and p99 within acceptable bound.- Throughput: meets required QPS with headroom (e.g., 30%).- Recovery: failover RTO < SLO (e.g., <60s), RPO = acceptable (near-zero/minutes).- Durability: successful backup + PITR restore within expected time.- Observability: vendor provides required metrics/logs and integrates with our monitoring.- Security: meets compliance checklist and uses our KMS/IAM patterns.- Cost: projected monthly cost within budget ±10% and no unexpected egress/lock-in blockers.- Support: responsiveness per SLA and clear runbooks for incident scenarios.

I would score vendors against these criteria and prioritize those meeting SLOs, operational transparency, portability, and predictable commercial terms.

HardTechnical

43 practiced

Your observability bill is growing rapidly due to high-cardinality telemetry. Propose architectural and product-level changes to lower costs while preserving SLO-relevant signal. Discuss sampling, aggregation, metric down-sampling, streaming pipelines, cardinality limits, and developer incentives.

Sample Answer

Requirements & constraints:- Preserve SLO-relevant signal (latency, error-rate, saturation) with alerting fidelity.- Significantly reduce cost from high-cardinality labels (user_id, request_id, feature flags).- Incremental rollout with ability to debug incidents when necessary.

High-level approach:1. Enforce cardinality hygiene (product level)- Define allowed label namespaces (service, endpoint, env, stage) and forbidden high-cardinality keys (user_id, request_id) in agent/client libraries.- Linting + CI checks: telemetry schema validation fails builds if services emit banned tags.- Developer incentives: quota-backed dashboards, show cost per tag, and require PRs for exceptions with TTL and owner.

2. Multi-tier telemetry pipeline (architecture)- Edge agents perform cheap pre-aggregation and deterministic sampling: - Metrics: aggregate histograms / summary per host/instance; emit pre-bucketed quantiles. - Traces/Spans: use probabilistic and tail-sampling. Always keep all error spans; sample normal spans at low rate.- Streaming system (Kafka → stream processor like Flink/Beam) implements: - Dynamic down-sampling rules based on traffic and anomaly detection (sample less during steady-state, less aggressive during incidents). - Roll-up: aggregate by coarser dimensions periodically (e.g., user_id → cohort, region) and emit aggregated metrics.- Long-term storage: store raw high-cardinality data only for short retention (7d), store down-sampled roll-ups for long-term (90d+).

3. Sampling & aggregation strategies- Adaptive sampling: keep a fixed low-rate baseline, but increase retention for rare/error signals and tail latency buckets. Use reservoir sampling to preserve diversity.- Deterministic sampling keyed by trace id to ensure trace continuity.- Cardinality-limited metric ingestion: reject or caps new unique label-value pairs per metric per time window with throttling/alerts to owners.

4. Operational controls & visibility- Cost-aware dashboards showing cost per metric/label; alert when cardinality growth rate exceeds thresholds.- “Zoom-in” workflow: on-demand temporary full-fidelity capture for debugging (short-lived toggle with approval and automatic expiry).- Backpressure & graceful degradation: agents fallback to aggregated emission when upstream rejects high-cardinality payloads.

Trade-offs:- Some loss of per-actor signal (user-level) but preserved SLOs via cohorting and tail-preserving sampling.- Added complexity in pipeline and developer processes; justified by predictable cost and improved signal-to-noise.

Expected impact:- 5–10x reduction in ingestion cardinality from blocking forbidden tags + aggregation.- Maintain alerting fidelity for SLOs by preserving error/tail signals and cohort roll-ups.- Controlled, auditable exceptions for debugging.

MediumTechnical

56 practiced

You need to evaluate the business impact of moving to a managed CI/CD provider versus running your own. Identify both quantitative and qualitative impacts (e.g., deployment frequency, outage reduction, security, cost), and propose an experiment or pilot to collect the required evidence.

Sample Answer

Framework: compare costs/benefits across reliability, velocity, security, and operational overhead; measure baseline, run a controlled pilot, and evaluate against SLOs and cost targets.

Quantitative impacts & metrics:- Deployment frequency & lead time: deploys/week, mean time from PR merge to prod (target % improvement).- Failure rate & MTTR: incidents/month attributable to deployment, mean time to rollback/restore.- Change failure rate: % of deployments causing incidents.- Uptime / SLOs: error budget consumption, latency percentiles.- Cost: total cost of ownership (TCO) = infra + maint. hours (FTE) + vendor fees; cost per deployment.- Security/compliance: number of critical vuln exposures, time-to-patch.

Qualitative impacts:- Developer velocity and CX: perceived friction, time spent debugging CI infra, onboarding time for new engineers.- Operational risk: reliance on vendor SLA, vendor support quality, configurability limits.- Tooling & visibility: telemetry, traceability, audit logs, ease of rollbacks.- Team focus: less time on maintenance allowing SREs to work on reliability projects.

Pilot proposal:- Scope: select 2 services (one high-release-rate, one critical-low-change) for 8 weeks.- Setup: mirror current pipelines on managed provider; keep existing self-hosted as fallback.- Data collection: instrument pipelines to record pipeline duration, success/failure, resource usage; tag incidents to root cause (pipeline vs app).- Success criteria: >=20% reduction in mean pipeline time OR >=30% reduction in maintenance hours; no increase in change-failure rate; acceptable TCO within 12 months.- Controls & rollback: dual-run period where both systems run for safety; traffic cutover only after smoke tests; automated rollback to self-hosted if error budget exceeds threshold.- Risk mitigation: contract exit terms, encryption/key management review, runbook updates, run periodic security scans.

This approach yields measurable evidence to decide whether managed CI/CD improves reliability and developer velocity while fitting SRE and business constraints.

Unlock Full Question Bank

Get access to hundreds of Infrastructure Strategy and Platform Decisions interview questions and detailed answers.

Join thousands of developers preparing for their dream job.