Technical Vision and Infrastructure Roadmap Questions

This topic assesses a candidate's ability to define a multi year technical vision for infrastructure, platform, and systems and to translate that vision into a practical execution roadmap. Core skills include evaluating technology choices and architecture evolution, planning migration and modernization paths, anticipating scalability and capacity needs, and balancing cost performance with resilience and operational reliability. Candidates should demonstrate approaches to managing technical debt, sequencing investments across quarters and releases, estimating resources and timelines, establishing measurable infrastructure goals and key performance indicators, and implementing governance and standards. Discussion may also cover reliability and observability, security and compliance considerations, trade offs between short term stability and long term rearchitecture, prioritization to enable business outcomes, and communicating technical trade offs to both technical and non technical stakeholders.

MediumBehavioral

49 practiced

Tell me about a time you had to shift your team's multi-year infrastructure roadmap because of an unexpected business priority. Describe how you reprioritized work, communicated tradeoffs to stakeholders, preserved critical reliability work, and what the measurable outcome was.

EasyTechnical

40 practiced

Explain the difference between a strategic multi-year infrastructure roadmap and a tactical backlog of operational tasks. Give concrete examples of items that belong to each, and describe an approach to keep them aligned so short-term work does not derail long-term modernization goals.

MediumTechnical

36 practiced

Explain technical debt from an infrastructure perspective. How do you identify and quantify infrastructure technical debt, how do you prioritize what to pay down, and what approaches do you use to balance paying technical debt versus delivering features in the roadmap?

MediumTechnical

47 practiced

You expect 3x traffic growth over the next 18 months. Describe your capacity forecasting approach: what data sources you would use (telemetry, business forecasts), statistical or heuristic models, how you would account for burstiness and tail traffic, how much buffer to provision, and how to translate the forecast into budget and procurement actions.

Sample Answer

Approach summary: combine telemetry + business inputs, build time-series growth + burst models, convert capacity units to dollars, and plan procurement with layered commitments (reserve baseline, autoscale bursts). Reforecast monthly and validate with load tests.

Data sources- Telemetry: QPS, requests/sec per endpoint, concurrent connections, CPU/mem/io utilization, latency distributions (p50/p95/p99), queue lengths, autoscaler events, histograms and per-minute granularity.- Business: product roadmap, marketing calendar, expected user growth, SLAs, geo expansion dates, historical campaign lift factors.- External: market/industry seasonality, third-party API constraints.

Modeling- Baseline growth: fit recent data with a time-series model (Prophet or ETS) including trend + weekly/daily seasonality and business-change covariates. Validate with backtests.- Capacity mapping: convert predicted QPS/concurrency to compute/network/storage using performance profiles (throughput per instance type, p95 latency SLO).- Burst/tail modeling: measure peak-to-median and short-window percentiles (1m, 5s). Model bursts with empirical distributions or Poisson/compound models; use extreme value (block maxima) to size for tail percentiles (e.g., 99.99% short-window).- Heuristics: use percentile-based sizing: reserve for 95th-percentile sustained load, plan autoscaling for 99.9th short-window peaks.

Buffer & SLO alignment- Buffer tied to risk tolerance and error budget: if SLO allows 0.1% error budget, size capacity to cover predicted 99.9th short-window load; otherwise apply a percentage buffer (20–50%) on top of modeled need. For 3x growth, aim baseline reserved capacity = modeled 3x * (1 + buffer). For high-risk services (auth, payment) choose larger buffer and lower autoscale reliance.

Procurement & budgeting- Layered procurement: convert capacity units to cost-per-instance/hour or vCPU/hour. - Commit/Reserve for baseline (e.g., 70–80% of expected steady-state) to capture discounts (RIs/Savings Plans). - Use on-demand/spot for scaling and transient bursts. - Plan reserved purchase cadence: buy in tranches (e.g., 6- and 12-month) tied to reforecast checkpoints to avoid overcommitment.- Translate to budget: compute forecasted hourly instance count * hours * unit cost → monthly/annual cost. Add buffer and contingency line items (e.g., +10% ops contingency).- Procurement actions: list SKU types, discount targets, contract lead times, procurement approval timeline, and rollback plan (ability to sell/convert commitments).

Operational steps- Validate with load tests and chaos engineering (simulate 3x + burst scenarios).- Implement autoscaler tuning (cooldown, target metric e.g., p95 latency or CPU), queueing/backpressure, rate limits for tail protection.- Monitoring & governance: monthly reforecast, alert when utilization trends deviate >10% from forecast, tie to SLO error-budget dashboards.

Example (quick math): current baseline 100 instances; 3x growth → 300. With 30% buffer for safety → 390. If reserved discounts save 30% and reserved target = 80% of baseline → reserve 312 instances, keep ~78 on-demand/spots for bursts; translate instance-hours × unit cost to dollars and submit procurement for reserved commitments in two tranches aligned to 6- and 12-month re-evals.

HardSystem Design

51 practiced

Architect an observability pipeline capable of ingesting 5 million metrics/sec, 100k spans/sec, and petabytes of logs per day while controlling cost. Describe tiers for ingestion, aggregation, sampling strategies for traces and logs, tiered storage and retention (hot/warm/cold), indexing strategy to enable queries, and how to keep the observability system itself reliable.

Sample Answer

Requirements & constraints:- Ingest: 5M metrics/s, 100k spans/s, petabytes of logs/day.- Cost-sensitive, high availability, queryable (alerts, debug), multi-tenant.

High-level architecture:- Edge tier (collectors) → Ingestion & validation → Fast aggregation/sampling → Write paths to hot tier + cold tier offload → Index/Query layer → UI/Alerting.

Edge/ingest tier:- Lightweight, autoscaling collectors (e.g., Fluent Bit/Vector for logs, Prometheus remote-write or Pushgateway for metrics, OpenTelemetry collectors for traces) deployed per region/K8s cluster. Perform TLS, auth, basic schema validation, tenant routing, rate-limits and backpressure (429 + buffer).

Aggregation & downsampling:- Metrics: stream processors (Kafka/Fluent/KS) + windowed rollups at ingestion (1s→10s→1m) using Flink/Trino or Cortex/Thanos-style ingesters. Store high-cardinality raw for short hot retention; store rollups for long-term.- Traces: perform tail-based and probabilistic sampling. Keep 100% of error/slow traces (instrument rule engine), sample normal spans at dynamic rates (e.g., 1-5%) using adaptive reservoir sampling keyed by service, endpoint, or trace id. Aggregate span-level metrics (p95/p99) into metrics store.- Logs: apply structured parsing at edge, redact sensitive fields, route high-value logs (errors, audit, alerts) to hot path; apply content-based sampling for verbose sources (e.g., debug at 0.1%). Use log aggregation to produce metrics and traces correlations.

Tiered storage & retention:- Hot (minutes → days): scalable TSDB/NoSQL (Cortex/Prometheus long-term + Dynamo/Cassandra) and fast object storage (SSD-backed) for recent logs/traces; enable low-latency queries and alerts. Retention: metrics raw 7–14d, traces raw 7d, logs 3–7d.- Warm (weeks → months): aggregated metrics & sampled traces on cheaper HDD-backed object store (S3/Blob) with coarse resolution (1m) and limited indexes.- Cold (months → years): archived aggregates and compressed logs/traces on S3 Glacier/Long-term buckets with minimal index; charge retrieval cost acceptable.

Indexing & query strategy:- Avoid indexing full logs; index metadata: tenant, service, trace_id, timestamp, log_level, error_flag, and a small set of user-defined tags. Use inverted index (Elasticsearch/Opensearch) for hot queries on metadata only. For full-text search, maintain a short hot window; for long-term, support background rehydration. Use TSDB for metric queries with label-based indexes (Cortex/Thanos) and a trace store (e.g., Jaeger/Tempo) that stores spans in object storage and builds lightweight secondary indexes (trace id → object location) in a scalable KV store (Bigtable/Dynamo).

Cost controls:- Early aggregation reduces storage and ingest costs.- Adaptive sampling and retention tiers reduce long-term storage.- Per-tenant quotas and billing, backpressure and throttling, ingest cost-based ingestion policies (e.g., deny or sample extremely noisy datasets).- Use object storage for long-term; compact and compress traces/logs (gzip, parquet, ORC).

Reliability & operations:- Run components in active-active multi-AZ with autoscaling, leaderless designs where possible.- Use durable queuing (Kafka or cloud pub/sub) between tiers for buffering and replay on downstream outages.- Health checks, SLOs for pipeline components, automated failover, and chaos testing.- Observability of the observability system: emit internal metrics, traces, and synthetic logs; run dedicated alerting and escalation paths; store critical system alerts in separate, high-priority channel with retained hot copy.- Disaster recovery: cold backups of indexes, periodic restore drills.- Security: tenant isolation, encryption at rest/in transit, RBAC, PII redaction.

Trade-offs:- Heavy indexing increases cost but improves queries — mitigate by limiting indexed fields and hot window.- Tail-based tracing increases CPU but yields high-fidelity for errors; combine with probabilistic sampling for baseline.

This design balances throughput, queryability and cost by pushing work to the edge, aggregating early, sampling intelligently, and using tiered storage and lightweight indexing while keeping the pipeline resilient and observable.

Unlock Full Question Bank

Get access to hundreds of Technical Vision and Infrastructure Roadmap interview questions and detailed answers.

Join thousands of developers preparing for their dream job.