Evaluation and justification of technologies services and platforms used to implement systems across the stack. Candidates should be able to select compute options including virtual machines containers and serverless platforms as well as orchestration and workflow engines messaging systems batch and streaming processing engines object and block storage data warehouses and other data platforms. The topic encompasses comparing managed services and self managed deployments cloud versus on premise hosting and choosing frameworks runtimes and overall stacks based on workload characteristics. Assessment focuses on weighing trade offs across cost operational overhead reliability latency and throughput scaling characteristics vendor lock in development velocity team familiarity and learning curve maturity and community support security and compliance and monitoring and debugging complexity. Candidates should demonstrate how system requirements map to service capabilities justify build versus buy decisions and managed service choices design proof of concept experiments and outline migration and rollout planning while making pragmatic choices that balance performance cost and operational risk.
EasyTechnical
61 practiced
Describe the trade-offs between adopting managed services versus self-managed deployments for core infrastructure components (databases, streaming systems, Kubernetes). Include operational overhead, upgrade and patching responsibilities, customization limits, cost differences, vendor support, and scenarios where you would prefer self-managed despite higher ops cost.
Sample Answer
At a high level, managed services (DBaaS, managed Kafka/Confluent, EKS/GKE/AKS) shift operational responsibility to a vendor; self-managed keeps control but requires your ops team. Key trade-offs:Operational overhead- Managed: Low day-to-day ops — provisioning, backups, monitoring, and basic HA are handled. Good for teams with limited SRE capacity.- Self-managed: High overhead — you must run clusters, backup/restore, HA, monitoring, runbooks, and incident response.Upgrade & patching responsibilities- Managed: Vendor handles OS/patches and often minor upgrades; you still own application-level migrations and major version decisions.- Self-managed: You are responsible for patch windows, can test upgrades in staging, and control timing.Customization limits- Managed: Constraints on kernel flags, custom plugins, storage drivers, or non-standard configuration; sometimes limited access to internals.- Self-managed: Full control — tune for latency, add custom plugins/extensions, choose storage and networking stack.Cost differences- Managed: Higher direct service cost but lower personnel cost; predictable OPEX.- Self-managed: Lower vendor fees but higher CapEx/OPEX for staff, on-call, tooling, and hidden costs (downtime, toil).Vendor support & SLAs- Managed: SLA-backed support and documented recovery paths; useful for compliance.- Self-managed: You control SLAs internally; may buy vendor support separately but still rely on your ops.When to choose self-managed despite higher ops cost- Regulatory or data residency requirements preventing managed offerings.- Need for uncommon customizations (e.g., custom storage engine, uncommon Kafka plugins, specific kernel tunings).- Cost at extreme scale where vendor margins are prohibitive and you can amortize engineering costs.- Legacy systems with complex integrations that require deep runtime access.- Strategic desire to avoid vendor lock-in for critical IP or to migrate across clouds.Recommendation as a Solutions Architect: evaluate total cost of ownership (people + tooling + downtime), required customizations, compliance, time-to-market, and team maturity. For most greenfield use cases prefer managed to accelerate delivery; choose self-managed when compliance/customization/scale justify the ops investment.
HardTechnical
54 practiced
Design experiments for a PoC that compares a managed cloud database service to a self-managed cluster, focusing on operational overhead and mean time to recover (MTTR). Specify metrics to collect, failure-injection tests to run (node failure, region failure, backup corruption), measuring human-hours, and success criteria for choosing managed versus self-managed.
Sample Answer
Overview: Run parallel PoCs (same DB version/config, identical dataset & workload) for a managed cloud database and a self-managed cluster. Focus on operational overhead and MTTR under realistic failures. Repeat each test multiple times (≥5) to capture variance.Experiment plan:1. Baseline: measure steady-state metrics for 24–72h under production-like load.2. Failure-injection tests (each run: inject, detect, remediate, recover; measure times & human effort): - Node failure: kill DB process or isolate a node (simulate disk full + restart). - Region failure: blackhole network to entire AZ/region or fail replicas and force failover. - Backup corruption: corrupt latest backup manifest or restore to wrong snapshot. - Upgrades/patch: rolling upgrade with one faulty node that needs rollback. - Operator error: accidental schema drop or misconfiguration requiring restore.Metrics to collect:- MTTR components: time-to-detect (automated alert), time-to-respond (human acknowledge), time-to-repair (actions to restore), time-to-validate (consistency/lag checks). Record timestamps automatically and manually.- Availability: downtime seconds and SLA percentage.- Data loss: bytes/rows lost, RPO observed.- Recovery correctness: application-level validation queries.- Operational overhead: human-hours per incident (log time spent by each responder), number of manual steps, runbook length & complexity, context switches.- Effort for routine ops: provisioning time, patch/upgrade time, backup validation time per period.- Automation coverage: percent of remediation automated vs manual.- Observability: time to root cause, number of alerts, false positives.- Cost: cloud costs + estimated staff cost (human-hours × fully loaded hourly rate).Measurement methodology:- Instrument with centralized logging, alerts, incident timer. Use screen recordings and post-incident surveys to capture human effort and cognitive load.- Define exact playbooks beforehand; for self-managed, require operators to follow their usual steps; for managed, follow provider procedures and support escalation.- Run each failure scenario at least 5 times at different times of day/week to capture variability.Success criteria:- If managed reduces median MTTR by ≥50% and reduces average human-hours per incident by ≥60% while meeting availability and RPO requirements, prefer managed.- If self-managed delivers comparable MTTR/human-hours but at materially lower TCO (including staffing) and gives required control/compliance, prefer self-managed.- Consider qualitative factors: vendor support responsiveness, runbook complexity, regulatory constraints, and long-term operational scaling.- Produce a decision matrix with weighted scores (MTTR 35%, human-hours 25%, cost 20%, control/compliance 15%, observability 5%) to pick the option.Deliverables:- Test harness, scripts, runbooks, raw metrics, incident transcripts, cost model, decision matrix, and recommended architecture with mitigation steps for chosen option.
MediumTechnical
53 practiced
Decide between serverless orchestration (e.g., AWS Step Functions, Durable Functions) and container-based scheduled jobs (Kubernetes CronJobs or Fargate tasks) for complex workflows that include human approval steps and occasional multi-hour tasks. Discuss cost, error handling, retries, long-running limits, observability, and developer ergonomics.
Sample Answer
Situation: A client needs to implement complex workflows with human approvals and occasional multi-hour tasks. Below I compare serverless orchestration (Step Functions / Durable Functions) vs container-based scheduled jobs (K8s CronJobs / Fargate) across the requested dimensions and give a recommendation.Cost- Serverless: Pay-per-use for state transitions and execution time; very cost-effective for infrequent workflows and idle waiting (human approvals incur minimal cost).- Containers: You pay for allocated CPU/memory while tasks run; for long idle waits you still pay unless you implement polling tricks or pause/resume.Error handling & retries- Serverless: Built-in retry policies, catch/fallback constructs, and versioned state machines make error handling declarative and consistent. Easier to model compensating transactions.- Containers: You must implement retries and backoff in application logic or rely on platform-level restarts; more flexible but more engineering effort.Long-running limits- Serverless: Many offerings support long-running orchestrations (Step Functions Standard and Durable Functions can model days/months via checkpoints). Beware of vendor-specific execution time or state size limits.- Containers: Naturally handle multi-hour tasks as long as node/task lifecycles and quotas allow; better for heavy compute that must run continuously.Observability- Serverless: Good built-in tracing, step-level execution history, and integrations with CloudWatch/X-Ray; state tracing simplifies audits (who approved what and when).- Containers: Use distributed tracing and logging stacks (Prometheus, Grafana, ELK). More control but requires more setup to get step-level business visibility.Developer ergonomics- Serverless: High-level declarative models (ASL, function workflows), quick to implement approval flows and retries, less infra ops. Steeper learning if complex branching needed.- Containers: Familiar dev workflow, full language/runtime control, easier to run local end-to-end tests, but you must design orchestration glue and infra.Recommendation (Solutions Architect perspective)- For workflows with human approvals and long idle waits but modest continuous compute: prefer serverless orchestration (Step Functions/Durable) for lower cost, simpler error handling, native long-running support, and better auditability.- If tasks require heavy continuous CPU/GPU, custom runtime dependencies, or strict runtime control: use container-based tasks (K8s CronJobs / Fargate) and layer an orchestration/state component (e.g., a lightweight Step Function or a message-based coordinator) to handle approvals and retries.Notes/Best practices- Hybrid approach often wins: orchestrate approvals and business state in serverless, kick off containers for heavy processing via tasks/events. Ensure idempotency, correlation IDs, and centralized observability across both worlds.
MediumTechnical
57 practiced
For a retail analytics platform that requires near-real-time dashboards (within 30 seconds) and complex historical queries across petabytes, recommend whether to use a traditional data warehouse, a lakehouse, or a streaming OLAP system. Discuss ingestion patterns, storage format, latency trade-offs, schema evolution, cost, and tool compatibility.
Sample Answer
Recommendation: use a hybrid lakehouse with streaming OLAP components — i.e., a Delta/Apache Iceberg-backed data lake for historical petabyte-scale storage, combined with a streaming OLAP layer (e.g., Apache Druid/ClickHouse/Materialized views via Pinot or StarRocks) to serve sub-30s dashboards. This balances cost, scale, and latency.Ingestion patterns:- Streaming for near-real-time: Kafka → stream processors (Flink/Structured Streaming) that produce micro-batches/append-only parquet/Delta files and push aggregates into the OLAP engine.- Batch for historical backfill: scheduled ETL jobs that write to the lakehouse and recompute aggregates.Storage format:- Columnar open formats (Parquet/ORC) managed by Delta Lake or Iceberg for ACID, time-travel, compaction and efficient predicate pushdown.- OLAP store keeps denormalized, pre-aggregated segments for low-latency queries.Latency trade-offs:- Dashboard freshness 10–30s: achievable by writing small parquet/Delta micro-batches + streaming ingestion into OLAP; lower latency increases write amplification and CPU.- Historical analytic queries: lakehouse offers cost-efficient scans but higher query latency vs OLAP; use caching, partition pruning, Z-ordering to improve performance.Schema evolution:- Delta/Iceberg: support schema evolution, column adds, safe merges and time travel. Enforce governance with schema registry and automated migrations in stream jobs.- OLAP layer: require controlled rollouts for schema changes; use view layers or column aliases to avoid reindexing where possible.Cost:- Store hot recent data in OLAP (higher cost per GB but fast); cold historical in lakehouse (cheaper object storage).- Optimize by retention policies, compaction, and tiered storage; minimize small file overhead to reduce compute costs.Tool compatibility & integration:- Works with BI tools via SQL endpoints (Presto/Trino, Druid SQL); supports data science via Spark/Pandas and metadata via Hive Metastore or Lakehouse catalog.- Use CDC connectors (Debezium) for upstream systems and orchestration (Airflow, Argo) for pipelines.Trade-offs summary:- Pure DW: good for consistent SQL/BI but cost-prohibitive at petabyte scale and limited streaming.- Pure streaming OLAP: excellent low-latency but expensive for deep historical scans and complex joins.- Hybrid lakehouse + streaming OLAP: best compromise—scalable, cost-effective historical analytics plus sub-30s dashboards when using careful ingestion, compaction, and pre-aggregation strategies.
MediumSystem Design
60 practiced
Design a network architecture for a hybrid cloud with an on-prem datacenter and two cloud regions. Include VPC/VNet layout, subnet segmentation, security zones, options for connectivity (VPN, Direct Connect/ExpressRoute), transit architecture, DNS design, routing, and how to implement failover under bandwidth constraints.
Sample Answer
Requirements & constraints (clarify): business apps across on‑prem + two cloud regions; low-latency intra-site where possible; secure zones; must support VPN + Direct Connect/ExpressRoute; predictable failover when bandwidth constrained.High-level architecture:- On‑prem datacenter connected to a Transit Layer (Transit Gateway / Virtual WAN / Hub VNet).- Two cloud regions each with a hub VPC/VNet and spoke VPCs for apps, DB, shared services.- Transit hubs peered/attached via cloud transit (Transit Gateway peering, Azure Virtual WAN) and an inter‑region backbone (cloud provider backbone preferred).VPC/VNet & subnet segmentation:- Hub VPC (per region): transit gateway attachment, shared services (DNS, NTP, Bastion), edge firewalls.- Spoke VPCs: App (private subnets), Data (private subnets with stricter NACLs/SGs), DMZ (public subnets for load balancers), Management (restricted).- Subnet sizing: /24 or larger per subnet; separate AZ subnets for HA.Security zones:- DMZ: external LB + WAF; only HTTP/HTTPS and health checks inbound.- Application zone: only accepts from DMZ and specific management ranges.- Data zone: no direct internet; DB port locked to app subnets only.- Management zone: admin access via bastion + MFA + Just-in-Time.- Enforcement: NGFWs in hub, Security Groups/NSGs per subnet, NACLs as extra layer, flow logs + IDS/IPS.Connectivity options:- Primary high-throughput: Direct Connect / ExpressRoute to hub (prefer private VIF / circuit) with BGP.- Secondary: Site-to-site IPSec VPN over internet (BGP) for failover or remote sites.- For inter-region: use cloud provider backbone (inter-region VPC peering or Transit Gateway peering). If cross-cloud, use encrypted VPN tunnels or SD-WAN overlay.Transit architecture:- Centralized Transit Hub design per cloud region with Transit Gateway connecting to spokes and on‑prem via Direct Connect + VPN.- Use route tables per attachment to enforce spoke isolation and egress control.- Optionally a shared services hub for DNS, security, and logging with VPC sharing or peering.DNS design:- Split-horizon DNS: internal authoritative zone in cloud (Route 53 private hosted zones / Azure Private DNS) linked to VPCs/VNets.- Forwarders/Resolvers in hub that forward on‑prem queries to local DNS and vice versa via conditional forwarding.- Use health checks and weighted/latency-based records for application endpoints; private endpoints for databases.Routing:- BGP over Direct Connect/ExpressRoute and VPN for dynamic routing and failover.- Prefer route propagation from on‑prem to transit hub; use route filters/prefix lists to control advertisement.- Use route priorities (local preference/AS-path prepending) to prefer Direct Connect and fall back to VPN.- Enforce egress via hub NGFWs using forced-tunnel where required.Failover under bandwidth constraints:- Prioritize traffic using QoS/traffic classes: critical app traffic (DB replication, auth) gets higher priority; bulk sync/backups scheduled off-peak or throttled.- Configure BGP local-pref and AS-path prepending so primary Direct Connect carries traffic; upon failure, VPN takes over with reduced bandwidth.- Implement active/active where feasible (dual Direct Connects in different providers/AZs) or active/passive with health probes in transit hub.- Use application-level graceful degradation: degrade non-critical features, redirect analytics to local cache, use read replicas in each region to reduce cross-site bandwidth.- Continuous monitoring: NetFlow/sFlow + CloudWatch/Network Watcher and automated runbooks to throttle or reroute bulk traffic.Trade-offs:- Centralized transit simplifies security and DNS but is a potential bottleneck—mitigate with autoscaling virtual appliances or distributed egress.- Direct Connect adds cost but lowers latency and increases stability; VPN is cheaper but variable.This design balances security, manageability, and resilience while enabling clear failover behavior under constrained bandwidth.
Unlock Full Question Bank
Get access to hundreds of Technology and Platform Selection interview questions and detailed answers.
