Cloud Platforms and Infrastructure Questions

Comprehensive understanding of cloud computing platforms and core infrastructure concepts. Candidates should know service models including Infrastructure as a Service, Platform as a Service, and Software as a Service, and be familiar with major providers such as Amazon Web Services, Google Cloud Platform, and Microsoft Azure. Core technical knowledge includes compute models, storage systems, networking fundamentals such as domain name system and load balancing, virtual private networks and network segmentation, virtualization, containerization for example Docker, orchestration with Kubernetes, serverless architectures, and microservices. Candidates should be able to evaluate trade offs between managed services and self managed solutions with respect to cost, reliability, operational burden, scalability, performance, security, and vendor lock in, and reason about when to choose platform managed services versus building custom infrastructure. The topic also covers system design considerations for high availability and fault tolerance, capacity planning and autoscaling, monitoring and observability, deployment strategies, and operational practices such as infrastructure as code and continuous integration and continuous delivery. This knowledge is critical for backend engineers, site reliability engineers, and DevOps roles and is increasingly relevant across many engineering positions.

HardSystem Design

25 practiced

Design an automated capacity planning and predictive autoscaling system that consumes historical telemetry and business signals (campaign schedules, forecasts) to scale infra across regions. Define data collection, feature engineering, model selection and retraining cadence, anomaly detection to avoid bad predictions, safety margins, and how predictions should be applied to cloud autoscalers while preventing cascading scale storms.

Sample Answer

**Clarifying goals & constraints**- Predict regional infra capacity (CPU, mem, RPS, concurrency) ahead of time (minutes→days) using telemetry + business signals (campaigns, forecasts). Respect SLOs, cost targets, and multi-region replication.

**High-level architecture**- Ingest: Metrics (Prometheus), traces, events, business signals (campaign calendar, forecasts) into data lake (Kafka → Parquet on object store).- Offline/online pipeline: feature store, model training, real-time inference service, autoscaler controller, monitoring & feedback loop.

**Data collection**- Granularity: 1m and 5m windows. Store raw + aggregates (p95/p99 latencies, error rates, active connections).- Business signals: campaign id, start/end, expected traffic multiplier, marketing spend, holiday flags.- Label: actual capacity used, scaling events, region identifiers, topology metadata.

**Feature engineering**- Time features: hour/day/week, seasonality flags.- Rolling stats: 5/15/60m means, diffs, trends, burstiness (std, kurtosis).- Categorical encodings for campaigns, region, instance types.- Lagged features and exogenous business signals; quantile targets for uncertainty.

**Model selection & retraining**- Ensemble: probabilistic time-series + ML: - Baseline: Prophet/SARIMA for long-term seasonality. - Gradient boosting (LightGBM/CatBoost) for mixed features & exogenous signals. - LSTM/Temporal CNN for complex temporal patterns if data volume justifies.- Predict quantiles (e.g., 50%, 90%, 99%) to drive safety margins.- Retrain cadence: daily incremental retrain + weekly full retrain. Trigger ad-hoc retrain on concept drift detection.

**Anomaly detection & safeguarding bad predictions**- Pre-filter inputs: online anomaly detectors (EWMA, Median Absolute Deviation) and ML detectors (Isolation Forest) to flag sensor or campaign data anomalies.- Model outputs checked by a validator: compare predicted vs short-term extrapolation; if disagreement exceeds threshold, reject or fall back to conservative model.- Circuit breaker: if many regions show concurrent large corrections, pause predictive updates and revert to reactive autoscaling.

**Safety margins & risk control**- Use quantile forecasts. Apply margin = max(configured_min_margin, percentile_gap + operational_buffer). Margin configurable per region/SLO.- Add cooldown windows and target utilization thresholds (e.g., target 60% utilization with upper safety margin).

**Applying predictions to cloud autoscalers**- Controller obtains horizon forecasts and computes desired capacity per region per instance class.- Smoothing: apply rate-limited steps (token-bucket) and progressive scale-up plans (staged increases) to avoid sudden large provisioning.- Leader election per region to avoid duplicate actions; use optimistic locking on central state.- Canary rollouts: small pre-warmed capacity before full scale; observe metrics for 5–15 minutes.- Integrate with provider APIs (ASGs, GKE node pools) via idempotent reconcile loops.

**Preventing cascading scale storms**- Global coordination: stagger scaling across availability zones and regions with jittered start times.- Upper bound on concurrent scaling operations; prioritize critical services.- Backoff strategies: exponential backoff on repeated scaling failures.- Safety thresholds: persistent high error rates or network anomalies trigger freeze.

**Monitoring & feedback**- Dashboards and alerts for prediction accuracy, cold starts, cost delta.- Automated KPI tracking (SLO violations, overprovisioning cost).- Closed-loop: use post-hoc reconciliation to label data for retraining and improve models.

This design balances predictive efficiency, operational safety, and cost control—use conservative rollout, strong validation, and cross-region coordination to avoid cascading failures.

HardTechnical

39 practiced

Technical coding exercise: Write a production-safe Python script that performs a rolling restart of a fleet of virtual machines in a cloud region using the cloud provider API of your choice (specify provider). Requirements: ensure at least N healthy instances remain available during the rolling restart, implement exponential backoff to respect API rate limits, be idempotent (safe to re-run), provide a dry-run mode and verbose logging, and handle crashes and retries gracefully. Describe how you'd test this script and common failure modes to handle.

Sample Answer

**Approach (brief)**

- Use AWS EC2 (boto3). Script lists instances in a target autoscaling group or tag, checks health, and restarts instances one-by-one while ensuring at least N healthy remain. Implements exponential backoff, idempotency (instance-level state tags), dry-run and verbose logging, and retry on failures.

**Code (production-safe, annotated)**

python

#!/usr/bin/env python3
import boto3, time, logging, sys, argparse, backoff
ec2 = boto3.client('ec2')
asg = boto3.client('autoscaling')

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger("rolling-restart")

def parse():
    p = argparse.ArgumentParser()
    p.add_argument('--region', required=True)
    p.add_argument('--tag', required=True)
    p.add_argument('--min-healthy', type=int, default=2)
    p.add_argument('--dry-run', action='store_true')
    p.add_argument('--verbose', action='store_true')
    return p.parse_args()

@backoff.on_exception(backoff.expo, Exception, max_time=60)
def describe_instances(region, tag):
    return boto3.client('ec2', region_name=region).describe_instances(
        Filters=[{'Name': f"tag:{tag.split('=')[0]}", 'Values': [tag.split('=')[1]]},
                 {'Name':'instance-state-name','Values':['running']}]
    )

def tag_instance(iid, key, value, dry_run, region):
    if dry_run: logger.info("[dry-run] tag %s=%s on %s", key,value,iid); return
    boto3.client('ec2', region_name=region).create_tags(Resources=[iid], Tags=[{'Key':key,'Value':value}])

def reboot_instance(iid, dry_run, region):
    if dry_run: logger.info("[dry-run] reboot %s", iid); return
    boto3.client('ec2', region_name=region).reboot_instances(InstanceIds=[iid])

def is_instance_healthy(iid, region):
    # simple: check status checks
    resp = boto3.client('ec2', region_name=region).describe_instance_status(InstanceIds=[iid])
    states = resp.get('InstanceStatuses', [])
    return bool(states) and states[0]['InstanceStatus']['Status']=='ok' and states[0]['SystemStatus']['Status']=='ok'

def main():
    args = parse()
    if args.verbose: logger.setLevel(logging.DEBUG)
    resp = describe_instances(args.region, args.tag)
    instances = [i['InstanceId'] for r in resp['Reservations'] for i in r['Instances']]
    logger.info("Found %d running instances", len(instances))
    for iid in instances:
        # idempotency: skip if tag Restarted=true
        # ensure min healthy remain
        healthy = sum(1 for x in instances if is_instance_healthy(x, args.region))
        if healthy - 1 < args.min_healthy:
            logger.info("Skipping %s to preserve min healthy (%d)", iid, args.min_healthy); continue
        # mark in-progress
        tag_instance(iid, 'RestartState', 'in-progress', args.dry_run, args.region)
        try:
            reboot_instance(iid, args.dry_run, args.region)
            # wait and verify
            for _ in range(12):
                time.sleep(10)
                if is_instance_healthy(iid, args.region): break
            else:
                raise RuntimeError("Instance failed health checks after reboot")
            tag_instance(iid, 'RestartState', 'done', args.dry_run, args.region)
            logger.info("Rebooted %s", iid)
        except Exception:
            logger.exception("Failed for %s, leaving RestartState=in-progress for retry", iid)
            raise

if __name__ == '__main__':
    main()

**Key concepts & reasoning**

- Idempotency: uses tags RestartState=in-progress/done so re-runs skip completed items and can resume.- Rate-limiting: uses exponential backoff decorator for API calls.- Safety: checks health before and after, respects min-healthy.- Dry-run & verbose: flags change behavior and logging.

**Testing & failure modes**

- Tests: unit-test functions with moto (mocked AWS), integration tests in staging with small fleet, chaos tests (force instance failures), simulate API throttling.- Common failures: API throttling (handled by backoff), instance failing to become healthy (alert, rollback or replace via ASG), partial progress/crash (idempotent tags allow resume), network partitions (retries), permission errors (fail fast with clear logs).

This script is a concise, production-minded pattern; in production add metrics, alerting, stronger health checks (load balancer/ASG), concurrency control, and IAM least-privilege.

EasyTechnical

31 practiced

Explain autoscaling concepts for cloud infrastructure. Differentiate horizontal autoscaling and vertical scaling. Describe autoscaling triggers (CPU, memory, request latency, custom app metrics), warm-up considerations, scaling cooldowns, and scenarios where scheduled or predictive scaling is preferable.

EasyTechnical

30 practiced

Define Kubernetes and its core control plane and node components: API server, etcd, scheduler, controller-manager, kubelet. Explain Pod, Deployment, ReplicaSet, StatefulSet, and Service. As a Systems Engineer, how do you decide whether to run stateful workloads on Kubernetes or use managed stateful services?

Sample Answer

**Kubernetes — short definition**Kubernetes is an open-source container orchestration platform that automates deployment, scaling, and management of containerized applications across clusters of machines.

**Core control-plane & node components**- API Server: central REST/gRPC endpoint; validates requests and serves cluster state.- etcd: distributed key-value store holding cluster state (source of truth); must be backed up and secured.- Scheduler: assigns Pods to nodes based on resource requests, taints/tolerations, affinity and policy.- Controller-Manager: runs controllers (Replica, Deployment, Node, Endpoints) to reconcile desired vs actual state.- Kubelet (node agent): ensures Pods/containers are running per PodSpec on each node; reports status to API server.

**Key workload abstractions**- Pod: smallest deployable unit (one or more co-located containers sharing network and storage).- ReplicaSet: ensures a specified number of Pod replicas are running; usually managed by Deployments.- Deployment: declarative updates for ReplicaSets/Pods; supports rolling updates and rollbacks.- StatefulSet: manages stateful apps with stable identities, ordered rollout/scale, persistent volumes per Pod.- Service: stable network abstraction (ClusterIP/NodePort/LoadBalancer) to expose Pods; Service + Endpoint handles discovery.

**Decision: run stateful on K8s vs managed services**Consider:- Data durability & SLAs: choose managed DBs for strong SLAs, automatic backups, multi-AZ replication.- Operational overhead: managed reduces ops burden (patching, upgrades, backups).- Performance & latency: colocating stateful on nodes may improve locality but increases complexity.- Scaling & lifecycle: StatefulSets handle stable identity but complex scaling and recovery; managed services better for horizontal scaling and failover.- Compliance & cost: sometimes on-prem or self-hosted needed for compliance or cost reasons.

Rule of thumb for Systems Engineer: prefer managed stateful services for production databases and mission-critical storage unless specific requirements (custom storage, latency, cost, compliance) justify running stateful workloads on Kubernetes with hardened ops (PV strategy, operator, backups, monitoring, etc.).

MediumTechnical

32 practiced

Create a cost-optimization plan for a mixed AWS workload (scheduled batch jobs, long-running web services, numerous dev/test environments). Include rightsizing, using reserved instances/savings plans, spot instances/spot fleets for batch processing, instance scheduling for dev/test, storage lifecycle policies, and tagging practices for chargeback and cost visibility.

Sample Answer

**Approach (brief)** As a systems engineer I’d create a phased cost-optimization plan combining rightsizing, commitment plans, spot for batch, scheduling for dev/test, storage lifecycle, and strong tagging + visibility so teams can be accountable.

**Plan (actions)**

- Rightsizing - Run AWS Compute Optimizer + CloudWatch metrics 30–90 days; identify underutilized vCPU/RAM/IOPS. - Move steady low-burst workloads to smaller families (T3/T4) or c5→c6 where appropriate; consolidate with ASG mixed instances.

- Reserved Instances / Savings Plans - Purchase regional Compute Savings Plans for baseline steady-state usage (1–3 year, convertible). - Use instance RIs for specific capacity needs (zonal) if required by HA.

- Spot for batch - Convert scheduled batch jobs to Spot Instances/Spot Fleets with diversified instance types and capacity-optimized allocation; use checkpoints or S3-backed state to tolerate interruptions. - Use EC2 Fleet or Batch with bid strategies and fallback to On-Demand for SLA-critical runs.

- Instance scheduling (dev/test) - Implement AWS Instance Scheduler or Lambda-driven start/stop policies by tag (e.g., workhours Mon–Fri 8–18). - Provide self-service portal (EventBridge + Lambda) for teams to request exemptions.

- Storage lifecycle - S3 lifecycle: frequent→infrequent→glacier/deep archive; enable Intelligent-Tiering for variable access. - EBS: identify unattached volumes and snapshots; lifecycle policies to delete old snapshots; right-size GP2→GP3 where possible. - RDS: review instance sizes, use automated backups retention policy, consider Aurora Serverless for variable DB load.

- Tagging & chargeback - Enforce required tags: CostCenter, Project, Environment, Owner, SLA. - Use AWS Organizations + SCPs to require tag keys at creation (via Service Catalog). - Enable Cost Allocation Tags, AWS Cost Explorer, and create per-project dashboards; export CUR to a data lake for detailed reporting and automated chargeback.

**Monitoring, automation & governance**- Implement automated reports and alerts (Cost Explorer budgets, CloudWatch Alarms) for monthly overspend thresholds. - Run Trusted Advisor and monthly audits; automated remediation via Lambda (e.g., stop orphaned resources).

**Metrics & expected outcomes**- KPIs: % reduction in On-Demand spend, % utilization improvement, monthly cost variance by project. - Target: 20–40% cost reduction within 3–6 months depending on baseline.

**Trade-offs**- RIs/Savings Plans require commitment; use phased purchases after observing 30–60 days of telemetry. - Spot reduces cost but adds complexity—limit to batch and non-critical services.

This plan balances immediate wins (rightsizing, scheduling) with longer-term governance (tagging, chargeback) to sustain cost discipline.

Unlock Full Question Bank

Get access to hundreds of Cloud Platforms and Infrastructure interview questions and detailed answers.

Join thousands of developers preparing for their dream job.