Cloud Platforms and Tooling Questions

Practical tooling, automation, and platform specific considerations for operating and deploying workloads in public cloud environments. Topics include infrastructure as code tools and templates including vendor native templates and third party tools, account and organization structure and role based access design, deployment automation and pipeline patterns for continuous integration and continuous delivery, platform specific software development kits and command line tooling, container orchestration and serverless deployment tooling, monitoring and logging stacks and alerting, cost and billing tooling and cost optimization methods, and guidance on single cloud versus multi cloud approaches. Candidates should be able to discuss tooling selection criteria, integration between developer and operational tooling, and how tooling choices impact deployment velocity, reliability, and operational overhead.

MediumTechnical

145 practiced

Explain how to implement policy-as-code across infrastructure and Kubernetes: example tools (OPA/Gatekeeper, Sentinel, Azure Policy), where to enforce policies (pre-commit, CI, admission controllers), and how to present denied changes to developers with actionable feedback.

EasyTechnical

120 practiced

Design a high-level account and organization structure for a mid-sized company (100 engineers) on a public cloud (choose AWS/GCP/Azure). Explain how you'd separate accounts/projects for development, staging, and production, how you'd implement role-based access control, and what guardrails (SCPs, org policies) you'd apply to balance autonomy and security.

Sample Answer

I’d choose AWS and implement a multi-account landing zone using Organizations + Control Tower (or Terraform + Organizational Units) to balance isolation, security, and team autonomy.

High-level account/OU layout- Root Org: no workloads, only org management.- Security OU: - Security Account: SIEM, GuardDuty, Security Hub, Secrets Manager, centralized IAM audit. - Logging/Audit Account: centralized CloudTrail, Config, S3 audit bucket (immutable).- Network OU: - Network Account: shared VPCs, NAT, Transit Gateway, VPC endpoints; use VPC sharing (RAM).- Platform/Tooling OU: - CI/CD Account: build runners, artifact registry, ECR, GitHub Actions runners, container registry.- Workloads OU: - Prod Account(s): one per critical product or single prod account with separate prod projects (for 100 engs typically 2–4 prod accounts to limit blast radius). - Staging Account(s): mirrors prod for testing. - Dev Account(s): per-team or per-feature sandbox accounts (self-service but ephemeral).

RBAC and access patterns- Central identity: AWS IAM Identity Center (SSO) federated to corporate IdP (Okta/AzureAD).- Groups & roles: map IdP groups to IAM roles (e.g., SRE-Prod-Admin, SRE-Prod-ReadOnly, Dev-Write). No direct long-lived user creds; use short-lived SSO sessions.- Cross-account access: grant assume-role patterns for CI/CD and SRE (audit/read/admin) with MFA and justification tags.- Least privilege: start with narrow roles and expand with permission boundaries and managed policies.- Just-in-time elevation: use AWS IAM Access Analyzer or a ticketed workflow + temporary elevation (e.g., AWS SSO session with approver).

Guardrails / Org Policies (SCPs and Config rules)- SCP examples (deny unless explicitly allowed): - Deny iam:CreateUser/PutUserPolicy and prevent root usage. - Deny public S3 buckets and enforce bucket encryption (SSE). - Deny creation of resources in disallowed regions. - Deny use of instance types that bypass security (e.g., disable classic EC2-Classic).- AWS Config rules & Service Control Policies: - Enforce encryption at rest and in transit. - Require resource tagging (owner, cost-center, environment). - Require CloudTrail enabled & integrity validation. - Enforce approved AMIs and baseline security groups (no 0.0.0.0/0 for sensitive ports).- Network guardrails: - Centralize egress controls via NAT/proxy, restrict internet gateways for prod subnets. - Prevent public IP assignment in prod by default.- Resource lifecycle: enforce auto-termination/expiration tags in dev accounts.

Operational practices- Centralized logging and metrics: all accounts send CloudTrail, VPC flow logs, app logs to Logging/Audit account; use cross-account CloudWatch/Prometheus federation.- CI/CD pipeline runs in CI account and assumes least-privilege deploy roles into target accounts.- IaC & policy-as-code: Terraform/CloudFormation + automated pre-commit policy checks (tfsec, cfn-lint) and policy enforcement with AWS Config/OPA/Gatekeeper.- Incident access & runbooks: documented assume-role playbooks; emergency break-glass with stricter audits and temporary keys tracked centrally.

Trade-offs / reasoning- Multiple accounts reduce blast radius and limit scope of misconfigurations; small overhead in cross-account orchestration mitigated by central tooling (VPC sharing, SSO).- SCPs provide safety net without micromanaging teams; use deny-by-exception rules to preserve autonomy.- Centralized logging/security enables faster incident response and compliance reporting, while delegated accounts allow teams to iterate fast in dev.

This design gives SREs clear admin and audit paths, lets devs self-serve in sandboxed spaces, and enforces security guardrails to keep production reliable and compliant.

HardSystem Design

88 practiced

Plan a migration from 50 self-managed Kubernetes clusters to a managed Kubernetes service (e.g., EKS/GKE/AKS) with zero downtime. Cover control plane migration, node pooling, network and CNI differences, data plane compatibility, cluster config drift, and a phased migration plan including testing.

Sample Answer

Requirements & constraints:- Zero user-visible downtime, preserve SLOs, GDPR/data residency, maintain security posture, support 50 clusters with minimal disruption, cost and rollback capability.

High-level strategy:- Adopt a blue-green-style migration per cluster: create a managed cluster (green), replicate workloads/data, switch traffic; do this progressively across clusters. Automate and standardize with IaC and CI.

Key areas & plan:

1) Control plane migration- Managed control plane (EKS/GKE/AKS) replaces self-managed masters. Create managed cluster with same k8s version or compatible minor version.- Use identical RBAC, OIDC/IdP integrations, and audit/log export. Validate admission controllers (PodSecurityPolicy -> PSP replacements like OPA/Gatekeeper).

2) Node pooling & workloads- Recreate node pools (instance types, taints/tolerations, autoscaling groups) using managed node groups or node pools + Karpenter for scale-to-zero/fast scale.- Deploy workloads gradually: use dual-cluster Deployments with traffic split at the service/proxy level (Ingress controller, Service meshes like Istio/Linkerd) or DNS/Global LB weights.- Ensure PodDisruptionBudgets and readiness/liveness probes are configured to avoid evicting all replicas.

3) Network & CNI differences- Inventory existing CNI (Calico/Weave/Flannel) and networking (overlay vs VPC-native). Choose managed CNI compatible with cloud (AWS VPC CNI, GKE’s VPC-native) or run Calico in policy-only mode.- Map IP addressing: ensure pod CIDR sizes and node subnet allocations avoid overlaps; reserve VPC CIDR. For inter-cluster connectivity, use VPC peering/VPN/Transit Gateway or service mesh multi-cluster.- Validate NetworkPolicy translation—if using Calico policy primitives, ensure managed environment supports them.

4) Data plane compatibility- Stateful apps: replicate PersistentVolumes by using cloud storage classes (EBS/PD/ManagedDisks) and migrate data via: - Application-level replication (DB replicas, streaming), - Volume snapshot and restore (CSI snapshots), - Temporarily run cross-cluster read replicas, then promote.- Ensure PVC access modes (RWX vs RWO) supported; for RWO, use application replication instead of moving single PV.

5) Cluster config drift & policy alignment- Implement GitOps (Flux/ArgoCD) with cluster-specific overlays to apply identical configs. Use a policy-as-code gating pipeline: OPA/Gatekeeper tests, Conftest for manifests.- Run config-drift scans (kube-bench, kubescape, custom comparator) pre/post migration.

6) Phased migration plan & testingPhase 0 — Prep (2-4 weeks):- Inventory: workloads, PVs, CNIs, ingress, external DNS, secrets, custom CRDs.- Build landing zone IaC: VPCs, IAM, logging, monitoring (Prometheus, Grafana, CloudWatch/Stackdriver), image registries, CI/CD.Phase 1 — Pilot (1-2 clusters):- Create managed cluster, replicate infra, deploy non-critical stateless apps.- Smoke tests: e2e, chaos (pod/node kill), latency, throughput, SLO checks.Phase 2 — Stateful & network test (3-5 clusters):- Migrate a stateful service using snapshot/replication.- Test cross-cluster connectivity, NetworkPolicies, service discovery.Phase 3 — Incremental cutover (rolling by 5–10 clusters/week):- For each cluster: create managed cluster, sync manifests via GitOps, deploy mirrored apps, enable dual-write or replication, perform canary traffic shift (10%→50%→100%) using traffic manager or global LB.- Monitor KPIs, rollback if errors (traffic revert to old cluster, scale down new).Phase 4 — Decommission & optimize:- After stable period, drain old nodes, revoke access, archive logs, and retire cluster infra.- Rightsize node pools, enable autoscaler optimizations, and enable managed features (spot/interruptible, node auto-provisioning).

Validation & rollback:- Pre-cutover: automated test suite, SLO gating, runbooks.- During cutover: health-check automation, automated rollback on threshold breach (error rate/latency).- Post-cutover: full audit, security scans, performance baselining.

Observability & runbooks:- Centralized logging/metrics/alerts; synthetic transactions for critical paths.- Playbooks for DNS, DB promotion, PV failback, and emergency cutback to self-managed clusters.

Trade-offs & risks:- Risk: CNI incompatibilities and IP exhaustion — mitigate via careful network planning and small pilots.- Risk: stateful migrations — prefer app-native replication where possible.- Cost vs operational simplicity: managed control plane costs but reduces ops burden.

This phased, automated, GitOps-driven approach preserves availability, gives safe rollback paths, and standardizes configuration to prevent drift while migrating 50 clusters with minimal operational risk.

EasyTechnical

77 practiced

Compare GitOps and traditional push-based CI/CD pipeline models for deploying to cloud platforms and Kubernetes. Describe typical tooling for GitOps (e.g., ArgoCD/Flux), core benefits, and potential operational challenges for an SRE team adopting GitOps.

HardSystem Design

73 practiced

Define SLOs and an error budget policy for a payments microservice with hard business requirements (e.g., >99.95% success for payment completion). Include how to attribute errors across dependencies, how to structure alerts for SLO burn, and the operational response when an error budget is exhausted.

Sample Answer

Requirements & constraints:- Business: payment completion success ≥99.95% over rolling 30-day window (max 0.05% failures).- Hard constraints: regulatory/accounting correctness, no silent loss, traceability, bounded latency (p95 < 500ms).- Multi-dependency: payment gateway, auth, ledger, queueing, DB.

SLOs:- Primary SLO: Successful end-to-end payment completions ≥99.95% (rolling 30d). Success = idempotent payment processed and ledger updated, confirmation delivered.- Secondary SLOs (for isolation): gateway-ack ≥99.9%, ledger-write success ≥99.99%, instrument-auth ≥99.99%, API latency p95 < 500ms.- Error budget: 0.05% failures over 30 days → ~43.2 minutes of allowed bad time for a 30d window (for requests weighted by volume).

Error attribution across dependencies:- Implement distributed tracing (W3C TraceContext) + unique payment IDs. Instrument each hop with standardized outcome tags (success, transient-error, permanent-error, throttled).- For each failed payment, record root-cause field: first non-retryable failure, or if retries exhausted, the dependency that returned terminal error.- Use probabilistic sampling + full trace capture for failures. Aggregate by dependency and error-type to compute error budget burn per component using request-weighted counts.

Alerting / SLO-burn structure:- Multi-tier alerts on SLO burn rate: - INFO: projected burn → 25% of budget at current rate (investigate, triage). - WARNING: 50% burn (page on-call lead, runbook review). - CRITICAL: 75% burn (eng + product stakeholders, mitigations). - EMERGENCY: 100% budget exhausted (immediate action).- Alert signals: absolute error rate spike, sustained elevated burn projection (e.g., burn rate > 4× baseline leads to exhaustion before window end), correlated dependency degradation.- Provide dashboards: burn countdown, error-by-dependency, top traces, latency distribution.

Operational response when budget exhausted:- Immediate measures (first 30–60 min): - Auto-mitigation: switch to circuit-breakers/backpressure, degrade non-critical features, throttle non-payments traffic, route to secondary gateway if healthy. - Halt risky deploys: freeze releases, revert recent deploys that correlate with burn. - Engage incident response: page SRE + dev + product; stand up war room; run dependency-specific playbooks.- Remediation (next steps): - Apply targeted fixes (retry tuning, configuration, scaling), increase capacity for throttled deps, apply temporary feature flags to reduce load. - Communicate: notify customers per SLA; internal stakeholder updates.- Post-incident: - Blameless postmortem with timeline, attribution, RCA, and action items: improve instrumentation, changegate rules, resiliency (circuit-breakers, bulkheads), revise SLOs if unrealistic. - Reset policy: error budget re-evaluated; until budget partly replenished, enforce stricter release/feature controls.

Trade-offs:- Strict SLO protects business but increases ops overhead. Use secondary SLOs to balance internal ownership. Prioritize automated attribution to avoid slow human triage.

Unlock Full Question Bank

Get access to hundreds of Cloud Platforms and Tooling interview questions and detailed answers.

Join thousands of developers preparing for their dream job.