Personal account of hands on experience using public cloud providers and the concrete results delivered. Candidates should describe specific services and patterns they used for compute, storage, networking, managed databases, serverless and eventing, and explain their role in architecture decisions, deployments, automation and infrastructure as code practices, continuous integration and continuous delivery pipelines, container orchestration, scaling and performance tuning, monitoring and incident response, and cost management. Interviewees should quantify outcomes when possible with metrics such as latency reduction, cost savings, availability improvements or deployment frequency and note any formal training or certifications. This topic evaluates depth of practical experience, ownership, and the ability to operate and improve cloud systems in production.
HardSystem Design
39 practiced
You must migrate a 100TB encrypted dataset from Cloud A to Cloud B while preserving encryption policies and compliance controls using customer-managed keys (CMKs). Describe the architecture and process to securely transfer data, how you would manage and rotate keys, whether re-encryption is necessary, how to preserve audit trails, minimize downtime, and ensure compliance during and after migration.
Sample Answer
Requirements & constraints:- Transfer 100TB encrypted data from Cloud A → Cloud B, preserve encryption policies and compliance (e.g., PCI, HIPAA), use customer-managed keys (CMKs), maintain auditability, minimize downtime, and ensure no regulatory gaps.High-level architecture:- Source: Cloud A storage (object/block) encrypted with Cloud A CMK (KMS-A).- Transit: Dedicated secure transfer network (VPN/Direct Connect equivalent) + multipart encrypted data streaming.- Target: Cloud B storage (object/block) encrypted with Cloud B CMK (KMS-B).- Orchestration: Migration controller (stateless service) running in a secure VPC, using worker fleet to copy shards in parallel.- Key/brokering layer: Key wrapping/unwrapping service to handle envelope encryption metadata and mapping between KMS-A and KMS-B.- Audit & monitoring: Centralized SIEM + immutable audit logs (WORM) and hash-based integrity store.Process (step-by-step):1. Discovery & classification: Inventory objects, classify by sensitivity, identify legal constraints (data residency).2. Choose transfer mode: Online incremental sync (recommended) — initial bulk transfer followed by delta sync to minimize downtime.3. Prepare KMS: - In Cloud B, create CMKs (KMS-B) with matching policy controls and roles. Mirror access principals and rotation policies. - Establish trust/peering so migration controller can call both KMSs using least-privilege IAM roles.4. Envelope approach (no mass re-encryption during transit): - For each object, retrieve its data encryption key (DEK) ciphertext from Cloud A (requires KMS-A decrypt permission). Decrypt DEK in-memory within a secure, short-lived HSM-backed environment or ephemeral instance that has transient permission. - Use the plaintext DEK to stream the object from Cloud A (or better: request object in encrypted form and perform rewrap: re-encrypt DEK with KMS-B without writing plaintext to disk). - Re-wrap (encrypt) the DEK with KMS-B producing DEK_B; store object in Cloud B encrypted with the same DEK but with updated metadata referencing DEK_B. This avoids decrypting object payload to disk. - For objects already using an envelope scheme with separate DEKs, this is efficient. If objects are directly encrypted with CMK, perform server-side copy APIs that allow re-encryption or use a secure staged re-encrypt.5. Integrity and audit: - Calculate and store per-object hashes (SHA-256) pre-transfer and post-transfer in an immutable ledger (append-only store or blockchain-like ledger) tied to audit logs. - Log KMS operations, object access, and rewrap operations to centralized SIEM and native KMS audit (CloudTrail/Audit Logs) with retention aligned to compliance.6. Delta sync & cutover: - After initial bulk copy, run continuous change-capture (object versioning, event notifications) to synchronize deltas. - For immutability, mark final sync window; redirect readers/writers to Cloud B endpoints with dual-write or brief maintenance window.7. Validation & decommission: - Validate checksums, audit trails, key provenance. - Keep KMS-A and original data under retention policy until legal/store delete grace periods; only decommission after legal sign-off.Key management & rotation:- Use envelope encryption: short-lived DEKs per object, wrapped by CMKs.- Maintain separation: KMS-A controls remain authoritative for past history; KMS-B becomes authoritative for new writes.- Rotation strategy: - Rotate KMS-B per org policy (e.g., annually) using automated rewrap jobs that re-encrypt DEK wrappers without touching payload. - Maintain key version metadata for provenance. Keep old KMS keys in a recoverable state if audits require historical decryption; disable rather than delete immediately.- Access controls: enforce least privilege, use KMS grants, require multi-approval for key deletion, enable HSM protection where required.Re-encryption necessity:- Re-encryption of payload is unnecessary if envelope DEKs can be re-wrapped (rewrap/reencrypt DEK under KMS-B). This minimizes data movement, exposure, and downtime.- If Cloud A does not expose DEK metadata or wrapping APIs, you must perform secure in-memory decrypt-and-reencrypt for payloads—plan HSM-backed ephemeral nodes and tight audit.Preserving audit trails & compliance:- Centralize and forward all Cloud A and Cloud B audit logs (KMS operations, storage access, IAM changes) to a secure, immutable logging store with retention per compliance.- Record per-object provenance: original key id/version, rewrap timestamp, operator identity, hash, and migration job ID.- Generate compliance artifacts (reports) showing key policies, rotation history, access logs, and integrity verification for auditors.Minimizing downtime:- Bulk initial transfer in parallel (sharding objects), then continuous delta sync using event streaming or CDC.- Support dual-read mode: route reads to Cloud B while writes are dual-wrapped to both clouds during cutover window, or short maintenance window to switch endpoints.- Use versioning to reconcile conflicts; for write-heavy datasets, coordinate with application owners to throttle writes or schedule migration during low-traffic windows.Security controls during transfer:- Use encrypted tunnels (TLS 1.3), mutual TLS endpoints, private connectivity (Direct Connect / Interconnect) with egress filtering.- Run migration controller in hardened, monitored hosts with ephemeral credentials (short-lived tokens).- No persistent plaintext at rest or on disk — in-memory operations, encrypted swap, disk encryption on instances.- Apply data loss prevention (DLP) scanning where required.Operational considerations & trade-offs:- Performance: parallelism vs KMS rate limits; use batched rewraps and request quotas.- Key exposure risk: prefer rewrap over decrypt-then-encrypt; if plaintext needed, use HSMs and narrow time windows.- Legal constraints: if cross-border transfer is restricted, consider hybrid or tokenization instead.Metrics & validation:- Track bytes transferred, throughput, error rates, reconciliation mismatches.- Post-migration: run sample decryption tests, model retraining to validate data integrity and feature parity.This architecture preserves encryption policies by keeping CMK provenance, uses envelope rewrap to avoid mass payload re-encryption, maintains full audibility, minimizes downtime with bulk + delta sync, and meets compliance through retained immutable logs and controlled key lifecycle.
MediumSystem Design
36 practiced
Design a production model-serving architecture that must handle 10,000 requests per second with a P95 latency under 100ms for a classification model. Specify cloud services (compute, load-balancing, caching), autoscaling strategy, concurrency model, caching and batching approaches, ingress and network considerations, and the approach to testing and load-validation to prove the design meets SLA.
Sample Answer
Requirements & constraints:- 10,000 RPS steady/peak, P95 latency <100ms end-to-end for a classification model.- Cloud-managed components preferred; resilient, observable, cost-conscious.High-level architecture:- Ingress: Cloud HTTP(S) Load Balancer (GCP LB / AWS ALB + NLB hybrid for gRPC) terminating TLS, with CDN/edge caching for static responses.- API layer: Kubernetes (GKE/EKS) with an ingress controller; stateless API pods accept requests and forward to model-serving pods via internal gRPC.- Model serving: Dedicated inference pods running optimized runtime (TensorFlow Serving / TorchServe / Triton) inside K8s node pool (CPU or GPU depending on model).- Cache & queue: Redis (managed: Memorystore/ElastiCache) for low-latency response caching and small request de-dup; Redis Streams or Kafka for optional asynchronous micro-batching pipeline.- Observability: Prometheus + Grafana, OpenTelemetry traces, logging to managed sink.Compute & sizing (example numbers — validate with perf tests):- Target per-inference latency budget: 60ms model + 20ms network + 20ms overhead.- If optimized model on CPU handles ~200 RPS (P95 under budget) per pod, need 50 pods (10,000/200). Plan headroom: provision autoscale up to 120 pods; for GPU-heavy models, estimate higher RPS per GPU and adjust.- Node pools: mixed instance types; node auto-provisioning (GKE Autopilot or Cluster Autoscaler) with PodDisruptionBudgets.Load balancing & networking:- Use internal gRPC / HTTP pooling; connection reuse to reduce latency.- Use NLB in front of node pools for stable TCP performance if using gRPC streaming.- Ensure subnets, VPC peering, and private endpoints for model storage and Redis to avoid public hops.- Use mTLS or service mesh (Istio/linkerd) selectively for tracing and routing.Autoscaling & concurrency model:- Horizontal Pod Autoscaler (HPA) driven by custom metrics: - Primary metric: real-time throughput per pod (requests/s) and P95 latency. - Secondary: CPU/GPU utilization.- KEDA for event-driven scaling if using Redis Streams/Kafka for batching.- Cluster Autoscaler to add nodes when pods pending.- Concurrency model: use multiple worker threads per pod (async gRPC/uvicorn with gunicorn for Python) with a per-pod concurrency cap to keep tail latency low (e.g., 32–64 concurrent requests/pod). Prefer async/non-blocking runtimes.Caching & batching:- Caching: - LRU Redis cache keyed by request features hash for deterministic classification requests; TTL tuned to data staleness. - Edge caching for idempotent requests (CDN).- Batching: - Synchronous micro-batching inside model server with a short timeout (e.g., max batch size 8–16, max wait 2–5ms) to increase GPU/CPU utilization without adding tail latency. - If high variance in arrival, use async queue (Redis Streams) + pool of batch workers; KEDA scales workers based on queue depth.Testing & load-validation to prove SLA:- Create representative load tests with realistic request distributions and payloads.- Tools & process: - Use k6 / locust / Fortio to generate 10k RPS from multiple load generators across regions to avoid client-side bottlenecks. - Run incremental ramp tests (start 1k -> 10k) while measuring P50/P95/P99 latency, error rates, CPU/GPU, queue depths. - Chaos tests: node termination, network latency injection, Redis failover, model cold-start tests. - Canary deploys with traffic shaping: shift 1%→10%→100% while validating metrics.- Success criteria: - Sustained 10k RPS with P95 <100ms and error rate <0.1%. - Auto-scaling completes within SLA (scale-out latency measured) and sufficient headroom to absorb spikes.Operational considerations:- Warm pools or pre-warmed container images to avoid cold-starts.- Model versioning in S3/GCS and rolling update strategy.- CI pipeline: benchmark new model artifacts in staging with automated perf tests before production rollout.- Cost trade-offs: evaluate CPU vs GPU per-inference cost and choose the most cost-effective while meeting latency.Why this works:- Combines low-latency path (no extra queue for typical requests) with optional async batching when throughput needs to be maximized.- Autoscaling tied to both utilization and latency/queue metrics ensures SLA-driven scaling.- Realistic performance validation and chaos testing prove the design meets the SLA before full rollout.
EasyTechnical
37 practiced
Compare serverless functions (e.g., AWS Lambda, GCP Cloud Functions) versus container-based deployments on Kubernetes (EKS/GKE/AKS) for serving ML models. Discuss cold-starts, concurrency limits, model size limits, latency characteristics, state management, operational complexity, and cost behavior at low and high traffic volumes. Provide recommendations for small batch inference and for serving large transformer models.
Sample Answer
Situation: Comparing serverless functions vs. Kubernetes for serving ML models across cold-starts, concurrency, model size, latency, state, ops, and cost.Comparison (by topic):- Cold-starts: Serverless can have noticeable cold starts (100ms–seconds) especially if large runtimes or cold-account. Kubernetes with always-on pods avoids cold starts.- Concurrency limits: Serverless platforms often limit per-instance concurrency (or scale by invocations); Kubernetes lets you control concurrency with pod counts and thread/process settings (e.g., Gunicorn, Uvicorn workers).- Model size limits: Serverless has hard package/container size and memory limits (e.g., Lambda /tmp, deployment package limits). Kubernetes supports large images and GPUs for big models.- Latency characteristics: Serverless is good for bursty, low-throughput endpoints where occasional latency is acceptable. Kubernetes gives more predictable, lower tail latency when pods are warmed and right-sized.- State management: Serverless is stateless; use external stores (Redis, S3). Kubernetes can run stateful services or sidecars, but best practice is still externalizing persistent state.- Operational complexity: Serverless is low ops (provider-managed), fewer infra skills needed. Kubernetes requires cluster management, networking, autoscaling, observability expertise.- Cost behavior: At low traffic, serverless is usually cheaper (pay-per-request). At sustained high traffic, Kubernetes can be more cost-efficient due to reserved instances and packed utilization.Recommendations:- Small batch inference (occasional, small models): Use serverless for fast setup and low cost at idle. Mitigate cold-starts with provisioned concurrency (if supported) or lightweight runtimes.- Serving large transformer models (GPU, large memory, low tail-latency requirements): Use Kubernetes (EKS/GKE/AKS) with node pools including GPU instances, model sharding, batching, and autoscaling. Consider managed inference services (SageMaker, Vertex AI) or model-optimized servers (TorchServe, Triton) on Kubernetes for best performance and control.Trade-offs summary: Choose serverless for simplicity and spiky low-volume workloads; choose Kubernetes for large models, strict latency, custom scaling, and cost efficiency at scale.
HardTechnical
34 practiced
You discover that a production model has been trained with features that leaked future information, inflating offline metrics. Outline an incident response plan: immediate mitigation (rollback or disable), scope analysis to identify affected models and data, remediation and retraining plan, how to estimate business impact and rollback cost, and how to update pipelines and tests to detect similar leakage in the future.
Sample Answer
Situation: A production model was found to use features that leaked future information, inflating offline metrics and risking bad decisions.Immediate mitigation (first 24 hours)- Disable the model endpoint or route traffic to a safe fallback (previous validated model or rule-based heuristic) to stop further bad decisions.- Put a short freeze on automated retraining or deployment pipelines to prevent compounding.- Notify stakeholders (product, engineering, compliance, ops) with a concise incident summary and mitigation steps.Scope analysis (24–72 hours)- Forensic feature audit: identify which features contain leakage (time-shifted joins, target-derived aggregations, label overlap).- Inventory affected artifacts: model(s), training datasets, feature stores, feature engineering code, feature-views, and deployment timestamps.- Evaluate serving vs. training discrepancy: confirm whether leaked features exist in production serving logic.- Run targeted tests: replay recent inputs through model to quantify divergence between expected and actual predictions.Remediation & retraining plan (3–14 days)- Fix root cause in feature pipeline: correct joins, ensure causality, add event-time semantics, use proper backfills.- Create a clean, provenance-tracked training dataset (immutable snapshots) and regenerate features with strict time-awareness.- Retrain model with corrected features; run strict validation: temporal cross-validation, label leakage tests, and synthetic leakage injection tests.- Stage the new model in canary with shadow traffic, monitor calibration, business KPIs, and data drift before full rollout.- Document changes and approvals for audit trail.Estimating business impact & rollback cost- Quantify decisions affected: count transactions/events during exposure window and proportion routed by model.- Sample and label outcomes where possible to estimate error rate uplift and financial impact (revenue loss, costs, compliance risk).- Estimate rollback cost: engineering hours to switch traffic, customer remediation, and opportunity cost; compare to cost of continuing vs. rollback.- Present conservative and optimistic impact ranges to stakeholders to make informed trade-offs.Preventing recurrence (pipeline & tests)- Enforce feature governance: register features with metadata (creation time, event-time semantics, allowed joins).- Automated static checks in CI: detect usage of target columns or future timestamps in feature code, enforce window constraints.- Add dynamic tests: time-travel/unit tests that validate that for each training row, no feature uses events after prediction time; synthetic leakage tests.- Expand validation suite: temporal cross-validation, permutation importance by time, and monitoring alerts for sudden offline/online metric divergence.- Add lineage and reproducibility: immutable data snapshots, feature-store versioning, and retention of training seeds and artifacts.- Run regular blameless postmortems and update runbooks with clear playbooks for leakage incidents.This plan balances immediate protection, forensic clarity, and long-term prevention while providing measurable business impact for stakeholder decisions.
EasyBehavioral
43 practiced
Describe a specific production project where you used a public cloud provider to deploy a data-science solution. Name the cloud provider(s) and the concrete services you used for compute, storage, networking, managed databases, serverless/eventing, and orchestration. Explain your role in architecture decisions, deployments, automation and Infrastructure-as-Code practices, CI/CD, monitoring, and quantify outcomes where possible (for example latency reduction, cost savings, availability improvements, or deployment frequency). Mention any cloud certifications you hold.
Sample Answer
Situation: At my last company I led a production project to deliver real-time churn prediction for a SaaS product using AWS.Task: Build, deploy and operate a low-latency, cost-effective ML service that scores users in near real-time and supports batch re-scoring.Action:- Architecture decisions: selected AWS SageMaker for model training and hosting (low ops, built-in A/B), S3 for raw/feature storage, Redshift for analytical warehouse, RDS (Postgres) for serving metadata, VPC + ALB for secure networking, Lambda + SNS/SQS for eventing, Step Functions to orchestrate retraining pipelines, and EKS (Kubernetes) for ancillary microservices.- I designed the pipeline, wrote Terraform modules for VPC, SageMaker domains, IAM, RDS and EKS to enforce reproducible infra-as-code.- CI/CD: GitHub Actions for pipeline (unit tests, model validation), automated Terraform plans/applies in CI with approvals, and a CD step to push new model artifacts to SageMaker (endpoint update with blue/green deployment).- Automation: Step Functions triggered monthly retrain; Lambda validated metrics and promoted model if AUC improved.- Monitoring: CloudWatch + SageMaker model monitoring for drift, Prometheus/Grafana for service metrics, alerting via PagerDuty.Result:- Achieved 120ms median prediction latency per request; availability >99.9% for the endpoint.- Reduced operational cost 35% vs. previous EC2-based approach by using SageMaker managed hosting + spot instances for training.- Deployment frequency: model updates automated from weekly to daily experimentation; mean time to deploy model decreased from 2 days to ~2 hours.- Business impact: 8% reduction in monthly churn within 3 months of model rollout.Certifications: AWS Certified Machine Learning – Specialty.
Unlock Full Question Bank
Get access to hundreds of Cloud Platform Experience interview questions and detailed answers.
