Compute and Cloud Native Architecture Questions

Covers high level compute architecture decisions when building cloud native systems, including how microservices, containerization, and orchestration fit together. Topics include cloud native principles, serverless versus container trade offs, how containerized workloads change application and infrastructure design, deployability considerations, observability and telemetry, operational concerns when running services in managed cloud environments, and patterns for designing systems for independent scalability and fault isolation.

HardTechnical

58 practiced

For a high-volume microservices environment generating millions of traces per minute, explain strategies to sample, store, and query traces without losing business-critical signals. Discuss adaptive sampling, tail-based sampling, storage tiering, and how you decide what to keep at full fidelity.

Sample Answer

Situation: In a high-volume microservices environment producing millions of traces/min, the goal is preserving business‑critical signals (errors, latency spikes, customer-impacting flows) while keeping costs and query latency manageable.

Approach (high-level):- Instrument to attach business context (customer id, transaction type, SLA tier) and semantic tags to spans so sampling decisions can be business-aware.- Combine sampling strategies: lightweight front-line adaptive sampling, edge filters, and a backend tail-based sampler.

Adaptive sampling:- On ingress do rate-based or probabilistic sampling per service or route with targets tuned per throughput and importance. Use dynamic adjustment (feedback loop) driven by service-level error/latency metrics so sampling rate increases when anomalies appear.- Keep a small reservoir of recent full-fidelity traces per key (e.g., per customer/SLA) to ensure representation.

Tail-based sampling:- Run in collector/ingest tier after traces are assembled. Evaluate completed trace attributes (error presence, high latency, cardinality of user IDs, or business tags). Persist traces that match rules (errors, p99>threshold, known VIPs). For non-matching traces apply probabilistic drop.- Advantage: preserves signals visible only once trace finishes (e.g., downstream errors).

Storage tiering:- Hot tier: full-fidelity traces for recent window (e.g., 24–72h) in fast store (object store + indexed metadata in time-series DB or search engine). Supports low-latency queries and debugging.- Warm tier: sampled traces & aggregated telemetry (service-level spans, metrics, histograms) for 30–90 days in cheaper object storage with indices for common queries.- Cold tier: long-term archival (compressed traces, aggregated rollups) for compliance in blob cold storage.

What to keep at full fidelity (decision criteria):- Always keep: traces with errors/exceptions, traces from VIP customers or high-dollar transactions, traces that breach SLOs/p99, traces matching active investigation/alerts, and traces from newly deployed code paths.- Probabilistically keep: representative traces for baseline behavior (stratified by route/customer/region) to enable sampling unbiasedness.- Drop or aggregate: low-value repetitive background work (e.g., internal heartbeats) after aggregation into metrics.

Implementation & trade-offs:- Use collectors that support tail-based sampling (Jaeger, Tempo with OTel), and storage solutions that separate span data and trace indices (Elasticsearch/ClickHouse for index; object blob for spans).- Monitor sampling bias: compute and surface sampling ratios per dimension; provide replay/temporary full capture when investigating.- Cost vs fidelity: higher fidelity increases storage & query cost; mitigate by selective full fidelity, compression, and aggregations.- Compliance/retention: ensure archival meets legal requirements; encrypt PII and use tokenization when storing.

Operational considerations:- Expose sampling controls via feature flags/API for on-demand trace ramp-up.- Instrument telemetry for sampling decisions to validate no blind spots.- Automate rule updates driven by ML anomaly detection for proactive tail-capture.

Result: This layered, business-aware approach preserves signal fidelity for critical business and troubleshooting needs while bounding cost and maintaining query performance.

HardTechnical

57 practiced

How would you design a chaos engineering program focused on compute and orchestration layers (container runtime crashes, kubelet restarts, node failures, control plane latency) to increase system resilience? Include scoping, blast radius control, automation, safety checks, reporting, and metrics to evaluate success.

Sample Answer

Requirements & constraints:- Target: compute + orchestration layers (container runtime, kubelet, node HW, control plane latency).- Non-functional: minimize customer impact, measurable SLOs, automated safety and rollback, auditability, multi-tenant isolation.- Phased rollout: dev → staging → canary customers → prod.

High-level design:1. Scoping & experiment catalog- Build a catalog mapping experiment types to objectives, e.g., "containerd crash" → validate pod restart and state reconciliation; "kubelet restart" → validate node cordon/drain behavior; "api-server latency" → validate control-plane timeouts and client-side retries.- Classify experiments by risk (low/medium/high) and required preconditions (no-DB writes, single-tenant).

2. Blast radius control- Namespaces/labels & node selectors to target only non-prod or canary nodes.- Rate limits & concurrency caps (max X nodes / Y minutes).- Circuit breaker service that stops further experiments if error thresholds hit.

3. Automation & orchestration- Implement experiments as code (e.g., Kubernetes jobs/Daemons + operator) in a chaos platform (Litmus/ChaosMesh) wrapped by CI pipelines.- Schedules tied to maintenance windows; experiments require signed approvals for prod.- Automatic rollback: operators reverse actions (restart services, uncordon nodes), and remediation runbooks invoked.

4. Safety checks- Pre-checks: validate backups, cluster health (etcd quorum, control plane pod restarts), SLO headroom.- Canary verification: run smoke tests post-experiment; if failures exceed thresholds, abort and remediate.- Rate-limited credentials and RBAC for chaos orchestration.

5. Observability, reporting & metrics- Collect: control-plane latency, API error rates, pod restart counts, nodeNotReady events, reconciliation latencies, application-level SLOs (p99 latency, error budget burn).- Define KPIs: mean time to detect (MTTD) regressions, mean time to recover (MTTR) from induced faults, successful experiment rate, SLO impact per experiment.- Dashboards and automated reports (per-experiment summary, incident timeline, logs, and traces). Store experiment metadata for audit.

6. Governance & rollout- Governance board approves new experiments; playbooks document intent, rollback, and business impacts.- Training for on-call and SRE teams; runbooks and runbooks-as-code.

Trade-offs & rationale:- Start narrow (control-plane latency via simulated network delay) to gain confidence; expand to high-risk experiments with stricter blast limits.- Automation increases repeatability but needs robust safety and RBAC to prevent misuse.

Success criteria:- Reduced production incident recurrence for failure modes tested.- Decreased MTTR and improved confidence (able to reclaim >80% error budget in staged tests without customer-visible impact).- Evidence of improved automation and documented fixes discovered via chaos experiments.

HardSystem Design

56 practiced

Describe the sidecar pattern's advantages and disadvantages for adding cross-cutting concerns such as logging, telemetry, and auth to microservices. Provide a migration plan to move existing services to a sidecar model with minimal downtime and how you'd mitigate performance and operational overhead.

Sample Answer

Clarify requirements & constraints: target platform (Kubernetes, VMs), SLAs, telemetry volume, security posture, and deployment pipeline. Assume Kubernetes-first but include non-K8s options.

Advantages:- Consistency: centralizes cross-cutting concerns per pod/service instance, ensuring uniform logging, metrics, tracing, auth enforcement without changing app code.- Isolation: sidecar process keeps concerns out of app runtime, reducing blast radius for bugs.- Language/platform agnostic: works for polyglot services.- Faster feature rollout: update sidecar independently of app.- Observability locality: collects per-instance telemetry with local buffering, improving fidelity.

Disadvantages:- Resource overhead: extra CPU/memory per instance; increased pod density impacts cluster sizing.- Operational complexity: more images to manage, security patches for sidecars, additional networking (localhost proxies).- Latency and throughput impact: network hops and proxying can add latency and head-of-line effects.- Debugging complexity: distributed behavior across app + sidecar.- Non-K8s lift: running sidecars on VMs requires process supervisors or proxies (systemd, DaemonSets, or per-host agents).

Migration plan (minimal downtime):1. Pilot: pick low-risk service; deploy sidecar as optional (transparent) proxy; enable telemetry in read-only mode.2. Canary per instance: - Add sidecar next to pods but keep traffic path unchanged (e.g., sidecar listens on separate port). - Use sidecar to mirror traffic (tap) for telemetry only; validate logs/metrics/traces.3. Gradual traffic shift: - Switch service mesh / ingress rules to route a small % through sidecar path (feature flag). - Monitor latency, error rates, resource usage, and end-to-end traces.4. Incremental rollout: - Increase traffic progressively (5%, 25%, 50, 100%), with automatic rollback triggers (SLO breaches). - For auth: start with “audit” mode (validate but accept), then enforce.5. Cutover and cleanup: - When stable, remove legacy collectors/agents and finalize config. - Automate sidecar image updates via CI/CD and include security scanning.

Mitigations for performance & ops overhead:- Right-size resources: baseline and vertical-provision sidecar CPU/memory; use node autoscaling and taints for telemetry-heavy workloads.- Batching & backpressure: use async buffering, batch exports (OTLP gRPC with batching), circuit-breakers to prevent cascading failures.- Use lightweight sidecars: choose performant proxies (Envoy/Linkerd) or single-purpose agents; strip unnecessary features.- Offload heavy processing: perform aggregation/processing in centralized collectors (DaemonSets) or downstream services to avoid per-pod CPU spikes.- Observability of the observability layer: instrument sidecars themselves and add health probes, rate limits, and metrics.- Operational automation: image signing, automated CVE scanning, centralized config (GitOps), templated Helm charts, and runbooks for incident response.- Cost control: estimate per-pod overhead and model TCO; consider shared DaemonSet agents for low-cardinality services.

Trade-offs: sidecars give consistency and rapid rollout at the cost of per-instance overhead and added operational surface. For extremely high-throughput, latency-sensitive services, consider host-level agents or selective in-process libraries. For enterprise clients, propose a hybrid: sidecars for most services, host agents or library integrations for critical low-latency paths.

EasyTechnical

72 practiced

Explain the core cloud-native principles (for example: microservices, immutable infrastructure, declarative APIs, automation, and observable systems). For each principle, describe how it influences compute architecture decisions for a medium-sized customer-facing web application and give one concrete design implication.

Sample Answer

Microservices- Explanation: Small, independently deployable services each owning a single business capability.- Influence: For a medium customer-facing app this favors breaking monolith functionality (auth, catalog, payments) into services to enable independent scaling and faster releases.- Design implication: Use service-per-domain containers (Kubernetes Deployments) with API gateway for routing and per-service autoscaling policies.

Immutable infrastructure- Explanation: Replace, don’t mutate, servers/containers — artifacts are rebuilt and redeployed.- Influence: Drives ephemeral compute (containers/VM images) so environments are reproducible and drift is eliminated.- Design implication: Build CI pipelines that produce versioned container images (immutable tags) and deploy by replacing pods/instances rather than in-place edits.

Declarative APIs (infrastructure and app)- Explanation: Desired state expressed declaratively; controllers reconcile actual state to desired state.- Influence: Enables platform-driven deployments and consistent environments across stages.- Design implication: Use Kubernetes manifests/Helm or GitOps (ArgoCD) to declare deployments, services, and config; rollbacks become simple state changes.

Automation- Explanation: Automate builds, tests, deployments, scaling and recovery to reduce manual error and accelerate delivery.- Influence: Encourages CI/CD, automated provisioning and autoscaling for predictable operations and faster feature delivery.- Design implication: Implement CI/CD pipelines (e.g., GitHub Actions/Jenkins) that run tests, build images, run security scans, and trigger automated deploys to staging/production.

Observable systems- Explanation: Instrumentation, centralized logs, metrics, and distributed tracing to understand runtime behavior.- Influence: Influences choice of runtime and middleware that emit useful telemetry; essential for SLAs and customer experience.- Design implication: Deploy Prometheus + Grafana for metrics, ELK/Opensearch or Loki for logs, and Jaeger/Zipkin for tracing with standardized instrumentation libraries.

Together these principles lead to a compute architecture based on containerized microservices on Kubernetes, immutable image-based deployments, GitOps-driven declarative manifests, automated CI/CD and autoscaling, and full-stack observability — balancing agility, reliability, and operational efficiency for a medium-sized customer-facing web app.

MediumSystem Design

67 practiced

Design a storage strategy for a microservices platform that requires both high-throughput object storage for media and low-latency block storage for databases. Discuss compute placement, data locality, caching layers, backup/restore, and cost trade-offs between managed and self-managed storage options.

Sample Answer

Requirements:- High-throughput object storage for media (uploads, streaming, immutable objects)- Low-latency block storage for OLTP DBs- Multi-region scale, tenancy, predictable cost, RTO/RPO targets

High-level architecture:- Object store (S3-compatible) for media + CDN fronting- Block storage (network attached block devices) for DBs hosted on stateful VMs/K8s with persistent volumes- Separate access patterns and tiers: hot/cold object tiers; high-IOPS and standard block tiers

Core components & responsibilities:1. Object storage: managed S3 (AWS S3/GCS/Azure Blob) or on-prem Ceph/Rook — handles high throughput, lifecycle policies, multipart upload.2. Block storage: managed volumes (EBS/PD/Managed Disks) or SAN with iSCSI for VMs; use NVMe/SSD for hot DBs.3. CDN (edge caching) for media delivery.4. Caching: Redis/Memcached for DB query results, edge caching for frequently accessed objects.5. Orchestration: Kubernetes with StatefulSets + CSI drivers for block volumes.6. Backup & snapshot service: point-in-time snapshots for block; versioning + lifecycle + cross-region replication for objects.

Compute placement & data locality:- Place DB compute in same AZ/rack as block storage to avoid cross-AZ latency and egress costs.- Place application pods/VMs serving media in AZs close to object storage or use signed URLs + CDN to reduce origin hits.- For multi-region, replicate objects to nearest region; use DB read-replicas regionally where low-latency reads are required.

Caching layers:- CDN for static media, with origin shielding.- Edge-object prefetch + Cache-Control headers.- In-memory caches for DB-hot paths; write-through or cache-aside depending on consistency needs.- Local ephemeral cache on app hosts for microsecond-level reads.

Backup, restore & DR:- Block: automated incremental snapshots, periodic full backups, test restores; maintain backup retention and cross-region replication for RTO/RPO.- Object: versioning, lifecycle to archive tiers (Glacier/Archive), cross-region replication for DR.- Disaster plan: runbook for failover (promote read replicas, re-point DNS/CDN), regular DR drills.

Cost trade-offs: managed vs self-managed- Managed (S3, managed volumes): higher OPEX, lower operational risk, built-in durability, scalability, security, multi-region replication — recommended for faster time-to-market and predictable SLAs.- Self-managed (Ceph, MinIO, on-prem SAN): lower raw storage cost at scale, finer control, but higher capital expense, staffing, backup complexity, and operational risk. Good when regulatory/compliance or existing datacenter investments mandate control.- Hybrid: use managed cloud object storage for public media + self-managed for sensitive data, or use cloud-managed offerings with reserved capacity for cost savings.

Recommendations:- Use managed S3 + CDN for media; configure lifecycle + cross-region replication.- Use managed SSD block volumes colocated with compute; encrypt at rest/in transit.- Implement multi-layer caching and regional read-replicas.- Favor managed services unless strict cost or compliance drivers justify self-managed; model TCO including staffing and DR costs.

Unlock Full Question Bank

Get access to hundreds of Compute and Cloud Native Architecture interview questions and detailed answers.

Join thousands of developers preparing for their dream job.