Senior Site Reliability Engineer Interview Preparation Guide - Amazon

Site Reliability Engineer (SRE)

Amazon

Senior

6 rounds

Updated 6/17/2026

While search results confirm SRE role openings exist at Amazon, detailed official Amazon career page data for SRE-specific interview processes was not available in search results. This guide is constructed using industry-standard SRE interview patterns for senior-level positions combined with AWS/cloud infrastructure practices, aligning with Amazon's technology ecosystem.

Amazon's senior-level SRE interview process typically involves a recruiter screening call, a technical phone screen focusing on systems thinking and coding, followed by 4-5 onsite interviews covering system design, technical deep dives, operational scenarios, and behavioral assessment aligned with Amazon Leadership Principles. The process emphasizes reliability engineering, incident response, infrastructure automation, and scalable system design.

Interview Rounds

Recruiter Screening

30 min3 focus topicsculture fit

Technical Phone Screen

60 min5 focus topicstechnical

What to Expect

A technical interview focusing on systems thinking, infrastructure knowledge, and light coding/scripting ability. The interviewer will present a scenario or problem involving system reliability, monitoring, or infrastructure and assess how you approach it. Expect questions about AWS services, basic system design thinking, and troubleshooting strategies. This round filters for core technical competency before onsite.

Tips & Advice

Think out loud and explain your reasoning - the interviewer wants to understand your approach to problems, not just the final answer. If presented with a scenario (e.g., 'Your RDS instance has high CPU usage'), walk through a structured troubleshooting process: clarify scope, identify metrics to check, list potential causes, explain how to validate each, then recommend solutions. Mention monitoring and observability specifics (CloudWatch metrics, logs, distributed tracing). Have concrete AWS knowledge - understand the difference between various service types, their failure modes, and scaling limitations. If coding is involved, it's typically scripting in Python or bash for automation tasks; focus on readability and practical functionality over cleverness.

Focus Topics

Scripting and Automation Basics

Comfortable writing simple scripts in Python or Bash for infrastructure tasks, log analysis, or metric aggregation. Code should be clean, readable, and practical. Focus on solving real SRE problems (parsing logs, querying APIs, automating checks) rather than algorithmic complexity.

Practice Interview

Study Questions

Scenario-Based Problem Solving

When presented with infrastructure scenarios (e.g., 'EC2 instance not reachable', 'RDS high CPU', 'intermittent latency spikes'), demonstrate systematic troubleshooting. Show understanding of layers (application, OS, network, infrastructure). Ask clarifying questions. Prioritize high-impact areas first.

Practice Interview

Study Questions

Monitoring, Observability, and Alerting

Practical experience with monitoring tools (CloudWatch, Datadog, Prometheus, ELK stack). Understanding the three pillars of observability (metrics, logs, traces). Designing meaningful alerts that reduce false positives while catching real problems. Understanding SLOs, SLIs, and error budgets. Knowing what to monitor and why certain metrics matter for reliability.

Practice Interview

Study Questions

Systems Troubleshooting and Diagnostics

Structured approach to diagnosing failures: establishing a clear problem statement, identifying relevant metrics (latency, error rates, resource utilization), understanding system topology, forming hypotheses about root causes, and validating through evidence. Knowledge of common failure patterns (cascading failures, resource exhaustion, configuration errors, DNS issues).

Practice Interview

Study Questions

AWS Infrastructure and Services

Deep knowledge of core AWS services relevant to SRE work: EC2 (instance types, autoscaling, capacity), RDS (read replicas, failover, performance tuning), S3 (consistency models, regional replication), CloudWatch (metrics, alarms, logs), CloudTrail (audit logging), VPC (networking, security groups, NAT gateways). Understanding of AWS limits, failure scenarios, and cost implications.

Practice Interview

Study Questions

Onsite Round 1 - System Design for Reliability

75 min6 focus topicssystem design

What to Expect

A deep technical interview focused on designing large-scale, reliable systems. You'll be given a system design prompt (e.g., 'Design a global payment processing system', 'Design a real-time analytics pipeline', 'Design a message queue system') and asked to architect it with emphasis on reliability, availability, monitoring, and operational concerns. This differs from generic system design interviews by focusing on SRE-specific aspects: failure modes, monitoring strategy, deployment approach, incident response plan, and operational complexity.

Tips & Advice

Start by clarifying requirements and constraints (scale, geography, consistency requirements, acceptable downtime). Then explicitly discuss reliability aspects: How do components fail? What's your approach to redundancy? How do you handle cascading failures? Design your monitoring and alerting strategy - what metrics matter? How do you detect and respond to problems? Discuss deployment and rollback strategies. Explain operational runbooks for common scenarios. Use AWS services appropriately but don't default to using managed services for everything - show you understand trade-offs (using RDS vs. self-managed database, managed vs. unmanaged container orchestration, etc.). Draw diagrams and explain your reasoning for each architectural choice. For senior level, expect probing follow-up questions about trade-offs, and be prepared to defend your choices with technical reasoning. Mention graceful degradation and chaos engineering validation.

Focus Topics

AWS Service Selection and Trade-Offs

Appropriate use of AWS services for the problem (managed vs. self-managed trade-offs, when to use Lambda vs. EC2 vs. containers, RDS vs. DynamoDB, etc.). Understanding service limits, failure modes, and cost implications. Being able to articulate why you chose one service over another based on reliability and operational requirements.

Practice Interview

Study Questions

Deployment, Testing, and Incident Response Strategy

How the system gets deployed safely (blue-green, canary, rolling updates). Testing strategy to prevent reliability regressions (chaos engineering, load testing, integration testing). Incident response procedures: how to detect, page, diagnose, and resolve issues. Rollback strategy when things go wrong.

Practice Interview

Study Questions

Distributed Systems Fundamentals

Understanding core distributed systems concepts: eventual consistency, CAP theorem, consensus protocols, replication strategies, network partitions, time synchronization challenges. Knowing when these concepts apply to real systems (database replication, multi-region setup, service mesh, etc.) and being able to reason about their trade-offs.

Practice Interview

Study Questions

Operational Complexity and Manageability

Designing systems that are operationally tractable. Understanding that elegant design on a whiteboard may be operationally complex in reality. Discussing trade-offs between feature richness and operational simplicity. Considering capacity planning, resource allocation, configuration management. Explaining how your design can be monitored, debugged, and patched in production.

Practice Interview

Study Questions

Monitoring, Alerting, and Observability Architecture

Designing comprehensive observability for the system you're architecting. Defining key metrics and SLIs for reliability. Designing alert thresholds that catch problems while minimizing false positives. Planning for log aggregation, distributed tracing. Explaining how operations teams would actually use your monitoring to detect and debug issues.

Practice Interview

Study Questions

Reliability-Focused System Architecture

Designing systems that maintain acceptable availability and performance. Understanding redundancy (geographic, service-level), failover strategies, and handling of partial failures. Knowledge of reliability patterns: circuit breakers, bulkheads, retries with exponential backoff, timeouts, graceful degradation. Ability to articulate Mean Time Between Failures (MTBF), Mean Time To Recovery (MTTR), and how architecture impacts these metrics.

Practice Interview

Study Questions

Onsite Round 2 - Incident Response and Troubleshooting Deep Dive

70 min6 focus topicstechnical

What to Expect

A technical interview simulating real-world incident response. You'll be presented with a complex system problem or incident scenario and asked to troubleshoot systematically. The interviewer may present additional information as you progress (new error messages, new metrics, unexpected findings) to simulate real incident response where information emerges over time. The goal is to assess your ability to diagnose complex problems under pressure, ask the right questions, understand system behavior, and work toward resolution.

Tips & Advice

Treat this like a real incident: stay calm, think systematically. Start by understanding the scope: what's broken, who is affected, what's the business impact? Ask for key metrics (error rates, latency, CPU, memory, disk). Build a mental model of the system and potential failure points. Form hypotheses and explain how you'd validate each. Think about layers: application layer, service layer, infrastructure layer, network layer, storage layer. As new information is revealed, incorporate it and revise your understanding. Don't fixate on one hypothesis if the evidence doesn't support it. Explain your reasoning clearly - interviewers want to hear your thought process. Discuss both immediate mitigation (how to stop the bleeding) and root cause investigation (why did this happen). Mention relevant tools you'd use (CloudWatch, CloudTrail, application logs, SSH to instances, database query logs, etc.). For senior level, show understanding of trade-offs (fix quickly vs. fix correctly, rollback vs. forward fix, etc.).

Focus Topics

Communication During Incidents

How you'd communicate with stakeholders during troubleshooting: providing status updates, setting expectations, escalating when needed, collaborating with other teams. Explaining technical issues clearly to non-technical people. Documenting your work for post-incident review.

Practice Interview

Study Questions

Database Troubleshooting and Performance

Diagnosing common database issues: high CPU usage, slow queries, replication lag, connection exhaustion, deadlocks, memory pressure. Using RDS-specific tools (Performance Insights, Enhanced Monitoring). Understanding query optimization, indexing, and when to consider scaling (read replicas, sharding, etc.). Knowledge of different database types and their failure modes.

Practice Interview

Study Questions

Incident Mitigation vs. Root Cause Analysis

Understanding the difference between stopping an incident (mitigation - quick fix, workaround, rollback) and understanding why it happened (root cause). Being able to articulate trade-offs between speed of recovery and proper investigation. Knowing when to do quick fixes vs. when to spend time understanding root cause. Planning post-incident reviews.

Practice Interview

Study Questions

Application and Infrastructure Layer Diagnosis

Understanding how to diagnose issues at different layers: application-level errors (logs, exceptions, performance), OS-level issues (CPU, memory, disk, network), database issues (queries, locks, replication lag, connection pooling), container/orchestration issues (Kubernetes pod failures, resource limits, scheduling). Knowing what symptoms point to each layer.

Practice Interview

Study Questions

Deep AWS Diagnostics and Tools

Using AWS tools to diagnose problems: CloudWatch metrics and alarms, CloudTrail for API audit logs, VPC Flow Logs for network diagnosis, RDS Performance Insights, CloudWatch Logs Insights queries, X-Ray for distributed tracing. Understanding what each tool shows and when to use each. Being comfortable with AWS CLI for querying service state.

Practice Interview

Study Questions

Systematic Troubleshooting Methodology

Structured approach to diagnosing failures: establishing clear problem statement, identifying impacted components, collecting relevant data, forming and testing hypotheses, iterating based on evidence. Understanding different failure categories (application failures, infrastructure failures, dependency failures, misconfiguration) and tailoring diagnosis accordingly.

Practice Interview

Study Questions

Onsite Round 3 - Amazon Leadership Principles and Behavioral Interview

50 min6 focus topicsbehavioral

What to Expect

A behavioral interview assessing cultural fit and alignment with Amazon Leadership Principles. The interviewer will ask questions about your past experiences and use the STAR method (Situation, Task, Action, Result) to evaluate behaviors. Amazon specifically evaluates leadership principles regardless of level: Ownership, Bias for Action, Customer Obsession, Deliver Results, Learn and Be Curious, Earn Trust, Think Big, Have Backbone/Disagree and Commit, Invent and Simplify, Hire and Develop the Best, and Insist on Highest Standards. For SRE specifically, you may face questions about on-call experience, incident response team dynamics, how you handle pressure, and how you balance reliability with development velocity.

Tips & Advice

Prepare 5-7 concrete stories from your career that demonstrate Amazon Leadership Principles. Use STAR format but keep stories concise (2-3 minutes each). Choose stories that show: taking ownership of complex problems, acting decisively with incomplete information, learning from failures, earning trust in teams, simplifying complicated systems, maintaining high standards, and delivering results under pressure. For SRE-specific stories, emphasize incidents you've resolved, systems you've improved, automation you've built, and how you influenced reliability culture on your team. When asked about conflicts or disagreements, show how you focused on the best solution rather than being right. Discuss your philosophy on on-call responsibilities and how you stay engaged during rotations. Be honest about failures - Amazon values learning from mistakes. Connect your stories back to why you're excited about SRE at Amazon specifically. Avoid generic answers; use specific metrics and outcomes when possible.

Focus Topics

Handling Pressure and On-Call Rotations

Experience with on-call rotations, your philosophy on availability, how you stay engaged during night shifts or weekends, how you handle the stress of being responsible for production systems, and how you maintain work-life balance. Examples of challenging incidents you've handled.

Practice Interview

Study Questions

Insist on Highest Standards

Examples of when you've maintained high standards for system reliability, code quality, or operational practices even when it would be faster to cut corners. Stories about pushing back on risky deployments, advocating for better monitoring, or refusing to ship without proper automation. Amazon leadership principle: 'Insist on Highest Standards'.

Practice Interview

Study Questions

Earn Trust and Collaboration

Examples of how you've built trust with development teams, gained credibility for reliability recommendations, handled disagreements with teammates, or collaborated across functions. Stories about influencing others toward reliability best practices. Amazon leadership principle: 'Earn Trust'.

Practice Interview

Study Questions

Learn and Be Curious - Growth and Technical Excellence

Stories about technologies you learned to solve problems, systems you studied deeply to understand failures, courses you took, mentors you learned from, or how you've stayed current with infrastructure trends. Examples of how you've grown technically and how you apply new knowledge to improve systems. Amazon leadership principle: 'Learn and Be Curious'.

Practice Interview

Study Questions

Ownership in SRE - Taking Responsibility for Systems

Stories demonstrating how you've owned system reliability, taken initiative beyond your immediate job description, driven improvements without being asked, and seen projects through to completion. Examples: adopting a struggling service, improving its reliability metrics, and staying accountable for sustained improvement. Amazon leadership principle: 'Ownership'.

Practice Interview

Study Questions

Bias for Action - Making Decisions and Delivering Results

Examples of times you made quick decisions with incomplete information, took action to resolve issues, and delivered results under time pressure. Stories about incidents you resolved quickly, experiments you ran, features you shipped despite obstacles. Amazon leadership principle: 'Bias for Action', 'Deliver Results'.

Practice Interview

Study Questions

Onsite Round 4 - Operational and Infrastructure Automation Deep Dive

70 min6 focus topicstechnical

What to Expect

A technical interview focusing on infrastructure automation, tooling, deployment practices, and operational excellence. You may be asked to design an automation solution, discuss your approach to Infrastructure as Code, explain deployment strategies, or troubleshoot configuration management issues. This round evaluates your depth in operational tooling and automation practices that reduce toil and improve reliability. You might discuss Terraform, CloudFormation, Kubernetes, container image security, CI/CD pipelines, or monitoring automation.

Tips & Advice

Demonstrate hands-on experience with infrastructure automation tools. If discussing Infrastructure as Code, explain your philosophy (state management, testing, version control, disaster recovery). Show understanding of idempotency - why it matters and how you ensure it. Discuss deployment automation: how you minimize manual steps, enable frequent deployments, and safely roll back when needed. For container topics, discuss image security, supply chain security, scanning, and runtime security. Talk about your approach to configuration management at scale and handling configuration drift. Explain trade-offs between centralized vs. decentralized control, fully automated vs. semi-automated processes. Discuss how automation enables reliability (faster recovery, fewer human errors, better visibility). Share specific examples of automation you've built and measurable impact (MTTR improvement, error reduction, time saved). Show understanding of orchestration tools and container platforms.

Focus Topics

Automation Trade-Offs and Simplicity

Understanding when to automate vs. when to accept manual toil. Recognizing that complex automation itself becomes a liability. Prioritizing automation by impact (frequency and severity of the problem). Keeping automation maintainable and understandable. Knowing when simpler manual processes are better than over-engineered automation.

Practice Interview

Study Questions

Container Image Security and Supply Chain

Securing container images across their lifecycle: base image selection, vulnerability scanning, software composition analysis, signing and attestation, registry security, supply chain security considerations. Understanding Docker content trust, image scanning tools, secrets management in containers. Runtime security considerations.

Practice Interview

Study Questions

Monitoring and Observability Automation

Automating instrumentation and alerting setup, managing monitoring configuration as code, detecting anomalies automatically, automated runbook generation, self-healing automation. Tools and practices for dynamic environments (especially with container orchestration). Understanding infrastructure that scales elastically and how monitoring must adapt.

Practice Interview

Study Questions

CI/CD Pipelines and Deployment Automation

Designing and implementing deployment pipelines that automate building, testing, and deploying code safely. Understanding blue-green deployments, canary releases, rolling updates. Implementing gates and approvals. Integrating security scanning, integration testing, and chaos testing into pipelines. Tools like Jenkins, GitLab CI, AWS CodePipeline. Balancing automation with safety - how to prevent bad code from reaching production.

Practice Interview

Study Questions

Container Orchestration and Kubernetes

Deep hands-on experience with Docker and Kubernetes. Understanding Kubernetes architecture (control plane, kubelet, service mesh), deployments, stateful sets, DaemonSets, job scheduling. Networking in Kubernetes (service discovery, ingress). Storage considerations. Resource limits and requests. Scaling strategies. Managing updates and rolling deployments. Troubleshooting container and pod issues. If familiar with Amazon ECS as well, understanding the trade-offs between ECS and EKS.

Practice Interview

Study Questions

Infrastructure as Code and Configuration Management

Practical experience with Terraform, CloudFormation, or similar IaC tools. Understanding of state management, version control for infrastructure, testing infrastructure code, documenting infrastructure. Philosophy on infrastructure definition: declarative vs. imperative, idempotency, repeatability. Handling infrastructure changes and rollbacks. Managing secrets and sensitive data. Experience with configuration management tools (Ansible, Chef, Puppet).

Practice Interview

Study Questions

Frequently Asked Site Reliability Engineer (SRE) Interview Questions

Incident Communication and DocumentationMediumTechnical

70 practiced

In Python, implement a script that converts a structured incident timeline (JSON array of timestamped events) into a human-readable update suitable for posting on an internal incident channel. Assume input schema: [{ 'ts': '2025-01-01T12:00:00Z', 'author': 'alice', 'note': 'service A 503s' }]. Show core parsing/formatting logic and describe how you would handle noisy or duplicate events (e.g., aggregation by minute).

Sample Answer

Approach: parse timestamps into datetime, normalize to UTC, deduplicate exact duplicates, group events into 1-minute buckets to aggregate noisy repeats, and format each bucket into a human-friendly update line (time range, authors, summarized notes + counts). Below is a compact, practical Python script showing core parsing/formatting logic.

python

from datetime import datetime, timezone
from collections import defaultdict, Counter

def parse_iso(ts):
    # handle "Z" suffix
    if ts.endswith('Z'):
        ts = ts[:-1] + '+00:00'
    return datetime.fromisoformat(ts).astimezone(timezone.utc)

def summarize_timeline(events, bucket_minutes=1):
    # normalize and dedupe exact duplicates
    seen = set()
    normalized = []
    for e in events:
        key = (e.get('ts'), e.get('author'), e.get('note'))
        if key in seen:
            continue
        seen.add(key)
        dt = parse_iso(e['ts'])
        normalized.append({'dt': dt, 'author': e.get('author',''), 'note': e.get('note','')})
    # bucket by minute
    buckets = defaultdict(list)
    for e in normalized:
        bucket_key = e['dt'].replace(second=0, microsecond=0)
        # optionally round to nearest minute if bucket_minutes>1 (not shown)
        buckets[bucket_key].append(e)
    # format human-readable updates
    lines = []
    for bucket_time in sorted(buckets):
        items = buckets[bucket_time]
        authors = sorted({it['author'] for it in items if it['author']})
        notes = Counter(it['note'] for it in items if it['note'])
        note_parts = [f"{n} (x{c})" if c>1 else n for n,c in notes.items()]
        time_str = bucket_time.strftime('%Y-%m-%d %H:%M UTC')
        lines.append(f"{time_str} — authors: {', '.join(authors) or 'unknown'}; events: {'; '.join(note_parts)}")
    return "\n".join(lines)

# Example
events = [
  {'ts':'2025-01-01T12:00:00Z','author':'alice','note':'service A 503s'},
  {'ts':'2025-01-01T12:00:10Z','author':'alice','note':'service A 503s'},
  {'ts':'2025-01-01T12:01:05Z','author':'bob','note':'restarted pod'}
]
print(summarize_timeline(events))

Key points / reasoning:- Exact dedupe via tuple key removes repeated identical reports.- 1-minute buckets collapse noisy rapid-fire alerts into single update lines; bucket size can be tuned (e.g., 5m) or made adaptive.- Counter aggregation gives counts for repeated notes, preserving visibility into frequency.- You can extend fuzzy deduplication (normalize note text, regex strip IDs) or cluster similar messages (Levenshtein / embedding) for smarter noise reduction.

Complexity: O(n log n) due to sorting buckets; O(n) space. Edge cases: missing/invalid timestamps (validate), different timezones (normalize to UTC), very large timelines (stream/batch processing).

Observability Fundamentals and AlertingEasyTechnical

94 practiced

What is OpenTelemetry and what are its main components (SDKs, Collector, instrumentations)? Describe how you would enable auto-instrumentation for a Python web service and what configuration choices you must consider for exporting metrics, logs, and traces to a backend.

Sample Answer

OpenTelemetry is an open standard and set of tools for collecting traces, metrics, and logs from applications and services so you can send them to observability backends. For an SRE it's useful because it standardizes telemetry, reduces vendor lock-in, and lets you centralize processing/forwarding.

Main components- SDKs: Language-specific libraries (opentelemetry-api + opentelemetry-sdk) that generate telemetry, provide context propagation, sampling, and batching.- Instrumentations: Libraries (automatic or manual) that hook into frameworks, DB drivers, HTTP clients, etc., to produce traces/metrics with minimal code changes.- Collector: A standalone agent/service (receivers → processors → exporters) that can receive OTLP (or other) telemetry, enrich/filter/aggregate it, and forward to one or more backends.

Enabling auto-instrumentation for a Python web service1. Install packages:

bash

pip install opentelemetry-api opentelemetry-sdk \
     opentelemetry-instrumentation opentelemetry-exporter-otlp

2. Run your app with the auto-instrument runner:

bash

opentelemetry-instrument --service-name myservice python app.py

This auto-enables supported instrumentations (Flask/Django/requests/DBs). For explicit control, install specific instrumentation packages (e.g., opentelemetry-instrumentation-flask) or initialize instrumentation in code.3. Optionally provide a small bootstrap config (environment variables or SDK code) for resources, sampling, and exporter settings.

Configuration choices for exporting metrics, logs, traces- Transport & endpoint: OTLP over gRPC/HTTP — set OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_HEADERS. Decide whether agents send directly to backend or via a Collector.- Use a Collector when you need local buffering, protocol translations, batching, sampling, enrichment, or to forward to multiple backends.- Sampling: configure sampler (trace percentage or parent-based) to limit high-cardinality trace volume.- Batching and retries: tune batch size, timeout and retry/backoff to avoid blocking app threads and to handle backend outages.- Security & auth: TLS, mTLS, API keys or headers — configure in exporter/collector.- Metrics specifics: choose aggregation temporality (cumulative vs delta), scrape vs push model, and histogram buckets if supported.- Resource attributes & semantic conventions: set service.name, environment, version for correct filtering/alerting.- Correlation: ensure trace IDs are attached to logs (configure log instrumentation or use Collector to correlate).- Cardinality control & attribute filtering: drop high-cardinality attributes before export (in SDK or Collector) to control cost.

In practice: run opentelemetry-instrument locally pointing exporters at your Collector (OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317), tune sampling and batching in Collector config, and enable attribute filtering in the Collector to protect backend cost and performance.

Learning Agility and Growth MindsetHardTechnical

59 practiced

You're evaluating adoption of a new distributed tracing standard (e.g., OpenTelemetry). Outline a plan to learn the technology, pilot instrumenting a small set of critical services, train engineers, and measure whether tracing reduced time-to-detection or MTTR. Include risks and sampling strategies.

Sample Answer

Requirements & goals:- Reduce time-to-detection (TTD) and mean time to repair (MTTR) by instrumenting services with OpenTelemetry (OTel).- Pilot on 3–5 critical services that span front-end, backend, and data plane.- Minimal runtime overhead, secure trace data, measurable ROI.

Learning plan (2 weeks):- Week 1: Hands-on labs — install OTel Collector, export to chosen backend (Jaeger/Tempo/Datadog). Follow official OTel SDKs for our stack (Java, Go, Python). Build simple end-to-end trace in a sandbox.- Week 2: Deep dive on context propagation, sampling, baggage, resource attributes, and metrics/logs correlation. Create runbooks for instrumentation patterns and performance/cost constraints.

Pilot instrumentation (4–6 weeks):- Select services: API gateway, core business service, async worker.- Step 1: Add OTel SDK with automatic instrumentation where available (HTTP, DB clients) and manual spans for key business ops.- Step 2: Deploy OTel Collector as a sidecar/DaemonSet; enable batching and retry.- Step 3: Export to staging tracing backend; validate traces end-to-end and latency/overhead.- Step 4: Promote to prod with feature-flagged rollout per service.

Sampling & cost control:- Start with tail-based sampling for production to keep error/slow traces (captures traces with high latency or errors). Complement with adaptive head-based sampling (e.g., 1% default).- Use rate limits and dynamic sampling rules by service and environment.- Keep high sampling for critical services, lower for noisy low-value services.

Training & enablement (ongoing):- Two 2-hour workshops: instrumentation patterns + debugging with traces.- Create templates, SDK examples, lint/CI checks for missing spans.- Office hours and an internal playbook (how to read traces, common antipatterns).

Measuring impact:- Baseline TTD and MTTR for 3 months pre-pilot using incident timelines.- After pilot, measure TTD and MTTR per incident class weekly for 3 months. Track: - Median/95th TTD - Median/95th MTTR - Number of incidents resolved without paging escalation - Time to root cause identification- Use A/B by service if possible (instrumented vs non-instrumented).

Risks & mitigations:- Performance overhead — measure added latency and CPU; tune sampling and batching.- Data leakage — redact PII at instrumentation and use collector processors.- Storage/cost growth — enforce retention/sampling, archive traces.- Developer friction — provide libraries, examples, and CI checks to minimize manual work.

Decision criteria to expand:- Statistically significant reduction in median TTD/MTTR (e.g., >20%)- Low operational cost/overhead and positive developer feedback- Ability to automate alerts from traces (slow spans → SLO alerting)

This plan balances fast learning, controlled rollout, measurable outcomes, and operational safety.

Infrastructure as Code and Configuration ManagementMediumTechnical

26 practiced

Provide a Helm values.yaml snippet to set CPU/memory requests and limits, enable liveness and readiness probes for a web service, and configure a PodDisruptionBudget. Explain how Helm supports environment overrides and how to design chart values for multiple environments.

Sample Answer

Approach: provide a clear values.yaml snippet covering resources, liveness/readiness probes and a PodDisruptionBudget (PDB), then explain how Helm supports environment overrides and best practices for designing values for multiple environments.

values.yaml:

yaml

replicaCount: 3

resources:
  requests:
    cpu: "200m"
    memory: "256Mi"
  limits:
    cpu: "500m"
    memory: "512Mi"

livenessProbe:
  enabled: true
  httpGet:
    path: /healthz
    port: http
  initialDelaySeconds: 30
  periodSeconds: 10
  timeoutSeconds: 3
  failureThreshold: 3

readinessProbe:
  enabled: true
  httpGet:
    path: /ready
    port: http
  initialDelaySeconds: 5
  periodSeconds: 5
  timeoutSeconds: 2
  failureThreshold: 3

podDisruptionBudget:
  enabled: true
  minAvailable: 2
  maxUnavailable: null
  labels: {}

Key points / reasoning:- Resources set requests to guarantee scheduling and limits to avoid noisy-neighbor issues.- Probes tuned: readiness fast to stop traffic quickly, liveness with longer initialDelay to avoid false kills during startup.- PDB uses minAvailable to ensure at least 2 pods remain during voluntary disruptions (use minAvailable OR maxUnavailable, not both).

Helm environment overrides:- Use separate values files per environment (values-dev.yaml, values-staging.yaml, values-prod.yaml) and pass with --values (-f): helm upgrade --install mysvc ./chart -f values.yaml -f values-prod.yaml. Later files override earlier keys.- Use --set for small overrides (e.g., --set replicaCount=5) but avoid for many keys.

Designing chart values for multiple environments:- Keep chart defaults generic and safe (reasonable requests/limits).- Make environment-specific differences explicit in per-env values files (replicas, resources, probe timings, PDB policy).- Structure nested values (resources, probes, pdb) so CI/CD can patch only the keys needed.- Parameterize SLO-related values (e.g., target latency thresholds) separately to align SRE runbooks.- Validate with automated helm lint and test hooks in CI to ensure env overrides produce expected manifests before deploy.

On Call and Production ReadinessMediumTechnical

79 practiced

Implement an idempotent exponential-backoff retry decorator in Python that supports jitter, max_retries, base_backoff_seconds, max_backoff_seconds, and max_elapsed_seconds. Provide the function signature, key implementation sketch, and explain how to make tests deterministic.

Sample Answer

To implement an idempotent exponential-backoff retry decorator, make it configurable and testable by injecting time, sleep, and jitter sources. Signature:

def retry( max_retries: int = 5, base_backoff_seconds: float = 0.1, max_backoff_seconds: float = 10.0, max_elapsed_seconds: float = 60.0, jitter: Callable[[float], float] | None = None, retry_on: Callable[[Exception], bool] | None = None, clock: Callable[[], float] | None = None, sleeper: Callable[[float], None] | None = None,):

Implementation sketch (deterministic-friendly):

python

import time, random, functools

def retry(...):  # params as above
    jitter = jitter or (lambda x: random.uniform(0, x))
    clock = clock or time.monotonic
    sleeper = sleeper or time.sleep
    retry_on = retry_on or (lambda e: True)

    def decorator(fn):
        @functools.wraps(fn)
        def wrapped(*args, **kwargs):
            start = clock()
            attempt = 0
            while True:
                try:
                    return fn(*args, **kwargs)
                except Exception as e:
                    attempt += 1
                    elapsed = clock() - start
                    if not retry_on(e) or attempt > max_retries or elapsed >= max_elapsed_seconds:
                        raise
                    backoff = min(max_backoff_seconds, base_backoff_seconds * (2 ** (attempt - 1)))
                    wait = backoff + jitter(backoff)
                    # cap to remaining allowed time
                    wait = min(wait, max(0, max_elapsed_seconds - elapsed))
                    if wait > 0:
                        sleeper(wait)
        return wrapped
    return decorator

Key points:- Use time.monotonic for monotonic elapsed measurement.- Jitter function receives computed backoff and returns additional/randomized offset; default uses random.uniform(0, backoff).- Idempotency: decorator assumes the wrapped operation is safe to retry; caller must ensure idempotency (or use idempotency keys).

Making tests deterministic:- Inject deterministic jitter (e.g., lambda x: 0 or lambda x: x*0.5), a fake clock (advance on each call) and a no-op or recorder sleeper to avoid real sleeps.- Example test setup: clock = iter([0.0, 0.1, 0.2, ...]).__next__ or a small FakeClock with advance(seconds).- Assert number of attempts, accumulated wait times, and that expected exception or success occurs.

Edge cases:- Respect max_elapsed_seconds to avoid indefinite retries- Ensure attempt counting aligns with desired semantics (attempt 1 = first try)- Avoid integer overflow in backoff calculation by capping at max_backoff_seconds.

Monitoring Tools and ObservabilityEasyTechnical

96 practiced

You have a set of inconsistent metric names: app.requestTime, requests_total, http:errors, db_conn_count. As SRE, propose a metric naming convention and show how you would rename or map these four metrics to follow your convention. Explain label usage for dimensions like service, method, and region.

Observability and Monitoring ArchitectureMediumSystem Design

30 practiced

Design an aggregation and partitioning strategy for a horizontally scalable TSDB that supports long-term retention and efficient single-metric queries. Discuss sharding keys (for example metric name vs specific label sets), replication, read/write path, and how to ensure even distribution and hot-shard mitigation.

Sample Answer

Requirements:- Efficient single-metric queries (e.g., metric + label match) at low latency- Long-term retention (cold/hot tiers) and horizontal scale- Even distribution, high availability, and mitigation of hot shards during bursts

High-level architecture:- Ingest tier (load balancers + ingestion proxies) → Sharding layer (consistent-hash + shard manager) → Storage nodes (hot TSDB + cold object store) → Query API + cache.

Sharding key strategy:- Use a two-tier key: 1) Primary shard key = hash(metric_name) to co-locate same metric family (keeps single-metric queries cheap). 2) Secondary partitioning inside shard = hash(sorted_label_pairs) to split heavy cardinality within that metric across stripe buckets.- Rationale: metric_name groups common queries; label-set hashing gives even distribution when a metric has high cardinality.

Replication:- Cross-availability-zone replication factor (RF=3). Use leader-follower per shard for writes; followers accept reads for replicas with slightly stale data (configurable consistency). Use quorum writes for durability.

Read/write path:- Write: Client -> ingestion proxy -> shard lookup (consistent-hash ring) -> leader node writes to WAL and in-memory TSDB segment, replicates to followers asynchronously with write-ahead ACK from majority.- Read (single-metric): Query router routes by metric_name to the set of nodes owning that metric. If label-filter covers specific label-keys, router targets only relevant stripe partitions; query fan-in merges time-series lists. Use per-shard LRU/bitmap cache and index bloom filters to avoid touching irrelevant partitions.

Even distribution & hot-shard mitigation:- Virtual nodes (vnodes) per physical node to smooth distribution.- Adaptive re-sharding: monitor per-shard QPS, series count, and size. When hot, split the metric’s secondary partitions across additional vnodes or migrate some label-hash ranges to other nodes.- Rate-limiting and ingest-tier backpressure per-metric to protect cluster.- Query-side sharding awareness + request hedging to reduce tail latency.- Hot metrics can be multi-written to temporary read-replicas (read-only fanout) and later compacted.

Tiering & retention:- Hot data retained on time-series DB for recent windows; compaction/aggregates (downsampling) produced during rollups into cold object store (Parquet/ORC). Query planner routes long-range queries to compacted aggregates.

Trade-offs:- Hashing by metric_name favors single-metric queries but risks hotspots for very popular metrics — mitigated by secondary partitioning and adaptive splitting.- Quorum writes incur latency but give durability; async replication faster but requires careful failover.

Operational notes:- Metrics: per-shard CPU, IO, memory, cardinality; alert on imbalance.- Automation: shard rebalancer with safety throttles, graceful reassignments, and canary moves.- SLOs: define consistency/latency tiers and strategies (strong read vs eventual) per tenant/metric.

Container Orchestration and Kubernetes OperationsHardSystem Design

74 practiced

Discuss architectural limits and challenges of very large Kubernetes deployments (10k nodes, 100k pods). Cover etcd size and latency implications, API server throughput, scheduler scaling, controller managers, object counts and namespace strategy, and propose strategies such as sharding, multiple clusters, or virtual clusters to handle scale.

Sample Answer

Requirements & constraints:- Target scale: ~10k nodes, ~100k pods; high availability (RPO/RTO), acceptable control-plane latency (<500ms for common reads), and strong operational simplicity.

etcd: size & latency- etcd is the slowest centralized dependency. Large object counts (ConfigMaps, Secrets, Endpoints) increase DB size and compaction/backup costs; leader elections and snapshotting become heavy.- Keep etcd working set small: avoid storing large blobs (Secrets/configs) per pod. Monitor etcd db size, watch queue length, compaction lag, and disk I/O.- Aim for <8GB working set per cluster control-plane member; beyond that, read/write latencies rise nonlinearly. Use SSDs, tuned keepalive/heartbeat, and separate etcd disks.

API Server throughput- API server CPU/network bound at high QPS (thousands/s). High watch counts amplify cost.- Use API server horizontal replicas behind a load-balancer; enable aggregated API servers for CRDs; enable watch-cache and resourceVersion pruning.- Rate-limit noisy clients and use bursty client detection. Prefer PATCHs over PUTs for churn.

Scheduler scaling- Default scheduler becomes CPU-bound with many nodes/pods. Scheduling complexity O(nodes * pods).- Options: multiple schedulers partitioned by label/namespace, scheduler sharding (node/namespace partition), or custom fast-path schedulers for simple workloads.- Use scheduling profiling, prefiltering (node affinity), and cache tuning.

Controller managers- Controllers interact heavily with API and etcd. Controller contention and leader-election bottlenecks occur.- Split controllers: run multiple controller-manager instances scoped by namespace or workload type; use leader-election namespaces to avoid hot-spotting.- Reduce reconciliation frequency for non-critical resources.

Object counts & namespace strategy- Avoid global namespace with millions of objects. Use many namespaces to partition object lists and reduce list/watch fanout.- Limit per-namespace object churn; colocate related workloads to reduce cross-namespace interactions.- Consolidate Endpoints by using EndpointSlices (recommended).

Strategies to handle scale1. Sharding within cluster: - Node/namespace-based scheduler/controller shards. Reduces per-instance load but increases operational complexity and cross-shard networking concerns.2. Multiple clusters (recommended for SRE): - Split by environment, team, or tenancy. Smaller clusters keep etcd and control plane healthy and simplify upgrades. Use cross-cluster service mesh or federation for global services. - Trade-off: more clusters to operate, network/service discovery complexity.3. Virtual clusters (vcluster, namespace virtualization): - Provide many virtual control planes backed by fewer physical clusters. Good for multi-tenancy and isolation; watch overhead can still be high but manageable.4. Hybrid: - Global control plane for infra-critical functions, many tenant clusters for workloads.

Operational & SRE practices- Strong observability: etcd metrics, apiserver latencies, scheduler QPS, watch counts, leader-election conflicts.- Quotas & admission control to stop noisy tenants.- Automation for cluster lifecycle, failover, and disaster recovery (etcd backups, restore tests).- Performance testing at scale: chaos engineering, simulated watch/list churn, and upgrade rehearsals.

Trade-offs summary:- Sharding reduces centralized bottlenecks but complicates networking and global policies.- Multiple clusters reduce blast radius and keep control planes small but increase operational overhead.- Virtual clusters give tenant isolation with fewer physical clusters but add abstraction complexity and potential performance overhead.

Recommendation:- For 10k nodes/100k pods, prefer multiple clusters (each sized to healthy etcd working set) combined with virtual clusters for tenancy and sharded controllers/schedulers for cross-cluster heavy workloads. Enforce quotas, reduce watch/list churn, and invest in automation and observability.

Incident Communication and DocumentationMediumTechnical

55 practiced

You are about to hand off an ongoing incident to the next on-call shift that spans 8 hours. Draft the handoff notes that include current status, outstanding tasks, open hypotheses, contact points, exact commands to check status (with expected outputs), any rate-limited actions, and explicit escalation criteria so the incoming responder can resume quickly.

Sample Answer

Incident Handoff — 2025-11-22 14:00 UTC — Handed off to next 8h shift

Summary / Current status- Incident: Payment API 502 errors from service-pay (customers see checkout failures ~10%).- Status: Degraded service, mitigation in progress. Traffic routed via degraded instance pool (~90% capacity). Error rate down from 18% to 9% after circuit-breaker rollouts; latency still ~600ms P95 (target <300ms).- Severity: Sev2. No customer data loss.

Outstanding tasks (priority order)1. Verify canary rollout of pay-v2 (nodes 3/5) completes without error -> if healthy, increase weight 20% every 30m.2. Investigate root cause of DB connection spikes (see hypotheses).3. Restore circuit-breaker config to default only after error rate <2% and P95 <350ms for 60m.4. Prepare rollback plan for pay-v2 (image: registry/prod/pay:v20251120) if errors increase >3% in any 30m window.

Open hypotheses- H1: New pay-v2 connection pool config too small causing retries -> leads to upstream 502s.- H2: DB replica lag causing timeouts during peak writes.- H3: Third-party gateway sustaining intermittent failures (10:45–13:00 UTC correlation).

Contact points- On-call SRE (current): alice@company / +1-555-0101 (Slack #oncall)- SRE Lead: bob@company / +1-555-0102 (PagerDuty)- Service owner (payments): carla@company / Slack @carla- DB SME: dba-team@company

Commands to check status (exact + expected output)- Service health (k8s):

bash

kubectl -n payments get pods -l app=service-pay -o wide

Expected: pods: pay-v2-* READY 2/2 STATUS Running; ready count should be >=3.

- Pod logs (last 200 lines):

bash

kubectl -n payments logs deploy/service-pay --tail=200

Expected: look for repeated "connection refused" or "upstream error 502".

- Deployment rollout status:

bash

kubectl -n payments rollout status deployment/service-pay

Expected: "deployment \"service-pay\" successfully rolled out" or shows failing pods.

- Metrics (Prometheus):

bash

curl -s 'http://prometheus/api/v1/query?query=rate(http_requests_total{job="service-pay",code="502"}[5m])'

Expected: numeric rate trending down; current expected <0.02 (per-second) to consider healthy.

- DB replica lag:

bash

psql -h db-replica -c "SELECT pg_last_xact_replay_timestamp(), now(), extract(epoch from now() - pg_last_xact_replay_timestamp()) as lag_seconds;"

Expected: lag_seconds < 5

- Upstream gateway status:

bash

curl -i https://gateway.partner/status

Expected: 200 OK with JSON status "ok".

Rate-limited / destructive actions (explicit)- Restart policy: only restart pods if they are CrashLoopBackOff or not Ready for >10m. Limit restarts to 2 per pod per hour to avoid cascading failures.- DB failover: NOT allowed without DB SME approval (high risk).- Increase canary weight: do at most +20% every 30 minutes. Document each step in incident channel.- Feature-flag toggles: only by service owner (Carla) or SRE Lead.

Escalation criteria (what to do, when)- If 502 error rate rises above 15% for 15 consecutive minutes -> page SRE Lead (PagerDuty) and Service Owner.- If P95 latency >1000ms for 10m OR payment throughput drops >40% vs baseline -> page PagerDuty immediately.- If DB replica lag >30s -> page DB SME immediately and stop canary rollout.- If rollout fails (any pod crashloop more than 3 times during rollout) -> execute rollback to registry/prod/pay:v20251119 and notify Service Owner (rollback command below).

Rollback command (if needed)

bash

kubectl -n payments set image deployment/service-pay service-pay=registry/prod/pay:v20251119
kubectl -n payments rollout status deployment/service-pay

Notes/Logs- Full logs and timeline: /var/log/incidents/INC-20251122-payments.log- Incident channel: #inc-payments (link pinned to runbook)

Runbook references- Payments runbook: Confluence > SRE > Payments Runbook (link)- DB failover SOP: Confluence > DB > Failover (link)

Handoff acceptance- Please confirm in #inc-payments when you've read this and run the health checks. If anything deviates from "Expected" outputs above, escalate per criteria.

Learning Agility and Growth MindsetMediumTechnical

50 practiced

Design a lightweight (1–2 day) internal certification for core SRE practices for your team. Specify the learning objectives, hands-on exercises, assessment methods, and a follow-up practice to ensure retention of skills after certification.

Sample Answer

Overview: A compact 1–2 day internal certification that ensures every SRE on the team can reliably operate, observe, and troubleshoot our services. Day 1 = fundamentals + hands-on labs. Day 2 (optional) = assessments + simulation.

Learning objectives:- Define SLOs, SLIs, and error budgets for a service and map them to alerts.- Implement and tune monitoring & alerting (metrics, dashboards, runbooks).- Execute incident response: triage, mitigation, escalation, and postmortem.- Automate a simple operational task (CI/CD rollback or auto-heal script).- Perform root-cause analysis using logs/traces/metrics.

Hands-on exercises:1. SLO workshop (45m): Given service data, set SLI/SLO, compute error budget, propose alert thresholds.2. Observability lab (90m): Use our Prometheus/Grafana stack to write a dashboard, create two alert rules, and attach runbook steps.3. Incident sim (2h): Time-boxed fire-drill on a staging cluster: identify the noisy signal, apply mitigation (scale/feature-flag/rollback), document timeline.4. Automation mini-task (60m): Implement a small script/playbook to auto-restart a failed pod and add to repo with tests.

Assessment methods:- Practical rubric-based evaluation for each lab (correctness, reproducibility, runbook quality, automated test presence).- Short written quiz (20 minutes) covering SRE theory and tool-specific commands.- Live oral debrief (10 minutes) where candidate explains one incident decision and trade-offs.Pass criteria: ≥80% on combined rubric + quiz.

Follow-up practice to ensure retention:- 6-week rotating “on-call shadow” where certified engineers lead a real minor incident post with a mentor reviewing runbook adherence.- Monthly 30-minute lightning lab: one micro-scenario (alert tuning, postmortem review) with checklist.- Maintain a public “certified playbooks” repo; require each certified SRE to contribute one improvement every quarter.

This design balances theory, realistic practice, measurable assessment, and built-in reinforcement to keep skills fresh.

Practice Site Reliability Engineer (SRE) questions across all topics

Additional Information

Want to create your own tailored preparation guide using our deep research?

Get Started for Free

Interview-Ready Courses

Visual-first, interactive, structured learning paths

Browse Site Reliability Engineer (SRE) jobs

AI-enriched listings across hundreds of company career pages

Explore Jobs

Senior Site Reliability Engineer Interview Preparation Guide - Amazon

Interview Process Overview

Interview Rounds

Recruiter Screening

What to Expect

Tips & Advice

Focus Topics

Communication and Team Collaboration

Practice Interview

Study Questions

Motivation for SRE and Amazon

Practice Interview

Study Questions

Professional Background and SRE Journey

Practice Interview

Study Questions

Technical Phone Screen

What to Expect

Tips & Advice

Focus Topics

Scripting and Automation Basics

Practice Interview

Study Questions

Scenario-Based Problem Solving

Practice Interview

Study Questions

Monitoring, Observability, and Alerting

Practice Interview

Study Questions

Systems Troubleshooting and Diagnostics

Practice Interview

Study Questions

AWS Infrastructure and Services

Practice Interview

Study Questions

Onsite Round 1 - System Design for Reliability

What to Expect

Tips & Advice

Focus Topics

AWS Service Selection and Trade-Offs

Practice Interview

Study Questions

Deployment, Testing, and Incident Response Strategy

Practice Interview

Study Questions

Distributed Systems Fundamentals

Practice Interview

Study Questions

Operational Complexity and Manageability

Practice Interview

Study Questions

Monitoring, Alerting, and Observability Architecture

Practice Interview

Study Questions

Reliability-Focused System Architecture

Practice Interview

Study Questions

Onsite Round 2 - Incident Response and Troubleshooting Deep Dive

What to Expect

Tips & Advice

Focus Topics

Communication During Incidents

Practice Interview

Study Questions

Database Troubleshooting and Performance

Practice Interview

Study Questions

Incident Mitigation vs. Root Cause Analysis

Practice Interview

Study Questions

Application and Infrastructure Layer Diagnosis

Practice Interview

Study Questions

Deep AWS Diagnostics and Tools

Practice Interview

Study Questions

Systematic Troubleshooting Methodology

Practice Interview

Study Questions

Onsite Round 3 - Amazon Leadership Principles and Behavioral Interview