Software Development Engineer in Test Role Questions

Assess understanding of the SDET role as a hybrid software engineering and testing discipline. Topics include building automated testing frameworks, writing automated tests and test infrastructure, integrating testing into continuous integration and continuous delivery pipelines, developing tooling to improve testability, performance and reliability testing, and distinguishing SDET responsibilities from manual QA or pure software engineering roles. Candidates should explain how SDETs contribute to quality, test strategy, and CI pipeline reliability.

HardTechnical

67 practiced

Problem solving: Design a test prioritization algorithm that, given a set of changed files in a pull request and a historical mapping from tests to touched files or components, selects the top N tests to run first to maximize fault detection within limited test runners. Describe data structures, scoring function, cold-start handling (new tests or files), and how you would evaluate effectiveness.

Sample Answer

**Approach (high level)** I’d build a data-driven ranking that scores tests by relevance to changed files and historical fault-detection signal, then pick top N to run first. The system runs in CI before full test suite.

**Data structures** - Tests table: {test_id, last_run_ts, avg_duration, failure_rate, touched_files_set} - File→tests inverted index: Map<file_path, Set<test_id>> for O(1) lookup of candidate tests - Component graph: Map<component, Set<file_path>> to handle coarse mapping when file-level data sparse - Lightweight LRU cache for recent PR mappings

**Scoring function** Score(test, PR) = w1 * file_overlap_score + w2 * recency_score + w3 * failure_score - w4 * duration_penalty - file_overlap_score = | touched_files_set ∩ changed_files | / |changed_files| - recency_score = exp(-Δt / T) where Δt is time since test last covered those files (favors recent evidence) - failure_score = normalized historical fault-finding precision (true positives / runs) - duration_penalty = avg_duration / expected_runner_budget Weights tuned via offline experiments (start w1 high).

**Cold-start handling** - New tests: bootstrap from file-level component heuristics and static analysis (call graph, imports) to assign provisional touched_files_set; give moderate default failure_score and conservative duration estimate. Use exploration-exploitation: epsilon-greedy — occasionally schedule unobserved tests to collect signal. - New files: map to component via path heuristics; if unmapped, select a diverse small set of tests that maximize code coverage heuristic.

**Algorithm (pseudo)**

python

candidates = union(file_index[f] for f in changed_files)
for t in candidates:
    score[t] = compute_score(t, changed_files)
ranked = sort_descending(score)
select top N such that sum(duration) <= runner_budget (or top N by count)

**Evaluation & metrics** - Primary: defect-detection rate among top-K (time-to-detect), mean time to failure detection, and recall@K (percent of failing tests included). - Secondary: CPU-hours saved, slowdown of PR merges, false-negative rate (missed failures that run later). - Method: A/B testing in CI (control: random/heuristic baseline). Use historical replay: simulate PRs with held-out failures to measure recall@K and expected detection time. Tune weights with Bayesian optimization.

**Complexity & operational notes** - Lookup: O(number_changed_files + |candidates| log |candidates|). In practice candidates << total tests. - Maintain indexes incrementally during test runs. Monitor drift: retrain failure_score monthly.

This balances precision (historical signal) with robustness (cold-start exploration) and operational constraints (runner budget).

EasyTechnical

85 practiced

Describe the sequence of steps and components required to integrate automated tests into a CI pipeline (for example GitHub Actions or Jenkins). Cover build, test execution (unit, integration, E2E), artifact storage, test result reporting, notifications, and what SDETs should monitor in the pipeline.

EasyTechnical

93 practiced

A team adds a new REST endpoint for processing payments. As the SDET, list which automated tests you would write at each level: unit, integration, contract, and end-to-end. For each level include example test cases, test data considerations, flakiness risks, and where they should run in CI.

Sample Answer

**Overview (SDET perspective)** I’d structure automated tests across unit, integration, contract, and end-to-end levels to catch defects early and keep CI fast.

**1) Unit tests** - Purpose: validate business logic in payment processing classes (validation, fee calc, retry backoff). - Example cases: valid card number formats, invalid CVV, amount boundary (0.01, max), rounding of fees, timeout handler. - Test data: mocked card DTOs, edge amounts, negative/zero values. - Flakiness risks: none if fully mocked; avoid real-time waits. - CI stage: run in fast unit test job on every commit (pre-merge).

**2) Integration tests** - Purpose: component interactions — controller -> service -> persistence (DB) with a sandboxed DB. - Example cases: successful charge writes transaction record; idempotent replays; DB rollback on failure. - Test data: test DB fixtures, seeded merchant accounts, sandbox API keys. - Flakiness risks: DB state leakage; network latency. Use test containers and deterministic seeds. - CI stage: run on PR and nightly; parallelizable but slower than unit.

**3) Contract tests** - Purpose: ensure API contract between payment service and clients (or upstream gateway) stays stable. - Example cases: request/response schemas, required headers (idempotency-key), status codes for common flows (200, 400, 402, 500). - Test data: example payloads, OpenAPI/JSON Schema definitions, mock gateway responses. - Flakiness risks: mismatched mock behavior; version drift. Automate schema validation and publish pacts. - CI stage: run in PR and as part of provider/consumer pipelines; gated before deployments.

**4) End-to-end (E2E) tests** - Purpose: full workflow: client -> service -> payment gateway sandbox -> bank simulator -> reconciliation. - Example cases: successful payment, declined card, network failure retry, refund flow, large volume batch. - Test data: sandbox gateway accounts, test card numbers (3DS if applicable), realistic payloads, time windows for reconciliation. - Flakiness risks: external sandbox instability, timing issues, rate limits. Mitigation: isolate to nightly runs, use retries, backoff, and observability. - CI stage: run in nightly or pre-production pipeline, not on every commit.

Notes on observability and maintenance: log test artifacts, use test tags (smoke/regression), and fail fast for unit/integration while keeping E2E and flaky external-dependent tests isolated and retried with limits.

HardSystem Design

88 practiced

System design: Describe how you would integrate chaos engineering and continuous chaos testing into a CI/CD workflow for microservices, including safe experiment rollouts in staging and production, steady-state hypotheses, guardrails, automated rollback, and ensuring test harnesses and observability capture degradations without causing customer impact.

Sample Answer

**Clarify goals & constraints**- Goal: continuously validate resilience of microservices without customer impact; integrate chaos into CI/CD with safe rollouts, automation, and observability.- Constraints: low blast radius, compliance, automated rollback, test repeatability.

**High-level approach**- Add a “chaos stage” to CI/CD that runs in parallel in staging and progressively in production via controlled canaries.- Automate experiments as first-class test artifacts (YAML + test code) stored in repo and run by Chaos Orchestration service (e.g., Litmus, Chaos Mesh, Gremlin).

**Staging flow**- On merge to main: run unit/integration → deploy to ephemeral stage cluster → run functional tests → run continuous chaos suite.- Chaos suite uses test harnesses (fixture that injects failures) and asserts steady-state hypotheses before/after experiments.

**Steady-state hypotheses**- Define measurable SLO-backed hypotheses per service (e.g., error rate < 0.5%, p95 latency < 300ms, throughput within 10%).- Express as automated assertions using observability queries (PromQL/OTel). Run baseline measurement window then run experiment and re-evaluate.

**Safe production rollout**- Canary phases: Canary-0 (internal staging), Canary-1 (1% traffic, maintenance window), Canary-2 (5% traffic), then wider.- Use traffic steering (service mesh like Istio) and feature flags to target subsets.- Each phase must pass hypothesis checks for a hold period before advancing.

**Guardrails & automated rollback**- Implement hard guardrails: circuit-breaker that halts experiment if SLO breach observed, automated rollback of experiment or service deployment.- Orchestrator subscribes to alerting/metrics. On breach: stop injection, revert routing, trigger pipeline job to rollback deployment, create incident ticket.

**Test harness & observability**- Harness provides deterministic fault injection APIs, synthetic load generators, and log correlation IDs.- Instrument services with distributed tracing, metrics, and custom chaos tags.- Use dashboards and anomaly detectors (baseline + ML) to spot silent degradations.- Capture snapshots (traces, flamegraphs, heap) on failures for postmortem.

**Automation & governance**- Experiments as code reviewed via PRs with metadata (blast radius, owner, run schedule).- RBAC so only authorized experiments run in prod windows.- Schedule recurring smoke chaos tests in off-peak times and continuous low-risk experiments otherwise.

**Example tools / stack**- CI: GitHub Actions/Jenkins, Orchestrator: Chaos Mesh/Gremlin, Mesh: Istio, Observability: Prometheus/Jaeger/OTel, Feature flags: LaunchDarkly.

This design ensures reproducible chaos tests, measurable hypotheses, automated safety stop/rollback, and observability that captures degradations without customer impact.

EasyTechnical

76 practiced

Technical coding (Python): Implement a function summarize_test_results(results) where results is a list of dictionaries with keys name (string), duration_ms (integer), status (one of pass, fail, skipped). The function should return a dictionary summarizing total count per status and total_duration_ms for each status. Handle empty input and invalid statuses gracefully.

Sample Answer

**Approach**- Aggregate counts and durations per status ("pass", "fail", "skipped"). - Validate each item; skip or categorize invalid statuses under "unknown". - Handle empty input by returning zeros.

**Code (Python)**

python

def summarize_test_results(results):
    # Initialize summary with expected statuses
    statuses = ["pass", "fail", "skipped", "unknown"]
    summary = {s: {"count": 0, "total_duration_ms": 0} for s in statuses}

    if not results:
        return summary

    for item in results:
        # Defensive extraction with defaults
        name = item.get("name")
        duration = item.get("duration_ms", 0)
        status = item.get("status", "").lower()

        # Validate types
        if not isinstance(duration, int) or duration < 0:
            # treat invalid duration as 0 (could also log)
            duration = 0

        if status not in ("pass", "fail", "skipped"):
            status = "unknown"

        summary[status]["count"] += 1
        summary[status]["total_duration_ms"] += duration

    return summary

**Complexity**- Time: O(n) single pass. Space: O(1) fixed-size summary.

**Edge cases & SDET notes**- Handles empty list, missing keys, negative/ non-int durations, unexpected statuses.- In CI pipelines, log or metric-export unknowns for visibility.

Unlock Full Question Bank

Get access to hundreds of Software Development Engineer in Test Role interview questions and detailed answers.

Join thousands of developers preparing for their dream job.