Testing Strategy and Test Pyramid Approach Questions

Understand test pyramid (unit, integration, E2E), testing types (functional, performance, security, usability, compliance), optimal ratios, and how to balance coverage vs. effort. Know when to prioritize manual vs. automated testing and justify decisions based on risk and ROI.

MediumTechnical

41 practiced

In Python using pytest, implement a marker/decorator named `@pytest.mark.slow` and a small integration that skips tests marked slow when the environment variable SKIP_SLOW is set to 'true'. Provide the minimal code and describe how you would configure this behavior in CI.

Sample Answer

**Approach**- Add a pytest marker `slow` and a hook that checks SKIP_SLOW env var; if 'true', skip tests marked slow.- Provide minimal conftest.py and example test; explain CI configuration.

**conftest.py**

python

# conftest.py
import os
import pytest

def pytest_configure(config):
    # register the marker so pytest doesn't warn
    config.addinivalue_line("markers", "slow: mark test as slow to optionally skip")

def pytest_collection_modifyitems(config, items):
    skip_slow = os.environ.get("SKIP_SLOW", "").lower() == "true"
    if not skip_slow:
        return
    skip_marker = pytest.mark.skip(reason="Skipping slow tests (SKIP_SLOW=true)")
    for item in items:
        if item.get_closest_marker("slow"):
            item.add_marker(skip_marker)

**Example test**

python

# test_example.py
import pytest
import time

@pytest.mark.slow
def test_slow_task():
    time.sleep(0.01)
    assert True

def test_fast_task():
    assert True

**How it works / reasoning**- pytest_configure registers the marker to avoid warnings.- pytest_collection_modifyitems inspects collected tests; when SKIP_SLOW=true it adds a skip marker to any item with @pytest.mark.slow.- This keeps tests discoverable but conditionally skipped at collection time.

**CI configuration**- In CI (GitHub Actions / Jenkins / GitLab CI) set SKIP_SLOW=true for fast pipelines: - GitHub Actions example:

yaml

env:
      SKIP_SLOW: "true"
    steps:
      - run: pytest -q

- For nightly/integration runs, omit or set SKIP_SLOW=false to include slow tests.

**Notes / best practices**- Use consistent marker docs in repository README.- Consider a CLI flag (pytest --skip-slow) via pytest_addoption for more flexibility.

MediumTechnical

38 practiced

You inherit a monolithic application with 10% unit-test coverage but many slow, brittle integration tests. Create a phased plan (0–3 months, 3–6 months) to increase confidence using test-pyramid principles while keeping feature delivery uninterrupted. Include tooling, quick wins, and ways to measure progress.

Sample Answer

**High-level approach**Increase confidence by shifting left on the test pyramid: add fast, deterministic unit and component tests, introduce contract tests for service boundaries, and shrink brittle end-to-end (E2E) suites. Keep delivery flowing by quarantining flaky tests, incremental rollouts, and CI gating.

**0–3 months (stabilize + quick wins)**- Triage & baseline: - Run full test suite, record failures, runtimes, and flaky tests. - Capture current metrics: unit % coverage, avg CI time, flaky-rate, escaped bugs.- Quick wins: - Quarantine flaky/integration tests with clear tickets and labels. - Add lightweight smoke tests that run on every PR (critical paths). - Introduce per-module coverage reports and enforce minimal thresholds for new code.- Tooling/setup: - Add test reporting (JUnit, Allure), coverage (Coveralls/Codecov), flaky-test dashboard. - Choose automation tools aligned with stack (pytest/jest, Playwright/Testcontainers).- Deliverables: triage board, smoke suite in CI, baseline dashboards.

**3–6 months (shift left & scale)**- Increase unit/component coverage: - Implement test templates, DI-friendly helpers, and mocking utilities. - Add contract tests (Pact) for external/internal APIs to reduce full-stack flakiness. - Create repeatable component tests using Testcontainers or lightweight in-memory servers.- Improve CI: - Parallelize tests, run unit/component on every PR, nightly integration/E2E. - Auto-quarantine/retry flaky tests and track fixes.- Measure progress: - Metrics to track weekly: unit coverage %, integration test runtime, flaky-test rate, mean time to detection (MTTD), PR feedback time. - Target examples: +20–30% unit coverage, reduce E2E count by 50% in favor of contract/component tests, reduce CI full run by 40%.- Long-term: introduce mutation testing for quality, and map tests to risk to prioritize coverage.

Why this works: faster, deterministic tests accelerate feedback for developers; contracts and component tests preserve confidence in integrations while E2E surface-level checks ensure end-to-end paths remain guarded.

MediumBehavioral

34 practiced

Describe a time when you led a cross-functional effort to shift testing left (earlier) in the development lifecycle. Explain strategy, stakeholder engagement, training, tool changes, pilot outcomes, and the KPIs you used to measure success. Use concrete metrics where possible.

MediumTechnical

34 practiced

Compare bottom-up (unit-first) and top-down (e2e-first) approaches for starting a test strategy on a mature, unstable codebase. From an SDET point-of-view, discuss pros/cons, which aligns better with the test pyramid, and concrete signals that would make you choose one approach over the other.

HardTechnical

36 practiced

Implement a Python function `prioritize_tests(tests)` where each test record contains historical_failure_rate (0..1), recent_churn_score (0..1), and execution_time_seconds. Devise a scoring formula that prioritizes flaky and recently-changed tests but penalizes very long tests. Return the tests sorted by descending priority and explain your weight choices.

Sample Answer

**Approach**Design a composite score: higher for historical_failure_rate and recent_churn_score, but divided by a time penalty to deprioritize very long tests. Use tunable weights and a smooth penalty (log) to avoid harsh drops.

**Implementation (Python)**

python

import math

def prioritize_tests(tests, w_fail=0.6, w_churn=0.4, time_scale=30.0):
    """
    tests: list of dicts with keys:
      - historical_failure_rate (0..1)
      - recent_churn_score (0..1)
      - execution_time_seconds (float >0)
    Returns tests sorted by descending priority (adds 'priority' field).
    """
    prioritized = []
    for t in tests:
        f = max(0.0, min(1.0, t['historical_failure_rate']))
        c = max(0.0, min(1.0, t['recent_churn_score']))
        time = max(0.1, float(t['execution_time_seconds']))

        raw = w_fail * f + w_churn * c
        # time penalty: logarithmic scaling to avoid zeroing long tests
        time_penalty = 1.0 / (1.0 + math.log1p(time / time_scale))
        priority = raw * time_penalty

        t2 = dict(t)
        t2['priority'] = priority
        prioritized.append(t2)

    return sorted(prioritized, key=lambda x: x['priority'], reverse=True)

**Why these weights & choices**- w_fail=0.6, w_churn=0.4: historical flakiness slightly more predictive of failures; recent churn still important for likely regressions.- Logarithmic time penalty: penalizes long tests progressively but keeps expensive high-risk tests runnable for gated runs.- time_scale tunes where penalty steepens (30s suits many suites; adjust per org).

**Complexity & Edge Cases**- O(n log n) due to sort. Handles invalid ranges and zero/very small times. Alternative penalties: linear, exponential — chosen log for stability and interpretability.

Unlock Full Question Bank

Get access to hundreds of Testing Strategy and Test Pyramid Approach interview questions and detailed answers.

Join thousands of developers preparing for their dream job.