Testing Strategy and Test Pyramid Approach Questions

Understand test pyramid (unit, integration, E2E), testing types (functional, performance, security, usability, compliance), optimal ratios, and how to balance coverage vs. effort. Know when to prioritize manual vs. automated testing and justify decisions based on risk and ROI.

EasyTechnical

51 practiced

Technical task (Python, pytest): write pytest unit tests for the following function signature:

def calculate_discount(price: float, percentage: float) -> float

Requirements: cover a normal case (e.g., price 100, percentage 15), a zero percentage (returns original price), and an invalid negative price (should raise ValueError). Provide only the test code (no implementation), using pytest conventions and parameterization where appropriate.

Sample Answer

python

import pytest
from discounts import calculate_discount  # adjust import to target module

@pytest.mark.parametrize(
    "price, percentage, expected",
    [
        (100.0, 15.0, 85.0),    # normal case: 15% off 100 -> 85
        (199.99, 0.0, 199.99),  # zero percentage -> original price
    ],
)
def test_calculate_discount_normal_and_zero(price, percentage, expected):
    """
    Covers:
    - normal percentage reduction
    - zero percentage returns original price
    """
    result = calculate_discount(price, percentage)
    # use approximate comparison to tolerate floating point rounding
    assert pytest.approx(result, rel=1e-9) == expected

def test_calculate_discount_negative_price_raises():
    """
    Negative price is invalid and should raise ValueError.
    """
    with pytest.raises(ValueError):
        calculate_discount(-10.0, 10.0)

EasyTechnical

35 practiced

Describe common test coverage metrics (line/statement, branch, function/method, and mutation). Explain their strengths and limitations, and give concrete examples where high line coverage still misses bugs. Briefly describe how mutation testing differs and why it can expose weak tests.

MediumTechnical

39 practiced

You're QA for a microservices-based e-commerce platform with a rich Single-Page Application front-end. Propose an initial test pyramid ratio (approximate percentage distribution across unit, integration/service, and end-to-end tests). Justify your choices based on maintainability, test speed, confidence, and the heavy UI nature of the product, and describe how you'd validate and iteratively adjust the ratios over time.

MediumSystem Design

51 practiced

Design how you would integrate performance testing into a CI/CD pipeline for a web service that must support 10k requests per second (RPS) at peak. Explain when to run smoke load tests vs full load tests vs regression benchmarks, required environments, data isolation strategies, pass/fail criteria, and how to contain costs while getting reliable results.

HardSystem Design

39 practiced

How would you test runtime feature flags that can be toggled per-tenant and rolled out to partial percentages of users? Design a testing strategy that ensures no leakage between tenants, correct rollout percentages, safe rollbacks, interactions with cached data, and how to validate behavior at unit, integration, canary, and e2e levels. Also include monitoring and rollback automation recommendations.

Sample Answer

**Approach overview**I’d build layered tests and automation to guarantee tenant isolation, correct percentage rollouts, safe rollback, and cache correctness — from unit tests to automated canary + production monitoring and rollback.

**Unit tests**- Mock flag service; assert decision logic: tenant-scoped lookup overrides global, percentage hashing consistent and deterministic.- Test edge cases (missing tenant ID, malformed flags, SDK fallbacks).- Fast, isolated, CI-gated.

**Integration tests**- Spin test environment with real flag service (or local emulation) + auth to simulate multiple tenants.- Create tenants A/B, toggle flags per-tenant and at global level; assert responses and storage semantics.- Test cache layer: ensure TTL, invalidation API and headers cause re-read of flags; include concurrency tests.

**Canary tests**- Deploy to canary subset of hosts/users. Use synthetic users that emulate percentage hashing to validate rollout matches intended % (statistical sampling).- Run smoke flows and behavioral assertions; measure error rates and latency differences.

**End-to-end tests**- Run e2e scenarios for representative tenants (high-privilege, low-privilege). Validate feature behavior, data writes, and that no tenant can access another’s feature state.- Include regression suites for cascade effects (DB schema, downstream services).

**Validating rollout percentages**- Use deterministic hash function in tests; generate large sample of user IDs, compute expected % and assert observed distribution within confidence intervals.- In integration/canary, collect telemetry on served users vs enabled users; assert tolerance (e.g., ±1–2%).

**Cache & consistency**- Test cache hit/miss paths, TTL expiry, explicit invalidation, and stale-read windows.- Simulate race conditions: toggle flag during active session; verify eventual consistency and user-visible behavior.

**Monitoring**- Metrics: flag-evaluation rate, % enabled per-tenant, error rate, latencies, cache hit ratio.- Logs: include tenant-id, flag-version, decision-reason.- Alerts: deviation from expected rollout %, spike in errors, auth failures, cross-tenant access anomalies.

**Rollback automation**- Define automated rollback triggers (error rate threshold, % mismatch, security alert).- Implement playbook: automated traffic shift (feature toggle to off for affected tenant/global) + CI/CD rollback job + verification probe.- Canary auto-rollback: automated promotion only after health checks pass for time window; otherwise revert.

**Governance & test data**- Use synthetic tenant/test users and production-safe feature gating.- Periodically run chaos tests (flag-store unavailability, cache eviction) to validate resilience.

Result: layered safety (unit→integration→canary→e2e), measurable rollout validation, cache correctness, and automated rollback to prevent leakage or user impact.

Unlock Full Question Bank

Get access to hundreds of Testing Strategy and Test Pyramid Approach interview questions and detailed answers.

Join thousands of developers preparing for their dream job.