Test Automation Concepts and Script Logic Questions

Basic conceptual understanding of how automated test scripts work. Ability to read simple test code and understand its purpose and logic at a high level. Basic knowledge of assertions (how automation validates expected results) and how test frameworks evaluate test results. Recognition of common automation patterns and terminology. No deep programming knowledge required, but understanding automation fundamentals.

MediumTechnical

85 practiced

You have a large suite of 5,000 tests. Describe how you would use test tags/annotations and CI pipeline stages to provide quick feedback on pull requests while still running a full regression nightly. Include example grouping rules (e.g., 'smoke', 'fast', 'integration', 'slow') and how to enforce them.

Sample Answer

**Approach (brief)**

I’d tag tests by intent and cost, run small fast groups on PRs for quick feedback, and run full regression in nightly pipelines. Enforce tags in test code and CI so ownership stays consistent.

**Tagging scheme & examples**

- smoke — critical user journeys (~50 tests)- fast — unit + small integration, < 2s each (~1,000)- integration — external systems, DB, network (~2,000)- slow — long-running/perf, > 5s or flaky (~1,950)

Use framework markers (pytest: @pytest.mark.smoke / @pytest.mark.integration), or JUnit categories.

**CI pipeline stages**

- PR pipeline (fast feedback) - job: test:smoke — run smoke (parallel shards) — required, fail PR on failure - job: test:fast — run fast + smoke (parallel) — optional required for merge - gating: if tests fail, block merge; report failures in PR comments

- Extended PR (optional) - test:integration — run only on labeled PRs or scheduled merge queue

- Nightly full regression - job: regression:full — run all tags, longer timeout, nightly schedule; alerts & dashboard

**Enforcement & automation**

- Pre-commit/CI lint to ensure new tests include a tag; fail if untagged.- PR merge rule: require successful smoke + fast jobs.- Test runtime guard: CI job checks cumulative runtime per tag; if a "fast" test exceeds defined time threshold, fail and require retagging to "slow".- Use test discovery to list test counts per tag; fail if tags are misbalanced.- Maintain dashboards (Pass rate, flakiness by tag) and require triage for failing smoke tests before merge.

**Example commands (pytest)**

bash

# run smoke only
pytest -m smoke -n auto

# run fast and smoke
pytest -m "fast or smoke" -n 4

# run full regression
pytest -m "smoke or fast or integration or slow" --maxfail=1

This gives rapid PR feedback on critical flows while preserving comprehensive nightly regression, with automated checks to keep tagging accurate and enforce SLAs for quick tests.

EasyTechnical

76 practiced

What are common signs that a test in your automation suite is flaky? List at least five indicators you would look for in CI history, logs, or dashboards, and describe one quick confirmation step you would take to prove a test is flaky.

MediumTechnical

72 practiced

Write a short parameterized pytest test (3–8 lines) that validates an is_palindrome(s) function for these cases: 'radar' -> True, 'hello' -> False, '' -> True. Show how to use @pytest.mark.parametrize to pass inputs and expected outputs.

Sample Answer

**Answer (QA Engineer perspective)**

- I would write a concise parameterized unit test to cover normal, negative and edge cases.- This uses pytest.mark.parametrize to pass (input, expected) pairs and keeps tests isolated and readable.

python

import pytest

@pytest.mark.parametrize("s,expected", [("radar", True), ("hello", False), ("", True)])
def test_is_palindrome(s, expected):
    assert is_palindrome(s) == expected

- This 4-line test is easy to extend; as a QA engineer I'd add property-based or unicode tests if needed.

HardTechnical

60 practiced

For a QA organization scaling automation across multiple product teams, outline governance policies and technical practices to ensure test suite maintainability and quality. Include items such as code review rules for tests, flakiness thresholds, ownership model, shared libraries and utilities, versioning of test frameworks, CI resource allocation, and enforcement mechanisms or incentives for teams.

EasyTechnical

80 practiced

Explain the difference between an assertion and a verification in automated testing. Describe what typically happens to test execution flow when a hard assertion fails compared to when a verification (soft assertion) records an error but allows the test to continue. Give one example use-case for each.

Unlock Full Question Bank

Get access to hundreds of Test Automation Concepts and Script Logic interview questions and detailed answers.

Join thousands of developers preparing for their dream job.