Security Testing Questions

Overview / Goal
Design an end-to-end security testing strategy embedded into the SDLC so QA enforces proactive detection, continuous validation, and measurable remediation with clear SLAs and exception handling.

Phases & Activities

• Requirements / Design: run threat modeling workshops (STRIDE) with product/dev/QA; produce attacker stories and prioritized risk register.
• Implementation: integrate SAST (e.g., SonarQube, Semgrep) in PR pipeline; fail build on high/critical findings.
• Pre-merge: secrets detection (git-secrets, truffleHog) + dependency & license scanning (OSS tools like Snyk/Dependabot + SPDX checks).
• CI & Nightly: DAST (OWASP ZAP, Burp CI) against deployed test env with authenticated scans; generate findings mapped to threat model.
• Release: scheduled pentests (quarterly + major release) and targeted retests of previously failed items.
• Regression: automated security regression tests (API fuzzing, auth flows, endpoint validation) executed in CI.

Integration & Automation

• CI hooks: SAST/secret/dependency checks on PR; DAST and regression suites on merge to develop/staging.
• Policy engine: gating rules per severity; auto-create tickets in Jira with labels, severity, remediation steps, CVSS mapping.
• Metrics: mean time to remediate (MTTR) by severity, leak rate, false positive rate.

Remediation & SLAs

• SLA examples: Critical = 72 hours, High = 14 days, Medium = 30 days, Low = next sprint.
• Exception management: formal risk acceptance workflow in Jira — requires owner, compensating controls, expiry date, and exec approval.

Ownership & Feedback

• QA maintains pipelines, regression suites, and triage first pass; Dev owns fix; Security reviews exceptions and pentest outputs.
• Continuous improvement: weekly triage, monthly metrics review, update threat model when features change.

Example

• PR blocks if SAST finds CWE-78 or secrets; otherwise PR creates ticket for medium findings. Nightly DAST discovers auth bypass → auto-ticket created, triaged, assigned, SLA starts.

This approach balances automation, manual assessment, measurable SLAs, and clear exception controls while keeping QA central to validation.

Approach (QA Engineer perspective)

I integrate SAST, DAST, and dependency composition analysis into CI/CD as complementary gates that surface OWASP Top 10 issues early while keeping velocity high.

Pipeline placement & scan cadence

• Pre-commit / local developer: lightweight SAST linters (e.g., ESLint with security rules, Semgrep) — immediate feedback.
• CI on pull request: full SAST (SonarQube/Snyk Code/Checkmarx) + dependency scan (Snyk, Dependabot, OWASP Dependency-Check) — block PRs for high/critical findings only.
• Nightly/full build: deeper SAST + DAST (OWASP ZAP, Burp CI integration, Snyk IaC) + interactive scans — reports for medium/low findings.
• Pre-production/staging: authenticated DAST + composition analysis + manual pen-test cadence (quarterly).

Tools (examples)

• SAST: SonarQube, Semgrep, Checkmarx, Snyk Code
• DAST: OWASP ZAP (CI plugin), Burp Suite Enterprise, Arachni
• Dependency / composition: Snyk, Dependabot, OSS Index, OWASP Dependency-Check
• Triage & tracking: Jira, GitHub Issues, GitLab issues, DefectDojo

False positives & triage

• Triage workflow: QA triages new findings daily — reproduce, classify (true/false/accepted risk), map to OWASP category, assign severity.
• Reproduce with minimal test case and include proof (request/response, stack trace, code pointer).
• Maintain a false-positive whitelist and suppression reasons in the scanning tool or as documented exemptions with timestamps and owner.

Developer feedback & remediation without blocking velocity

• Fail CI only on high/critical issues tied to OWASP Top 10 (e.g., injection, auth bypass). Medium/low create automated tickets and add PR comments with remediation links.
• Provide actionable remediation: code snippet, CWE reference, secure-code example, and unit/e2e test suggestions.
• Integrate scan output into PR view (annotations), and include a remediation checklist in PR template.
• Use security champions & rotation: QA creates knowledge sessions and small runbooks for common fixes to reduce cycle time.

Metrics & continuous improvement

• Track MTTR for security defects, false-positive rate, and percent of issues fixed before merge.
• Iteratively tune rules to reduce noise; add automated unit tests to prevent regressions.

Approach — goals & structure

• Lightweight libFuzzer-style harness that feeds mutated byte buffers to a parser which reads integers and performs arithmetic; enable ASAN+UBSAN to catch UB, and use sanitizers' integer-overflow reports plus crash signatures to triage exploitable cases.

Harness (annotated C++ pseudocode)

// fuzz_harness.cpp
#include <cstdint>
#include <cstring>
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
    // simple parser: parse ASCII integers separated by spaces/newlines
    // and perform operations that may overflow
    std::string s((const char*)data, size);
    std::istringstream in(s);
    long long a, b;
    while (in >> a >> b) {
        // Example op that can overflow/underflow
        long long sum = a + b;           // UBSAN will catch signed overflow
        long long prod = a * b;          // may overflow
        // use results to influence control flow (expose more UB)
        if (sum > 1000000 || prod < -1000000) {
            volatile long long sink = sum ^ prod; // prevent optimizations
            (void)sink;
        }
    }
    return 0;
}

Compilation / sanitizer flags

• Compile with: clang++ -fsanitize=address,undefined -fno-sanitize-recover=undefined -g -O1 fuzz_harness.cpp -fsanitize=fuzzer,address,undefined
• Options: -fno-sanitize-recover=undefined to turn UBSAN faults into crashes for easy fuzzing; keep -O1 to avoid removing checks but allow UB detection.

Mutation strategy

• Use libFuzzer or AFL; seeds: "0 0", "2147483647 2", "-2147483648 -1", hex bytes.
• Mutators: integer-focused mutations (flip sign bit, add/sub constants like INT_MAX, INT_MIN, ±1, ±2^k), length extenders, byte splicing.
• Guided fuzzing: add dictionary of patterns ("-"," ","0x","e+").

Detecting exploitable overflow vs benign UB

• UBSAN reports signed integer overflow locations; examine stack trace + instruction pointer.
• Exploitable if overflow influences memory access (index, pointer arithmetic) or control flow (function pointer, return address). Benign if only arithmetic result truncated but not used in memory bounds.
• Use ASAN to see heap/stack corruption that follows; combine with sanitizer logs.

Triage distinct crashes

• Automate canonicalization: normalize stack traces, record sanitizer type + backtrace + input.
• De-dup by top frame + sanitizer msg + crashpc.
• Reproduce minimal testcase with sanitizer logs; run under gdb/rr for deterministic replay.
• For potential exploitability: check whether overflow leads to pointer/index used in memory access or code path enabling memory write.
• Report: steps to reproduce, sanitizer output, minimized testcase, suggested severity (CRITICAL if control-flow/memory corruption; MEDIUM/LOW otherwise).

Why this works (QA perspective)

• Focused integer mutations expose edge arithmetic; sanitizers provide precise diagnostics; triage process groups findings into actionable defects for developers with minimized testcases and reproducible evidence.