Cross-Browser and Cross-Platform Testing Questions

Approach summary (SDET perspective)
Design a layered, risk-first test pyramid that balances quick feedback with periodic full-matrix assurance.

Targeted sampling (who/what to test)

• Build a prioritized matrix: weight browser/OS combos by real-user telemetry (top 80% traffic), business-critical flows, known fragility, and vendor market share.
• Define Core (top 5 combos), Extended (next 10), and Full (all combos).

CI cadence & scope

• PR canary/smoke: run fast headless/unit/integration + UI smoke on Core combos (parallelized, ~2–5 min). Purpose: catch regressions before merge.
• Nightly: run Extended matrix with more end-to-end scenarios.
• Weekly/overnight full-matrix: run exhaustive tests (longer, lower priority jobs).

Device-lab vs headless

• Use headless/browserless for Core smoke and most unit/functional tests for speed and cost.
• Reserve real device-lab / cloud VMs for Extended/Full runs and triage of flaky/visual/browser-specific bugs. Cache session images/videos for debugging.

Escalation policy

• Failure on Core in PR: block merge and open high-priority ticket.
• Intermittent failures in headless: auto-retry once, then run on device-lab combo.
• If nightly uncovered regression affecting >X% traffic or critical flow: trigger ad-hoc full-matrix run and rollback policy.

Automation & stability practices

• Parallelize test shards, smart test selection (git-diff -> affected tests), and stable selectors.
• Add visual diffs only on device-lab runs to save cost.

Metrics to evaluate effectiveness

• Release escape rate (bugs found in production per release) — primary metric.
• Mean time to detect and fix (MTTD/MTTR) for cross-browser regressions.
• Test coverage of user traffic (percentage of real sessions covered by Core+Extended).
• Flake rate per combo and cost per successful run (dollars per stable pipeline).
• Econometric: cost vs number of production regressions prevented (trend over time).

This strategy gives fast developer feedback, targeted real-device assurance, and measurable trade-offs between cost and risk.

Approach (brief)

• Use TestNG DataProvider to supply browser names and capabilities.
• Use ThreadLocal<WebDriver> for thread-safe driver per test.
• Annotate tests and DataProvider to run in parallel.

Code example

import org.testng.annotations.*;
import org.openqa.selenium.*;
import org.openqa.selenium.chrome.*;
import org.openqa.selenium.firefox.*;
import org.testng.Assert;

public class CrossBrowserTest {
  private static ThreadLocal<WebDriver> tlDriver = new ThreadLocal<>();

  private WebDriver getDriver() { return tlDriver.get(); }

  @DataProvider(name = "browsers", parallel = true)
  public Object[][] browsers() {
    return new Object[][] {
      {"chrome"}, {"firefox"}
    };
  }

  @BeforeMethod
  @Parameters("browser")
  public void setUp(Object[] params) {
    String browser = (String) params[0];
    WebDriver driver;
    if ("chrome".equalsIgnoreCase(browser)) {
      ChromeOptions opts = new ChromeOptions();
      opts.addArguments("--headless=new"); // example capability
      driver = new ChromeDriver(opts);
    } else {
      FirefoxOptions opts = new FirefoxOptions();
      opts.setHeadless(true);
      driver = new FirefoxDriver(opts);
    }
    tlDriver.set(driver);
    getDriver().manage().window().maximize();
  }

  @Test(dataProvider = "browsers")
  public void sampleE2ETest(String browser) {
    WebDriver d = getDriver();
    d.get("https://example.com");
    Assert.assertTrue(d.getTitle().contains("Example"));
  }

  @AfterMethod
  public void tearDown() {
    if (getDriver() != null) { getDriver().quit(); tlDriver.remove(); }
  }
}

Notes

• Run TestNG with parallel="methods" or "tests" in testng.xml or configure suites in CI.
• Use remote WebDriver/DesiredCapabilities for Selenium Grid/Cloud providers.