Mobile Application Testing Questions

Overview (goal)
I would build a layered, signal-driven test pyramid that balances fast feedback with broad-device reliability: heavy unit tests, focused integration tests on emulators, and curated UI tests on real devices.

Test coverage targets

• Unit: 70–80% logic/code paths (view models, utilities, validators)
• Integration: 15–25% (network layer, DB, platform-API glue)
• UI/E2E: 5–10% (critical user flows: auth, payments, onboarding, camera/location)

Mocking & stubbing

• Use dependency injection to swap real services for mocks in unit tests (Mockito/KMock for Android, Cuckoo/Mockingbird for iOS, or Dart mockito for Flutter).
• For integration tests, use contract-mocks and a local mock server (WireMock, MockWebServer) simulating latency, errors, auth tokens.
• For hardware (camera/GPS), provide platform-level stubs and simulated sensor data.

Device farm strategy

• Emulators/simulators run on every PR for fast feedback.
• Device farm (Firebase Test Lab, AWS Device Farm, BrowserStack) runs nightly and pre-release suites covering: top 30 devices by usage + 3 OS versions per platform, at least one low-end device per major OEM.

Test data management

• Use deterministic seeded test DBs and tokenized test accounts.
• Isolate state per test using fixtures and teardown; use feature flags to enable test-only endpoints.
• Encrypt and rotate sensitive fixtures; store in CI secrets.

Flaky detection & quarantine

• Track flaky tests via CI history (failure rate >5% flagged).
• Auto-quarantine flaky tests to a “flake” pipeline and notify owner; require flaky-fix within SLA (e.g., 2 sprints) or move to manual regression.
• Capture full device logs, screenshots, and video on failure to aid triage.

CI reporting to developers

• Enrich CI with structured reports: per-test pass/fail, stack traces, device model/OS, logs, screenshots, repro steps.
• Post PR comments with unit/integration summary; block merges on failing unit tests, allow flaky-flagged PRs but create a blocking issue.
• Daily dashboard (Grafana) for test health, device coverage, and flaky trends; integrate with Slack for alerts.

Trade-offs & maintenance

• UI tests are slow/expensive—keep them small and high-value. Regularly prune tests and update device matrix based on analytics.
  This strategy gives fast local feedback while ensuring reliability across diverse real devices.

Overview / goals
Design a test pyramid that gives fast feedback (unit), realistic behaviour (integration with simulated backend), and full-user validation (E2E on devices), while making tests deterministic and CI-reliable under flaky networks.

Unit tests (fast, deterministic)

• Tools: iOS XCTest + URLProtocol-based stubs or OHHTTPStubs; Android JUnit + Mockito + OkHttp MockWebServer.
• Strategy: isolate networking layer by injecting an HTTP client interface; mock network responses and clocks; assert request formation, retries, and error handling.
• Example: use MockWebServer to enqueue JSON and verify headers and retry counts.

Integration tests (simulated backend)

• Tools: WireMock standalone / Docker, MockWebServer for Android, local Swift/Vapor server for iOS.
• Strategy: run app build pointing to WireMock host in a test profile; test real serialization, persistence, and retry/backoff logic against scripted latency, 500s, auth flows.
• Use contract tests (Pact) to verify API expectations between mobile and backend teams.

Deterministic replay of server responses

• Tools: WireMock recordings, VCR-like recordings, or saved MockWebServer enqueues; Charles/mitmproxy session dumps for complex flows.
• Strategy: record golden interactions, store mappings in repo, run replay mode in CI to reproduce specific bugs. Pin timestamps/UUIDs via dependency injection to avoid nondeterminism.

End-to-end on device farms with flaky networks

• Tools: Firebase Test Lab, AWS Device Farm, BrowserStack App Automate; Espresso/XCUITest for automation.
• Network conditions: use Network Link Conditioner (iOS), tc/netem on device farm VMs, or device-farm network shaping to simulate latency, jitter, loss, and throttling.
• Strategy: run E2E suites under multiple profiles (good/3G/edge/packet-loss). Capture logs, videos, and HARs. Validate UX: offline flows, sync, resume-after-reconnect.

Flaky-test mitigation & observability

• Mark long-running flaky scenarios as nightly; for PRs run fast smoke E2E.
• Auto-rerun transient failures (1–2 retries) with strict logging; quarantine persistent flaky tests.
• Collect telemetry: screenshots, network traces, device logs, and deterministic seeds.

CI strategy

• Pipeline tiers:
  1. Pre-merge: unit tests + static analysis (Xcode Analyzer, detekt/ktlint), run in parallel on GitHub Actions / CircleCI / Bitrise.
  2. Merge gate: integration tests against WireMock in Docker, run in CI matrix (iOS/Android, API versions).
  3. Post-merge/nightly: E2E on device farms (Firebase/AWS/BrowserStack), network-shaping matrix, contract verification.
• Artifacts: store recordings, device logs, videos for debugging. Use caching for builds, and split tests to parallel device shards.
• Rollout tie-in: use staged rollouts and feature flags to limit blast radius; run canary E2E before full release.

Trade-offs

• Full device E2E is slow/costly; mitigate with targeted smoke tests on PRs and more exhaustive nightly runs.
• Recordings speed up reproducibility but may drift—maintain periodic re-recording and contract tests.

This strategy balances speed, realism, determinism, and CI reliability for mobile networked features.

Approach (overview)
Design tests that run deterministically across sizes/orientations by parameterizing device configs, using stable selectors, and isolating external dependencies (network, animations).

Frameworks

• iOS: XCUITest for end-to-end UI flows; SnapshotTesting or iOSSnapshotTestCase for pixel/layout comparisons; use XCTest expectations + XCUI Accessibility Identifiers for stability. Use URLProtocol-based stubs or OHHTTPStubs for network.
• Android: Espresso for interactions + Espresso IdlingResources to wait for async work; androidx.test.uiautomator for cross-app checks; Shot or Paparazzi for screenshot assertions. Use MockWebServer or OkHttp interceptors for network.

Test structure to reduce flakiness

• Use accessibility identifiers / resource IDs (avoid text-based selectors).
• Parameterize tests over device sizes and orientations (JUnit Parameterized / XCTest loops) so same test runs for iPhone SE → iPad and portrait/landscape.
• Centralize setup/teardown and test data; reset app state between tests.
• Synchronize async work with IdlingResources (Android) / expectations (iOS) and disable animations on CI.
• Keep tests small and focused (one assertion per behavior), avoid sleeping; use retries only sparingly.

Mocking network (deterministic tests)

• Inject network layer (protocol/interface) so tests can replace real client with a mock.
• Android example: use MockWebServer to enqueue deterministic JSON responses.

// Kotlin: enqueue a fixed response
val server = MockWebServer()
server.enqueue(MockResponse().setBody(fixtureJson).setResponseCode(200))
val baseUrl = server.url("/").toString()

• iOS example: register a custom URLProtocol that returns fixture data for matching requests.

// Swift: URLProtocol subclass returns local JSON fixture for given request
URLProtocol.registerClass(MockURLProtocol.self)

• Use fixtures/versioned responses, test error and latency scenarios, and run both mocked and a small set of integration tests against a staging server.

This combination ensures reproducible layout and behavior verification across device fragmentation while keeping tests fast and reliable.