Test Strategy, Planning, and Risk-Based Prioritization Questions

Direct answer

Validating an application's error handling as an SRE means testing not just that errors are caught, but that the system's response to each class of error (a bad request, a downstream timeout, a hard dependency outage) is the correct, safe behavior at every layer, unit tests for the error-handling logic itself, integration and contract tests for how the application reacts to a real dependency's actual failure modes, and chaos tests for how the whole system behaves under a genuinely injected failure.

Structured elaboration

• Unit tests: verify the error-handling logic in isolation, does a specific invalid input produce the correct, specific error response rather than an unhandled exception or a misleading generic error.
• Integration tests: verify the application's behavior against a real (or realistically simulated) dependency returning actual error conditions, a timeout, a connection refusal, a malformed response, confirming the application handles each distinctly rather than treating all failures identically.
• Contract tests for third-party APIs: verify the application correctly interprets the third-party API's documented error format and status codes, catching a mismatch (the application expecting one error shape while the actual API returns a different one) before it manifests as a real incident.
• Chaos tests: inject an actual failure (kill a dependency, introduce network latency or a partition) into a running system and confirm the observed behavior matches what unit and integration tests predicted in isolation, since a chaos test validates the INTEGRATION of all the individual error-handling pieces under a real, systemic failure condition, which isolated tests cannot fully guarantee on their own.

Worked example

For an application that calls a third-party payment API: a unit test confirms that a locally-raised validation error (an invalid amount) returns a specific, correct 400-style response to the caller. An integration test against a sandboxed version of the payment API confirms that when the sandbox returns a simulated timeout, the application retries according to its documented policy and eventually returns a specific, correct error to the user rather than hanging indefinitely. A contract test confirms the application correctly parses the payment API's actual documented error response shape, catching a case where the application's error-handling code assumed a slightly different field name than the API actually returns. A chaos test, run periodically in a controlled environment, actually blocks network access to the payment API for a defined window and confirms the application surfaces the same correct, user-facing error and does not, for instance, silently hang or return an incorrect success response, validating end to end that the pieces tested individually above actually work together under a real failure.

Automating this in a CI/CD pipeline: unit and integration tests run on every commit, since they are fast; contract tests run on every commit against a maintained mock reflecting the third-party API's documented contract, with a separate, less frequent (nightly or weekly) run against the real sandboxed API to catch contract drift; chaos tests run on a scheduled cadence (for example weekly) in a dedicated environment rather than per-commit, since they are more expensive and disruptive to run constantly, with results reported to the team and any regression treated as a release-blocking finding.

Trade-offs and pitfalls

The most common gap is testing error handling only at the unit level and assuming that guarantees correct behavior under a real, systemic failure, when the actual integration between error-handling logic, retry policy, and dependency behavior can behave differently than any individual piece suggests in isolation. The second pitfall is letting the contract-test mock drift from the real third-party API's actual current behavior without a periodic real-sandbox check, which silently invalidates the confidence the contract tests were supposed to provide.

Direct answer

For a legacy repository with little to no automated tests, the safest way to introduce testing is to start with unit tests on the most stable, highest-value logic and consumer-driven contract tests at service boundaries, since both let teams refactor with real safety without requiring the risky, expensive step of building full end-to-end coverage first.

Structured elaboration

Why this order: unit tests are the cheapest to write and give the fastest feedback on core logic correctness, so they are the natural starting point for any codebase. Consumer-driven contract tests come next specifically because they protect the riskiest part of introducing tests into a legacy system, the interfaces between components or services, without requiring a full integration environment; a contract test catches a breaking change to how one part of the system calls another well before a slow, expensive end-to-end test would.

Safe first targets: pick the modules or services that are both frequently changed (so tests pay off quickly by catching regressions on every future change) and relatively well-understood (so writing a correct test does not itself require reverse-engineering undocumented behavior first). Avoid starting with the most tangled, least-understood part of the legacy system, even if it feels like the highest-risk area, since writing a wrong test there does more harm (false confidence) than writing no test at all.

CI integration: wire the new tests into the build pipeline from day one, even if coverage starts small, so the habit of running tests on every change is established early rather than retrofitted later once dozens of untested changes have already been merged without a safety net.

Success metrics to track: the number of previously-untested modules that now have baseline coverage, the number of contract tests protecting service boundaries, and, critically, the number of refactors or changes that were caught being unsafe by a new test before reaching production, which is the actual proof this effort is paying off, not raw test count alone.

Worked example

Concretely, for a legacy repository with a dozen loosely-coupled modules: phase 1 (first month), identify the two most frequently-changed, best-understood modules and add unit test coverage for their core logic, plus consumer-driven contract tests for the two service boundaries those modules expose to the rest of the system. Phase 2 (months 2-3), expand to the next tier of frequently-changed modules, and begin using the contract tests as a real safety net for a planned refactor of one of the covered boundaries, demonstrating the approach's value concretely (the refactor ships with confidence because the contract test would have caught an accidental breaking change). Track success as: 2 modules covered by month 1, 5 by month 3, and at least one real instance where a contract test caught a genuine breaking change before it reached production, which becomes the concrete evidence used to justify continuing the investment.

Trade-offs and pitfalls

The most common mistake is starting with the most complex, highest-risk-looking part of the legacy system out of a sense of urgency, when in practice a wrong or superficial test there provides false confidence and can be worse than having no test. The other mistake is delaying CI integration until "enough" tests exist, which misses the compounding value of catching regressions from day one, however small the initial coverage.