Direct answer
Validating an application's error handling as an SRE means testing not just that errors are caught, but that the system's response to each class of error (a bad request, a downstream timeout, a hard dependency outage) is the correct, safe behavior at every layer, unit tests for the error-handling logic itself, integration and contract tests for how the application reacts to a real dependency's actual failure modes, and chaos tests for how the whole system behaves under a genuinely injected failure.
Structured elaboration
- Unit tests: verify the error-handling logic in isolation, does a specific invalid input produce the correct, specific error response rather than an unhandled exception or a misleading generic error.
- Integration tests: verify the application's behavior against a real (or realistically simulated) dependency returning actual error conditions, a timeout, a connection refusal, a malformed response, confirming the application handles each distinctly rather than treating all failures identically.
- Contract tests for third-party APIs: verify the application correctly interprets the third-party API's documented error format and status codes, catching a mismatch (the application expecting one error shape while the actual API returns a different one) before it manifests as a real incident.
- Chaos tests: inject an actual failure (kill a dependency, introduce network latency or a partition) into a running system and confirm the observed behavior matches what unit and integration tests predicted in isolation, since a chaos test validates the INTEGRATION of all the individual error-handling pieces under a real, systemic failure condition, which isolated tests cannot fully guarantee on their own.
Worked example
For an application that calls a third-party payment API: a unit test confirms that a locally-raised validation error (an invalid amount) returns a specific, correct 400-style response to the caller. An integration test against a sandboxed version of the payment API confirms that when the sandbox returns a simulated timeout, the application retries according to its documented policy and eventually returns a specific, correct error to the user rather than hanging indefinitely. A contract test confirms the application correctly parses the payment API's actual documented error response shape, catching a case where the application's error-handling code assumed a slightly different field name than the API actually returns. A chaos test, run periodically in a controlled environment, actually blocks network access to the payment API for a defined window and confirms the application surfaces the same correct, user-facing error and does not, for instance, silently hang or return an incorrect success response, validating end to end that the pieces tested individually above actually work together under a real failure.
Automating this in a CI/CD pipeline: unit and integration tests run on every commit, since they are fast; contract tests run on every commit against a maintained mock reflecting the third-party API's documented contract, with a separate, less frequent (nightly or weekly) run against the real sandboxed API to catch contract drift; chaos tests run on a scheduled cadence (for example weekly) in a dedicated environment rather than per-commit, since they are more expensive and disruptive to run constantly, with results reported to the team and any regression treated as a release-blocking finding.
Trade-offs and pitfalls
The most common gap is testing error handling only at the unit level and assuming that guarantees correct behavior under a real, systemic failure, when the actual integration between error-handling logic, retry policy, and dependency behavior can behave differently than any individual piece suggests in isolation. The second pitfall is letting the contract-test mock drift from the real third-party API's actual current behavior without a periodic real-sandbox check, which silently invalidates the confidence the contract tests were supposed to provide.
