Safe Deployment and Rollback Strategies Questions

Direct answer
A strong answer here follows STAR: what tipped you off that something was wrong, what you actually did to execute the rollback, and what changed afterward so the same failure mode doesn't recur. The interviewer is listening for concrete detection signals and concrete actions, not a vague "we noticed issues and rolled back."

Structured elaboration

• Situation/Task: name the service, the scale (traffic volume matters for how fast things degraded), and what the deploy changed.
• Action - detection: was it a dashboard alert, a customer report, a synthetic check? Specificity here (a named metric crossing a named threshold) is what separates a real story from a generic one.
• Action - execution: what commands or automation did you actually run? Redeploy previous image tag, flip a feature flag, revert a config? Did you have to coordinate a database rollback too, or was code-only sufficient?
• Action - safety checks: how did you confirm the rollback itself was safe before running it (was there a schema dependency you had to check first)?
• Result: how long did it take from detection to resolution, and what was the actual customer impact?
• Follow-up: what changed afterward: a new automated rollback trigger, a canary gate that would have caught it earlier, a runbook that didn't exist before?

Worked example
"We shipped a change to our checkout service that introduced a null-pointer path under a rare cart configuration. Fifteen minutes after full rollout, our error-rate alert fired at 3% (baseline 0.1%). I confirmed via the dashboard the spike started at the deploy timestamp, then ran our rollback script to redeploy the previous image tag, which took about ninety seconds including health-check verification. Error rate returned to baseline within two minutes of the redeploy completing. Afterward we added that cart configuration as an explicit test case and lowered our canary's automated error-rate threshold so a similar regression would be caught at 1% traffic instead of 100%."

Trade-offs and pitfalls
A common weak answer stops at "we rolled back and it was fixed" without naming a detection signal or a concrete command, which reads as secondhand rather than lived experience. Another common gap is skipping the "what changed afterward" beat entirely, which is often what the interviewer is most interested in, since it signals whether you learn from incidents systemically or just fight fires one at a time.

Direct answer
This is the entry-level version of the deployment behavioral question: what was your role, how did you prepare, what surprised you, and what was the measurable outcome, even without a dramatic rollback story attached.

Structured elaboration

• Role: were you the one deploying, reviewing, on-call for it, or supporting? Be specific rather than vague about your actual involvement.
• Preparation: what did you do before shipping (tests written, a runbook checked, a rollback plan confirmed, a smaller-than-usual rollout percentage chosen because it was a first-time change)?
• A surprise, even a small one: interviews aren't looking for a disaster; a benign surprise (a metric moved differently than expected, a dependency behaved unexpectedly) still shows you were paying attention rather than deploying and walking away.
• Measurable outcome: a number if you have one (adoption rate, performance change, error rate before/after), or a concrete qualitative outcome if not.

Worked example
"I deployed a caching layer change for a read-heavy endpoint. I prepared by running the change through our staging load test first and setting up a dashboard specifically for the metrics I expected to move, latency and cache-hit rate, before shipping. The surprise was that cache-hit rate improved less than modeled, about 15 points instead of the 30 I'd projected, because a chunk of traffic had more request-parameter variability than our test data captured. The outcome was still a real 15-point improvement and a genuinely useful lesson about how our synthetic test traffic didn't reflect production request diversity, which changed how we built test fixtures afterward."

Trade-offs and pitfalls
The weakest version of this answer is generic ("it went well, no issues") with no specificity, which gives the interviewer nothing to probe and reads as either inexperience or a lack of real engagement with the deploy. Even a smooth, uneventful deployment has SOMETHING specific worth naming: a metric you watched, a decision you made about rollout size, a thing you learned.

Direct answer
A rollback redeploys the previous, already-tested version of the artifact; a revert is a NEW forward commit that undoes the change in source control and then gets built and deployed like any other change. After discovering a production regression, rollback is almost always the faster, safer first move, since it restores a known-good state immediately, while a revert (even though it also "undoes" the change conceptually) still has to go through the normal build-and-deploy pipeline before it takes effect.

Structured elaboration

• Rollback: uses infrastructure/deployment tooling (redeploy the previous artifact, kubectl rollout undo, switch a blue-green environment back) to restore the PREVIOUS RUNNING STATE directly, without rebuilding anything; it's fast precisely because the previous version is already built, tested, and known-good.
• Revert: a source-control operation (git revert) that creates a new commit undoing the change; this new commit then needs to go through CI, build, and deploy like any normal change, which takes real time even if every step passes cleanly, and it's not automatically faster just because it "undoes" something.
• When you'd reach for each: rollback for the immediate, fast restoration of service; revert as the FOLLOW-UP action that keeps the source-control history clean and honest about what's actually running, and as the mechanism for making the "undo" permanent once you've confirmed the rollback fixed the problem (otherwise the next normal deploy, built from a source tree that still contains the bad change, would silently reintroduce the regression).
• Why both matter, not just one: rolling back WITHOUT eventually reverting means the next deploy from the current source tree reintroduces the bug, since the source code still contains the bad change even though the RUNNING version has been reverted; reverting without rolling back first means waiting through a full build-and-deploy cycle before service actually recovers, when a faster path was available.

Worked example
A regression discovered five minutes after a deploy: immediately kubectl rollout undo restores the previous, known-good version running in production within seconds. Separately, and not blocking that fast recovery, git revert <bad-commit> is pushed to keep the source tree consistent with what's actually running, so the next unrelated deploy (which will build from the current source tree) doesn't accidentally reintroduce the regression.

Trade-offs and pitfalls
The common mistake is treating these as interchangeable or doing only one: rolling back without ever reverting leaves a latent landmine in the source tree that resurfaces on the next deploy; reverting without rolling back first needlessly extends the outage while waiting for a full pipeline run when a faster path existed. The strongest practice is rollback FIRST for immediate recovery, revert SECOND (often within the same incident) to make the fix permanent in source control.

Direct answer
Blue-green deployment runs two full, identical production environments, "blue" (currently live) and "green" (the new version), and switches all traffic from one to the other at once via a DNS change or load-balancer reconfiguration, rather than gradually. Because the previous environment stays fully running and untouched, rollback is just switching traffic back, which is close to instant.

Structured elaboration

1. Prepare: deploy the new version into the idle environment (green) while blue continues serving all production traffic.
2. Validate: run smoke tests and synthetic checks against green directly, without exposing it to real users yet, since it's on its own environment.
3. Cut over: repoint the load balancer or update DNS so all new traffic goes to green. A load-balancer switch is effectively instant; a DNS switch is delayed by however long clients cache the old DNS TTL, which is why teams that need a fast cutover keep TTLs short or use a load balancer instead of DNS for the switch.
4. Monitor and hold: keep blue running, unmodified, for some period after cutover specifically so rollback is available without a redeploy.
5. Decommission: once green is proven stable, blue becomes the new idle environment for the next release (roles swap).

Session affinity and database state: stateless web frontends are the easy case: any request can go to either environment. Sticky sessions complicate the cutover, since a user mid-session on blue needs to either finish there or have their session state migrated to green. The database is usually the real complication, because you typically can't run two full database copies and keep them in sync cheaply, so both blue and green usually share ONE database, which means any schema change has to be compatible with BOTH the old and new application code during the window when either might be live.

Worked example
A stateless API service with a shared Postgres database: blue-green here is mechanically simple because the database is shared and unaffected by the switch. If the release also renamed a column, that rename would need to happen as a separate, backward-compatible expand-contract migration BEFORE the blue-green cutover, not as part of it, precisely because a straight rename would break whichever environment is still live during validation.

Trade-offs and pitfalls
Blue-green roughly doubles your running infrastructure cost for the duration both environments exist, which is the main reason teams don't leave it running indefinitely. The most common mistake is treating the database as if it participates in the blue-green switch the same way stateless compute does; it doesn't, and schema changes need their own compatibility discipline layered on top.

Direct answer
A full rollback reverts every service or component involved in a release back to its previous version; a partial rollback reverts only some of them, leaving others on the new version. You choose partial when only some of the changed components are actually implicated in the problem and reverting the rest would be unnecessary churn or would itself introduce risk (for example, if a later component now depends on an earlier one's new behavior).

Structured elaboration

• When full rollback makes sense: a single service deployed a bad change, or several services were released together as one coupled unit where partial reversion would leave them in an inconsistent, untested combination.
• When partial makes sense: a coordinated multi-service release where only ONE service's new version is misbehaving, and the others are backward-compatible enough to keep running against either the old or new version of the problem service.
• Risk in partial rollback: you have to be confident the services you're leaving on the new version don't assume the now-reverted service's new behavior; if they do, a partial rollback trades one outage for a different, subtler one.

Worked example
Full: three tightly coupled microservices (auth, session, and API-gateway) deploy together as one release because the API gateway's new version assumes session's new token format; a bug anywhere means reverting all three together, since a partial revert would leave a token-format mismatch between them.
Partial: a release touches a recommendations service and a completely independent notifications service in the same deploy window purely for scheduling convenience; if only recommendations regresses, rolling back just that service and leaving notifications on its (unrelated, unaffected) new version is the right call, since reverting notifications too would be pure unnecessary churn.

Trade-offs and pitfalls
Partial rollback is faster and less disruptive when it's safe, but the judgment call of "is it actually safe to leave X on the new version" is exactly where mistakes happen; teams that don't explicitly map service dependencies before a coordinated release often discover the hard way, mid-incident, that two "independent" services weren't as independent as assumed.