Safe Deployment and Rollback Strategies Questions

Direct answer
The recreate deployment strategy tears down every instance of the old version before starting any instance of the new one, accepting a period of full downtime in exchange for the simplest possible mental model: at any given moment, either all-old or all-new is running, never a mix.

Structured elaboration
Teams still choose recreate, despite the downtime, when:

• Mixed-version compatibility is genuinely hard or unsafe to reason about: if old and new code can't safely coexist even briefly (a breaking, non-backward-compatible change with no feasible way to make it compatible for a transition window), recreate avoids ever creating that mixed-version state at all, at the cost of downtime instead.
• The service can tolerate a maintenance window: an internal tool, a batch-processing system, or a service with well-understood low-traffic periods can schedule the downtime somewhere it genuinely doesn't matter much, making the simplicity worth the (planned, contained) cost.
• Resource constraints: environments without spare capacity for even a small surge (an embedded system, a resource-constrained edge deployment, a small on-premises cluster with no slack) may not have room to run old and new simultaneously even briefly, making recreate the only option that fits the available resources.
• Simplicity as a deliberate choice for a low-stakes service: not every service justifies the operational complexity of a rolling update or canary; a low-traffic internal dashboard might reasonably just accept a 30-second restart rather than investing in zero-downtime tooling for something that doesn't need it.

Worked example
A nightly batch-processing job that only runs during a defined off-peak window: recreate is not just acceptable but arguably the RIGHT choice, since there's no live traffic to protect during that window anyway, and the operational simplicity (no need to reason about version-skew compatibility, no readiness-probe tuning) reduces real complexity for no actual downside given the service's usage pattern.

Trade-offs and pitfalls
The common mistake is defaulting to recreate purely out of inertia or unfamiliarity with rolling/canary patterns, on a service that actually DOES have meaningful live traffic and would benefit from a more careful strategy; recreate should be a deliberate choice matched to a specific service's tolerance for downtime, not the path of least resistance for every service regardless of its actual traffic pattern.

Direct answer
This is the entry-level version of the deployment behavioral question: what was your role, how did you prepare, what surprised you, and what was the measurable outcome, even without a dramatic rollback story attached.

Structured elaboration

• Role: were you the one deploying, reviewing, on-call for it, or supporting? Be specific rather than vague about your actual involvement.
• Preparation: what did you do before shipping (tests written, a runbook checked, a rollback plan confirmed, a smaller-than-usual rollout percentage chosen because it was a first-time change)?
• A surprise, even a small one: interviews aren't looking for a disaster; a benign surprise (a metric moved differently than expected, a dependency behaved unexpectedly) still shows you were paying attention rather than deploying and walking away.
• Measurable outcome: a number if you have one (adoption rate, performance change, error rate before/after), or a concrete qualitative outcome if not.

Worked example
"I deployed a caching layer change for a read-heavy endpoint. I prepared by running the change through our staging load test first and setting up a dashboard specifically for the metrics I expected to move, latency and cache-hit rate, before shipping. The surprise was that cache-hit rate improved less than modeled, about 15 points instead of the 30 I'd projected, because a chunk of traffic had more request-parameter variability than our test data captured. The outcome was still a real 15-point improvement and a genuinely useful lesson about how our synthetic test traffic didn't reflect production request diversity, which changed how we built test fixtures afterward."

Trade-offs and pitfalls
The weakest version of this answer is generic ("it went well, no issues") with no specificity, which gives the interviewer nothing to probe and reads as either inexperience or a lack of real engagement with the deploy. Even a smooth, uneventful deployment has SOMETHING specific worth naming: a metric you watched, a decision you made about rollout size, a thing you learned.

Direct answer
A rolling update replaces old-version instances with new-version ones gradually, a few at a time, so the service stays available with a mix of old and new versions running simultaneously during the transition. A recreate deployment, by contrast, terminates ALL old instances first and only then starts the new ones, which means a period of full downtime but avoids ever running mixed versions.

Structured elaboration
For a stateless microservice in Kubernetes, a rolling update:

1. Kubernetes creates a batch of new-version pods (controlled by maxSurge, how many extra pods above the target replica count are allowed).
2. Waits for those new pods to pass their readiness probe before routing traffic to them.
3. Terminates an equivalent batch of old-version pods (controlled by maxUnavailable, how many pods can be down at once).
4. Repeats until all pods are on the new version.

Common failure modes to watch for during a rollout:

• Readiness probe misconfigured too loosely: traffic gets routed to a pod that's technically "ready" but not actually able to serve correctly yet (cache not warmed, connection pool not established).
• Version skew during the mixed-version window: old and new pods running simultaneously both talk to the same downstream dependencies (shared database, shared cache), so if the new version isn't backward-compatible with what the old version expects, you get intermittent failures purely from which version happened to handle a given request.
• Resource exhaustion from surge: if maxSurge allows too many extra pods at once relative to available cluster capacity, new pods can fail to schedule, stalling the rollout partway.
• A bad new version rolling out gradually still affects a growing fraction of traffic before anyone notices, unlike blue-green where the bad version is fully isolated until an explicit cutover.

Worked example
A 20-replica deployment with maxSurge: 25% and maxUnavailable: 25% creates up to 5 extra pods (25 total temporarily) while taking down up to 5 old pods at a time, cycling through until all 20 are on the new version. If the new version has a subtle bug that only manifests under a specific downstream response, roughly a quarter of traffic is exposed to it at any point mid-rollout, growing toward 100% as the rollout proceeds, unless something halts it.

Trade-offs and pitfalls
Rolling update avoids downtime and extra infrastructure cost (no duplicate fleet, unlike blue-green) but accepts a mixed-version window where compatibility between old and new has to hold, and it doesn't isolate a bad release the way canary or blue-green does; it just gradually replaces capacity regardless of whether the new version is actually healthy, UNLESS combined with a readiness-probe-based or metrics-based halt condition.

Direct answer
A rollback redeploys the previous, already-tested version of the artifact; a revert is a NEW forward commit that undoes the change in source control and then gets built and deployed like any other change. After discovering a production regression, rollback is almost always the faster, safer first move, since it restores a known-good state immediately, while a revert (even though it also "undoes" the change conceptually) still has to go through the normal build-and-deploy pipeline before it takes effect.

Structured elaboration

• Rollback: uses infrastructure/deployment tooling (redeploy the previous artifact, kubectl rollout undo, switch a blue-green environment back) to restore the PREVIOUS RUNNING STATE directly, without rebuilding anything; it's fast precisely because the previous version is already built, tested, and known-good.
• Revert: a source-control operation (git revert) that creates a new commit undoing the change; this new commit then needs to go through CI, build, and deploy like any normal change, which takes real time even if every step passes cleanly, and it's not automatically faster just because it "undoes" something.
• When you'd reach for each: rollback for the immediate, fast restoration of service; revert as the FOLLOW-UP action that keeps the source-control history clean and honest about what's actually running, and as the mechanism for making the "undo" permanent once you've confirmed the rollback fixed the problem (otherwise the next normal deploy, built from a source tree that still contains the bad change, would silently reintroduce the regression).
• Why both matter, not just one: rolling back WITHOUT eventually reverting means the next deploy from the current source tree reintroduces the bug, since the source code still contains the bad change even though the RUNNING version has been reverted; reverting without rolling back first means waiting through a full build-and-deploy cycle before service actually recovers, when a faster path was available.

Worked example
A regression discovered five minutes after a deploy: immediately kubectl rollout undo restores the previous, known-good version running in production within seconds. Separately, and not blocking that fast recovery, git revert <bad-commit> is pushed to keep the source tree consistent with what's actually running, so the next unrelated deploy (which will build from the current source tree) doesn't accidentally reintroduce the regression.

Trade-offs and pitfalls
The common mistake is treating these as interchangeable or doing only one: rolling back without ever reverting leaves a latent landmine in the source tree that resurfaces on the next deploy; reverting without rolling back first needlessly extends the outage while waiting for a full pipeline run when a faster path existed. The strongest practice is rollback FIRST for immediate recovery, revert SECOND (often within the same incident) to make the fix permanent in source control.

Direct answer
Feature flags are runtime switches that let you turn a piece of behavior on or off without a redeploy, which decouples DEPLOYING code from RELEASING a feature to users. The common types are a release flag (temporarily gates a feature during rollout, meant to be removed once it's fully live), an experiment flag (drives an A/B test, removed once the experiment concludes), an operational flag (a longer-lived control, like a rate limiter toggle or a maintenance-mode switch), and a kill-switch (an emergency, instant off-switch for something risky).

Structured elaboration

• Release flags: owned by the engineer shipping the feature; short-lived by design, meant to be deleted once the feature is fully rolled out and stable.
• Experiment flags: owned jointly by product/data science and engineering; lifetime tied to the experiment's duration, removed once a winner is chosen.
• Operational flags: owned by the team operating the service; can be long-lived (a genuinely permanent operational lever), which is the ONE type where "long-lived" isn't automatically a problem.
• Kill-switches: owned by whoever's on-call; meant to be exercised rarely but tested regularly so it's trustworthy when actually needed.
• Common pitfalls as flags accumulate: flag debt (release flags that were never cleaned up after the feature fully shipped, cluttering the codebase with dead branches), flag sprawl (so many flags that nobody can reason about which combinations of flag states are even possible, let alone tested), and stale defaults (a flag's fallback value drifts out of sync with what's actually safe as the surrounding code evolves).

Worked example
A team ships a new checkout flow behind a release flag, ramps it from 5% to 100% of users over two weeks while watching conversion metrics, then deletes the flag and the old code path once it's fully rolled out and stable for a monitoring period. If that deletion step gets skipped (a common failure under deadline pressure to move to the next feature), six months later the codebase has a flag nobody remembers the purpose of, defaulting to a value nobody's sure is still correct.

Trade-offs and pitfalls
Flags buy real safety (instant, redeploy-free control over risky behavior) at the cost of code complexity: every flag is effectively a branch that has to be reasoned about and eventually tested in both states. The single most common failure mode in practice isn't misusing a flag while it's active, it's forgetting to remove it once its job is done, which is why healthy flag programs bake in an explicit cleanup step (an expiration date, a dashboard of stale flags, or a policy that blocks new flags until old ones are retired) rather than relying on developers to remember.