Safe Deployment and Rollback Strategies Questions

Direct answer
Blue-green deployment runs two full, identical production environments, "blue" (currently live) and "green" (the new version), and switches all traffic from one to the other at once via a DNS change or load-balancer reconfiguration, rather than gradually. Because the previous environment stays fully running and untouched, rollback is just switching traffic back, which is close to instant.

Structured elaboration

1. Prepare: deploy the new version into the idle environment (green) while blue continues serving all production traffic.
2. Validate: run smoke tests and synthetic checks against green directly, without exposing it to real users yet, since it's on its own environment.
3. Cut over: repoint the load balancer or update DNS so all new traffic goes to green. A load-balancer switch is effectively instant; a DNS switch is delayed by however long clients cache the old DNS TTL, which is why teams that need a fast cutover keep TTLs short or use a load balancer instead of DNS for the switch.
4. Monitor and hold: keep blue running, unmodified, for some period after cutover specifically so rollback is available without a redeploy.
5. Decommission: once green is proven stable, blue becomes the new idle environment for the next release (roles swap).

Session affinity and database state: stateless web frontends are the easy case: any request can go to either environment. Sticky sessions complicate the cutover, since a user mid-session on blue needs to either finish there or have their session state migrated to green. The database is usually the real complication, because you typically can't run two full database copies and keep them in sync cheaply, so both blue and green usually share ONE database, which means any schema change has to be compatible with BOTH the old and new application code during the window when either might be live.

Worked example
A stateless API service with a shared Postgres database: blue-green here is mechanically simple because the database is shared and unaffected by the switch. If the release also renamed a column, that rename would need to happen as a separate, backward-compatible expand-contract migration BEFORE the blue-green cutover, not as part of it, precisely because a straight rename would break whichever environment is still live during validation.

Trade-offs and pitfalls
Blue-green roughly doubles your running infrastructure cost for the duration both environments exist, which is the main reason teams don't leave it running indefinitely. The most common mistake is treating the database as if it participates in the blue-green switch the same way stateless compute does; it doesn't, and schema changes need their own compatibility discipline layered on top.

Direct answer
Feature flags are runtime switches that let you turn a piece of behavior on or off without a redeploy, which decouples DEPLOYING code from RELEASING a feature to users. The common types are a release flag (temporarily gates a feature during rollout, meant to be removed once it's fully live), an experiment flag (drives an A/B test, removed once the experiment concludes), an operational flag (a longer-lived control, like a rate limiter toggle or a maintenance-mode switch), and a kill-switch (an emergency, instant off-switch for something risky).

Structured elaboration

• Release flags: owned by the engineer shipping the feature; short-lived by design, meant to be deleted once the feature is fully rolled out and stable.
• Experiment flags: owned jointly by product/data science and engineering; lifetime tied to the experiment's duration, removed once a winner is chosen.
• Operational flags: owned by the team operating the service; can be long-lived (a genuinely permanent operational lever), which is the ONE type where "long-lived" isn't automatically a problem.
• Kill-switches: owned by whoever's on-call; meant to be exercised rarely but tested regularly so it's trustworthy when actually needed.
• Common pitfalls as flags accumulate: flag debt (release flags that were never cleaned up after the feature fully shipped, cluttering the codebase with dead branches), flag sprawl (so many flags that nobody can reason about which combinations of flag states are even possible, let alone tested), and stale defaults (a flag's fallback value drifts out of sync with what's actually safe as the surrounding code evolves).

Worked example
A team ships a new checkout flow behind a release flag, ramps it from 5% to 100% of users over two weeks while watching conversion metrics, then deletes the flag and the old code path once it's fully rolled out and stable for a monitoring period. If that deletion step gets skipped (a common failure under deadline pressure to move to the next feature), six months later the codebase has a flag nobody remembers the purpose of, defaulting to a value nobody's sure is still correct.

Trade-offs and pitfalls
Flags buy real safety (instant, redeploy-free control over risky behavior) at the cost of code complexity: every flag is effectively a branch that has to be reasoned about and eventually tested in both states. The single most common failure mode in practice isn't misusing a flag while it's active, it's forgetting to remove it once its job is done, which is why healthy flag programs bake in an explicit cleanup step (an expiration date, a dashboard of stale flags, or a policy that blocks new flags until old ones are retired) rather than relying on developers to remember.

Direct answer
The recreate deployment strategy tears down every instance of the old version before starting any instance of the new one, accepting a period of full downtime in exchange for the simplest possible mental model: at any given moment, either all-old or all-new is running, never a mix.

Structured elaboration
Teams still choose recreate, despite the downtime, when:

• Mixed-version compatibility is genuinely hard or unsafe to reason about: if old and new code can't safely coexist even briefly (a breaking, non-backward-compatible change with no feasible way to make it compatible for a transition window), recreate avoids ever creating that mixed-version state at all, at the cost of downtime instead.
• The service can tolerate a maintenance window: an internal tool, a batch-processing system, or a service with well-understood low-traffic periods can schedule the downtime somewhere it genuinely doesn't matter much, making the simplicity worth the (planned, contained) cost.
• Resource constraints: environments without spare capacity for even a small surge (an embedded system, a resource-constrained edge deployment, a small on-premises cluster with no slack) may not have room to run old and new simultaneously even briefly, making recreate the only option that fits the available resources.
• Simplicity as a deliberate choice for a low-stakes service: not every service justifies the operational complexity of a rolling update or canary; a low-traffic internal dashboard might reasonably just accept a 30-second restart rather than investing in zero-downtime tooling for something that doesn't need it.

Worked example
A nightly batch-processing job that only runs during a defined off-peak window: recreate is not just acceptable but arguably the RIGHT choice, since there's no live traffic to protect during that window anyway, and the operational simplicity (no need to reason about version-skew compatibility, no readiness-probe tuning) reduces real complexity for no actual downside given the service's usage pattern.

Trade-offs and pitfalls
The common mistake is defaulting to recreate purely out of inertia or unfamiliarity with rolling/canary patterns, on a service that actually DOES have meaningful live traffic and would benefit from a more careful strategy; recreate should be a deliberate choice matched to a specific service's tolerance for downtime, not the path of least resistance for every service regardless of its actual traffic pattern.

Direct answer
Blue-green, canary, and rolling all reduce the risk of a bad release, but through different mechanisms: blue-green switches ALL traffic at once between two full environments, canary exposes a small SLICE of traffic to the new version before widening it, and rolling replaces instances gradually IN PLACE. A plain recreate deployment, by contrast, tears down the old version entirely before starting the new one, accepting downtime in exchange for simplicity.

Structured elaboration

Rollback complexity nuance: blue-green's rollback is fastest because the old environment never stopped running; canary and rolling both have to actively redeploy or re-route, which takes real time even if it's automated; recreate's "rollback" is simple mechanically but means accepting a second period of downtime.

Stateful services: all three of blue-green/canary/rolling get significantly harder with state (a database, in-memory session data, local disk), because you can't just duplicate or partially expose the data layer the way you can stateless compute; the deployment strategy for the STATELESS layer often decouples from a separate, more careful strategy for the DATA layer.

Worked example
A stateless API fronting a shared database: canary is a strong default, since it limits blast radius on the code change while the shared database (which doesn't get canaried the same way) stays constant underneath. Blue-green would be a better fit if the team's top priority is minimizing time-to-rollback over minimizing blast radius, since flipping back to the old environment is close to instant.

Trade-offs and pitfalls
There's no universally "best" strategy: the choice trades off blast radius, rollback speed, infrastructure cost, and operational complexity, and the right answer depends on which of those the specific service and change profile cares about most. A common mistake is picking a strategy based on what's trendy (everyone reaches for canary) rather than what the actual risk profile of the change calls for; a low-risk config change might not need any of this ceremony at all.

Direct answer
A full rollback reverts every service or component involved in a release back to its previous version; a partial rollback reverts only some of them, leaving others on the new version. You choose partial when only some of the changed components are actually implicated in the problem and reverting the rest would be unnecessary churn or would itself introduce risk (for example, if a later component now depends on an earlier one's new behavior).

Structured elaboration

• When full rollback makes sense: a single service deployed a bad change, or several services were released together as one coupled unit where partial reversion would leave them in an inconsistent, untested combination.
• When partial makes sense: a coordinated multi-service release where only ONE service's new version is misbehaving, and the others are backward-compatible enough to keep running against either the old or new version of the problem service.
• Risk in partial rollback: you have to be confident the services you're leaving on the new version don't assume the now-reverted service's new behavior; if they do, a partial rollback trades one outage for a different, subtler one.

Worked example
Full: three tightly coupled microservices (auth, session, and API-gateway) deploy together as one release because the API gateway's new version assumes session's new token format; a bug anywhere means reverting all three together, since a partial revert would leave a token-format mismatch between them.
Partial: a release touches a recommendations service and a completely independent notifications service in the same deploy window purely for scheduling convenience; if only recommendations regresses, rolling back just that service and leaving notifications on its (unrelated, unaffected) new version is the right call, since reverting notifications too would be pure unnecessary churn.

Trade-offs and pitfalls
Partial rollback is faster and less disruptive when it's safe, but the judgment call of "is it actually safe to leave X on the new version" is exactly where mistakes happen; teams that don't explicitly map service dependencies before a coordinated release often discover the hard way, mid-incident, that two "independent" services weren't as independent as assumed.