API Versioning, Deprecation and Backward Compatibility Questions

Requirements & constraints:

• Minimize churn for many consumers while splitting monolith into services.
• Maintain backward compatibility, enable incremental rollout, and allow independent deploys.

High-level approach:

1. Define coarse-grained bounded contexts as service boundaries (business domain first). Design APIs around stable use-cases (e.g., Orders, Customers) rather than internal DB tables.
2. Contracts as explicit API spec (OpenAPI) stored in a contract repo; changes require review and CI checks.

Versioning strategies:

• Prefer evolution without major versions: additive, backwards-compatible changes use "vNextless" on same URI; breaking changes use major versioning (e.g., /v1/, /v2/) or header versioning when needed.
• Use feature flags and consumer-driven contracts (CDC) to coordinate migrations.

Idempotency patterns:

• For non-idempotent operations (create/order), require client-supplied idempotency-key (Idempotency-Key header) stored in service for TTL; return same result for same key.
• For retries, design operations to be idempotent where possible (PUT for upsert, PATCH for partial updates with resource version).

Backward compatibility:

• Make all changes additive: add new fields, new endpoints, new optional query params.
• Avoid removing fields; deprecate via API responses/headers and docs; plan for at least 2 release cycles before removal.
• Tolerant readers: clients ignore unknown fields; servers tolerate missing optional fields.

Schema migration strategies:

• Database: expand-then-contract pattern: add new columns, backfill, switch reads to new model, then remove old schema after stable.
• API: expand request/response; support both old and new shapes for period; use translation/adaptor layer at service boundary if necessary.

API compatibility testing plan:

• Maintain contract tests (Pact or Spring Cloud Contract) run in CI for provider and consumers.
• Automated integration tests: run consumer tests against provider stub and provider verification.
• Regression tests for idempotency and retry scenarios.
• Smoke tests for both /v1 and /v2 during rollout.

Concrete evolving JSON example and incremental consumer adaptation:
Initial response (v1):

{
  "orderId": "123",
  "amount": 49.99,
  "currency": "USD",
  "status": "NEW"
}

Evolve to include shipping and customer summary (additive):

{
  "orderId": "123",
  "amount": 49.99,
  "currency": "USD",
  "status": "NEW",
  "shipping": {
    "address": "1 Main St",
    "eta": "2025-12-01"
  },
  "customer": {
    "id": "c789",
    "name": "Alice"
  }
}

How consumers adapt:

• Phase 1: Consumers ignore new fields (tolerant reader) — no change required.
• Phase 2: Consumers that need shipping info opt-in by reading new fields and adding parsing/DTO updates; deploy consumer changes while provider continues to send both shapes.
• Phase 3 (if structure changes): Provider introduces a minor schema rename only after deprecation notices and ensures /v1 continues until all consumers migrated, or offers /v2 for breaking rename.

Key practices to minimize churn:

• Use explicit contract testing + CI gating
• Make changes additive and document deprecations
• Use idempotency keys for safe retries
• Backward-compatible DB migrations (expand-then-contract)
• Coordinate via consumer-driven contracts and feature flags for incremental rollout.

Definition and trade-offs (brief):

• URI versioning (/v1/...): version in path. Simple, discoverable (easy for clients and docs), cache/CDN friendly (URL is cache key). Tends to encourage coarse-grained breaking changes; requires duplicate endpoints for old versions.
• Header / content negotiation: version in a custom/header (e.g., X-API-Version) or Accept header. Cleaner URIs, better for evolving resources, but less discoverable and harder for simple clients (browsers, CDNs) because caches often key on URL only; proxies may strip headers.
• Media-type versioning: encode version in media type (Accept: application/vnd.myapi.v1+json). Good for resource-level versioning and semantics, supports multiple representations, but same discoverability and caching issues as header-based approaches.
• Semantic-versioned contracts: follow semver for fields/behaviors (MAJOR.MINOR.PATCH). Emphasizes backward-compatibility rules (only MAJOR breaks), encourages additive/non-breaking changes and formal contract management (change logs, compatibility guarantees). Requires clear governance and client upgrade guidance.

Recommendation for a public API with many third parties:
Use URI versioning for major breaking changes + semantic-versioned contract discipline within each major version.
Why:

• Discoverability: /v1/... URLs are explicit in docs, code samples and easy for developers to test.
• Caching/CDN/proxies: URLs make caching reliable; CDNs and intermediaries will cache responses per-version without extra config.
• Compatibility: Combine with semver rules so MINOR/PATCH remain backward-compatible; reserve new major path (e.g., /v2/) for breaking changes. This gives predictable upgrade windows.
• Operational complexity: URI versioning minimizes special header handling and reduces client support burden; semantic governance requires process but saves long-term churn.

Implementation notes:

• Document version migration guides and deprecation timelines.
• Route old /v1/ to stable handlers; use feature flags and API gateways to route and monitor usage.
• Provide client libraries and SDKs tied to version to ease upgrades.

Requirements & principles:

• Fast-moving teams must own APIs; governance must be lightweight, automated, and predictable.
• Goals: consistency, safety, discoverability, backward-compatibility, and minimal blocking.

High-level model:

1. Standards & artifacts

• Mandatory: OpenAPI spec, semantic versioning, SLAs (latency, availability), auth model (OAuth2/JWT), and error-format.
• Provide templates: OpenAPI starter, SDK generator configs, API changelog template.

2. CI checks (automated guardrails)

• Pre-merge CI runs:
  • openapi-lint (spectral) & style enforcement
  • contract/consumer-driven tests (PACT/contract tests) against staging mocks
  • schema diffing tool that flags breaking changes
  • security scanners (snyk/OWASP)
  • performance smoke tests (simple latency/throughput)
• PR must include updated spec and compatibility report. Non-breaking changes auto-approve; breaking changes require review.

3. Approval workflow

• Triage: PR opens with "API-change" label -> automated checks run.
• If all checks pass and compatibility = non-breaking -> auto-merge after owner approval (team owner).
• If breaking or high-risk -> create an API Change Request in lightweight governance board (web UI) with metadata: impact, migration plan, rollout strategy, deprecation timeline.
• Governance Board (rotating engineers from teams + API steward) meets twice weekly; urgent items have 48h SLA.

4. Deprecation policy

• Deprecation announced via registry and release notes; maintain deprecated endpoint for minimum N weeks (e.g., 12) with monitoring on usage.
• Provide automated traffic shadowing and client-usage reports to prioritize retirements.
• Enforce hard removal only after usage < threshold and stakeholders sign-off.

5. Scaling without blocking teams

• Maximize automation: auto-lint, auto-compatibility checks, auto-documentation, policy-as-code in gateway.
• Tiered governance: Tier 1 (internal low-risk) — self-serve; Tier 2 (public/internal high-impact) — lightweight review; Tier 3 (platform/core infra) — full board review.
• Decision SLA: auto-approve on policy compliance; human review minimized and batched.
• Self-service platform: API registry, spec templates, migration helpers, client SDK generation, and one-click staging deployments.
• Observability & feedback: telemetry, breaking-change alerts, consumer subscriptions.

Trade-offs

• Automation upfront costs (tooling, templates) but prevents slow human gating.
• Rotating board reduces bias and spreads knowledge; SLAs ensure progress.

Example technologies

• OpenAPI, Spectral, Pact, CI (GitHub Actions/Jenkins), API Gateway (Kong/Apigee/Istio), API registry (Backstage), policy-as-code (Open Policy Agent), monitoring (Prometheus, Grafana).

Requirements & constraints:

• Two teams must evolve API independently; avoid breaking consumers; allow rolling release cadence; minimize synchronous coordination.

Governance model (people/process):

• API Steering Board: one engineering lead from each team + API platform owner; meets weekly for escalations and approves major version bumps.
• API Change Proposal (ACP): any breaking change requires a short RFC with migration plan, deprecation timeline, and contract test updates. Post to shared repo and Slack channel; 3-business-day comment window.
• Release windows: teams can release non-breaking changes anytime; breaking changes gated by ACP approval and deprecation rules.

Versioning policy:

• Semantic versioning for public API surface: MAJOR.MINOR.PATCH.
• MAJOR = breaking changes; requires new route/path or explicit Accept header versioning; MINOR = backward-compatible additions; PATCH = fixes.
• Prefer additive changes and feature flags. Use header-based versioning if path changes are expensive.

Deprecation schedule:

• When a MAJOR is planned, publish deprecation notice at least 90 days before removal; provide compatibility layer for minimum 90 days where feasible.
• Provide automated migration guides and client library shims for common languages.
• Track consumer adoption via telemetry; removal allowed once <5% legacy usage or after 120 days—whichever later.

Technical strategy:

• Backwards-compatibility enforcement: keep old handlers active; implement compatibility adapters in API gateway or facade.
• Consumer-driven contract tests: consumers publish Pact/OpenAPI consumer tests in a shared CI pipeline. Provider runs those tests on every PR against a mock and staging provider.
• CI gating: Provider CI must run:
  • Static API schema validation (OpenAPI lint)
  • All consumer-driven contracts
  • Integration tests in ephemeral environments
  • Performance smoke tests for changed endpoints
  • If any contract fails, PR blocked and auto-notifies owning consumer team.
• Contract registry: store canonical OpenAPI spec and consumer pacts in a versioned repo; CI pulls relevant consumer contracts for testing.

Deployment & delivery:

• Canary + feature flags for new behavior; route a small % of traffic to new code.
• Blue/green or shadow deployments for major changes while running compatibility adapters.
• Automate client SDK generation from OpenAPI for major languages; publish to internal package registry with version mapping.

Communication & observability:

• Dedicated Slack channel + weekly sync; ACPs published in a Confluence page with migration checklist.
• Telemetry: log API version headers, consumer IDs; dashboards for legacy vs new usage; automated alerts when legacy usage drops below thresholds or when error rates rise after deploy.
• Post-mortem policy: any compatibility incident triggers a blameless PM and remediation within 48 hours.

Trade-offs:

• Extra short-term work (adapters, contracts) for long-term independence and fewer hotfixes.
• 90–120 day deprecation is a balance between consumer velocity and operational cost; may be shortened for internal-only tight teams.

Example flow (breaking change):

1. Team A files ACP describing MAJOR change + migration guide.
2. Steering Board approves; Team A implements change behind feature flag and compatibility adapter.
3. Consumers update contracts; provider CI runs consumer pacts — failures block merge and notify consumers.
4. Deploy canary; monitor telemetry for regressions.
5. Publish deprecation notice (90 days), auto-generated migration scripts and SDK update.
6. After threshold met, remove legacy code and increment MAJOR.

This model enforces technical safety (contracts, CI), clear timelines (deprecation), and lightweight governance to enable independent delivery with minimal friction.

Situation: We maintain a public backend API used by several product teams. We need a breaking change (contract change) and I own the technical plan to make it safe and low-impact.

Proposal & negotiation

1. Stakeholder map: list product owners, SDK teams, mobile/web teams, SRE, legal/security.
2. Draft a design doc: motivation, API diff, backwards compatibility matrix, migration path, timeline, risk assessment, test plan, rollback criteria. Share in an engineering RFC and schedule a 1-hour sync for feedback.
3. Negotiate: collect required features from consumers, agree on versioning strategy (major-versioned REST endpoint or header-based versioning + semantic versioning), finalize migration window and SLAs.

Compatibility & technical strategy

• Non-breaking-first: add new fields/endpoints where possible. If breaking, introduce v2 alongside v1.
• Use an adapter layer in the gateway to support both shapes for a transition period.
• Consumer-driven contract testing (Pact or equivalent) so teams can run provider/consumer tests in CI.
• Deprecation header and response warnings (Retry-After / Deprecation dates).

Testing & rollout

1. Implement feature in a branch with adapter + flag.
2. Automated tests: unit, integration, contract tests; end-to-end tests against staging; performance/load tests for both versions.
3. Canary rollout: enable v2 for small percentage of traffic (--gateway routing or header override). Observe metrics for 72 hours.
4. Dark launch (optional): route requests to v2 without exposing to users to validate telemetry.

Monitoring & rollback criteria

• Monitor: 5xx error rate, latency P95/P99, error types, business KPIs (conversion), consumer test failures, support tickets.
• Automatic rollback thresholds: >1% increase in 5xx over baseline for 15m, >20% increase in P99 latency, critical consumer test failure, or significant business KPI regression. Rollback via gateway config/feature-flag; run postmortem.

Communication plan

• Pre-release: RFC + migration guide + sample SDK patches + code snippets + quickstart in docs; announce timeline and migration deadline at least 8 weeks in advance; host migration office hours weekly.
• During rollout: daily status updates in a shared channel, runbook accessible, notify product owners of canary results.
• Deprecation: mark v1 deprecated in responses and docs, remind at 4w/2w/1w/1d intervals via email/Slack and API portal. Provide automated tooling to detect use of deprecated endpoints (analytics + alerts).

Deprecation & final cutover

• After migration window, run consumer check: if outstanding critical consumers exist, escalate; otherwise schedule final shutdown during low traffic with immediate rollback plan and support on standby.
• Keep archived read-only v1 logs for debugging; update SLA and billing if applicable.

Outcome & learnings

• Use versioning + gateway adapters and contract tests to minimize friction.
• Clear timelines, automation, and objective rollback criteria reduce risk and build trust with consumer teams.