API and Interface Design for Distributed Services Questions

Direct answer

Use consumer-driven contract testing: each consuming team publishes the exact requests and expected responses it actually depends on, and the provider team's own continuous integration (CI) pipeline verifies the real API against every published consumer contract before a change can merge or deploy. This inverts the usual failure mode, where a consumer discovers a break after the provider has already shipped; here the provider's own pipeline fails first, and the consumer team is notified automatically through the same system that stores the contracts, not by someone noticing something broke in production.

Structured elaboration

The contract and where it lives:

• Each consumer team writes a small set of interactions ("when I call this endpoint with this request, I expect a response shaped like this") and publishes them to a shared, versioned contract registry (a broker), tagged with which environment or release they apply to.
• This is deliberately narrower than a full schema: the consumer only asserts the exact fields and shapes it actually reads, so the contract reflects real usage rather than the provider's entire response shape.

Automating verification in the provider's CI:

• On every pull request to the provider's API code, a CI step fetches the latest contracts tagged for the relevant environment, spins up (or points at a running instance of) the provider, replays each consumer's recorded requests against it, and asserts the actual responses still satisfy what each consumer expects. Any mismatch fails the build and blocks the merge.
• Before an actual deploy (not just before merge), a second gate checks whether the version about to be deployed is compatible with whichever consumer versions are currently running in production, not just the newest consumer contract, since a provider change can be compatible with a consumer's latest contract while still breaking a consumer instance that hasn't picked up its own latest release yet.

How the affected team finds out:

• The contract broker's webhook posts directly to the consumer team's own channel (chat integration or issue tracker) whenever a provider verification against their specific contract fails, naming the exact interaction that broke and linking to a diff between what was expected and what the provider now returns. The team learns about the break from an automated, targeted message tied to their own contract, not from a shared build dashboard they'd have to be watching.

Worked example

A consumer's published contract (one interaction, in a contract-broker style format):

{
  "consumer": { "name": "checkout-web" },
  "provider": { "name": "orders-api" },
  "interactions": [
    {
      "description": "fetching an order by id",
      "request": { "method": "GET", "path": "/orders/500" },
      "response": {
        "status": 200,
        "body": { "id": "500", "status": "PAID", "total_cents": 4200 }
      }
    }
  ]
}

Provider CI step (illustrative, tool-agnostic pseudocode) that fails the build on a mismatch:

verify-contracts:
  script:
    - fetch-contracts --provider orders-api --tag production
    - contract-verify --provider-base-url http://localhost:8080 --contracts ./contracts/*.json
  # non-zero exit code on any interaction mismatch blocks the merge

Suppose a developer on the provider team renames total_cents to totalCents in a refactor. The verification step replays the recorded request, gets back a body missing the field the consumer's contract asserted, and the build fails with a message like: "interaction 'fetching an order by id' failed: expected field total_cents, field not present in actual response." The broker then posts that same message, with a link to the failing interaction, straight to the checkout-web team's channel, so they know about the incompatibility before it ever reaches a shared environment.

Trade-offs and pitfalls

• Contract testing only catches what a consumer actually asserted. If a consumer's published contract never checked the total_cents field in the first place, removing it silently passes; the technique is only as good as how faithfully each contract reflects real usage, which puts real responsibility on consumer teams to keep their contracts current as their own code changes.
• Verifying against the latest contract at merge time is necessary but not sufficient; gating the actual deploy against what is compatible with consumer versions currently running in production (not just their newest published contract) is what catches the case where two independently-safe changes combine badly because of deploy ordering.
• This approach is faster and cheaper than full end-to-end integration testing, but it is not a replacement for it: contract tests verify shape and basic behavior, not real-world concerns like load, timing, or business-logic correctness across a full user flow.
• A pitfall on the process side: if publishing and updating contracts is treated as optional or bolted on late, teams tend to let contracts drift stale, at which point the safety net silently degrades and nobody notices until a real break gets through anyway.

Direct answer

The two endpoints need genuinely different contracts because they fail differently. Sync inference has to answer inside the caller's request timeout, so its idempotency concern is narrow: protect against the caller re-sending a request when it can't tell whether the first one actually landed. Batch prediction runs long enough that the caller cannot hold a connection open for it, so the contract has to hand back a durable job handle immediately and give the caller a way to observe progress without blocking, and its idempotency concern is broader: protect against accidentally launching the same expensive job twice.

Structured elaboration

Endpoint 1: online single-request inference (synchronous)

POST /v1/models/{model_id}/infer

• Headers: Authorization: Bearer <token>, optional Idempotency-Key, Content-Type: application/json.
• The model is a pure function of its pinned version and input, so the computation is naturally idempotent; the Idempotency-Key exists purely to protect against duplicate execution when the caller retries after a timeout and can't tell if the first call succeeded. Server stores key -> response for a bounded window (documented, e.g., 24 hours) and replays the same response on a repeat with the same key and payload; a repeat with the same key but a different payload is a client error.

Endpoint 2: asynchronous batch prediction

POST /v1/models/{model_id}/batch-predict returns a job handle immediately (202 Accepted), never the result inline.

• Idempotency here is about job creation, not the underlying computation: if the caller retries the submit call (because the first response was lost, not because the job failed), the server must recognize the same Idempotency-Key and return the existing job_id rather than starting a second, redundant, possibly expensive job.
• Job status is observed with GET /v1/jobs/{job_id}, a plain state read, not a mutation, so it needs no idempotency key at all; polling it repeatedly is always safe by construction.
• Result delivery: the caller either polls the status endpoint until it reaches a terminal state, or supplies a webhook URL at submission time and the server pushes the terminal status once, with retries on delivery failure.

Error taxonomy shared by both endpoints:

Worked example

Sync inference request and response:

// POST /v1/models/churn-classifier/infer
{
  "model_version": "2026-05-01",
  "input": { "tenure_months": 14, "monthly_spend_cents": 3499, "support_tickets_90d": 2 }
}

// 200 OK
{
  "request_id": "req_a1f9",
  "model_version": "2026-05-01",
  "predictions": [{ "label": "churn", "score": 0.31 }]
}

Batch job submission, status, and completion, shown as a sequence so the polling contract is explicit:

sequenceDiagram
  participant C as Client
  participant A as API
  C->>A: POST /batch-predict (Idempotency-Key: b-7741)
  A-->>C: 202 Accepted, job_id=job_5820, status=queued
  loop poll every backoff interval
    C->>A: GET /jobs/job_5820
    A-->>C: 200, status=running, progress=40
  end
  C->>A: GET /jobs/job_5820
  A-->>C: 200, status=succeeded, result.output_location

// 202 Accepted (submit)
{ "job_id": "job_5820", "status": "queued", "submitted_at": "2026-07-18T15:00:00Z" }

// GET /jobs/job_5820 while running
{ "job_id": "job_5820", "status": "running", "progress": 40 }

// GET /jobs/job_5820 once finished
{
  "job_id": "job_5820",
  "status": "succeeded",
  "completed_at": "2026-07-18T15:04:00Z",
  "result": { "output_location": { "type": "s3", "uri": "s3://predictions/job_5820/" } }
}

A caller that supplied notify_webhook at submission time receives one POST of the same final body once the job reaches succeeded or failed, with retries on delivery failure, so it does not need to poll at all.

Trade-offs and pitfalls

• Retrying the sync endpoint blindly on every 5xx without an Idempotency-Key risks running the (cheap but not free) inference twice; retrying the batch submit endpoint blindly without one risks launching a genuinely expensive duplicate job. The stakes differ enough that batch submission should treat the key as effectively mandatory, not optional.
• Polling too aggressively on a long batch job wastes both client and server resources; the contract should hand the client a documented backoff hint (a growing poll interval) rather than leaving the client to guess, and the webhook path should be presented as the preferred option for anything longer than a few status checks.
• A common design mistake is putting the batch input/output data inline in the request body; for anything past a small payload, the contract should take a reference to storage (as shown) rather than embed the data, since inline batch payloads hit request-size limits and make retries expensive.
• 409 on a reused idempotency key with a different payload must be a hard error, not a silent overwrite. Silently accepting a changed payload under the same key is what causes a client bug (accidentally mutating the request before a retry) to produce a confusing, hard-to-reproduce result days later.

Split the API into a lightweight list endpoint that returns only preview fields (last message snippet, unread count, small avatar thumbnails) and a separate detail endpoint that returns the full thread, and never embed binary attachment data in either response. Attachments are represented only as metadata (id, type, size, a small thumbnail URL) in both responses; the actual file bytes are fetched through a separate, time-limited URL only when the user opens that specific attachment.

Endpoints

• GET /conversations: list view, returns an array of conversation previews.
• GET /conversations/{id}: thread detail, returns paginated messages plus attachment metadata only.
• GET /attachments/{attachmentId}/download: returns a short-lived signed URL for the actual file bytes; called only on explicit user action.

Field-selection strategy

Support a sparse-fieldset query parameter (e.g., ?fields=id,title,lastMessage,unreadCount) so a client that only needs to render a badge count is not forced to receive fields it will discard. For nested resources, support scoped includes (e.g., ?include=messages.attachments(fields=id,type,thumbnailUrl)) so the detail endpoint can be shaped per screen instead of always returning the maximal thread payload.

Why sparse fields instead of one endpoint per screen: a fixed "preview" and "full" shape covers the two screens named in the question, but a client roadmap eventually adds a third screen (for example a "shared media" view showing only attachments). Field selection generalizes without adding a new endpoint per screen. The trade-off is that the server must validate and whitelist which fields are selectable, so a buggy or malicious client cannot request an unindexed, expensive field on every request.

Attachment handling

Represent attachments as {id, type, sizeBytes, thumbnailUrl}, never inline binary. Thumbnails are themselves small pre-generated images served from a content delivery network (a CDN: a globally distributed cache of static assets placed close to the requesting client), not computed per request. The signed download URL is scoped to one attachment, expires quickly (minutes, not hours), and is issued only after the server re-checks the requesting user's access to that conversation, since a still-valid but stale URL should never become a durable bypass of authorization.

Worked example

List response for one conversation, everything a client needs to render a row:

{
  "conversations": [
    {
      "id": "c_9f2",
      "title": "Design Review",
      "lastMessage": { "id": "m_881", "snippet": "Sent the updated mocks", "sentAt": "2026-07-18T14:02:00Z" },
      "unreadCount": 3,
      "participants": [{ "id": "u_12", "name": "Rae", "avatarThumbUrl": "https://cdn.example.com/av/u_12_32.jpg" }]
    }
  ]
}

Detail response for the same conversation, messages included, attachments as metadata only:

{
  "id": "c_9f2",
  "messages": [
    {
      "id": "m_881",
      "senderId": "u_12",
      "body": "Sent the updated mocks, let me know if the spacing works",
      "attachments": [
        { "id": "a_44", "type": "image", "sizeBytes": 812000, "thumbnailUrl": "https://cdn.example.com/att/a_44_thumb.jpg" }
      ]
    }
  ],
  "nextCursor": "m_881"
}

The 812000-byte image is never present in this response, only its thumbnail URL string is. A client that opens the image calls GET /attachments/a_44/download, which returns { "url": "https://signed.example.com/a_44?exp=1752854520&sig=...", "expiresIn": 300 }, a URL valid for 300 seconds.

Trade-offs and pitfalls

Sparse fieldsets add server-side complexity (field whitelisting, and cache-key fragmentation, since ?fields=a,b and ?fields=b,a should normalize to the same cache entry) in exchange for one flexible endpoint instead of many narrow ones. A common wrong turn is returning attachment thumbnails as base64-encoded strings inside the JSON body "to save a round trip": this still bloats every list or detail response for every user, even ones who never open the attachment, and it defeats ordinary HTTP-level caching of the image separately from the JSON. Keep binaries out of JSON always, even small ones. Signed URLs need a short expiry to limit exposure if leaked (through logs or a shared screenshot), but too short an expiry causes broken images on slow connections; pairing a short-lived URL with a client-side retry-on-expired pattern (re-request the URL, then retry the download) resolves this without lengthening the exposure window. GraphQL is a real alternative for the field-selection problem, since the client literally specifies the shape in the query, but it moves cost control (query depth and complexity limits) onto the server and adds cache-key complexity per unique query shape; for two well-known screens like this, REST with sparse fields and a couple of named include recipes is usually the simpler production choice.

Direct answer

Design this as a small, versioned, resource-oriented contract: one endpoint for synchronous scoring, one for asynchronous batch scoring, and one for model metadata, each with an explicit JSON schema, a URL-based major version, bearer-token authentication scoped by tenant, and a structured error body so every client in every region gets the same shape whether the request succeeds or fails. The goal is to promise the client just enough that the model behind the endpoint can change without the promise changing.

Framework

Endpoints

Sample request/response

POST /v1/models/reco-v2/score
{
  "request_id": "b6b8f8f0-6e2b-4a35-9b2e-6b6a2b6e2b6a",
  "features": {
    "user_id": "u_123",
    "country": "US",
    "candidate_items": ["A", "B", "C"]
  },
  "options": { "top_k": 2 }
}

200 OK
{
  "request_id": "b6b8f8f0-6e2b-4a35-9b2e-6b6a2b6e2b6a",
  "model_version": "2026-05-15",
  "predictions": [
    { "item_id": "B", "score": 0.92 },
    { "item_id": "A", "score": 0.77 }
  ]
}

Authentication and authorization

• Bearer tokens: JSON Web Tokens (JWT), issued by a central identity service, carrying a tenant_id claim and scopes such as predict:read and predict:batch.
• Authorization: role-based access control (RBAC). The gateway checks the token's scopes against the endpoint being called, and the tenant_id claim against the model's owning tenant, before forwarding the request.
• Transport: mutual Transport Layer Security (mTLS) between regional callers and the gateway, so both sides authenticate each other at the connection level, in addition to the per-request bearer token.
• Tokens are short-lived and refreshed through an OAuth 2.0 client-credentials flow (a machine-to-machine token exchange where the calling service, not a human user, proves its own identity to obtain a fresh token), so a leaked token has a bounded blast radius.

Versioning strategy

• The major version lives in the URL (/v1/) because it is the version of the contract shape, what fields exist and what they mean, not the version of the model. The model's own version is a separate response field (model_version), since a model can change on its own schedule while the contract stays stable for months.
• Minor, backward-compatible changes (new optional fields) ship without a version bump.
• A breaking change to the contract shape gets a new major version, and the old version is marked deprecated once the new one is stable.

Error codes

{
  "error_code": "MISSING_FEATURE",
  "message": "features.user_id is required",
  "request_id": "b6b8f8f0-6e2b-4a35-9b2e-6b6a2b6e2b6a",
  "retryable": false
}

Quota is communicated, not enforced, at this layer: each response carries X-RateLimit-Limit and X-RateLimit-Remaining headers (a widely used convention, though not from a single ratified standard), and a caller that exceeds its quota gets 429 with a Retry-After value telling it how long to wait. Which limiting algorithm sits behind that response is an implementation detail of the service, not part of the contract a client codes against.

Backward compatibility

• Additive only within a major version: new optional request fields, new optional response fields. Clients are expected to ignore fields they do not recognize (a tolerant reader).
• A field's meaning or type is never repurposed within a version.
• An OpenAPI document (a machine-readable API description) is published per major version, so client libraries can be generated and diffed automatically against it.

Worked example

A concrete, backward-compatible evolution: version 1's response is predictions: [{item_id, score}]. A later minor release adds an optional explanation field without bumping the version:

{
  "request_id": "b6b8f8f0-6e2b-4a35-9b2e-6b6a2b6e2b6a",
  "model_version": "2026-06-01",
  "predictions": [ { "item_id": "B", "score": 0.92 } ],
  "explanation": { "method": "shap", "top_features": ["country", "recency"] }
}

Old clients that only read predictions are unaffected. New clients that know to read explanation get more detail. This is what "additive, optional field" means in practice, and it is why either side treating the field's absence as an error would be a bug, not a feature.

Trade-offs and pitfalls

• Keeping model version only in the response makes it hard for a client to pin an exact model for reproducible results. If that matters, add an optional model_version request field and return 409 Conflict if the pinned version no longer exists.
• Tying authorization purely to a coarse scope like predict:batch is simple but blunt. If a tenant later needs per-model access control, retrofitting that into the token shape is a breaking change, so it is worth reserving a model-scoped claim from the start even if it is unused initially.
• A common pitfall is treating an async batch job as "the same request, just later." Batch has different failure semantics, partial success across many rows, that a single success/failure flag cannot represent, so the job-status endpoint needs its own richer schema with per-row outcome counts, not just a status string.

Direct answer

Anchor the pagination cursor to a specific item's identity in the ordering, never to a position count, so an insert at the head cannot shift what "the next page" means. Give the client explicit control over when new head items enter its view (a banner it opts into, not a silent reflow), and dedupe every incoming page against a client-side set of already-seen IDs so a race between a live push and a page fetch can never render the same item twice. This generalizes past feeds to any high-churn ordered list built on top of a mutable store.

Why offset-based paging breaks here

Offset pagination asks "give me the rows starting at position N," where position is counted fresh on every request. If a new row lands at the head between two calls, every existing row's position shifts by one: the client's next request for "position N" now returns a row it already saw (duplicate) while the row that used to sit at position N slides out of view entirely (skip). Cursor pagination instead asks "give me the rows after this specific row," so a head insert changes nothing about what "after this row" means.

Cursor contract and handling head inserts

• The cursor encodes the last-seen row's ordering key (for example created_at plus id as a tie-breaker) and is treated as opaque by the client.
• Real-time inserts arrive over a separate channel (stream or websocket) carrying the full item payload. The client never merges them straight into the currently rendered list; it buffers them and shows a "N new" affordance the user can tap.
• When the client does fetch the next page, it filters incoming rows against a local set of IDs it has already rendered (from either the page fetch or the live stream) before appending, so a row delivered by both paths only renders once.
• If the anchor row referenced by a cursor is later deleted, the server cannot resolve "after this row" any more. Signal that explicitly (for example a 410 Gone-style error on that specific cursor) so the client knows to discard its cursor and reload the current view, rather than the server silently guessing a nearby row.

Worked example

Assume feed posts are labeled by creation order, oldest to newest: p1 ... p9 already exist when the client loads page one, and p10 arrives from the live stream while the client is browsing.

sequenceDiagram
    participant C as Client
    participant F as Feed API
    participant R as Realtime stream
    C->>F: GET /feed?cursor=null&limit=2
    F-->>C: items=[p9,p8], next_cursor=after:p8
    R-->>C: push new item p10 (inserted at head)
    Note over C: buffer p10, show "1 new" banner, do not reorder current page
    C->>F: GET /feed?cursor=after:p8&limit=2
    F-->>C: items=[p7,p6], next_cursor=after:p6
    Note over C: dedupe against seen-id set before appending

p10's arrival never touches the cursor after:p8: it still means exactly what it meant before the insert, so the second page correctly returns [p7, p6] with no repeat of p9 or p8 and no skip past p7.

Trade-offs and pitfalls

Cursor pagination trades away random access (you cannot jump straight to "page 7") for stability under churn, which is almost always the right trade for a live feed. A common mistake is auto-merging live-pushed items straight into the rendered list the instant they arrive: this yanks the scroll position under the user's finger and is a worse experience than a controllable banner, even though it feels "more real-time."

If filters or sort order change mid-session, the old cursor and seen-ID set are no longer meaningful and must be discarded, not reused; treat that as a fresh pagination session. For most feed use cases, accepting eventual consistency (the client may briefly be a few items behind, reconciled by the next page fetch or a manual refresh) is the right call over paying for a strict per-client consistent snapshot, which adds real operational cost for a benefit most users never notice.