API and Interface Design for Distributed Services Questions

Direct answer

Frame the decision around the actual constraints rather than the protocols' reputations: which languages across the polyglot stack need first-class client support, whether the traffic is latency-sensitive service-to-service calling or something a human needs to inspect, and whether streaming is a hard requirement. Build a small prototype for the leading candidate against a real workload, compare it structurally rather than by a single benchmark number, and validate the decision with the teams who will own the resulting code before committing company-wide.

Evaluation criteria

• Wire efficiency: gRPC serializes with protocol buffers, a binary format that omits field names and uses variable-length integers, so a given message is typically meaningfully smaller on the wire than the equivalent JSON; REST/JSON is larger but human-readable without any tooling.
• Connection behavior: gRPC runs over HTTP/2, which multiplexes many concurrent requests over a single connection; this avoids head-of-line blocking, the delay where one slow request stuck at the front of a connection holds up every other request queued behind it, a problem plain HTTP/1.1 REST can suffer under high concurrency.
• Streaming support: gRPC has first-class client-streaming, server-streaming, and bidirectional streaming built into the protocol; REST needs bespoke chunking or a separate protocol (such as server-sent events or websockets) bolted on.
• Language and tooling coverage across the actual stack: check every language you run in production, not just the popular ones. gRPC's code generation and runtime maturity vary meaningfully by language.
• Debuggability: a REST/JSON payload is directly inspectable with a browser or curl; gRPC needs a reflection service and a tool like grpcurl to get the same visibility.
• Operational and observability tooling: both integrate with modern tracing systems, but gRPC's HTTP/2 transport sometimes needs extra configuration for proxies and load balancers that were written assuming HTTP/1.1.

What I would benchmark, and how

Build matching prototype services in the two or three languages that make up most of the real traffic, exercise both protocols with the same synthetic load generator and connection count, and compare relative, structural behavior (does one need meaningfully less CPU per request at the same throughput, does one handle the streaming case directly versus requiring a workaround) rather than quoting a single absolute number as if it generalizes. Any specific latency or throughput figure from a single environment is a snapshot of that environment, not a portable fact, so treat it as directional evidence for this specific decision rather than a reusable benchmark.

Worked example

Situation: a polyglot environment (Python data workers, Go services, Java-based batch jobs) needed a new internal path for high-throughput data ingestion. The requirement list included sustained streaming from Go into Python, and low overhead at high request volume. Action: scored REST and gRPC against the criteria above, built matching prototype services in Go and Python, ran both through the same load generator, and reviewed the comparative results with the two teams that would own the resulting code day to day. gRPC's built-in server-streaming matched the ingestion use case directly, while REST would have needed a bespoke chunked-transfer workaround; that single structural fit mattered more in the final call than any specific number either prototype produced. Result: adopted gRPC for the internal high-throughput and streaming paths, kept REST/JSON for a small admin API where human debuggability mattered more than raw efficiency, and put a translating gateway in front so a team that could not migrate immediately was not blocked.

Trade-offs and pitfalls

gRPC has a steeper learning curve than REST, and its debugging story is worse out of the box: budget time for a reflection service and team familiarity with grpcurl rather than assuming curl will still work. Verify gRPC client maturity for every language actually in your stack before committing, not just the two or three most common ones; an internal tool with weak gRPC support becomes a maintenance tax nobody accounted for at decision time.

REST remains the better default for public or third-party-facing APIs, where you cannot force external consumers to adopt a specific client library the way you can for an internal service. The most common mistake in this kind of evaluation is running a benchmark in one environment on one payload shape and generalizing that result to the whole company without a real pilot: always validate with a scoped rollout to one or two teams before treating a prototype's numbers as proof for every future use case.

Direct answer

Version the payload schema so that most evolution never requires a new API version at all: additive, optional-field changes ship into the existing version, and only a genuinely breaking change (removing or renaming a field, changing a type, changing meaning) earns a new major version. Communicate and execute deprecation through machine-readable signals, not just documentation and email, so that automated partner integrations, not just the humans who read the changelog, find out a version is going away and when.

Structured elaboration

Classifying changes so most of them don't need a version bump:

Rollout stages for a genuinely breaking change:

1. Preview. The new major version is available opt-in (a new URI path or an explicit request header) so partners can test against it before it is the default.
2. Dual-run. Both versions are served in production for a defined overlap window; usage of the old version is measured per partner so the team knows who still needs to migrate, not just that traffic to it is declining in aggregate.
3. Deprecation announced. The still-serving old version starts returning a Deprecation header on every response (standardized in RFC 9745, March 2025, on the IETF Standards Track) carrying the date deprecation took effect, plus a Sunset header (RFC 8594, Informational status) carrying the exact date the version will stop serving entirely. The Deprecation header's Standards Track status is actually the stronger of the two. Both are machine-readable, so a partner's own monitoring can alert on them without a human reading a changelog.
4. Sunset. After the Sunset date passes, the old version stops serving.

410 versus 404 after sunset, the definitional point: once a version is actually sunset, the correct response is 410 Gone, not 404 Not Found. 404 means "nothing is here," which is ambiguous to a caller debugging a broken integration, it could be a typo in the URL just as easily as a real removal. 410 says explicitly "this used to exist, on purpose, and it is not coming back," which is exactly the information a caller needs to know it must migrate rather than fix a typo. The 410 response body should still explain what happened and where to go next.

Worked example

While the old version is deprecated but still serving:

HTTP/1.1 200 OK
Deprecation: Wed, 01 Jul 2026 00:00:00 GMT
Sunset: Thu, 31 Dec 2026 23:59:59 GMT
Link: <https://api.example.com/v2/ingest>; rel="successor-version"

After the sunset date has passed:

HTTP/1.1 410 Gone
Content-Type: application/json

{
  "error": {
    "code": "endpoint_sunset",
    "message": "API v1 was sunset on 2026-12-31. Migrate to /v2/ingest; see the migration guide linked in this response.",
    "request_id": "req_9f21"
  }
}

The dates in the headers and the body agree, so a partner's monitoring, its on-call engineer reading a log line, and its API client's own error message all tell the same story.

Trade-offs and pitfalls

• Relying only on a changelog and email is the most common failure mode; automated partner integrations do not read email, and by the time a human notices the deprecation notice it may already be close to the sunset date. Machine-readable headers close that gap.
• Returning 404 for a sunset endpoint (instead of 410) is a small-looking choice with a real cost: it makes "this API version is gone on purpose" indistinguishable from "you mistyped the URL," which slows down every partner's own debugging.
• Dual-running two versions is real, ongoing engineering cost (two schemas, two sets of compatibility tests, two things that can each break), not a one-time expense; the overlap window needs a firm end date from the start, or "temporary" dual-running quietly becomes permanent.
• A subtler pitfall: bumping the major version for a change that could have been additive (for example, renaming a field instead of adding a new one alongside the old, deprecated one) creates breaking-change churn that erodes partner trust in the version number actually meaning something.

Expose two endpoint shapes, one for time-bucketed aggregated counts and one for top-k lists, and never let the client request an ungoverned ad-hoc aggregation. Every response carries an explicit freshness field (an asOf timestamp and a precision flag of exact or approximate) so the client-visible contract, not just the internal pipeline, states what freshness the caller is actually getting; the backend may route a request to a fast precomputed view or a slower exact path, but that routing choice is invisible to the caller except through this field.

Endpoints

• GET /metrics/counts?metric=events&start=...&end=...&granularity=hour&dim=region: a time series of pre-aggregated buckets.
• GET /metrics/top?metric=clicks&k=10&start=...&end=...&dim=category: a ranked top-k list.
• Both accept a freshness query parameter as a request-side contract: freshness=fast (accept approximate or slightly stale data, get the lowest latency) versus freshness=exact (route to an exact recompute, higher latency, and the contract documents a range cap or async fallback rather than letting the request hang indefinitely for a huge range).

The staleness contract, in the response itself

Every response includes asOf (the timestamp through which data is complete) and precision. This is the part of the design that belongs squarely to an API-contract discussion: it turns an internal trade-off (how fresh is the backing view) into an explicit, documented, client-visible field, so a caller building a dashboard can show an "as of 2 minutes ago" badge instead of silently trusting a number that might be stale.

Bounding and shaping the response

k is capped server-side (for example, a maximum of 100) regardless of what the client requests, and the response documents the cap via requestedK and returnedK, so a client cannot silently receive a truncated list without knowing it was truncated. The response is already the final shape the dashboard renders, bucketed counts or a ranked list; the client never receives raw event rows and aggregates them itself. That is the same overfetch-avoidance principle as any other read-heavy endpoint: the API promises a specific, small, pre-shaped payload, not a firehose the client has to post-process.

Worked example

GET /metrics/counts?metric=events&start=2026-07-18T00:00Z&end=2026-07-18T03:00Z&granularity=hour&freshness=fast

{
  "metric": "events",
  "granularity": "hour",
  "asOf": "2026-07-18T03:00:42Z",
  "precision": "approximate",
  "buckets": [
    { "start": "2026-07-18T00:00Z", "count": 48210 },
    { "start": "2026-07-18T01:00Z", "count": 51330 },
    { "start": "2026-07-18T02:00Z", "count": 49980 }
  ]
}

GET /metrics/top?metric=clicks&k=5&start=2026-07-18T00:00Z&end=2026-07-18T03:00Z&dim=category&freshness=fast

{
  "metric": "clicks",
  "asOf": "2026-07-18T03:00:42Z",
  "precision": "approximate",
  "requestedK": 5,
  "returnedK": 5,
  "items": [
    { "key": "footwear", "count": 9120 },
    { "key": "outerwear", "count": 7040 },
    { "key": "accessories", "count": 5210 },
    { "key": "electronics", "count": 4880 },
    { "key": "home", "count": 3990 }
  ]
}

Trade-offs and pitfalls

The biggest interview-relevant trade-off is that offering freshness=exact at all is a contract commitment: once a caller can request it, some caller eventually will, on a huge date range, and the API needs a documented, bounded answer (a hard range cap, a queued and pollable async job, or an explicit rejection) rather than an unbounded query that degrades the whole service. A common wrong turn is exposing a single endpoint that "just returns the data" with no freshness or precision field, leaving the client to guess whether a number is authoritative; once a dashboard has shown a number without a staleness caveat, a later "actually that was approximate" correction reads as the API being wrong, when the real defect was an underspecified contract. Enumerating allowed dim values, rather than accepting any column name, trades flexibility for the ability to document, cap, and index every supported query shape; a genuinely open-ended analytics need should be pointed at a dedicated data-warehouse query interface, not bolted onto this API.

Use keyset (cursor-based) pagination anchored to a point-in-time snapshot, not offset pagination. Take a snapshot marker at the start of the export (a monotonically increasing watermark, e.g., the highest committed transaction id at that instant), then page through rows ordered by an indexed key while filtering to "visible as of that watermark." This guarantees a stable, resumable page sequence: rows inserted after the export starts are invisible to it, and rows deleted after the watermark are still included as long as deletes are soft (tombstoned) rather than hard.

Why offset pagination fails here

OFFSET/LIMIT re-executes the query each page and counts rows from the start of the current result set. If a row is inserted or deleted before the current offset between page fetches, every row after it shifts by one position, which causes a row to be skipped or repeated on the next page. At billions of rows spread across millions of pages, this is not a rare edge case, it is the default outcome of any concurrent write during the job.

The snapshot marker

Pick one of:

• Database MVCC (multi-version concurrency control: the database keeps multiple versions of a row and gives each long-running transaction a consistent view of the data as of when that transaction started) snapshot, if the database supports it natively (e.g., a Postgres REPEATABLE READ transaction).
• A manual high-water mark (HWM): record max_committed_tx_id (or a reliable updated_at ceiling) at export start. Every subsequent page query filters committed_tx <= HWM.

Keyset pagination mechanics

Order by an indexed, unique, monotonic key (id ASC is simplest). Each page's query is WHERE id > :last_seen_id AND committed_tx <= :hwm ORDER BY id LIMIT :page_size. The export job persists last_seen_id after every page, so a crash or client disconnect resumes exactly where it left off with no re-scan of already-shipped rows.

Handling deletes and updates mid-export

A hard DELETE during the export can remove a row your watermark says should still be visible. Two ways to handle it:

1. Soft-delete with a deleted_at column and a delete_tx watermark, so the export can still include "not yet deleted as of my snapshot" rows even if the row is physically gone by the time a later page executes.
2. Accept a documented limitation that a delete racing the read of that specific row is out of scope, if the database's own MVCC already handles this transparently.

Two-phase export for "as of now, plus anything that lands during the job"

Phase 1 runs the consistent snapshot export described above. Phase 2 tails a change stream (or re-runs the export with a new, later watermark) for anything committed after the phase-1 watermark, deduplicated by primary key when merged with phase 1's output.

Worked example

A table has ids 1 through 12. The export starts with watermark tx_max=500 and page size 4.

• Page 1: WHERE id > 0 AND committed_tx <= 500 ORDER BY id LIMIT 4 returns rows 1,2,3,4. The job persists cursor last_id=4.
• Between page 1 and page 2, row 2 is soft-deleted at delete_tx=510, and a new row 13 is inserted at tx=505.
• Page 2: WHERE id > 4 AND committed_tx <= 500 ORDER BY id LIMIT 4 returns rows 5,6,7,8. Row 13 is invisible because its tx (505) is greater than the watermark (500). Row 2 already shipped in page 1 and stays correctly included, because its delete happened after the watermark was taken: the filter checks deleted_at IS NULL OR delete_tx > 500, and 510 > 500, so row 2 still counts as "present as of the snapshot."
  This shows the property that actually matters: whether row 2 belongs in the export is decided once, at watermark time, and never re-evaluated per page.

Operational impacts

Also persist job metadata (last_id, watermark, rows_exported_count) so a restart resumes cleanly, and offload the scan to a read replica to avoid competing with production traffic, confirming the replica's replication lag is smaller than your acceptable staleness window before paging starts.

Trade-offs and pitfalls

Offset pagination is simpler to implement but silently produces duplicates or gaps under concurrent writes, it should only be mentioned to explain why it is rejected, never offered as a real option here. A long-lived MVCC transaction is easy to wire up but can block cleanup on the primary if the export runs for hours; a manual watermark avoids that but requires the schema to already carry transactional or monotonic timestamp metadata, which is a real prerequisite, not something you get for free. Soft-deletes solve visibility-at-watermark cleanly but only if the application already uses soft-deletes everywhere the export touches; retrofitting that late in a large system is expensive. The most common wrong turn in an interview room is proposing "just take a snapshot" without wiring the resulting cutoff into every subsequent page query, that restates the goal instead of solving it.

Direct answer

Model each query as a durable resource with an explicit id and a small state machine, not as something the API tries to hold open. The client submits a query and gets back a job_id immediately; every other verb (status, partial results, cancel) operates on that id. Persist just enough to answer "what state is this job in and what has it produced so far" without persisting more than needed, bound concurrency at submission time as a documented quota rather than an invisible queue, and garbage-collect only jobs that are both in a terminal state and confirmed idle, never on a timer alone.

Structured elaboration

Endpoints:

• POST /queries submits a query, returns 202 Accepted with a query_id and initial state queued.
• GET /queries/{id} returns current state, progress, and (once available) a pointer to partial or final results.
• GET /queries/{id}/results?cursor=... streams results in pages; safe to call repeatedly, a pure read.
• POST /queries/{id}/cancel requests cancellation; returns immediately, cancellation is best-effort, not a guarantee the job stops mid-instruction.

Job state model, as a state machine the API exposes to the caller:

stateDiagram-v2
  [*] --> queued
  queued --> running
  running --> partial: first result page ready
  partial --> partial: more pages
  partial --> succeeded
  running --> succeeded
  running --> failed
  running --> canceled: cancel accepted
  partial --> canceled: cancel accepted
  succeeded --> [*]
  failed --> [*]
  canceled --> [*]

What gets persisted (durable store, e.g. a relational table, not just in-memory):

• query_id, user_id, submitted query text/params, state, progress (rows or bytes processed), created_at, last_heartbeat_at, result_manifest (list of completed result-page locations in object storage), expires_at.
• Result data itself does not belong in the same store as job metadata: pages are written to object storage as the job produces them, and the job row only tracks pointers (a manifest), so the metadata store stays small and fast to query regardless of result size.

Bounding concurrency per user, as a contract, not just an internal limiter:

• At submission time, the server checks the caller's currently-running/queued job count against a documented per-user limit (for example, a fixed number of concurrent queries per account tier). Over the limit, the API returns 429 with a Retry-After value and a body explaining which quota was hit, rather than silently queueing the request indefinitely. This gives the caller something to program against instead of guessing why the request never seems to make progress.

Streaming partial results:

• Once the query has produced at least one page, state moves to partial, and GET /queries/{id}/results returns whatever pages exist plus a cursor for the next page, the same pagination shape whether the job is still running or already succeeded. A client can start rendering rows well before the whole query finishes.

Cancellation:

• POST /queries/{id}/cancel is asynchronous and best-effort: the state moves to a transient "cancel-requested" condition, the worker checks for it between chunks of work and stops, and the final state the client observes could still legitimately be succeeded if the job finished before it noticed the cancellation. The contract should say this plainly rather than pretend cancel is instantaneous.

Safe garbage collection without touching running jobs:

• Only terminal-state jobs (succeeded, failed, canceled) are eligible for cleanup, and only once expires_at has passed AND last_heartbeat_at shows no recent activity, so a job that looks terminal in the metadata store but has a worker still flushing a last results page is not deleted out from under it.
• Cleanup removes the object-storage pages referenced by the manifest and then the metadata row, in that order, so a crash mid-cleanup leaves an orphaned-but-harmless metadata row rather than a manifest pointing at deleted data.
• expires_at is returned to the caller in the job status response, so a client polling late can see its results are about to disappear instead of getting a surprise 404.

Worked example

Submit and inspect a query:

// POST /queries
{ "sql": "SELECT region, SUM(revenue_cents) FROM orders WHERE order_date >= '2026-06-01' GROUP BY region", "priority": "normal" }

// 202 Accepted
{ "query_id": "q_5f31", "state": "queued", "submitted_at": "2026-07-18T10:00:00Z" }

// GET /queries/q_5f31 (mid-run)
{
  "query_id": "q_5f31",
  "state": "partial",
  "progress_pct": 55,
  "result_pages_ready": 2,
  "expires_at": "2026-07-20T10:00:00Z"
}

// GET /queries/q_5f31/results?cursor=page_2
{
  "rows": [
    { "region": "us-east", "revenue_cents": 48291000 },
    { "region": "us-west", "revenue_cents": 30112500 }
  ],
  "next_cursor": "page_3",
  "final": false
}

Once state becomes succeeded, the same results endpoint keeps working with final: true on the last page, so the client's polling and rendering code does not change between "still running" and "done."

Trade-offs and pitfalls

• A common mistake is coupling the results endpoint's shape to whether the job is done, forcing the client to special-case "partial vs final" results differently; keeping one paginated results contract regardless of job state avoids that branch entirely.
• Treating cancel as synchronous and guaranteed is a pitfall: a distributed worker may be mid-chunk when the cancel request arrives, so the honest contract is "cancellation requested, best-effort, check final state," not "cancelled means stopped immediately."
• Garbage collection racing a slow-to-report worker is the sharpest failure mode here: a heartbeat check in addition to a TTL is what prevents deleting a job's output while a worker is still writing to it, a pure clock-based TTL is not sufficient on its own.
• Per-user concurrency enforced only at submission time (not re-checked mid-run) is simpler but means a user who is throttled cannot "sneak in" more work by resubmitting after their first job finishes just below the limit; that is a deliberate simplicity trade-off worth naming rather than over-engineering a live-recheck.