API and Interface Design for Distributed Services Questions

Direct answer

Frame the decision around the actual constraints rather than the protocols' reputations: which languages across the polyglot stack need first-class client support, whether the traffic is latency-sensitive service-to-service calling or something a human needs to inspect, and whether streaming is a hard requirement. Build a small prototype for the leading candidate against a real workload, compare it structurally rather than by a single benchmark number, and validate the decision with the teams who will own the resulting code before committing company-wide.

Evaluation criteria

• Wire efficiency: gRPC serializes with protocol buffers, a binary format that omits field names and uses variable-length integers, so a given message is typically meaningfully smaller on the wire than the equivalent JSON; REST/JSON is larger but human-readable without any tooling.
• Connection behavior: gRPC runs over HTTP/2, which multiplexes many concurrent requests over a single connection; this avoids head-of-line blocking, the delay where one slow request stuck at the front of a connection holds up every other request queued behind it, a problem plain HTTP/1.1 REST can suffer under high concurrency.
• Streaming support: gRPC has first-class client-streaming, server-streaming, and bidirectional streaming built into the protocol; REST needs bespoke chunking or a separate protocol (such as server-sent events or websockets) bolted on.
• Language and tooling coverage across the actual stack: check every language you run in production, not just the popular ones. gRPC's code generation and runtime maturity vary meaningfully by language.
• Debuggability: a REST/JSON payload is directly inspectable with a browser or curl; gRPC needs a reflection service and a tool like grpcurl to get the same visibility.
• Operational and observability tooling: both integrate with modern tracing systems, but gRPC's HTTP/2 transport sometimes needs extra configuration for proxies and load balancers that were written assuming HTTP/1.1.

What I would benchmark, and how

Build matching prototype services in the two or three languages that make up most of the real traffic, exercise both protocols with the same synthetic load generator and connection count, and compare relative, structural behavior (does one need meaningfully less CPU per request at the same throughput, does one handle the streaming case directly versus requiring a workaround) rather than quoting a single absolute number as if it generalizes. Any specific latency or throughput figure from a single environment is a snapshot of that environment, not a portable fact, so treat it as directional evidence for this specific decision rather than a reusable benchmark.

Worked example

Situation: a polyglot environment (Python data workers, Go services, Java-based batch jobs) needed a new internal path for high-throughput data ingestion. The requirement list included sustained streaming from Go into Python, and low overhead at high request volume. Action: scored REST and gRPC against the criteria above, built matching prototype services in Go and Python, ran both through the same load generator, and reviewed the comparative results with the two teams that would own the resulting code day to day. gRPC's built-in server-streaming matched the ingestion use case directly, while REST would have needed a bespoke chunked-transfer workaround; that single structural fit mattered more in the final call than any specific number either prototype produced. Result: adopted gRPC for the internal high-throughput and streaming paths, kept REST/JSON for a small admin API where human debuggability mattered more than raw efficiency, and put a translating gateway in front so a team that could not migrate immediately was not blocked.

Trade-offs and pitfalls

gRPC has a steeper learning curve than REST, and its debugging story is worse out of the box: budget time for a reflection service and team familiarity with grpcurl rather than assuming curl will still work. Verify gRPC client maturity for every language actually in your stack before committing, not just the two or three most common ones; an internal tool with weak gRPC support becomes a maintenance tax nobody accounted for at decision time.

REST remains the better default for public or third-party-facing APIs, where you cannot force external consumers to adopt a specific client library the way you can for an internal service. The most common mistake in this kind of evaluation is running a benchmark in one environment on one payload shape and generalizing that result to the whole company without a real pilot: always validate with a scoped rollout to one or two teams before treating a prototype's numbers as proof for every future use case.

Direct answer

Split the contract into two endpoints threaded together by one opaque sync token: a push endpoint that accepts the client's local mutations tagged with the server version they were based on (so the server can detect conflicts), and a paginated pull endpoint that returns only the deltas since that token plus a new token to resume from. The client always pushes before it pulls, so its own pending changes are reflected in the server state it is about to reconcile against.

Contract shape

Push (client to server), sends local mutations:

POST /sync/push
{
  "clientId": "device-abc",
  "baseSyncToken": "tkn-123",
  "mutations": [
    {"localId": "c1", "type": "update", "resource": "note", "id": "srv-45", "baseVersion": 77, "payload": {"title": "New title"}}
  ]
}

Push response, tells the client what happened to each mutation:

{
  "applied": [
    {"localId": "c1", "serverId": "srv-45", "status": "applied", "serverVersion": 78}
  ],
  "conflicts": [],
  "newSyncToken": "tkn-124"
}

Pull (server to client), paginated deltas since the token:

GET /sync/pull?since=tkn-124&pageToken=null
{
  "items": [
    {"serverId": "srv-46", "resource": "note", "op": "upsert", "payload": {"title": "Meeting notes"}, "version": 12}
  ],
  "nextPageToken": null,
  "newSyncToken": "tkn-125"
}

Conflict signaling

Every mutation in the push request carries baseVersion, the server version the client last saw for that resource. The server compares it to the resource's current version before applying:

Whichever strategy is chosen, baseVersion is what makes the push idempotent and safe to retry after a network drop: replaying the same push twice against an already-applied localId returns the same applied result rather than double-applying it.

Worked example

sequenceDiagram
    participant C as Mobile client
    participant S as Sync API
    C->>S: POST /sync/push {baseSyncToken, mutations}
    S-->>C: {applied, conflicts, newSyncToken}
    C->>S: GET /sync/pull?since=newSyncToken&pageToken=null
    S-->>C: {items, nextPageToken=abc, newSyncToken}
    C->>S: GET /sync/pull?since=newSyncToken&pageToken=abc
    S-->>C: {items, nextPageToken=null, newSyncToken}
    Note over C: pull loop ends when nextPageToken is absent

Tracing the token through the exchange above: the push response's newSyncToken (tkn-124) becomes the since value on the first pull; each pull response echoes back the same newSyncToken (it only advances once a full pull pass completes) until nextPageToken comes back absent, which is the client's signal to stop paging and consider itself caught up as of that token.

Trade-offs and pitfalls

Deletes need a tombstone record (a marker saying "this resource was removed," not just its absence) so a client pulling deltas can distinguish "never existed" from "existed then got deleted." Tombstones cannot be kept forever: they need a retention window and a background cleanup pass that garbage-collects (permanently purges) tombstones older than the window, after which a client that reconnects past that window can no longer compute a delta and must fall back to a full resync instead of a partial pull.

A client that reconnects after a very long offline period is the sharpest edge case: if its baseSyncToken predates the server's retained history, the pull endpoint must detect that explicitly (rather than silently returning an incomplete delta) and tell the client to discard local state and re-fetch a full snapshot.

Monotonic version numbers per resource are enough for the common single-writer-per-record case. True multi-device, multi-writer scenarios (the same resource edited concurrently from two devices before either has synced) need richer causality tracking than a single version number can express; that machinery belongs to your consistency model, not this contract, so treat it as a known limitation to flag rather than something to solve inside the sync endpoints themselves.

Direct answer

Model the collection with a resource-oriented URL (GET/POST /v1/products), cursor pagination for listing, structured filter and sort query parameters, and a URI-path major version (/v1/). To roll out a breaking change without a hard cutover, let the old and new response shapes run side by side under different version paths, keep the new field additive first, and retire the old version only once usage telemetry shows it is safe, not on a calendar date alone.

Framework

Endpoints and shapes

GET /v1/products?limit=2&sort=price:asc
200 OK
{
  "data": [
    { "id": "p1", "name": "Widget", "price": 9.99, "status": "active" },
    { "id": "p2", "name": "Gadget", "price": 14.50, "status": "active" }
  ],
  "page": { "limit": 2, "next_cursor": "<opaque-cursor-for-p2>", "has_more": true }
}

POST /v1/products
{ "name": "New Widget", "price": 19.99, "category": "tools" }

201 Created
Location: /v1/products/p3
{ "id": "p3", "name": "New Widget", "price": 19.99, "category": "tools", "status": "active" }

Pagination strategy: cursor-based, for the same reason as any list that can grow and be written to concurrently: stable under inserts, roughly constant query cost at depth, at the cost of not supporting a direct "jump to page 43." Offer limit and an opaque after cursor rather than page/offset.

Filtering and sorting: plain query parameters for common fields (status=active, category=tools), and a single sort=field:direction parameter, comma-separated for multiple fields, for example sort=price:asc,name:desc. The filterable field vocabulary is documented per field in the OpenAPI spec (a machine-readable API description) so clients know exactly what is filterable rather than guessing.

Versioning: the major version lives in the URL path (/v1/), because it is simple to route at a gateway or load balancer without inspecting headers, and it is trivially cacheable per version. Additive changes, new optional fields, new optional query parameters, ship inside v1 without a version bump; anything that removes a field, changes a field's type, or changes default sort or filter behavior goes into v2.

Rolling out a breaking change without a hard cutover: suppose v2 needs to rename price to unit_price and change it from a decimal amount to an integer number of cents.

1. Ship v2 alongside v1 on the same deployment. v2's handler reads and writes the same underlying data as v1, so there is exactly one source of truth behind two response shapes.
2. Mark v1's price field deprecated via a response header and a documented sunset date, while it keeps working exactly as before. Nothing breaks yet.
3. Instrument both versions: tag every request with which version served it, and track the fraction of traffic still hitting v1.
4. Migrate internal clients first, since you can coordinate with them directly, then notify external and partner clients with a fixed migration window, for example 90 days, pointing at the v2 docs and a short code sample.
5. Remove v1 only once telemetry shows its remaining traffic has dropped to a level you have explicitly decided is safe to force-migrate, for example only a handful of clients you can contact individually, not on the calendar date alone. If a significant client is still on v1 at the deadline, extend the window rather than break them, and treat that as a signal the migration tooling or communication needs work.

Worked example

Concretely, at the moment v2 ships, v1 carries 100% of traffic. After internal clients migrate in the first month, v1's share drops to 70%. Announcing the 90-day window to external partners brings it down further; by day 60, v1 is at 8%, all from three named partner integrations already contacted directly about their remaining migration steps. That 8%-and-named state, not the passage of 60 days on its own, is what tells you it is close to safe to set a hard removal date, once those three integrations confirm.

Trade-offs and pitfalls

• Renaming price to unit_price and changing its unit, dollars to cents, in the same release conflates two changes into one migration. A client that only cared about the rename still has to handle the unit change, which raises the odds the migration is done wrong. Prefer landing one breaking change at a time when volume allows it.
• A common pitfall is announcing deprecation only in documentation and not in the response itself. Clients that never read a changelog will not notice until the sunset date arrives, so a machine-readable response header, not prose, is what actually drives safe removal.
• Cursor pagination combined with sort needs care: the opaque cursor usually encodes the sort key's value, so changing sort mid-pagination, fetching page 1 by price then asking for page 2 by name, should be rejected or restarted from page 1, since a cursor from one sort order is meaningless under a different one.

Expose two endpoint shapes, one for time-bucketed aggregated counts and one for top-k lists, and never let the client request an ungoverned ad-hoc aggregation. Every response carries an explicit freshness field (an asOf timestamp and a precision flag of exact or approximate) so the client-visible contract, not just the internal pipeline, states what freshness the caller is actually getting; the backend may route a request to a fast precomputed view or a slower exact path, but that routing choice is invisible to the caller except through this field.

Endpoints

• GET /metrics/counts?metric=events&start=...&end=...&granularity=hour&dim=region: a time series of pre-aggregated buckets.
• GET /metrics/top?metric=clicks&k=10&start=...&end=...&dim=category: a ranked top-k list.
• Both accept a freshness query parameter as a request-side contract: freshness=fast (accept approximate or slightly stale data, get the lowest latency) versus freshness=exact (route to an exact recompute, higher latency, and the contract documents a range cap or async fallback rather than letting the request hang indefinitely for a huge range).

The staleness contract, in the response itself

Every response includes asOf (the timestamp through which data is complete) and precision. This is the part of the design that belongs squarely to an API-contract discussion: it turns an internal trade-off (how fresh is the backing view) into an explicit, documented, client-visible field, so a caller building a dashboard can show an "as of 2 minutes ago" badge instead of silently trusting a number that might be stale.

Bounding and shaping the response

k is capped server-side (for example, a maximum of 100) regardless of what the client requests, and the response documents the cap via requestedK and returnedK, so a client cannot silently receive a truncated list without knowing it was truncated. The response is already the final shape the dashboard renders, bucketed counts or a ranked list; the client never receives raw event rows and aggregates them itself. That is the same overfetch-avoidance principle as any other read-heavy endpoint: the API promises a specific, small, pre-shaped payload, not a firehose the client has to post-process.

Worked example

GET /metrics/counts?metric=events&start=2026-07-18T00:00Z&end=2026-07-18T03:00Z&granularity=hour&freshness=fast

{
  "metric": "events",
  "granularity": "hour",
  "asOf": "2026-07-18T03:00:42Z",
  "precision": "approximate",
  "buckets": [
    { "start": "2026-07-18T00:00Z", "count": 48210 },
    { "start": "2026-07-18T01:00Z", "count": 51330 },
    { "start": "2026-07-18T02:00Z", "count": 49980 }
  ]
}

GET /metrics/top?metric=clicks&k=5&start=2026-07-18T00:00Z&end=2026-07-18T03:00Z&dim=category&freshness=fast

{
  "metric": "clicks",
  "asOf": "2026-07-18T03:00:42Z",
  "precision": "approximate",
  "requestedK": 5,
  "returnedK": 5,
  "items": [
    { "key": "footwear", "count": 9120 },
    { "key": "outerwear", "count": 7040 },
    { "key": "accessories", "count": 5210 },
    { "key": "electronics", "count": 4880 },
    { "key": "home", "count": 3990 }
  ]
}

Trade-offs and pitfalls

The biggest interview-relevant trade-off is that offering freshness=exact at all is a contract commitment: once a caller can request it, some caller eventually will, on a huge date range, and the API needs a documented, bounded answer (a hard range cap, a queued and pollable async job, or an explicit rejection) rather than an unbounded query that degrades the whole service. A common wrong turn is exposing a single endpoint that "just returns the data" with no freshness or precision field, leaving the client to guess whether a number is authoritative; once a dashboard has shown a number without a staleness caveat, a later "actually that was approximate" correction reads as the API being wrong, when the real defect was an underspecified contract. Enumerating allowed dim values, rather than accepting any column name, trades flexibility for the ability to document, cap, and index every supported query shape; a genuinely open-ended analytics need should be pointed at a dedicated data-warehouse query interface, not bolted onto this API.

Direct answer

This is the classic N+1 fan-out: fetching a list of related items triggers one call for the list plus one additional call per item, instead of a single call that fetches all the related data at once, so latency and downstream load scale with the size of the result set rather than staying flat. The fix is to collapse the per-item calls into a single batched call, or a request-scoped loader that performs that collapsing automatically, and only reach for a precomputed denormalized store when the read path is hot enough to justify the added consistency cost.

Diagnosing it

Confirm it is really N+1 and not just "a slow downstream service" by counting calls per request, not just measuring total time: instrument the aggregation layer to log how many downstream calls a single incoming request triggers, then check whether that count grows linearly with the size of the requested list. A request for 5 related items making 5 downstream calls and a request for 200 items making 200 calls is the signature; a flat call count regardless of list size means the problem is elsewhere.

Fix patterns

A "DataLoader" here means a request-scoped utility, popularized by GraphQL server implementations, that defers execution until every resolver in the current request has registered the key it needs, then fires one batched downstream call instead of many individual ones.

Worked example

flowchart LR
    subgraph Naive["Naive: one call per item"]
        G1[API Gateway] --> I1[Item 1 call]
        G1 --> I2[Item 2 call]
        G1 --> I3[Item N call]
    end
    subgraph Batched["Fixed: single batched call"]
        G2[API Gateway] --> B1[Batch call: all item IDs]
    end

Reasoning about the scaling, not wall-clock timing (real timings depend on hardware and network and are not reproducible from an answer alone): if a request needs data for $N$ related items and each downstream round trip costs a fixed unit $c$, a fully serial naive fan-out costs $N \times c$ total round-trip time. A single batched call collapses this to a constant cost of $c$ regardless of $N$. A concurrency-capped fan-out with cap $k$ costs $\lceil N/k \rceil \times c$: for example, with $N=200$ items and $k=20$ concurrent calls, that is $\lceil 200/20 \rceil = 10$ rounds, versus 1 round batched or 200 rounds fully serial. The batched call is the only option whose cost does not grow with the result set at all.

Trade-offs and pitfalls

Batching changes the failure model: a single bad ID in a batch call must not fail the entire batch. The batched response needs a shape that supports partial success (per-item status alongside a shared envelope), or one missing related record takes down an otherwise-successful response for every other item in the batch.

A request-scoped loader only helps inside one request's lifetime: if the same key is requested by two separate incoming requests seconds apart, the loader does not share that saved work across them, that is what a cache is for. Reaching for a denormalized store as the first fix, before trying batching or a loader, is a common overreach: it is the most expensive option operationally (a whole pipeline to build and keep in sync) and should be reserved for paths where the simpler fixes have already been tried and the traffic genuinely justifies it.