API and Interface Design for Distributed Services Questions
Designing the contracts between services and clients: REST, gRPC, and GraphQL tradeoffs, versioning and backward compatibility, pagination, rate limiting, and idempotent endpoints. Covers request/response modeling, error contracts, and API gateway responsibilities. Focuses on the interface layer that ties distributed components together, not internal data schemas.
Compare cursor-based pagination with offset-based pagination for APIs. For a frequently-updated feed with a high insert rate at the head, explain which approach you would pick and why. Provide a sample response shape for a cursor-based page and mention how you would encode and expire cursors.
Sample Answer
Direct answer
For a feed with a high insert rate at the head, pick cursor-based (keyset) pagination. It anchors each page to the last item's sort key instead of a row count, so it does not skip or repeat rows when new items appear ahead of the client's read position. Offset-based pagination (OFFSET n LIMIT m) is simpler but silently shifts under concurrent inserts, which is exactly what a frequently-updated feed does continuously.
Framework
What each approach actually does
- Offset pagination: the client asks for "skip N, take M." The server re-runs the ordering on every request and counts N rows in before returning results. A page's identity is a position in a list, and that position can move.
- Cursor (keyset) pagination: the client sends the sort-key value(s) of the last item it saw. The server issues a seek query (
WHERE (created_at, id) < (:last_created_at, :last_id) ORDER BY created_at DESC, id DESC LIMIT M). A page's identity is anchored to a specific row's key, not a position.
| Dimension | Offset (OFFSET n LIMIT m) | Cursor / keyset |
|---|---|---|
| Correctness under head inserts | Breaks: new rows shift every row's position, causing duplicates or skipped rows on the next page | Stable: the seek predicate is relative to a fixed key and is unaffected by inserts elsewhere in the ordering |
| Query cost at depth | Grows with the offset (the database still has to skip N rows before returning M) | Roughly constant, an index seek to the key followed by M rows, no matter how deep the client has paged |
| Jump to an arbitrary page | Trivial (OFFSET 4000) | Not supported directly; you can only move forward or backward from a known cursor |
| Implementation complexity | Low | Higher: needs a stable, unique sort key (usually a tie-breaker column) and a cursor encoding scheme |
| Best fit | Small, mostly-static result sets, admin UIs that want page numbers | Feeds, timelines, anything with concurrent writes or deep paging |
Sorting by created_at alone is not enough if two rows can share a timestamp. The seek predicate needs a secondary, unique tie-breaker column (commonly id) so (created_at, id) is a total order with no ties.
Worked example
A concrete trace of why offset duplicates a row when an item is inserted at the head, and why cursor pagination does not.
Five existing rows, ordered newest-first by created_at:
| id | created_at |
|---|---|
| 105 | 2026-01-01T10:04:00Z |
| 104 | 2026-01-01T10:03:00Z |
| 103 | 2026-01-01T10:02:00Z |
| 102 | 2026-01-01T10:01:00Z |
| 101 | 2026-01-01T10:00:00Z |
Offset pagination, page size 2:
- Page 1:
OFFSET 0 LIMIT 2returns positions 1-2:[105, 104]. - A new row
106(created_at2026-01-01T10:05:00Z) is inserted at the head. Every existing row shifts down one position:105is now position 2,104is position 3, and so on. - Page 2:
OFFSET 2 LIMIT 2returns whatever now sits at positions 3-4, which is[104, 103]. The client already saw104on page 1, so it is duplicated, and the client has no way to notice.
Cursor pagination, same scenario:
- Page 1: no cursor supplied, returns
[105, 104];next_cursorencodes{"id": "104", "created_at": "2026-01-01T10:03:00Z"}. - Row
106is inserted at the head. - Page 2:
WHERE (created_at, id) < ('2026-01-01T10:03:00Z', 104) ORDER BY created_at DESC, id DESC LIMIT 2. Row106'screated_atis greater than104's, so it fails the<predicate and is correctly excluded. The query still returns[103, 102], exactly the rows that follow104, regardless of what was inserted ahead of it.
Sample cursor-based response shape (page 1 above):
{
"items": [
{ "id": "105", "created_at": "2026-01-01T10:04:00Z" },
{ "id": "104", "created_at": "2026-01-01T10:03:00Z" }
],
"next_cursor": "eyJpZCI6IjEwNCIsImNyZWF0ZWRfYXQiOiIyMDI2LTAxLTAxVDEwOjAzOjAwWiJ9.f0e3e3b16f56ab33",
"has_more": true
}
How that cursor was actually built (reproducible, step by step):
- Payload:
{"id": "104", "created_at": "2026-01-01T10:03:00Z"}. - Compact JSON serialization:
{"id":"104","created_at":"2026-01-01T10:03:00Z"}. - Base64url-encode the bytes, no padding:
eyJpZCI6IjEwNCIsImNyZWF0ZWRfYXQiOiIyMDI2LTAxLTAxVDEwOjAzOjAwWiJ9. - Sign the raw JSON bytes with a server-side hash-based message authentication code (HMAC-SHA256) using a secret only the server holds, and take the first 16 hex characters of the digest as a tamper check:
f0e3e3b16f56ab33. - Join encoded payload and signature with a period:
<base64url>.<signature>.
A production cursor payload would also carry an issued-at timestamp and a time-to-live (TTL): the server checks issued-at plus TTL on every request and, if the cursor has expired, rejects it (for example with a 400 response) and asks the client to restart from the first page, rather than trying to resume mid-sequence from stale coordinates.
Trade-offs and pitfalls
- Cursor pagination cannot jump to an arbitrary page. If the product genuinely needs "go to page 43," you need offset, or a hybrid where a search step lands the client near a position and cursor paging takes over from there.
- If the tie-breaker column is not unique, or is not indexed together with the primary sort key, the seek query degrades toward a scan instead of an index lookup.
- Signing the cursor matters for more than tamper-proofing: an unsigned or unencrypted cursor can leak internal ids and ordering, and a forged cursor could be used to skip access checks that were only applied on page 1.
- A common pitfall is anchoring the cursor to a mutable field, such as an "updated at" timestamp instead of creation time. An edit to any row changes its position under that ordering, so items can be revisited or skipped after an edit, not just after an insert. Prefer an immutable creation-time key plus a stable id tie-breaker.
Design a contract-testing approach that catches a breaking API change before it reaches production, given that the consuming clients are owned by different teams than the API itself. Explain how you would automate this in CI and how a team would find out their contract test failed.
Sample Answer
Direct answer
Use consumer-driven contract testing: each consuming team publishes the exact requests and expected responses it actually depends on, and the provider team's own continuous integration (CI) pipeline verifies the real API against every published consumer contract before a change can merge or deploy. This inverts the usual failure mode, where a consumer discovers a break after the provider has already shipped; here the provider's own pipeline fails first, and the consumer team is notified automatically through the same system that stores the contracts, not by someone noticing something broke in production.
Structured elaboration
The contract and where it lives:
- Each consumer team writes a small set of interactions ("when I call this endpoint with this request, I expect a response shaped like this") and publishes them to a shared, versioned contract registry (a broker), tagged with which environment or release they apply to.
- This is deliberately narrower than a full schema: the consumer only asserts the exact fields and shapes it actually reads, so the contract reflects real usage rather than the provider's entire response shape.
Automating verification in the provider's CI:
- On every pull request to the provider's API code, a CI step fetches the latest contracts tagged for the relevant environment, spins up (or points at a running instance of) the provider, replays each consumer's recorded requests against it, and asserts the actual responses still satisfy what each consumer expects. Any mismatch fails the build and blocks the merge.
- Before an actual deploy (not just before merge), a second gate checks whether the version about to be deployed is compatible with whichever consumer versions are currently running in production, not just the newest consumer contract, since a provider change can be compatible with a consumer's latest contract while still breaking a consumer instance that hasn't picked up its own latest release yet.
How the affected team finds out:
- The contract broker's webhook posts directly to the consumer team's own channel (chat integration or issue tracker) whenever a provider verification against their specific contract fails, naming the exact interaction that broke and linking to a diff between what was expected and what the provider now returns. The team learns about the break from an automated, targeted message tied to their own contract, not from a shared build dashboard they'd have to be watching.
Worked example
A consumer's published contract (one interaction, in a contract-broker style format):
{
"consumer": { "name": "checkout-web" },
"provider": { "name": "orders-api" },
"interactions": [
{
"description": "fetching an order by id",
"request": { "method": "GET", "path": "/orders/500" },
"response": {
"status": 200,
"body": { "id": "500", "status": "PAID", "total_cents": 4200 }
}
}
]
}
Provider CI step (illustrative, tool-agnostic pseudocode) that fails the build on a mismatch:
verify-contracts:
script:
- fetch-contracts --provider orders-api --tag production
- contract-verify --provider-base-url http://localhost:8080 --contracts ./contracts/*.json
# non-zero exit code on any interaction mismatch blocks the merge
Suppose a developer on the provider team renames total_cents to totalCents in a refactor. The verification step replays the recorded request, gets back a body missing the field the consumer's contract asserted, and the build fails with a message like: "interaction 'fetching an order by id' failed: expected field total_cents, field not present in actual response." The broker then posts that same message, with a link to the failing interaction, straight to the checkout-web team's channel, so they know about the incompatibility before it ever reaches a shared environment.
Trade-offs and pitfalls
- Contract testing only catches what a consumer actually asserted. If a consumer's published contract never checked the
total_centsfield in the first place, removing it silently passes; the technique is only as good as how faithfully each contract reflects real usage, which puts real responsibility on consumer teams to keep their contracts current as their own code changes. - Verifying against the latest contract at merge time is necessary but not sufficient; gating the actual deploy against what is compatible with consumer versions currently running in production (not just their newest published contract) is what catches the case where two independently-safe changes combine badly because of deploy ordering.
- This approach is faster and cheaper than full end-to-end integration testing, but it is not a replacement for it: contract tests verify shape and basic behavior, not real-world concerns like load, timing, or business-logic correctness across a full user flow.
- A pitfall on the process side: if publishing and updating contracts is treated as optional or bolted on late, teams tend to let contracts drift stale, at which point the safety net silently degrades and nobody notices until a real break gets through anyway.
You are designing an API for a social feed consumed by both mobile and web clients, and the mobile team keeps complaining the responses are heavy and hard to work with. Walk through how you would shape the API to minimize client complexity, and justify your protocol choice given the mobile bandwidth constraints.
Sample Answer
Direct answer
The mobile team's complaint ("responses are heavy and hard to work with") is really two separate problems: the payload carries fields nobody on that screen needs, and the client has to reshape what it gets before it can render. The fix is to stop shipping one fixed shape to every consumer and let the server compute exactly the fields each client asks for, while keeping the underlying feed data model the same. Given that mobile and web genuinely want different shapes from the same feed and mobile is bandwidth-constrained, GraphQL's field-selection model is the better protocol fit here, as long as it is constrained (persisted, allow-listed queries; depth and cost limits) so no client can accidentally request an expensive, unbounded query.
Structured elaboration
Requirements to design against: less client-side reshaping work, smaller mobile payloads, predictable caching behavior, and one schema that serves both clients without diverging codepaths.
Protocol comparison for this consumer mix:
| Concern | REST (fixed resource shape) | GraphQL (client-selected fields) | gRPC |
|---|---|---|---|
| Matches "mobile wants less, web wants more" | Poor without ad hoc ?fields= params | Native fit, one query per client shape | Poor, schema is fixed per RPC method |
HTTP-level caching (CDN, Cache-Control) | Strong, stable URLs | Weak by default, needs persisted-query IDs to get cacheable URLs back | Not HTTP-cacheable in the browser sense |
| Browser/mobile-web client support | Native | Native (plain HTTP POST/GET) | Needs gRPC-Web plus a proxy translating HTTP/2 trailers, extra moving part for a browser client |
| Risk of one client hurting others | Low, server controls the shape | Real, a deep nested query can be expensive; needs depth/cost limits | Low |
| Payload compactness for text-heavy feed data | Medium (JSON) | Medium (JSON, but only requested fields) | Best (binary), but the win is smaller for a feed that's already mostly short strings and IDs |
Response-shaping principles regardless of protocol:
- Cursor-based pagination (opaque
next_cursortoken) instead of offsets, so the feed stays stable while new posts are inserted. - Embed hot-path fields the card renders unconditionally (author name, avatar URL, like count); reference cold-path data the card doesn't render by default (full bio, high-resolution media, comment thread) via a separate call or a nested field the client only selects when it needs it.
- Serve media as pre-generated size variants (thumbnail/medium/full URLs) rather than one large image URL the client has to resize client-side; this is a bigger bandwidth win for mobile than almost any protocol choice.
Making GraphQL safe for a bandwidth-constrained mobile client:
- Ship the mobile app with a fixed set of persisted queries (each identified by a hash the client sends instead of the query text). The server only executes registered query hashes and rejects anything else with a 400 response; this closes off arbitrary expensive queries and lets a CDN cache responses by persisted-query hash the same way it would cache a REST URL.
- Enforce query depth and complexity limits server-side so a client cannot nest
author.posts.author.posts...into an expensive fan-out. - Keep the REST fallback in mind: a much lower-effort alternative that gets most of the same benefit is REST with a sparse-fieldset query parameter (
?fields=id,text,author.name,author.avatar_url); it is worth naming as the pragmatic choice for a smaller team that isn't ready to run a GraphQL server.
Worked example
Same feed item, shaped two different ways from one underlying schema.
REST, default embedding (what the mobile team is complaining about):
{
"id": "post_9182",
"text": "Shipped the new onboarding flow today",
"created_at": "2026-07-18T14:02:00Z",
"author": {
"id": "u_4471",
"name": "Priya Shah",
"avatar_url": "https://cdn.example.com/avatars/u_4471_full.jpg",
"bio": "Product engineer, ex-Stripe, likes climbing",
"follower_count": 3821,
"following_count": 512
},
"media": [
{ "url": "https://cdn.example.com/media/m_1.jpg", "width": 4032, "height": 3024 }
],
"like_count": 214,
"comment_count": 18,
"geo": { "lat": 37.7749, "lng": -122.4194 }
}
Everything the card actually renders on mobile is five fields; the rest (bio, follower/following counts, full-resolution image, precise geo) is dead weight on that screen.
GraphQL query the mobile client sends for a feed row (persisted, hash-identified in production; shown here as text for clarity):
query MobileFeed($cursor: String) {
feed(cursor: $cursor, limit: 20) {
edges {
node {
id
text
createdAt
likeCount
author { id name avatarThumbUrl }
media { thumbnailUrl }
}
}
pageInfo { endCursor hasNextPage }
}
}
Response for one edge:
{
"id": "post_9182",
"text": "Shipped the new onboarding flow today",
"createdAt": "2026-07-18T14:02:00Z",
"likeCount": 214,
"author": { "id": "u_4471", "name": "Priya Shah", "avatarThumbUrl": "https://cdn.example.com/avatars/u_4471_thumb.jpg" },
"media": [{ "thumbnailUrl": "https://cdn.example.com/media/m_1_thumb.jpg" }]
}
The web client, on the same schema, asks a different query for the same node and legitimately gets bio, counts, and full-resolution media because its screen renders them; no server-side branching by client type, no separate mobile endpoint to maintain.
Trade-offs and pitfalls
- GraphQL's caching story is genuinely worse than REST's out of the box (an arbitrary query has no stable URL); persisted queries recover most of it by giving each allowed shape a stable, cacheable identifier, but that is an extra piece of infrastructure to build and operate, not a free win.
- The classic GraphQL trap is the N+1 resolver problem: naively resolving
authorper feed item issues one database call per post instead of one batched call for all authors in the page. This needs a batching layer (a per-request data loader that collects author IDs across the whole response and issues one query) or the fix does not show up until load testing. - Over-embedding (putting the full author profile in every feed item) reproduces the original complaint under a new protocol; under-embedding forces one extra round trip per screen, which is worse than a slightly larger response on a high-latency mobile network. The embed-vs-reference line should be drawn by "does this screen render it unconditionally," not by what's convenient to fetch.
- A REST sparse-fieldset compromise is a legitimate answer for a team not ready to operate a GraphQL server; naming it as the deliberately-rejected alternative (and why the full field-selection model wins here specifically because mobile and web diverge so much) is what separates a considered answer from a technology-preference answer.
Your microservices communicate using protobuf messages. Describe a safe process for evolving that schema over time: how you would add fields, remove fields, and handle a change that would otherwise break existing consumers, including the rollout sequence you would follow.
Sample Answer
Direct answer
Treat every protobuf message type as append-only: new fields get new, never-reused field numbers with sensible defaults, retiring fields get marked reserved (never deleted or renumbered) only after every consumer has stopped reading them, and any change that cannot be expressed additively goes through a new versioned message or endpoint instead of mutating the existing one in place.
Safe changes vs unsafe changes
| Change | Safe? | Why |
|---|---|---|
| Add a new field with a fresh field number | Yes | Old binaries silently ignore unknown field numbers; new binaries treat an unset field as its default until it is populated |
| Remove a field, then later reuse its number for something new | No | A newer message reusing field 3 for a different meaning looks like valid old data to a stale reader: silent data corruption, not a crash |
| Rename a field | Safe on the wire, risky elsewhere | The wire format only encodes field numbers, not names, but generated code and any JSON representation change, breaking anything that keyed off the old name |
Change a field's type (for example int32 to string) | No | The wire encoding differs by type; old readers misinterpret the bytes |
Change optional to repeated (or back) | No | The wire representation differs between the two |
Rollout sequence
- Add the new field to the
.protofile with a fresh field number; document it, but do not remove anything yet. - Ship the producer so it emits both the old and new fields simultaneously (a brief dual-write window).
- Migrate consumers to read the new field, verified through automated compatibility checks and a canary rollout before full deployment.
- Once telemetry shows zero consumers still reading the old field, stop populating it in the producer.
- After a sunset window with no incidents, mark the retired field
reservedin the schema so its number and name can never be reused by accident.
Worked example
// v1: original schema
message Order {
string order_id = 1;
string customer_id = 2;
int32 quantity = 3;
}
Step 2, additive change while quantity is being retired in favor of a field that supports multiple line items:
// v2: additive only, nothing removed yet
message Order {
string order_id = 1;
string customer_id = 2;
int32 quantity = 3 [deprecated = true]; // still populated during the migration window
string shipping_address = 4; // new field, new number
repeated int32 line_quantities = 5; // replaces "quantity" semantics going forward
}
Step 5, only after monitoring confirms field 3 has zero remaining readers:
// v3: retiring field 3, its number and name can never be reused
message Order {
reserved 3;
reserved "quantity";
string order_id = 1;
string customer_id = 2;
string shipping_address = 4;
repeated int32 line_quantities = 5;
}
reserved 3 and reserved "quantity" together stop the compiler from letting anyone accidentally reassign that number or that name to a future field, which is exactly the mistake row two of the table above warns against.
Tooling and CI safety net
Add a schema-compatibility check to continuous integration (CI, the automated pipeline that builds and tests every change before merge), such as buf breaking from the buf command-line tool, configured to fail the pull request if a diff removes a field number, changes a type, or otherwise breaks wire compatibility with the previous committed schema. This turns "did we just ship a breaking change" from a question answered in an incident review into one answered before merge.
Trade-offs and pitfalls
The dual-write, dual-read window costs real code complexity and, if the old field is expensive to keep populating, some resource cost too; it should have an explicit end date, not run indefinitely. Enum evolution deserves the same discipline: adding new enum values is safe only if every consumer treats an unrecognized value as its unspecified default rather than crashing, which is why proto3 conventions reserve zero as that default case.
Avro's evolution model differs in a way worth naming if the sub-area comes up: Avro compatibility is enforced by a schema registry comparing reader and writer schemas directly, and it requires an explicit default value for any field being added or removed to stay compatible, rather than protobuf's per-field-number independence. Confusing the two models (assuming Avro's "just add a default" rule applies to protobuf's numbered-field model, or vice versa) is a common mistake when a team works with both formats.
An API endpoint that aggregates data from several microservices for a single client request is making one call per related item, and latency balloons as the result set grows. Walk through how you would diagnose this and redesign the aggregation approach, and how you would weigh the trade-offs between the fixes available to you.
Sample Answer
Direct answer
This is the classic N+1 fan-out: fetching a list of related items triggers one call for the list plus one additional call per item, instead of a single call that fetches all the related data at once, so latency and downstream load scale with the size of the result set rather than staying flat. The fix is to collapse the per-item calls into a single batched call, or a request-scoped loader that performs that collapsing automatically, and only reach for a precomputed denormalized store when the read path is hot enough to justify the added consistency cost.
Diagnosing it
Confirm it is really N+1 and not just "a slow downstream service" by counting calls per request, not just measuring total time: instrument the aggregation layer to log how many downstream calls a single incoming request triggers, then check whether that count grows linearly with the size of the requested list. A request for 5 related items making 5 downstream calls and a request for 200 items making 200 calls is the signature; a flat call count regardless of list size means the problem is elsewhere.
Fix patterns
| Pattern | What it does | Best when | Trade-off |
|---|---|---|---|
| Single batched call | Replace N item calls with one call carrying all the IDs | The downstream service already supports (or can add) a bulk lookup endpoint | Requires that bulk endpoint to exist |
| Request-scoped loader (a "DataLoader"-style collector) | Collects every key requested during one incoming request's lifetime, then issues one batched call for all of them | Composition-heavy code (many independent resolvers each needing the same kind of lookup, common in GraphQL-style servers) | Only helps within a single request; needs its own coordination layer |
| Concurrency-capped fan-out | Keeps per-item calls but issues only a fixed number k at once | No bulk endpoint exists and cannot be added quickly | Still N total calls and downstream load, just paced instead of instantaneous |
| Denormalized read store | Precomputes the joined view ahead of time via events or change-data-capture, CDC (a stream of row-level changes from the source database) | Read-heavy, latency-critical path where some staleness is acceptable | Eventual consistency, plus a pipeline and storage to keep the copy in sync |
| Caching the aggregated response | Caches the whole assembled response or the individual entities | Same aggregation requested repeatedly with a low underlying write rate | Needs a real invalidation strategy; a wrong invalidation reintroduces staleness bugs |
A "DataLoader" here means a request-scoped utility, popularized by GraphQL server implementations, that defers execution until every resolver in the current request has registered the key it needs, then fires one batched downstream call instead of many individual ones.
Worked example
flowchart LR
subgraph Naive["Naive: one call per item"]
G1[API Gateway] --> I1[Item 1 call]
G1 --> I2[Item 2 call]
G1 --> I3[Item N call]
end
subgraph Batched["Fixed: single batched call"]
G2[API Gateway] --> B1[Batch call: all item IDs]
end
Reasoning about the scaling, not wall-clock timing (real timings depend on hardware and network and are not reproducible from an answer alone): if a request needs data for N related items and each downstream round trip costs a fixed unit c, a fully serial naive fan-out costs N×c total round-trip time. A single batched call collapses this to a constant cost of c regardless of N. A concurrency-capped fan-out with cap k costs ⌈N/k⌉×c: for example, with N=200 items and k=20 concurrent calls, that is ⌈200/20⌉=10 rounds, versus 1 round batched or 200 rounds fully serial. The batched call is the only option whose cost does not grow with the result set at all.
Trade-offs and pitfalls
Batching changes the failure model: a single bad ID in a batch call must not fail the entire batch. The batched response needs a shape that supports partial success (per-item status alongside a shared envelope), or one missing related record takes down an otherwise-successful response for every other item in the batch.
A request-scoped loader only helps inside one request's lifetime: if the same key is requested by two separate incoming requests seconds apart, the loader does not share that saved work across them, that is what a cache is for. Reaching for a denormalized store as the first fix, before trying batching or a loader, is a common overreach: it is the most expensive option operationally (a whole pipeline to build and keep in sync) and should be reserved for paths where the simpler fixes have already been tried and the traffic genuinely justifies it.
Unlock Full Question Bank
Get access to all 35 API and Interface Design for Distributed Services interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.