API and Interface Design for Distributed Services Questions

Direct answer

For a feed with a high insert rate at the head, pick cursor-based (keyset) pagination. It anchors each page to the last item's sort key instead of a row count, so it does not skip or repeat rows when new items appear ahead of the client's read position. Offset-based pagination (OFFSET n LIMIT m) is simpler but silently shifts under concurrent inserts, which is exactly what a frequently-updated feed does continuously.

Framework

What each approach actually does

• Offset pagination: the client asks for "skip N, take M." The server re-runs the ordering on every request and counts N rows in before returning results. A page's identity is a position in a list, and that position can move.
• Cursor (keyset) pagination: the client sends the sort-key value(s) of the last item it saw. The server issues a seek query (WHERE (created_at, id) < (:last_created_at, :last_id) ORDER BY created_at DESC, id DESC LIMIT M). A page's identity is anchored to a specific row's key, not a position.

Sorting by created_at alone is not enough if two rows can share a timestamp. The seek predicate needs a secondary, unique tie-breaker column (commonly id) so (created_at, id) is a total order with no ties.

Worked example

A concrete trace of why offset duplicates a row when an item is inserted at the head, and why cursor pagination does not.

Five existing rows, ordered newest-first by created_at:

Offset pagination, page size 2:

• Page 1: OFFSET 0 LIMIT 2 returns positions 1-2: [105, 104].
• A new row 106 (created_at 2026-01-01T10:05:00Z) is inserted at the head. Every existing row shifts down one position: 105 is now position 2, 104 is position 3, and so on.
• Page 2: OFFSET 2 LIMIT 2 returns whatever now sits at positions 3-4, which is [104, 103]. The client already saw 104 on page 1, so it is duplicated, and the client has no way to notice.

Cursor pagination, same scenario:

• Page 1: no cursor supplied, returns [105, 104]; next_cursor encodes {"id": "104", "created_at": "2026-01-01T10:03:00Z"}.
• Row 106 is inserted at the head.
• Page 2: WHERE (created_at, id) < ('2026-01-01T10:03:00Z', 104) ORDER BY created_at DESC, id DESC LIMIT 2. Row 106's created_at is greater than 104's, so it fails the < predicate and is correctly excluded. The query still returns [103, 102], exactly the rows that follow 104, regardless of what was inserted ahead of it.

Sample cursor-based response shape (page 1 above):

{
  "items": [
    { "id": "105", "created_at": "2026-01-01T10:04:00Z" },
    { "id": "104", "created_at": "2026-01-01T10:03:00Z" }
  ],
  "next_cursor": "eyJpZCI6IjEwNCIsImNyZWF0ZWRfYXQiOiIyMDI2LTAxLTAxVDEwOjAzOjAwWiJ9.f0e3e3b16f56ab33",
  "has_more": true
}

How that cursor was actually built (reproducible, step by step):

1. Payload: {"id": "104", "created_at": "2026-01-01T10:03:00Z"}.
2. Compact JSON serialization: {"id":"104","created_at":"2026-01-01T10:03:00Z"}.
3. Base64url-encode the bytes, no padding: eyJpZCI6IjEwNCIsImNyZWF0ZWRfYXQiOiIyMDI2LTAxLTAxVDEwOjAzOjAwWiJ9.
4. Sign the raw JSON bytes with a server-side hash-based message authentication code (HMAC-SHA256) using a secret only the server holds, and take the first 16 hex characters of the digest as a tamper check: f0e3e3b16f56ab33.
5. Join encoded payload and signature with a period: <base64url>.<signature>.

A production cursor payload would also carry an issued-at timestamp and a time-to-live (TTL): the server checks issued-at plus TTL on every request and, if the cursor has expired, rejects it (for example with a 400 response) and asks the client to restart from the first page, rather than trying to resume mid-sequence from stale coordinates.

Trade-offs and pitfalls

• Cursor pagination cannot jump to an arbitrary page. If the product genuinely needs "go to page 43," you need offset, or a hybrid where a search step lands the client near a position and cursor paging takes over from there.
• If the tie-breaker column is not unique, or is not indexed together with the primary sort key, the seek query degrades toward a scan instead of an index lookup.
• Signing the cursor matters for more than tamper-proofing: an unsigned or unencrypted cursor can leak internal ids and ordering, and a forged cursor could be used to skip access checks that were only applied on page 1.
• A common pitfall is anchoring the cursor to a mutable field, such as an "updated at" timestamp instead of creation time. An edit to any row changes its position under that ordering, so items can be revisited or skipped after an edit, not just after an insert. Prefer an immutable creation-time key plus a stable id tie-breaker.

Direct answer

A pagination cursor is just an opaque, URL-safe string the client passes back untouched. Encode the real ordering key, here created_at (an ISO 8601 timestamp, the standard sortable date-time text format) plus id as a tie-breaker, into a small JSON object, then base64url-encode it (a text-safe encoding using only URL-safe characters, no padding) so it survives cleanly in a query string. A version tag inside the payload lets the decoder recognize and translate older cursor shapes instead of breaking on them.

Approach

• Ordering key: sort by (created_at, id) so ties on created_at (common when many rows are written in the same instant) resolve deterministically by id.
• Payload: {"v": 2, "ts": <iso8601>, "id": <string>}, JSON-encoded then base64url-encoded with padding stripped.
• Decoding: try base64url plus JSON first; if that fails, fall back to a known legacy plain-text format ("<timestamp>::<id>", version 1) before giving up.
• Invalid input: never let a decode error surface as an unhandled exception. Raise a typed ValueError that the route handler turns into a 400 response with a clear message.

import base64
import json
from datetime import datetime, timezone
from typing import Tuple


def encode_cursor(created_at: datetime, item_id: str) -> str:
    if created_at.tzinfo is None:
        created_at = created_at.replace(tzinfo=timezone.utc)
    iso = created_at.astimezone(timezone.utc).isoformat()
    payload = {"v": 2, "ts": iso, "id": item_id}
    raw = json.dumps(payload, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
    return base64.urlsafe_b64encode(raw).rstrip(b"=").decode("ascii")


def decode_cursor(cursor: str) -> Tuple[datetime, str]:
    # Legacy v1 format: "<iso-ts>::<id>", never base64-encoded
    if "::" in cursor and not _looks_base64(cursor):
        ts_str, item_id = cursor.split("::", 1)
        return datetime.fromisoformat(ts_str.replace("Z", "+00:00")), item_id

    pad = "=" * (-len(cursor) % 4)
    try:
        raw = base64.urlsafe_b64decode(cursor + pad)
        data = json.loads(raw.decode("utf-8"))
    except Exception as e:
        raise ValueError(f"invalid cursor: {cursor!r}") from e

    version = data.get("v", 1)
    if version not in (1, 2):
        raise ValueError(f"unsupported cursor version: {version!r}")
    try:
        ts = datetime.fromisoformat(data["ts"].replace("Z", "+00:00"))
        item_id = str(data["id"])
    except (KeyError, ValueError) as e:
        raise ValueError(f"malformed cursor payload: {data!r}") from e
    return ts, item_id


def _looks_base64(s: str) -> bool:
    import re
    return bool(re.fullmatch(r"[A-Za-z0-9\-_]+", s))

Key points

• The client treats the cursor as opaque: it never parses the base64 payload itself, so the server is free to change the internal shape as long as it keeps decoding old versions.
• Including id as a secondary sort key solves the identical-timestamp problem: two rows written in the same instant get distinct, stable cursors because they differ on id.
• The v field is what makes backward compatibility possible. A cursor minted before that field existed (v absent) is treated as version 1; a legacy plain-text cursor is detected before any base64 decoding is even attempted.

Complexity and edge cases

Both functions run in $O(L)$ time and space, where $L$ is the cursor's length, bounded by the size of id plus a fixed timestamp width, so effectively constant per call.

Edge cases this design handles: identical timestamps (tie-break by id), invalid or corrupted cursor strings (raises ValueError instead of crashing), legacy "<ts>::<id>" cursors issued before the version field existed, and a cursor claiming an unsupported version number.

Edge cases this sketch does NOT handle, worth naming out loud in an interview: clock skew across writer instances (created_at should come from one authoritative clock, such as the database's transaction timestamp, not individual app-server wall time), and an id that does not sort consistently with insertion order within a tied timestamp (the tie-break assumes something like a UUID v7 or an auto-increment key, not a random UUID v4).

Worked example

Running the functions above with fixed inputs reproduces the exact same cursor every time:

t1 = datetime(2026, 7, 20, 12, 0, 0, tzinfo=timezone.utc)
print(encode_cursor(t1, "item-100"))
# eyJ2IjoyLCJ0cyI6IjIwMjYtMDctMjBUMTI6MDA6MDArMDA6MDAiLCJpZCI6Iml0ZW0tMTAwIn0

t_shared = datetime(2026, 7, 20, 12, 0, 0, tzinfo=timezone.utc)
print(encode_cursor(t_shared, "item-200"))
# eyJ2IjoyLCJ0cyI6IjIwMjYtMDctMjBUMTI6MDA6MDArMDA6MDAiLCJpZCI6Iml0ZW0tMjAwIn0
print(encode_cursor(t_shared, "item-201"))
# eyJ2IjoyLCJ0cyI6IjIwMjYtMDctMjBUMTI6MDA6MDArMDA6MDAiLCJpZCI6Iml0ZW0tMjAxIn0

item-200 and item-201 share the exact same created_at, but they encode to different cursor strings and decode back to their own distinct id, so paging past one never skips or repeats the other. Decoding the legacy string "2026-07-20T12:00:00+00:00::item-999" returns (2026-07-20 12:00:00+00:00, "item-999") without the caller needing to know that cursor predates the version field. Decoding a garbage string like "not-a-valid-cursor!!!" raises ValueError: invalid cursor: 'not-a-valid-cursor!!!' instead of propagating a raw base64 or JSON exception up to the caller.

Trade-offs and pitfalls

Base64-JSON is readable in a debugger and easy to version, but it is not tamper-proof: a client can decode and edit it. If the ordering key must not be forgeable (for example if it gates which rows a caller is allowed to see), sign the payload with an HMAC or keep the encoding fully server-opaque behind a lookup table instead.

A common mistake is sorting only by created_at and using plain numeric offsets to break ties: under write bursts (many rows landing in the same millisecond, common with low-resolution autogenerated timestamps) this reintroduces the exact duplicate-or-gap bug cursor pagination exists to avoid. The (created_at, id) compound key removes that ambiguity by construction.

Silently ignoring an unsupported cursor version, rather than raising, hides client bugs and can leave a client's list state inconsistent instead of failing loudly with a 400 response.

Direct answer

Anchor the pagination cursor to a specific item's identity in the ordering, never to a position count, so an insert at the head cannot shift what "the next page" means. Give the client explicit control over when new head items enter its view (a banner it opts into, not a silent reflow), and dedupe every incoming page against a client-side set of already-seen IDs so a race between a live push and a page fetch can never render the same item twice. This generalizes past feeds to any high-churn ordered list built on top of a mutable store.

Why offset-based paging breaks here

Offset pagination asks "give me the rows starting at position N," where position is counted fresh on every request. If a new row lands at the head between two calls, every existing row's position shifts by one: the client's next request for "position N" now returns a row it already saw (duplicate) while the row that used to sit at position N slides out of view entirely (skip). Cursor pagination instead asks "give me the rows after this specific row," so a head insert changes nothing about what "after this row" means.

Cursor contract and handling head inserts

• The cursor encodes the last-seen row's ordering key (for example created_at plus id as a tie-breaker) and is treated as opaque by the client.
• Real-time inserts arrive over a separate channel (stream or websocket) carrying the full item payload. The client never merges them straight into the currently rendered list; it buffers them and shows a "N new" affordance the user can tap.
• When the client does fetch the next page, it filters incoming rows against a local set of IDs it has already rendered (from either the page fetch or the live stream) before appending, so a row delivered by both paths only renders once.
• If the anchor row referenced by a cursor is later deleted, the server cannot resolve "after this row" any more. Signal that explicitly (for example a 410 Gone-style error on that specific cursor) so the client knows to discard its cursor and reload the current view, rather than the server silently guessing a nearby row.

Worked example

Assume feed posts are labeled by creation order, oldest to newest: p1 ... p9 already exist when the client loads page one, and p10 arrives from the live stream while the client is browsing.

sequenceDiagram
    participant C as Client
    participant F as Feed API
    participant R as Realtime stream
    C->>F: GET /feed?cursor=null&limit=2
    F-->>C: items=[p9,p8], next_cursor=after:p8
    R-->>C: push new item p10 (inserted at head)
    Note over C: buffer p10, show "1 new" banner, do not reorder current page
    C->>F: GET /feed?cursor=after:p8&limit=2
    F-->>C: items=[p7,p6], next_cursor=after:p6
    Note over C: dedupe against seen-id set before appending

p10's arrival never touches the cursor after:p8: it still means exactly what it meant before the insert, so the second page correctly returns [p7, p6] with no repeat of p9 or p8 and no skip past p7.

Trade-offs and pitfalls

Cursor pagination trades away random access (you cannot jump straight to "page 7") for stability under churn, which is almost always the right trade for a live feed. A common mistake is auto-merging live-pushed items straight into the rendered list the instant they arrive: this yanks the scroll position under the user's finger and is a worse experience than a controllable banner, even though it feels "more real-time."

If filters or sort order change mid-session, the old cursor and seen-ID set are no longer meaningful and must be discarded, not reused; treat that as a fresh pagination session. For most feed use cases, accepting eventual consistency (the client may briefly be a few items behind, reconciled by the next page fetch or a manual refresh) is the right call over paying for a strict per-client consistent snapshot, which adds real operational cost for a benefit most users never notice.

Direct answer

Frame the decision around the actual constraints rather than the protocols' reputations: which languages across the polyglot stack need first-class client support, whether the traffic is latency-sensitive service-to-service calling or something a human needs to inspect, and whether streaming is a hard requirement. Build a small prototype for the leading candidate against a real workload, compare it structurally rather than by a single benchmark number, and validate the decision with the teams who will own the resulting code before committing company-wide.

Evaluation criteria

• Wire efficiency: gRPC serializes with protocol buffers, a binary format that omits field names and uses variable-length integers, so a given message is typically meaningfully smaller on the wire than the equivalent JSON; REST/JSON is larger but human-readable without any tooling.
• Connection behavior: gRPC runs over HTTP/2, which multiplexes many concurrent requests over a single connection; this avoids head-of-line blocking, the delay where one slow request stuck at the front of a connection holds up every other request queued behind it, a problem plain HTTP/1.1 REST can suffer under high concurrency.
• Streaming support: gRPC has first-class client-streaming, server-streaming, and bidirectional streaming built into the protocol; REST needs bespoke chunking or a separate protocol (such as server-sent events or websockets) bolted on.
• Language and tooling coverage across the actual stack: check every language you run in production, not just the popular ones. gRPC's code generation and runtime maturity vary meaningfully by language.
• Debuggability: a REST/JSON payload is directly inspectable with a browser or curl; gRPC needs a reflection service and a tool like grpcurl to get the same visibility.
• Operational and observability tooling: both integrate with modern tracing systems, but gRPC's HTTP/2 transport sometimes needs extra configuration for proxies and load balancers that were written assuming HTTP/1.1.

What I would benchmark, and how

Build matching prototype services in the two or three languages that make up most of the real traffic, exercise both protocols with the same synthetic load generator and connection count, and compare relative, structural behavior (does one need meaningfully less CPU per request at the same throughput, does one handle the streaming case directly versus requiring a workaround) rather than quoting a single absolute number as if it generalizes. Any specific latency or throughput figure from a single environment is a snapshot of that environment, not a portable fact, so treat it as directional evidence for this specific decision rather than a reusable benchmark.

Worked example

Situation: a polyglot environment (Python data workers, Go services, Java-based batch jobs) needed a new internal path for high-throughput data ingestion. The requirement list included sustained streaming from Go into Python, and low overhead at high request volume. Action: scored REST and gRPC against the criteria above, built matching prototype services in Go and Python, ran both through the same load generator, and reviewed the comparative results with the two teams that would own the resulting code day to day. gRPC's built-in server-streaming matched the ingestion use case directly, while REST would have needed a bespoke chunked-transfer workaround; that single structural fit mattered more in the final call than any specific number either prototype produced. Result: adopted gRPC for the internal high-throughput and streaming paths, kept REST/JSON for a small admin API where human debuggability mattered more than raw efficiency, and put a translating gateway in front so a team that could not migrate immediately was not blocked.

Trade-offs and pitfalls

gRPC has a steeper learning curve than REST, and its debugging story is worse out of the box: budget time for a reflection service and team familiarity with grpcurl rather than assuming curl will still work. Verify gRPC client maturity for every language actually in your stack before committing, not just the two or three most common ones; an internal tool with weak gRPC support becomes a maintenance tax nobody accounted for at decision time.

REST remains the better default for public or third-party-facing APIs, where you cannot force external consumers to adopt a specific client library the way you can for an internal service. The most common mistake in this kind of evaluation is running a benchmark in one environment on one payload shape and generalizing that result to the whole company without a real pilot: always validate with a scoped rollout to one or two teams before treating a prototype's numbers as proof for every future use case.

Direct answer

For an internal microservices platform, prefer gRPC for service-to-service calls: its binary encoding and HTTP/2 transport give lower latency and smaller payloads at high call volumes, and its contract-first interface keeps many services in sync automatically. For anything a browser calls directly, prefer REST over plain HTTP and JSON, because a browser cannot open a native gRPC connection at all.

Framework

REST is an architectural style over HTTP that uses JSON bodies, standard HTTP methods, and standard status codes. gRPC is a contract-first remote procedure call framework that runs over HTTP/2 and encodes messages with Protocol Buffers (protobuf), a compact binary serialization format defined by a .proto schema file that generates client and server code in multiple languages.

For the internal platform: gRPC, because service-to-service calls are usually high in volume, latency-sensitive, and made by services the team already controls, so the cost of generating and deploying stubs from a shared .proto file pays off, and the smaller binary payload plus multiplexed streams cut both bandwidth and per-connection overhead compared to many REST connections.

What changes for a browser: a browser's networking stack cannot speak native gRPC, because it cannot control HTTP/2 framing and trailers the way a gRPC client library does. Two ways to still reach the same backend from a browser: put a REST/JSON facade in front of the gRPC service (a translation layer sometimes called a backend for frontend), or use gRPC-Web, a variant browsers can speak that still needs a proxy in front of the gRPC service (commonly Envoy) to translate wire formats, and that does not support the full bidirectional streaming native gRPC does. In practice, most teams pick REST/JSON for anything a browser calls directly and keep gRPC strictly behind that boundary.

A third style, GraphQL, is not really the comparison this question is asking for. It trades either extreme for a single flexible query language over HTTP, and it would be the right answer if the real problem were a client needing to shape its own response across many resources at once, not the internal-platform protocol choice being asked about here.

Worked example

Suppose an internal order service calls an inventory service 50 times per incoming user request, once per line item, to check stock. With REST/JSON, each call opens or reuses an HTTP connection and pays JSON parsing cost per call. With gRPC, all 50 calls can multiplex over a single HTTP/2 connection to the inventory service, and each message is a compact protobuf encoding instead of a JSON object with repeated field-name strings, so the marginal cost per call is lower at that fan-out.

If that same platform later needs to expose an inventory check directly to a web storefront, the storefront calls a REST endpoint instead:

GET /v1/inventory/A1
200 OK
{ "sku": "A1", "in_stock": 42 }

The storefront is never handed a .proto file or a gRPC client; it gets ordinary JSON over HTTP.

Trade-offs and pitfalls

• gRPC's stricter schema is also a cost: every field change means regenerating and redeploying stubs across every consuming service, whereas REST/JSON tolerates an unexpected extra field without anyone regenerating anything.
• A common pitfall is exposing gRPC-Web directly to third-party public API consumers to save engineering time. It forces every external integrator to adopt protobuf tooling and a compatible proxy, which is a worse experience than JSON for most public consumers.
• Debugging gRPC's binary frames is not as simple as opening a browser's network tab or running curl the way it is with JSON; teams that skip investing in gRPC-aware tracing early often regret it once they have many services talking to each other.