RESTful API Design Questions

Direct answer. Read the Idempotency-Key header; if a request with that key has already completed, replay the stored response verbatim instead of re-running the creation logic; if one is currently in flight, reject the duplicate with a distinct status instead of racing it.

Implementation (Python, Flask). This uses an in-memory store for a self-contained, runnable example; in production the same design sits on a Postgres table with a unique index on the idempotency key, so the have-I-seen-this-key check and the reserve-it step are atomic at the database level, not just inside a Python lock.

import threading
import uuid
from flask import Flask, request, jsonify

app = Flask(__name__)

# Stand-ins for a Postgres "idempotency_keys" table and an "orders" table.
idempotency_keys = {}
orders = {}
lock = threading.Lock()


@app.post("/orders")
def create_order():
    key = request.headers.get("Idempotency-Key")
    if not key:
        return jsonify({"error": "Idempotency-Key header is required"}), 400

    body = request.get_json(force=True)

    with lock:
        existing = idempotency_keys.get(key)
        if existing is not None:
            if existing["status"] == "in_progress":
                return jsonify({"error": "already being processed"}), 409
            return jsonify(existing["response_body"]), existing["response_status"]
        idempotency_keys[key] = {"status": "in_progress", "response_body": None, "response_status": None}

    order_id = str(uuid.uuid4())
    orders[order_id] = {"id": order_id, "item": body.get("item"), "qty": body.get("qty")}
    response_body = {"order_id": order_id, "item": body.get("item"), "qty": body.get("qty")}
    response_status = 201

    with lock:
        idempotency_keys[key] = {"status": "done", "response_body": response_body, "response_status": response_status}
    return jsonify(response_body), response_status


if __name__ == "__main__":
    client = app.test_client()
    r1 = client.post("/orders", json={"item": "widget", "qty": 3}, headers={"Idempotency-Key": "key-abc"})
    print("first request  ->", r1.status_code, r1.get_json())
    r2 = client.post("/orders", json={"item": "widget", "qty": 3}, headers={"Idempotency-Key": "key-abc"})
    print("retry request  ->", r2.status_code, r2.get_json())
    print("same order_id returned on retry:", r1.get_json()["order_id"] == r2.get_json()["order_id"])
    print("total orders actually created:", len(orders))

Output (actually run):

first request  -> 201 {'item': 'widget', 'order_id': '72664f4a-d7da-4618-8e70-38c7bff94a38', 'qty': 3}
retry request  -> 201 {'item': 'widget', 'order_id': '72664f4a-d7da-4618-8e70-38c7bff94a38', 'qty': 3}
same order_id returned on retry: True
total orders actually created: 1

Key points. The reservation of the key (marking it in_progress) happens before the real work starts, inside the same lock as the lookup, closing the race where two near-simultaneous retries both pass the have-I-seen-this check. The stored result is the exact response (body and status), replayed verbatim, not re-derived.

Complexity. Each request does O(1) dictionary work plus the actual creation work; the lock is held only for the cheap lookup-and-reserve step, not for the full creation logic, so concurrent requests with different keys are not serialized against each other in the real (database-backed) version, only requests racing on the same key are.

Edge cases. A second request with the same key but a different body (a client bug, not a legitimate retry) is not checked in this minimal example; a production version should also compare a hash of the request body against what was stored for that key, and reject the request with a 422 if they do not match, since replaying the original response to what looks like a different logical request would silently hide the client bug.

Direct answer. Expose a consistency signal directly in the API contract (a resource version or a last-write timestamp on every response, plus a way for a client to explicitly request the freshest possible read) rather than leaving clients to guess how stale a regional cache might be, and invalidate regional caches via an event propagated from the write region rather than a synchronous cross-region call on every write.

What the contract needs to expose. Every read response includes a version or timestamp field (updated_at or an opaque version token) representing when that specific piece of data was last written at the source of truth. A client that just performed a write (or needs a guaranteed-fresh read for some specific reason, a payment confirmation for instance) can pass that version back on a subsequent read via a header (something like X-Min-Freshness: <version>), and a region whose replica has not yet caught up to that version can either serve a slightly slower "read from the primary region" fallback, or return a distinct signal telling the client the data it has is not fresh enough yet, rather than silently serving a stale answer with no indication anything is off.

A concrete trace. Say product SKU SKU-48213 is priced at $24.99, and a write updates it to $29.99 in the write region (US-EAST) at 14:32:07 UTC, bumping the resource's version from v104 to v105. A read that hits the eu-west regional cache two seconds later, at 14:32:09 UTC, still returns version: v104, price: $24.99 -- the stale, pre-write value -- because eu-west's cache-invalidation event for this key has not landed yet. If the client that just performed the write immediately sends X-Min-Freshness: v105 on its next read, eu-west (still holding v104) recognizes its cached copy does not meet that minimum: it either forwards the read to the primary region for a guaranteed-fresh v105 answer (the slower fallback), or responds with a distinct "not fresh enough yet" signal instead of silently handing back the stale $24.99. This trace is illustrative, not a measured production timing -- the actual invalidation-lag window depends on your own event-propagation latency, not a fixed 2 seconds.

Regional cache invalidation on a central write. Writes commit centrally, then publish an invalidation (or an updated version marker) as an EVENT to all regions, rather than the write path synchronously waiting for every region's cache to be updated (which would defeat the entire point of regional caching by making every write pay a global round-trip cost). Each region's cache subscribes to this event stream and evicts or updates the affected entries asynchronously; the trade-off is explicit and honest: a regional read immediately after a write elsewhere may briefly return the OLD cached version until that region's invalidation catches up, and the contract's version/timestamp field is exactly what lets a client detect that condition rather than being surprised by it.

Communicating the trade-off to clients. Document plainly, per endpoint, which consistency guarantee it actually offers: this is not a single global answer for the whole API, some endpoints (a payment status check) may need to route to the primary region for a guaranteed-fresh read regardless of latency cost, while others (a public product listing) are entirely fine serving a regionally-cached, occasionally-a-few-seconds-stale response in exchange for much lower latency worldwide.

Trade-offs and pitfalls. The most common mistake is treating "eventually consistent" (meaning every region's cached copy will converge on the same value given enough time, just not immediately -- so a read right after a remote write, as in the v104/v105 trace above, can briefly return the old value) as an implementation detail that does not need to be surfaced in the contract at all; a client that is never told a read might be stale has no way to build correct behavior around it (for instance, retrying a read after a write until it observes the new version), and ends up discovering the staleness window in production as a confusing, hard-to-reproduce bug instead of a documented, designed-for property of the API.

Direct answer. Start from the idempotency store, not the database: look up what state is recorded against the key the client says it sent. If the store says succeeded with a specific result, but the client is seeing something different from what that stored result contains, the bug is in how the retry was matched or replayed, not in the original creation logic.

Investigation, step by step.

1. Confirm the key actually matched. Pull the exact Idempotency-Key header value from both the original request's logs and the retry's logs. A surprising number of duplicate-detection-failed bugs are actually the client generating a new key on each attempt instead of reusing the same one (defeating the whole mechanism before the server even runs), or a proxy or load balancer stripping or rewriting a header it does not recognize.
2. Check the database's unique constraint. If the idempotency key has a unique index, a genuine race between two near-simultaneous first-attempts should have produced a constraint violation on the second insert, which the code should catch and treat as someone-else-already-has-this-key-in-flight. If there is no unique constraint (only an application-level check-then-insert), a race window exists: two requests can both pass the check before either has written the key, and both proceed to create the resource. This is the single most common root cause of sending a request once but getting inconsistent results back.
3. Look for a malformed or reused key. A key reused across two logically different requests (say, a client bug that hardcodes one key instead of generating a fresh one per logical operation) makes the second, different, request incorrectly replay the first request's stored result, which looks exactly like inconsistent data from the client's point of view: they asked for B and got A back.
4. Replay the actual request logs. Reconstruct the exact sequence of requests and responses for this idempotency key from your logs; a genuine race is usually visible as two requests with overlapping timestamps and the same key, both reaching the not-seen-yet branch of the code.

Root causes to rule in or out, honestly. A race condition (missing unique constraint) is the most likely and most fixable. A malformed or client-reused key is second most likely and points to a client-side bug, not a server bug, worth confirming before spending time fixing server code that is not broken. A missing unique constraint on the underlying resource itself (not just the idempotency key) is a distinct, related bug: even with a correct idempotency-key check, a completely separate bug elsewhere could still let two orders be created for unrelated reasons.

Prevention. Add a unique database constraint on the idempotency key (not just an application-level check), so a race becomes a hard, catchable insert failure instead of a silent double-create; this is the fix that actually closes the class of bug, not just this instance of it.

Direct answer. HTTP/2's multiplexing removes the old "avoid too many small requests, batch them into one big one" advice almost entirely for requests to the SAME origin, since many requests can now share one connection with no head-of-line blocking at the HTTP layer; HTTP/3's move to QUIC (over UDP instead of TCP) additionally removes TCP-level head-of-line blocking and speeds up connection setup, which matters most on flaky mobile networks, and both change what your load balancer and CDN need to actually do.

Multiplexing changes API design guidance that predates it. Under HTTP/1.1, browsers opened a limited number of parallel TCP connections per origin, so an API client benefited from batching many small requests into fewer, larger ones (a classic piece of REST API advice: avoid chatty, many-small-calls API shapes). Under HTTP/2, many logical requests share ONE connection via multiplexed streams, so issuing several smaller, well-scoped requests to the same origin is no longer the performance problem it used to be; the old advice does not fully disappear (a request still has real per-call overhead: headers, server-side routing, business logic), but the SPECIFIC "too many TCP connections" penalty it was originally guarding against is gone. Concretely: under a common browser limit of about 6 concurrent connections per origin under HTTP/1.1, a page issuing 30 small API calls to the same origin would need to queue them into 5 sequential batches of 6; under HTTP/2's multiplexing, all 30 can be in flight at once over the single connection, with no batching required at all.

Header compression (HPACK/QPACK). HPACK is HTTP/2's header-compression format, and QPACK is its HTTP/3 counterpart, redesigned so header compression still works correctly when QUIC delivers streams out of order, which plain HPACK cannot tolerate. Both work by having each side maintain a shared table of previously-seen header name/value pairs: once a header (an auth token, a standard Accept or User-Agent value) has been sent once on a connection, later requests on that SAME connection can send a short reference into that table instead of the full header text again. Repeated headers across many requests to the same connection get compressed incrementally this way, which meaningfully reduces overhead for an API client making many small, frequent calls, exactly the pattern multiplexing now makes more attractive in the first place.

HTTP/3 and QUIC specifically. Because QUIC runs over UDP and manages its own stream multiplexing independently, one lost packet on one logical stream no longer blocks delivery of data on OTHER streams sharing the same connection, unlike TCP where a single lost packet stalls the entire connection until it is retransmitted; this specifically helps a mobile client on a lossy network, which is exactly the scenario where TCP-level head-of-line blocking used to hurt the most. QUIC's connection setup also folds the transport and TLS handshakes together, cutting the round trips needed before the first real request can even be sent, which matters disproportionately on high-latency mobile connections where each round trip is expensive. Concretely: a fresh HTTPS connection over TCP + TLS 1.3 needs 2 round trips before the client can send its first request byte (1 to establish the TCP connection, 1 for the TLS 1.3 handshake); QUIC combines both into one combined handshake, needing just 1 round trip for a new connection (and its 0-RTT mode can send request data on the very first flight when resuming a connection to a host the client has already talked to). On a mobile connection, where a single round trip commonly costs on the order of 50-100ms, that one saved round trip is a real, directly-felt latency difference before any actual request-response work even starts.

What changes in load-balancing and CDN configuration. Load balancers and CDNs need to actually terminate and understand HTTP/2 or HTTP/3 themselves (not simply pass bytes through) to preserve these benefits end-to-end, and per-request load-balancing algorithms that assumed "one connection roughly equals one in-flight request" need rethinking, since one HTTP/2 or HTTP/3 connection can now carry many concurrent logical requests, changing what "an overloaded backend" or "a slow connection" even looks like from the load balancer's point of view. Many CDNs and cloud load balancers already handle this transparently, but a self-managed edge layer built assuming HTTP/1.1 semantics needs deliberate upgrading, not an assumption that the protocol change is invisible to it.

Trade-offs and pitfalls. The most common mistake is treating this purely as an infrastructure upgrade with no API-design implications; an API whose CLIENTS were specifically designed around HTTP/1.1's connection limits (aggressively batching everything into one mega-request "to save connections") may actually see LESS benefit from multiplexing than a more naturally-shaped API would, since it already paid the batching cost the new protocols were meant to make unnecessary.

Direct answer. Malformed JSON is 400 Bad Request. A business-rule validation error is also 400 (or 422 Unprocessable Entity if you distinguish syntactically-valid-but-semantically-wrong from not-even-parseable). An authentication failure (no or invalid credentials) is 401 Unauthorized. An authorization failure (valid credentials, insufficient permission) is 403 Forbidden. A missing resource is 404 Not Found. Work accepted for asynchronous processing is 202 Accepted. A successful creation is 201 Created; a successful update with nothing new to return is 200 OK (or 204 No Content if the body is genuinely empty).

Working through the scenario.

1. Malformed JSON in the request body: the server never even got a well-formed request to evaluate, so this is a client error before any business logic runs. 400.
2. A validation error on a business rule (say, a date field that must not be in the past): the JSON parsed fine, but the content violates a rule. 400 is defensible; many APIs use 422 specifically to separate "your request is not even shaped right" from "your request is shaped right but the values are invalid." Pick one convention and apply it consistently across the whole API.
3. Authentication failure: 401, and the response should include a WWW-Authenticate header naming the expected scheme.
4. Authorization failure: 403. The distinction from 401 matters for the client: 401 means try again with different credentials, 403 means these credentials are fine, you just cannot do this.
5. Resource not found: 404. For a public API, be deliberate about whether a 404 for does-not-exist versus exists-but-you-cannot-see-it leaks information; many APIs intentionally return 404 for both to avoid confirming a private resource's existence to an unauthorized caller.
6. Accepted for async processing: 202, ideally with a Location header pointing at a status resource the client can poll.
7. Successful creation vs. update-with-nothing-new: 201 with the new resource's URI in Location for creation; 200 with a body, or 204 with no body, for an update, depending on whether you have anything useful to return.

Common mistake. Returning 200 for every successful outcome regardless of what actually happened, including creations (should be 201) and validation failures caught late in a handler (silently returning {"success": false} with a 200 status). This breaks every generic HTTP client, cache, and monitoring tool that reasons about status codes rather than parsing your custom body shape, and it is one of the most common REST code-review findings.

Trade-offs and pitfalls. 401 vs. 403 is the pair engineers mix up most: 401 always means I do not know who you are, or I do not believe your credentials; 403 always means I know exactly who you are, and the answer is still no.