API Documentation and Developer Experience Questions

Direct answer

(Deflection means a developer solves their problem without opening a ticket.) Start by reading the actual tickets and sorting them by cause, because "cut tickets in half" is only reachable if a large share are the kind that docs, better errors or self-service tools can prevent. Then plan work in order of tickets removed per unit of effort, and measure deflection with a rate (tickets per 1,000 active developers) validated by a holdout, not with raw ticket counts or self-reported "article helped" clicks.

Step 1: find the removable tickets

Pull the last 3 to 6 months of tickets and tag a random sample of a few hundred by root cause and by what would have prevented it.

Worked example (illustrative numbers)

200 sampled tickets: 60 authentication and key setup, 44 unexplained error responses, 30 how-to (pagination, webhooks, which are calls our server makes to the developer's URL when something happens), 26 real bugs, 20 billing and account, 20 other.

docs / self-service addressable: 60 + 44 + 30 = 134
134 / 200 = 67%

Not all 134 will vanish, because some readers still will not read a page. If a good fix removes, say, 60% of a category, the docs-addressable 67% yields about 40% of total volume. Reaching 50% therefore needs more than docs: also fix the API itself (clearer error messages, bugs) and add self-service. State that honestly to support, do not promise 50% off docs alone.

Step 2: plan work as a prioritised roadmap

Order by (tickets in category times expected removal rate) divided by effort. Applying it to the sample, with removal rates and effort points (1 = a few days, 2 = a few weeks) as judgement calls you would revisit:

auth and key setup: 60 x 0.70 / 1 = 42.0
how-to guides:      30 x 0.50 / 1 = 15.0
unexplained errors: 44 x 0.60 / 2 = 13.2

Auth goes first, then the how-to guides, then error messages, which cost more engineering but stay in the first half of the year. Twelve-month shape: quarter one, docs and error messages (cheap, biggest categories); quarter two, self-service tools; quarters three and four, SDK improvements and whatever the re-tagged tickets show still remains. Re-sample tickets each quarter because fixing the top category reveals the next one.

Step 3: measure deflection honestly

• Normalise: raw ticket counts rise when adoption rises. Baseline 1,200 tickets per month with 4,000 active developers is 1,200 / 4,000 x 1,000 = 300 tickets per 1,000 developers. Halving means 150 per 1,000, even if total tickets stay flat because developers doubled.
• Do not count "article views without a ticket" as deflection. Most of those readers would never have filed a ticket anyway. Instead, run a holdout (a small group deliberately left on the old experience, as a control). Example: 2,000 new developers sign up in a month; randomly keep 200 on the old experience and give 1,800 the new error-page link. The held-out group files 30 tickets, 30 / 200 = 15 per 100 developers. The treated group files 180, 180 / 1,800 = 10 per 100. Honest deflection is (15 - 10) / 15 = 33% fewer tickets, measured against what would have happened anyway.
• Track guardrails (numbers that must not get worse while you chase the main goal): developer satisfaction, activation (share of signups who make a call) and time to first call (minutes from signup to the first successful API request) must not fall. A hidden contact form deflects tickets by making developers give up.
• Track category-level rates, so the auth-ticket rate should fall after the auth guide ships, and not just the total.

Pitfalls

• Counting tickets closed by an auto-reply bot as resolved.
• Cutting volume by making support harder to reach.
• Ignoring reopen rate (the share of closed tickets the developer reopens or refiles): a deflected ticket that returns as two tickets is not a win.

Direct answer

A new public endpoint ships with five things: a reference entry generated from the API's machine-readable description, a runnable example that a stranger can copy, the error and limit behaviour spelled out, a changelog entry with versioning notes, and a working sandbox path. It is "done" when someone who did not build it can call it correctly using only those materials, and when an automated check proves the docs match the running code.

What must ship

The same checklist for an internal shared service

Suppose the "public endpoint" is an internal model-serving service used by other teams. The downstream teams need the same items, weighted toward safe integration: rate limits and quotas per team, idempotency and retry rules, the full error-code table, the payload schema including maximum sizes, and versioning notes (which model version answers, when it changes). Internal consumers cannot ask you in a hallway at scale, so the docs replace that conversation.

Definition-of-done checks

1. Clean-room run: someone outside the team follows the docs on a fresh machine and makes a successful call.
2. Docs match code: a contract test in CI (continuous integration, the automated build run on every change) compares real responses to the OpenAPI schema. If the code changes and the spec does not, the build fails. This is how docs stay in sync with the implementation.
3. Examples execute: every sample is run automatically against the sandbox.
4. Failure paths documented: for each error code you can trigger, the docs show it.
5. Links and generated explorer work: the interactive page loads and "try it" reaches the sandbox.

Worked example

A new GET /orders endpoint. The reference says it returns data (a list) and next_cursor. The pagination example shows: first call GET /orders?limit=2 returns two orders and "next_cursor": "c_9"; second call GET /orders?limit=2&cursor=c_9 returns one order and "next_cursor": null, meaning done. The error table lists 401 (bad key), 429 (slow down, retry after the time given in the Retry-After header, a number of seconds or an HTTP date) and 422 (bad limit). If a reviewer's clean-room run cannot explain what null means, the doc is not done.

Sample changelog entry

2025-03-04  Added: GET /orders (list orders, cursor pagination). Stable.
            Affects: new endpoint only, no action needed for existing integrations.
2025-03-04  Deprecated: field `orders[].total` (use `total_cents`).
            Still returned until 2025-09-04. Affects: clients reading `total`.

A contract test in miniature. The OpenAPI file says next_cursor is a string or null. The test calls the sandbox GET /orders?limit=2 and checks that the response has exactly the documented fields with the documented types. If a developer renames next_cursor to cursor in the code but not in the spec, the check finds a missing documented field and fails the build. A clean-room run means a fresh machine with none of the team's existing keys, tools or setup, so hidden assumptions surface. Quotas are longer-horizon usage caps (for example, per month), distinct from per-minute rate limits, and a payload is the body of a request or response.

Pitfalls

• Reference without examples: complete but unusable for a beginner.
• Examples showing only the happy path: the first real problem is an error nobody documented.
• Hand-maintained docs beside a fast-changing API: they diverge quietly. Generate what can be generated and test the rest.

Direct answer

Write it in terms of business outcome, rules and consequences, not HTTP mechanics. A product manager needs to know what the endpoint lets the business do, what must be true for it to work, what happens after it succeeds, and what can go wrong for customers.

The paragraph

"Create Order is the call our storefront and partner systems make to place a customer's order. It needs a customer, at least one item that is in stock, and a delivery address; if any of these is missing, no order is created and the caller is told which one. When it succeeds, the order is saved in a 'pending' state, stock for those items is reserved, and the order appears in the customer's history and in the daily sales report. Payment is collected separately, so a pending order can still be abandoned if payment fails. It is safe for a caller to retry after a timeout without creating a duplicate order. Orders above our fraud-review threshold are held for manual approval, which can delay confirmation."

What I included and why

• Purpose in business words: who calls it and what it enables.
• Preconditions that change what customers experience (needs stock, needs address).
• What happens next: reserved stock, pending status, reporting. A PM plans features on these downstream effects.
• Consequences of failure: abandoned pending orders, held orders.
• Anything that changes an outcome: retry safety means no duplicate orders, so a customer is never asked to pay twice for one purchase by a retry (payment itself is collected by a separate call, which needs its own retry protection), which is a customer-facing promise.

What I left out on purpose

• Header names, authentication token format, field types and JSON structure.
• Exact status codes, the database tables touched, internal service names.
• Pagination, versioning details and other things the PM will never act on.

If the same summary were for an engineer, those omitted items become the core; the facts stay the same, the audience changes which ones matter.

Pitfalls

Copying the reference description ("POST /orders creates an order resource") gives the PM nothing to decide with. Also avoid promising business behaviour the code does not implement: check each claim (stock reservation, fraud threshold) with the owning team before it goes in the paragraph.

Direct answer

(Triage means sorting incoming items by type and urgency and deciding who handles each.) Collect feedback from three kinds of sources (asked-for, unprompted, and behavioural), route it into one tagged backlog owned by a named person, prioritise by how many developers it affects and how badly it blocks them, and close the loop publicly with a changelog and direct replies so developers see that reporting is worth their time.

Signals to collect

Behavioural data covers the silent majority: most stuck developers never write in.

Turning it into a prioritised backlog

1. One intake, one owner: every signal becomes an item in a single tracker, tagged by area (auth, errors, pagination, SDK) and type (docs gap, bug, feature). A named owner triages weekly.
2. Merge duplicates and count: each item carries the number of distinct developers affected and the source.
3. Score: rank by developers affected times severity (blocked entirely, worked around, cosmetic) divided by effort. Use simple numeric weights: severity 3 = blocked entirely, 2 = worked around, 1 = cosmetic; effort 1 = under a day, 2 = a few days, 3 = weeks. Boost items that hit the first-call path, and give an enterprise-account request a multiplier (for example x2, meaning its revenue counts double) agreed with product, not decided by whoever shouts.
4. Route: docs fixes go to the docs owner, bugs to engineering, feature requests into the product roadmap process.

Worked example (illustrative)

Docs search logs show "webhook signature" (the check that proves a webhook really came from us) returned no result 45 times in a month, three forum threads ask the same thing, and support has 9 tickets on it. De-duplicating by developer gives about 40 distinct developers. Blocking a common flow, so severity 3; one page to write, so effort 1.

webhook signature page: 40 x 3 / 1 = 120
cosmetic request from one enterprise customer: 5 developers x 1 / 2 = 2.5, x2 enterprise multiplier = 5

The docs page outranks the loud request by a wide margin. The enterprise item would only overtake it if its multiplier was enormous, which is a business decision to make openly.

Showing developers that feedback was acted on

• A public changelog entry that says "you asked, we did", linking the original issue where that is allowed.
• Reply directly to whoever reported it when the item ships, even a one-line message.
• A visible public roadmap or "top requested" list with status (planned, in progress, shipped, not planned), including an honest reason for "not planned".
• A quarterly "what we fixed from your feedback" note.

Pitfalls

• Prioritising by loudest voice: the single vocal developer, or the biggest account, always wins without a count.
• Collecting feedback and never replying, which teaches developers not to bother.
• Treating only ticket text as feedback and missing the search logs, where the silent majority speaks.
• Unowned backlog: without a weekly triage owner the list grows and stales within months.

Direct answer

The error {"error": "invalid request"} with a 400 tells the developer that they are wrong but not what, where or how to fix it. A good error names the field, says what was expected, gives a stable machine-readable code, carries a request ID for support, and links to help. The safe way to do this is to build errors from a controlled catalogue of messages you wrote, never from raw exception text (the internal error message your code throws, which can reveal how your system is built).

Critique

• Tautology. "Invalid request" restates the 400. No new information.
• No location. Which field or header? Maybe the JSON did not even parse.
• No cause or fix. What was wrong and what is valid?
• No machine-readable code, so client code cannot branch on the failure without string-matching the message.
• No request ID, so support cannot find the log line, which forces a ticket.
• No documentation link.

Rewrite

{
  "error": {
    "code": "invalid_parameter",
    "message": "amount_cents must be an integer of at least 1. Received the string \"25.00\".",
    "param": "amount_cents",
    "request_id": "req_8f3a2c",
    "doc_url": "https://docs.example.com/errors/invalid_parameter"
  }
}

The response also carries Content-Type: application/json (or application/problem+json, the media type of RFC 9457 "Problem Details for HTTP APIs", if you follow that standard; a media type is the label in the Content-Type header that says what format the body is in, and an RFC is a numbered internet standards document). For several problems at once, return an array of {param, code, message} items so the developer fixes everything in one pass rather than one round trip per mistake.

Keeping errors from leaking sensitive detail

1. Catalogue, not exceptions. Map each failure class to a code and a message a person wrote. A catalogue is just a lookup table kept in your code, for example:

invalid_parameter  ->  "{param} must be {rule}."
not_found          ->  "No such resource."
rate_limited       ->  "Too many requests. Retry after {n} seconds."

The code picks the row and fills in only safe values such as the field name. Anything not in the table becomes a generic internal_error plus the request ID. Never pass an exception's text to the client: it can contain SQL, file paths, hostnames or stack traces (the listing of internal code locations printed when a program crashes).
2. Log the detail server-side under the request_id. The developer gets the ID, support gets the internals.
3. Do not echo secrets. If the invalid value could be a token or password, name the field but do not repeat the value. Truncate long echoed input.
4. Avoid enumeration (letting an attacker learn what exists by trying guesses and reading the differences in your answers). Do not say "no account with that email" versus "wrong password" on login; return the same message. Return 404 for another tenant's resource rather than 403, so existence is not confirmed. A tenant is one customer account in a system shared by many. If customer A requests /invoices/1042 and it belongs to customer B, a 403 Forbidden tells A that invoice 1042 exists (just not for them), so A can probe IDs and map B's data. A 404 Not Found is the same answer A would get for an ID that never existed, so nothing leaks.
5. Keep validation messages about the caller's own input, never about internal structure ("column customers.tax_id is null").
6. Test it. Send malformed and hostile inputs in CI and assert responses never contain a stack trace, file path or internal hostname. CI (continuous integration) is the automated test run on every code change.

Pitfalls

• Messages helpful to attackers ("SQL syntax error near...").
• Changing messages breaks clients that parse them: that is why the code, not the message, is the stable part.