API Security, Authentication and Authorization Questions

Direct answer

At the scale described (hundreds of services, millions of requests per minute) the pipeline has
to split into a fast synchronous path that adds only a few milliseconds per request, and a
slower asynchronous path that does the expensive enrichment and model scoring off to the side
and feeds its verdicts back as a cache the fast path can check cheaply. You cannot run full
machine-learning scoring inline on every request at that volume without blowing the latency
budget, so the design's central decision is what stays synchronous (a cheap reputation lookup)
versus what happens asynchronously and only changes future requests (enrichment, scoring, model
updates).

Structured elaboration

Sizing the problem first. "Millions of requests per minute" is on the order of tens of
thousands of requests per second; for example 5,000,000 requests/minute is about 83,000
requests/second sustained. Any synchronous, per-request check has to fit inside a latency budget
of a few milliseconds at that rate, which rules out anything that calls out to a heavyweight
model or an external enrichment API inline.

Data collection and enrichment. Capture request metadata at the edge (source IP, user agent,
TLS fingerprint, authenticated identity if any) and enrich it: geolocation and ASN (autonomous
system number, which identifies the network/ISP a request came from) lookups from a local,
periodically-refreshed dataset rather than a live network call; device fingerprinting from
client-side signals where available. Do the enrichment lookups against an in-memory or
local-cache copy of the reference data, not a network round trip per request, since a network
call per request at 83,000 requests/second is its own outage risk.

Real-time detection and scoring. Split scoring into two tiers:

• A cheap, synchronous tier that checks a precomputed verdict (IP or identity already on a
  blocklist or flagged as high-risk from a shared, low-latency cache) and applies simple rules
  (velocity thresholds) that need no model inference.
• An asynchronous tier that aggregates streaming features (request velocity per identity, ratio
  of failed to successful auth attempts, geographic dispersion of a single credential's usage)
  over sliding windows, and periodically runs those aggregated features through a scoring model.
  The model's output updates the shared verdict cache that the synchronous tier reads, so the
  request path never blocks on model inference; it only ever blocks on a cache read.

Alerting to SIEM/SOAR. Route confirmed and borderline detections to the security team's SIEM
(security information and event management system, which centralizes security logs for
investigation) and SOAR (security orchestration, automation and response, which can trigger
automated response playbooks) rather than only auto-blocking. High-confidence, high-severity
patterns can trigger automated blocking directly; medium-confidence patterns should raise an
alert for a human or a scripted playbook to act on, since automated blocking on a weak signal
risks blocking real users (a false positive that costs revenue and trust).

Automated blocking and the feedback loop. Blocking decisions (IP bans, credential lockouts,
CAPTCHA challenges) need to be revisable: log every automated action with the signal that caused
it, and feed confirmed false positives (a legitimate user who got blocked and later proved it,
for example by successfully completing account recovery) back into the model's training data or
into rule exceptions, so the system's precision improves rather than accumulating permanent
mistakes.

Preserving low latency on request paths. The single most important architectural rule is
that nothing on the synchronous request path may depend on the availability or latency of the
detection pipeline's slower components. If the verdict cache is unreachable, the request path
should fail open to "no additional friction" (log the miss, do not block), not fail closed to
"block everything," unless the product's risk tolerance explicitly demands the opposite for a
specific high-value action (initiating a payment, for example).

Worked example

graph LR
    R[API Request] --> E[Enrichment: geo, ASN, device fingerprint]
    E --> F[Streaming Feature Aggregation]
    F --> ML[Real time ML Scoring]
    ML -->|high risk| B[Auto Block or Challenge]
    ML -->|medium risk| SIEM[Alert to SIEM and SOAR]
    ML -->|low risk| P[Pass Through]
    B --> FB[Feedback Loop]
    SIEM --> FB
    FB --> ML

Reading the diagram left to right: the synchronous request path is only the leftmost box, since
everything from "Streaming Feature Aggregation" onward runs asynchronously against buffered
data, not inline with the request. At 83,000 requests/second, if the synchronous enrichment plus
a verdict-cache read together cost 2ms, that is a fully absorbable addition to a typical API's
latency budget; if the same path instead waited on the "Real time ML Scoring" box per request,
the pipeline would need that box to sustain 83,000 scoring calls per second with sub-millisecond
latency each, which is why that box is drawn as feeding a cache asynchronously rather than
sitting inline.

Trade-offs and pitfalls

• Fail-open vs. fail-closed under detector outage. Failing open (letting requests through
  when the detection pipeline is down) protects availability and revenue but temporarily loses
  abuse protection; failing closed protects against abuse but can turn a detection-pipeline
  outage into a full product outage. State this trade-off explicitly per endpoint rather than
  picking one default for the whole system, since a login endpoint and a public read-only search
  endpoint have very different risk profiles.
• False positives have a real cost, not just a technical one. An overly aggressive
  auto-block tier degrades the experience for legitimate users and generates support load; this
  is why the design routes medium-confidence signals to an alert instead of an automatic block,
  and why the feedback loop exists at all.
• Common wrong turn: scoring every request synchronously "to be safe." This looks more
  thorough on paper but does not scale to the stated volume and turns the fraud pipeline into
  the system's latency bottleneck; the asynchronous, cache-backed design is not a shortcut, it is
  the only version of this architecture that survives the request rate.
• Common wrong turn: treating the feedback loop as optional. Without it, the model's
  precision decays as attackers adapt and as the false-positive rate silently rises, since
  nothing in the system is measuring or correcting for it.

Direct answer

Treat every third-party API integration as an extension of your own attack surface, not a black box you can trust by contract alone. The strategy has to cover three separate failure windows: before onboarding (vendor vetting), during normal operation (isolated credentials, rate limiting, monitoring), and the day the vendor itself gets breached (supply chain containment), because all three have actually happened to real, well-known companies.

Structured elaboration

Vendor vetting
Before integrating, assess the vendor's own security posture: a SOC 2 or ISO 27001 attestation (third-party audits that certify a vendor's security controls), their breach history, how they handle your data at rest and in transit, and whether they support scoped credentials and signed webhooks on their side. Treat this the same way you would vet a subprocessor, because that is functionally what they are.

Credential management
Never share one platform-wide secret across every tenant's connection to a vendor. Issue per-tenant, or per-integration, credentials, so a leak or compromise tied to one tenant's connection does not unlock every other tenant's data through the same key. Rotate on a fixed cadence and on any signal of compromise, and make rotation an operational non-event: run old and new secrets valid in parallel for a short overlap window, so partners are never forced through a hard, scary cutover the moment a secret changes.

Sandboxing and testing
Integrate against the vendor's sandbox or test environment first. Even in production, constrain what the integration's credential and network identity are actually allowed to touch, an egress rule permitting only the vendor's documented IP ranges or domains, not open outbound traffic from that service.

Rate limiting and circuit breakers
Apply outbound rate limits so a runaway retry loop on your own side does not get you throttled or banned by the vendor. Apply inbound circuit breakers so a slow or failing vendor does not cascade into your own API's availability, since a synchronous call to a vendor that hangs can otherwise block your own request threads or connections, turning a vendor incident into your incident.

Monitoring for anomalous behavior
Baseline the normal call volume and pattern per integration, and alert on deviation in either direction: a sudden spike could mean abuse routed through your platform, a sudden drop could mean the vendor silently changed something and you are now failing calls quietly. Track the vendor's own status page and security advisories as an input too, not only your own internal metrics.

Contractual SLAs (Service Level Agreements)
Define uptime expectations, data handling terms, breach-notification timelines (how fast the vendor must tell you if they get breached), and right-to-audit terms. A security control you cannot enforce technically still needs to exist as a contractual obligation with a real consequence attached.

Mitigating supply chain risk
Assume the vendor will eventually be compromised and design so that outcome is survivable: scoped, per-tenant credentials limit blast radius (as above); no standing write access beyond what is actually needed; alerting tuned specifically to catch abnormal use of the vendor-facing credential, not just general auth logs; and a documented, actually-tested kill switch that revokes the vendor's access immediately without requiring a deploy.

Worked example

A platform integrates a third-party shipping-label API. Each tenant gets its own API key issued at onboarding, scoped to that tenant's shipments only. Outbound traffic to the vendor is restricted to their documented IP range via an egress allowlist. A circuit breaker trips if the vendor's error rate crosses a threshold, falling back to a queued retry instead of blocking checkout requests. When the vendor discloses a breach of their own systems, the response is to immediately revoke every tenant's key for that vendor from a single admin action (the kill switch), re-issue fresh keys once the vendor confirms remediation, and check the anomaly-monitoring dashboard for any tenant whose usage pattern changed in the days before the disclosure, since that is the fastest way to spot whether the compromise was already being exploited through this platform.

Trade-offs & pitfalls

Per-tenant credentials multiply operational complexity, more secrets to rotate, store, and monitor, than a single shared key. That cost is worth paying at any meaningful scale because it bounds the blast radius of a single leak, but a small startup that skips it "for now" finds it much harder to retrofit later once hundreds of tenants already share one key. Over-trusting a vendor because they are large or well-known is a real, recurring trap: some of the most damaging third-party compromises in recent memory involved exactly that kind of vendor. Circuit breakers and timeouts that are not tuned aggressively enough still let a slow vendor degrade your own service before the breaker actually trips.

Direct answer

Store the refresh token in an HttpOnly, Secure, SameSite cookie, never in localStorage, where any Cross-Site Scripting (XSS) vulnerability on the page could read and exfiltrate it. Rotate the refresh token on every use, issuing a new one and invalidating the old one atomically, and treat the reuse of an already-rotated-away token as a theft signal that revokes the whole token family, not just that one token.

Structured elaboration

Storage
The refresh token lives in an HttpOnly, Secure, SameSite=Strict (or Lax, if legitimate cross-site navigation into the app is a real requirement) cookie, scoped to the specific refresh endpoint path so it is not sent on every request, shrinking its exposure to only the one call that actually needs it. The short-lived access token can live in memory (a JavaScript variable, never persisted storage) since it is replaced constantly anyway.

Rotation
Every call to the refresh endpoint issues a brand-new refresh token and invalidates the previous one, atomically, on the server. This bounds the value of a stolen refresh token to a single use before it is dead.

Revocation and theft detection
Keep a lightweight server-side record per refresh-token "family," a chain created at login where each rotation produces a new generation of the same family. If a token that has already been rotated away is presented again, that is a strong signal an attacker copied it and is now racing the legitimate user, or that the legitimate user's old token leaked somehow. The response is to revoke the entire family immediately, forcing full reauthentication, rather than trying to guess which of the two callers is the real one.

Minimizing user friction
Refresh silently in the background, using the HttpOnly cookie, proactively before the access token expires rather than only reacting to a 401 response, so the user never sees a login prompt during a normal active session. Rotation itself stays invisible to the user; only a detected theft forces a real reauthentication.

Offline access
For a browser-based SPA, "offline" mainly means the cookie persists across tab close and reopen for a defined session lifetime, with a sliding absolute cap (re-require login after some maximum session age regardless of activity), so a stolen-but-undetected token cannot live forever. This differs from a native or mobile offline-access story, which would rely on device-bound secure storage and a longer-lived credential instead.

Stateless vs stateful trade-off
A pure stateless JWT (JSON Web Token) refresh token, with no server-side record at all, cannot support real-time revocation or reuse detection, since there is nothing to check against. This design deliberately keeps some server-side state, the token-family record, specifically to make revocation and theft detection possible. The token-family record described above is effectively a lightweight, purpose-built revocation list, one row per active family rather than one row per token. The alternative pattern is introspection, where every refresh call is validated by a live call to a central token-issuing service instead of a local record lookup; introspection centralizes the decision even further and makes revocation instantaneous everywhere at once, but adds a network round trip to every single refresh and makes the introspection service itself a shared, latency-sensitive dependency at 1M-user scale. The token-family approach above is the better fit here because refreshes are already infrequent relative to access-token use, so the extra state stays small, cheap, and does not need the always-on network dependency introspection would add.

Worked example (token-family rotation and theft detection)

sequenceDiagram
    participant User as Legit browser
    participant Attacker
    participant API as Backend API

    User->>API: Login
    API-->>User: Refresh token (family F1, gen 0)
    User->>API: Refresh using gen 0
    API-->>User: New refresh token (gen 1)
    Note over API: gen 0 marked used, invalid

    Attacker->>API: Refresh using stolen gen 0
    API-->>Attacker: Reject: gen 0 already used
    Note over API: Reuse of a rotated-away token detected, revoke entire family F1
    API-->>User: Next refresh attempt also fails, forces full re-login

Whichever side, the legitimate user or the attacker, presents the already-rotated-away token second is treated identically: the whole family is revoked, and the legitimate user is forced to re-authenticate. This is a deliberate trade-off (a real user occasionally gets logged out due to a race, see pitfalls below) in exchange for reliably catching theft without guessing.

Trade-offs & pitfalls

Rotating on every call sounds airtight but can break under legitimate concurrency, two open tabs from the same real user issuing near-simultaneous refresh calls can look identical to an attack. Handle this with a short grace window that accepts the immediately-prior token once, rather than an absolute single-use rule that locks out a legitimate second tab. Storing the refresh token anywhere JavaScript can read it, localStorage or sessionStorage, defeats this entire design regardless of how good the rotation logic is, since an XSS vulnerability bypasses all of it in one step. At 1M users, the token-family store needs to be a fast, horizontally scalable keyed cache, not a heavy relational table scanned on every refresh call.

Direct answer

Push the checks that are cheap, universal, and identity-only to the gateway (authentication verification, transport security, coarse rate limiting, structural request validation), and keep the checks that need business or ownership context inside the service (fine-grained authorization, business-aware rate limiting, semantic validation). Never treat the gateway's decision as the only check: services should still re-verify authorization even for traffic they assume already passed the gateway, because a bypass or misconfiguration at the gateway should not be the single point of failure for data access.

Structured elaboration

Worked example

A checkout request hits the gateway without a valid token: the gateway rejects with 401 before the order service, inventory service, or payment service ever see it, saving three services from processing traffic that was never going to be allowed. A validly authenticated request for PATCH /orders/482 is forwarded with a verified user-id header; the order service still checks that the caller actually owns order 482 before applying the patch, because the gateway only confirmed "this is a real, authenticated user," not "this user owns this specific order." If that service-side ownership check were removed on the assumption the gateway already handled it, any authenticated user could edit any order by changing the ID in the URL, which is exactly the Broken Object Level Authorization pattern, one of the most reliably tested authorization failures in API interviews at any level.

Trade-offs & pitfalls

Availability: a gateway that is horizontally scaled and stateless is not a bigger single point of failure than any other tier, but a gateway that grows to hold business logic becomes a deploy bottleneck for every service behind it, so keep it thin on purpose.

Performance: the extra hop at the gateway adds latency, but it is repaid by filtering malformed or unauthenticated traffic before it costs any downstream service compute, which is a net win under load.

Security: defense in depth means the service re-checks authorization even though the gateway already made a coarse allow decision. A common pitfall is deleting service-side authorization checks because "the gateway already checks it," which turns a misrouted internal call, a compromised adjacent service, or a gateway misconfiguration into a full authorization bypass.

Testing and observability: run contract tests that assert the gateway's allowed routes and the service's own authorization rules agree, so they cannot silently drift apart. Run synthetic BOLA probes directly against services in a staging environment, bypassing the gateway entirely, to confirm services do not rely on the gateway as their only defense. Correlate logs across the gateway and every service hop with a shared request ID, track 401/403 rates and rate-limit rejections per route as standing metrics, and alert when the gateway's allow decision and a service's deny decision disagree for the same request, since that disagreement is itself a signal of policy drift between the two layers.

Direct answer

Sign every webhook payload so the receiver can prove it really came from the partner (a shared-secret HMAC, or an asymmetric signature if there is no shared secret), bind a timestamp and a unique event ID into that signature so a captured request cannot be replayed later or against a different event, and record processed event IDs so retried deliveries are absorbed as no-ops instead of double-processed. Handle secret rotation by accepting two valid secrets during a short overlap window instead of a hard cutover.

Structured elaboration

Signing schemes: HMAC vs asymmetric

• HMAC (Hash-based Message Authentication Code) is the common default: both sides share one secret, the sender computes a keyed hash (typically HMAC-SHA256) over the request, and the receiver recomputes the same hash and compares it. Fast, simple, and what most webhook providers (Stripe, GitHub) use.
• Asymmetric signing (the sender signs with a private key, the receiver verifies with the sender's public key) removes the need to ever share a secret at all, which matters when you cannot trust a channel to distribute a shared secret safely, or when many receivers need to verify the same sender's signature without each holding a copy of a secret that could leak. It costs more CPU per verification and adds public key distribution/rotation as its own problem.

Replay defenses

• A timestamp header, signed as part of the payload (not sent alongside unsigned), so an attacker who captures a valid request cannot replay it after altering the timestamp without invalidating the signature.
• The receiver rejects any request whose timestamp is outside a short tolerance window (a few minutes), which bounds how long a captured request stays useful even if replayed unmodified.
• A unique event ID per webhook delivery, checked against a short-lived dedup store (a cache keyed by event ID with a TTL slightly longer than the timestamp tolerance), so an identical request replayed within the tolerance window is still caught.

Idempotency for retries

• Partners retry on timeout or a 5xx response, which means the receiver will see the same event ID more than once by design, not by attack. The fix is the same dedup store used for replay defense: before doing any side effect, check whether this event ID has already been processed; if so, return success without repeating the work.
• The "already processed" check and the "mark as processed" write need to be atomic (a unique constraint in the datastore, or an atomic check-and-set in the cache) so two near-simultaneous deliveries of the same event cannot both pass the check before either marks it done.
• Respond fast: verify signature, check the dedup store, enqueue the actual business work, and return 200 immediately. Doing the real processing synchronously inside the request is what makes retries expensive and dedup races more likely under high volume.

Secret rotation

• Never a hard cutover. Accept signatures computed with either the current secret or the previous secret for a defined overlap window, and include a key identifier in the request headers so the receiver knows which secret to try first instead of testing both on every request.
• Communicate the rotation window to partners, retire the old secret only after the window closes, and treat "a partner is still signing with the retired secret past the deadline" as an operational alert, not a silent failure.

Worked example

A concrete HMAC-SHA256 signature, computed and verified exactly as shown (copy, run, and you will get the same digest since every input is pinned):

import hmac, hashlib

secret = b"whsec_5f8a3c9e2b7d4a1f6e0c8b2d9a4f7c1e"
timestamp = "1735689600"
body = b'{"event":"payment.succeeded","id":"evt_8f2a","amount":4999}'

signed_payload = timestamp.encode() + b"." + body
signature = hmac.new(secret, signed_payload, hashlib.sha256).hexdigest()
print(signature)

Output:

2e702838565a081c906544b3c8d22f6ce61ab8cedeb9babb9f7ae1245e596b14

On the receiving side, verification recomputes the same digest and compares in constant time, then applies the replay and idempotency checks:

def verify_and_should_process(headers, body, secret, seen_event_ids, now, tolerance_s=300):
    timestamp = headers["X-Timestamp"]
    if abs(now - int(timestamp)) > tolerance_s:
        return False, "timestamp outside tolerance"

    expected = hmac.new(secret, (timestamp + ".").encode() + body, hashlib.sha256).hexdigest()
    if not hmac.compare_digest(expected, headers["X-Signature"]):
        return False, "bad signature"

    event_id = headers["X-Event-Id"]
    if event_id in seen_event_ids:
        return True, "already processed, ack without reprocessing"

    seen_event_ids.add(event_id)
    return True, "process"

This is not run in this answer since it depends on request-time state (now, a shared dedup store); the HMAC computation above is the part with fully pinned inputs and a verifiable output.

Trade-offs & pitfalls

Using a plain string comparison (==) instead of a constant-time comparison (hmac.compare_digest) leaks timing information an attacker can use to guess the signature byte by byte. Trusting a client-supplied timestamp that is not itself signed into the HMAC input lets an attacker pair an old, still-valid signature with a fresh timestamp. If the idempotency store is local to one server instance instead of shared, two receiver replicas behind a load balancer can each independently "not have seen" the same event and both process it. Rotating a secret without an overlap window causes a hard outage for every in-flight or slightly-delayed delivery signed with the old secret.