Application Programming Interface Design and Scalability Questions

Designing application programming interfaces that remain reliable, performant, and maintainable at high scale. Candidates should understand how interface decisions affect scalability, availability, latency, and operational complexity and be able to reason about trade offs between client complexity and server responsibility. Core areas include stateless interface design, pagination and cursor strategies for large result sets, filtering and search optimization, payload minimization, batching and streaming, and techniques to reduce server load while preserving client experience. Resilience and operational controls include rate limiting and quota management, throttling, backpressure and flow control, retry semantics and idempotency patterns, error format design and explicit identification of retryable errors, and strategies for graceful degradation under overload. Evolution and compatibility topics include backward compatible versioning strategies, deprecation policies, contract design and testing approaches to avoid breaking consumers. Infrastructure and deployment considerations include API gateway and edge patterns, interaction with load balancers and traffic distribution, caching and content delivery, routing fault tolerance, health checks and canary rollout strategies, and observability through metrics, distributed tracing, and logging to support capacity planning and incident response. Security considerations such as scalable authentication and authorization, credential and key management, and permission models are also important. Candidates should be prepared to discuss concrete patterns, trade offs, algorithms, and operational playbooks for designing and running high traffic application programming interfaces.

EasyTechnical

70 practiced

Explain content negotiation for supporting multiple response formats such as JSON and Protobuf in an API. Describe how Accept and Content-Type headers interact with caching, with CDNs and with backward compatibility when introducing a new format.

HardTechnical

63 practiced

You must design an API that supports end-to-end encryption between clients and your service while still enabling server-side filtering or search over some fields. Discuss approaches such as tokenization, deterministic encryption, searchable encryption, client-side indices, and the security and privacy trade-offs of each approach.

Sample Answer

Requirements clarification:- End-to-end encryption (E2EE): server cannot read plaintext.- Server must still support filtering/search on some fields (e.g., email, tags) while minimizing leakage.- Consider latency, scalability, revocation, and regulatory requirements.

Approaches and trade-offs:

1) Deterministic encryption (DET)- What: Same plaintext → same ciphertext (e.g., AES-SIV with fixed IV derived deterministically).- Pros: Simple; supports equality queries and indexing on server.- Cons: Reveals frequency and equality patterns; vulnerable to dictionary/known-plaintext attacks for low-entropy fields.- Use when field has high entropy (UUIDs) or combined with other protections.

2) Tokenization / Format-preserving tokens- What: Client substitutes sensitive values with server-stored tokens via a token service or deterministic mapping.- Pros: Server indexes tokens; tokens can be revoked/rotated centrally.- Cons: Token service becomes a central plaintext vault; requires strong auth and auditing; breaks pure E2EE unless tokenization keys are client-held.

3) Searchable Encryption (SSE / PEKS / ORAM)- What: Cryptographic schemes allowing search over encrypted data (e.g., deterministic SSE for keyword search, structured SSE with encrypted indices, Public-key PEKS).- Pros: Stronger formal guarantees; supports richer queries (conjunctions, range in advanced schemes).- Cons: Complexity, higher latency, larger storage, potential leakage patterns (access/volume patterns). Constructions with minimal leakage exist but are expensive.

4) Client-side indices / Query tokens- What: Clients maintain encrypted indices or compute query tokens they send to server; server matches tokens against stored encrypted tokens.- Pros: Keeps primary secrets client-side; flexible; supports complex queries if client builds index.- Cons: Client storage/CPU overhead; may leak token-query patterns; synchronization complexity.

Hybrid recommendations (practical):- Classify fields by sensitivity & entropy. Use cleartext or DET for non-sensitive/high-entropy fields; use SSE or client indices for sensitive low-entropy fields.- Add defenses: blind indexes (salted HMAC-based tokens per tenant with rotation), padding and bucketing to hide frequencies, rate limits and query auditing.- Key management: client-held keys for E2EE; use hardware-backed storage (TPM/secure enclave). Server NEVER holds long-term plaintext keys.- Revocation: design token/key rotation and re-encryption workflows; consider forward secrecy for messages.

Final guidance:- Prefer layered approach: DET or tokenization for fields where equality search is acceptable and entropy is sufficient; use searchable encryption or client-side indices where leakage is unacceptable. Carefully document leakage profile for each mechanism and validate threat model (insider, passive observer, active attacker).

MediumTechnical

59 practiced

Provide pseudocode for an Express middleware in Node.js that enforces idempotency for POST endpoints using an Idempotency-Key header and Redis. Describe how the middleware handles concurrent identical requests, what to store in Redis (response or status), TTL choices, and how to protect Redis memory from unbounded growth.

Sample Answer

Approach: Use an Express middleware that reads Idempotency-Key from the request. For a new key, create a Redis lock (SET NX) and store a placeholder status; process the request, then atomically store the final response (status headers + body) with a TTL. For concurrent identical requests, other workers wait/poll or return 202 + retry-after until the first finishes, or fetch the stored response once ready.

javascript

// pseudocode (JavaScript/Node.js)
const idempotencyMiddleware = (redis) => async (req, res, next) => {
  const key = req.header('Idempotency-Key');
  if (!key || req.method !== 'POST') return next();

  const redisKey = `idem:${req.path}:${key}`;
  const lockKey = `${redisKey}:lock`;
  // Try to read stored result
  const stored = await redis.get(redisKey);
  if (stored) {
    // stored contains { status, headers, body }
    const parsed = JSON.parse(stored);
    res.set(parsed.headers).status(parsed.status).send(parsed.body);
    return;
  }

  // Try to acquire lock to process request
  const gotLock = await redis.set(lockKey, processId, 'NX', 'PX', 30_000);
  if (!gotLock) {
    // Another process is handling it: poll/backoff or return Retry-After
    // Simple approach: return 202 Accepted with Retry-After
    res.set('Retry-After', '2').status(202).send({ message: 'Request in progress' });
    return;
  }

  try {
    // Buffer the response: override res.send to capture output
    const originalSend = res.send.bind(res);
    let bodyBuffer;
    res.send = (body) => { bodyBuffer = body; return originalSend(body); };

    await next(); // let route handler run

    // After handler, capture status, headers, body
    const result = {
      status: res.statusCode,
      headers: filterHeadersForReplay(res.getHeaders()),
      body: bodyBuffer
    };
    // Store final response with TTL (e.g., 24h)
    await redis.set(redisKey, JSON.stringify(result), 'EX', 24 * 3600);
  } finally {
    // release lock
    await redis.del(lockKey);
  }
};

Key points:- Store full response (status, safe headers, body) so subsequent requests get identical replies. Storing only a status requires reprocessing or client logic.- Concurrency: use a short-lived lock (SET NX with PX) to ensure only one worker processes. Others either poll, wait with exponential backoff, or receive 202 + Retry-After.- TTL: choose based on business — 24h–7d common. Shorter for high-throughput APIs. TTL prevents indefinite storage.- Protect Redis memory: - Use key prefix and TTLs for all idempotency keys. - Run Redis eviction policy (volatile-lru) and size limits. - Limit max size of stored bodies (reject large requests or offload body to blob store and store reference). - Periodically archive/cleanup old keys if needed.- Security: validate/limit header length, authenticate key usage per client to avoid key conflicts across endpoints.

MediumTechnical

69 practiced

You are asked to design a GraphQL API. Explain the N+1 query problem and how data loader patterns and batching solve it. Describe additional mitigations such as persisted queries, query complexity analysis, depth limiting, and caching best-practices for GraphQL endpoints.

Sample Answer

The N+1 problem: when a GraphQL resolver fetches a list (N items) and for each item triggers another DB/API call, you do 1 query for the parent + N queries for children → O(N) extra calls, high latency and DB load. Example: query posts { id, author { id, name } } where author resolver queries DB per post.

Solution — DataLoader / batching:1) Batch requests within a single tick or request lifecycle.2) Cache results per-request to avoid duplicate loads.

Example (JavaScript using facebook/dataloader):

javascript

const DataLoader = require('dataloader')

// batch load function receives array of keys and returns Promise of values in same order
const userLoader = new DataLoader(async (ids) => {
  const users = await db('users').whereIn('id', ids) // single DB call
  const usersById = new Map(users.map(u => [u.id, u]))
  return ids.map(id => usersById.get(id) || null)
})

// in resolvers, use loader from context
const resolvers = {
  Post: {
    author(post, args, ctx) {
      return ctx.loaders.userLoader.load(post.authorId)
    }
  }
}

Why it works: converts N independent loads into one batched query and caches within request.

Additional mitigations:- Persisted queries: store allowed query hashes on server so clients send short hashes, reducing parsing overhead and preventing arbitrary heavyweight queries.- Query complexity analysis: compute a cost metric (fields × multipliers for expensive fields) and reject queries exceeding threshold to prevent abuse.- Depth limiting: cap AST depth to stop extremely nested queries that multiply work.- Caching best-practices: - Use per-field caching where safe (idempotent fields). - Use HTTP caching for persisted queries and GET requests; add Cache-Control, ETag. - Use CDN for public data; use short-lived caches for user-specific data and invalidate on writes. - Combine DataLoader (per-request cache) with a shared read-through cache (Redis) for expensive resolved entities.Trade-offs: aggressive caching and complexity rules can block legitimate queries; tune thresholds and provide monitoring/metrics (resolver call counts, DB query counts, latency).

MediumTechnical

65 practiced

Design an API mechanism that allows clients to request partial responses using a fields selector (for example '?fields=title,author.profile') while keeping cache keys predictable and minimizing combinatorial cache entries. Explain strategies to limit the explosion of cache variants and how you would document the feature for SDK users.

Sample Answer

Clarify requirements: clients can request partial responses via fields selectors (e.g. ?fields=title,author.profile). Goals: predictable cache keys, avoid exponential cache variants, preserve correctness and reasonable freshness.

Design:- Server accepts a normalized fields selector (comma-separated, dot notation). Parse and canonicalize: sort fields, remove duplicates, validate against schema.- Cache key = resource-id + canonical-fields-hash + other vary headers (e.g. Accept-Language). Example: /books/123 | fields=author.profile,title -> key: books:123:fields:sha1(author.profile,title):v3- Always store the full canonical resource in primary storage; caches may store either full responses or materialized partial responses.

Strategies to limit variant explosion:1. Canonicalization & hashing: treat equivalent selectors identically and use short hash in cache key.2. Limit allowed selectors: support only top-level fields + one-level nesting, or ban arbitrary deep selectors; return 400 on invalid.3. Feature flags / whitelists: only allow partial-response on endpoints where it yields significant benefit.4. Predefined field profiles: expose commonly used profiles (summary, detail, compact) and map selectors to profiles; cache per-profile rather than per-arbitrary-set.5. Cache the full object and derive partial view at edge: fetch full cached object and project fields at CDN/edge or gateway—one cache entry per resource.6. TTL alignment: use same versioning for all variants (version token in key) so invalidation is simple.

Consistency & invalidation:- Use a version token (resource ETag or numeric version) included in cache key rather than embedding timestamps.- On update, bump version to invalidate all cached variants.

SDK documentation for users:- Explain syntax, canonicalization rules, and limits (max fields, nesting depth).- Provide examples and recommended profiles (summary/detail).- Show cache semantics: when a selector produces a cached hit vs when server will project from the full object, and how ETag/versioning affects freshness.- Provide client helpers: builder functions to construct canonical selectors and validate against schema (with code samples in major languages).- Include migration notes and error examples (400 for invalid selectors, 413 for too-large selectors).

This balances flexibility for clients with predictable cache behavior and operational simplicity.

Unlock Full Question Bank

Get access to hundreds of Application Programming Interface Design and Scalability interview questions and detailed answers.

Join thousands of developers preparing for their dream job.