API Integration and System Connectivity Questions

Covers the end to end concepts and practices for connecting systems using application programming interfaces and integration middleware. Candidates should understand REST and HTTP fundamentals, webhooks versus polling, API authentication and authorization patterns, request and response formats, rate limiting, versioning, and reading API documentation. Include practical skills for building and troubleshooting integrations: data mapping and transformation, schema mapping, message formats, error handling and retry strategies, idempotency, logging and monitoring, and ensuring data consistency across synchronous and asynchronous flows. Also includes integration tooling and middleware platforms, real time versus batch synchronization, designing for partial failures and back pressure, and frontend concerns such as async data fetching, loading and error states, pagination, optimistic updates, and race condition mitigation.

MediumTechnical

97 practiced

How would you implement a globally-consistent rate limiter across API gateway instances in multiple regions for per-API-key quotas and burst allowances? Describe algorithms, storage choices (Redis, CRDTs, local caches), consistency trade-offs, and how to keep latency low while avoiding single points of failure.

Sample Answer

Requirements to clarify: per-API-key global quota (rate per second/minute + burst), low latency for API calls from any region, no single point of failure, tolerant to partitions (define strict vs soft quotas), operational visibility.

Recommended hybrid design (practical for multi-region, low-latency, scalable):

1) High-level approach- Use a multi-tiered token-bucket: local fast-path token buckets at each gateway instance (or regional edge) for low-latency decisions + a global authoritative store for replenishment and long-term accounting.- Local buckets allow bursts and immediate acceptance; the global store enforces long-term quotas and prevents abuse across regions.

2) Algorithms & implementation- Local: in-memory token bucket (leaky/token bucket) per API key with replenishment rate = global rate / N_regions and an extra local burst allowance. Use strict local atomic operations.- Global: centralized counters using Redis (clustered, highly-available) with Lua scripts to implement a global token bucket / GCRA (generalized token bucket) or sliding window counters. Use Redis Cluster for sharding and Sentinel/RAFT-backed Redis Enterprise or managed Redis for HA.- Sync protocol: on local token refill, periodically (or when local bucket depletes below threshold) attempt to acquire tokens from global Redis via atomic Lua script: decrement global allowance, return success/failure. Also support a background reconciliation task that returns unused local tokens to global accounting (approximate).- For strict global limits: use a consensus-backed service (small Raft quorum or strongly-consistent Redis via primary in each region with cross-region leader election) and route heavy hitters to that path.

3) Storage choices / CRDTs- Redis: good for low-latency atomic operations and Lua scripts. Use clustered Redis with persistence and replicas to avoid single point of failure.- CRDTs (PN-Counters / grow-only sets): useful if you want fully decentralized, eventually-consistent global counters that converge without coordination. Use when soft quotas are acceptable; they allow per-region increments merged later. Not suitable when strict instantaneous enforcement is required.- Local caches: required for latency; keep small in-memory buckets with TTLs.

4) Consistency trade-offs- Strong consistency (global single counter): enforces exact quotas but increases latency and introduces cross-region calls and potential availability loss during partitions.- Eventual / approximate (local buckets + periodic reconciliation, CRDTs): very low latency and high availability, but allows temporary overuse (bounded by local burst allowances). Use thresholds, heuristics, and monitoring to bound risk.- Practical compromise: enforce short-term limits locally (fast path) and long-term quotas globally (slow path). For high-value API keys, route all requests to strong-consistency path.

5) Avoid single points of failure & keep latency low- Redis Cluster with replicas and multi-AZ deployment; prefer managed services with cross-region replication.- Fallback: if global store unreachable, allow degraded local-only mode with conservative token budgets and alerting; fail-closed for critical keys if business requires.- Use client-side backoff and circuit-breakers, exponential retry for global token acquisition.- Consistent hashing of API keys to Redis shards to distribute load; local caches to reduce global reads.- Use batching and async writes for metrics and reconciliation.

6) Operational concerns- Rate-limit rules store in a config service with versioning and push to gateways via pub/sub (Kafka / CDN config propagation).- Metrics, alerts for overuse, reconciliation drift, and Redis latency.- Test via chaos (partition) to validate degraded behavior.

Summary: hybrid token-bucket with local fast-path + Redis-backed global authoritative accounting balances latency and correctness. Use CRDTs only when eventual consistency is acceptable. Provide strict paths (consensus-backed) for critical quotas and pragmatic fallbacks to avoid single points of failure.

MediumTechnical

93 practiced

Explain patterns to ensure eventual consistency when syncing data using asynchronous events. Cover the outbox pattern, change-data-capture (CDC), event sourcing, idempotent consumers, and compensating transactions. For each pattern describe typical use-cases, benefits, and operational challenges.

Sample Answer

Start with the goal: eventual consistency via async events requires reliable change propagation, ordering (when needed), deduplication, and clear failure/compensation strategies. Below are common patterns, when to use them, benefits, and operational challenges.

Outbox pattern- What: Write state change and outgoing event to the same transactional store (outbox table) in one DB transaction; a separate process reads outbox and publishes events.- Use-cases: Microservices needing atomic DB + event publish (order, inventory, billing).- Benefits: Guarantees no lost events, avoids distributed transactions.- Challenges: Need background poller/relay, ensure exactly-once publish semantics (dedupe, transactional producers), outbox pruning, schema migration complexity.

Change Data Capture (CDC)- What: Capture DB binlog/transaction stream (Debezium/Kafka Connect) and convert row changes into events.- Use-cases: Integrating legacy databases, populating data lakes, syncing read models.- Benefits: Low-intrusion, reliable, preserves DB ordering.- Challenges: Schema evolution, noisy events (filtering), initial snapshot, handling non-relational side effects, operational load on DB/log.

Event Sourcing- What: Persist domain events as primary source of truth; state is materialized by replaying events.- Use-cases: Complex business workflows, auditability, temporal queries, compensating workflows.- Benefits: Perfect audit trail, easy to rebuild state, supports CQRS.- Challenges: Event schema evolution, event versioning, replay performance, operational complexity, larger cognitive load for developers.

Idempotent Consumers- What: Design consumers so reprocessing same event is safe (use idempotency keys, upserts, dedupe stores).- Use-cases: Any distributed consumer facing at-least-once delivery (webhooks, queues).- Benefits: Simplifies reliability, supports retries.- Challenges: Choosing idempotency key, storage growth for dedupe metadata, need for concurrency control.

Compensating Transactions- What: When an operation can’t be strictly ACID across services, define compensating events/actions to undo or reconcile.- Use-cases: Long-running sagas: booking + payment + notification.- Benefits: Supports eventual consistency for multi-step processes, models business rollback.- Challenges: Designing compensations for partial failures, user-visible inconsistency, ordering and retries, complex failure modes.

Operational guidance / trade-offs- Combine patterns: Outbox + CDC + idempotent consumers is common (DB-level outbox then stream via CDC/Kafka).- Observability: Centralize tracing, include correlation IDs, monitor lag, DLQs, and metrics.- Schema/versioning: Adopt backward/forward-compatible schemas (Protobuf/Avro + schema registry).- Testing: Chaos/failure tests, replay tests, and clear runbooks for stuck messages or replays.

As a solutions architect, pick patterns aligned to business SLAs, operational maturity, and existing stack—prioritize simple, observable designs first and introduce event-sourcing or complex sagas only when domain requirements justify their costs.

EasyTechnical

97 practiced

Explain common API authentication and authorization patterns used in modern integrations: API keys, OAuth2 (authorization code, client credentials), JWTs, and mutual TLS (mTLS). For each pattern describe typical threat model, where you'd use it, pros/cons, and operational considerations (rotation, revocation, storage).

Sample Answer

API authentication and authorization patterns — quick, role-appropriate summary for solution design.

API Keys- Threat model: bearer token leaked -> anyone can call API; no cryptographic binding to client.- Use: server-to-server integrations with low risk, or simple public APIs with rate-limiting.- Pros: simple, easy to implement.- Cons: no built-in expiry/claims, weak if leaked, limited granularity.- Ops: store encrypted (KMS/secret manager), rotate regularly (automated deployments), revoke by blacklisting or changing key, enforce usage limits and IP/ACL restrictions.

OAuth2 — Authorization Code (with PKCE)- Threat model: protects against token theft in browser/mobile flows; attacker could intercept code if PKCE absent.- Use: user delegated access (web apps, mobile) where end-user consents.- Pros: user consent, refresh tokens, scope-based authorization.- Cons: complexity, requires auth server and secure redirect handling.- Ops: short-lived access tokens, rotate refresh tokens on use (refresh token rotation), revoke via auth server, store refresh tokens securely (HTTP-only cookies or secure enclave).

OAuth2 — Client Credentials- Threat model: client credentials (client id/secret or cert) compromise grants app-level access.- Use: machine-to-machine server-to-server flows without user context.- Pros: standardized, supports scopes, token lifecycle managed by auth server.- Cons: needs secure storage of client secret; less granular than per-user auth.- Ops: store secrets in vaults, rotate client secrets/certs regularly, revoke client registration when compromised, prefer mTLS or private network when possible.

JWTs (JSON Web Tokens)- Threat model: valid signed JWT misuse; replay of long-lived tokens; key compromise invalidates many tokens.- Use: stateless auth between services, conveying claims without central lookup.- Pros: self-contained, performant (no session DB), supports claims/audiences.- Cons: revocation hard if long-lived; large tokens increase payload; must validate signature, issuer, audience, expiry.- Ops: use short lifetimes, issue refresh tokens as needed, publish and rotate signing keys (JWKS) with key IDs, maintain token revocation list or use token introspection for critical systems.

Mutual TLS (mTLS)- Threat model: strong client and server certificate binding; mitigates impersonation and network-level MiTM.- Use: high-assurance service-to-service communication, B2B integrations, internal critical APIs.- Pros: cryptographic strong client identity, automatic key usage by TLS stack, no bearer tokens on wire.- Cons: operational complexity (PKI management), cert provisioning/rotation overhead, firewall/NAT issues.- Ops: use enterprise PKI or mTLS tooling, automate cert issuance and rotation (ACME/internal CA), maintain revocation via CRL/OCSP, store private keys in hardware/secure modules.

Design guidance- Combine patterns: e.g., OAuth2 + mTLS or JWTs signed by auth server; apply defense-in-depth.- Prefer short-lived credentials, centralized policy enforcement (API gateway), secure storage (vaults/KMS), automated rotation, and monitoring for abnormal usage.- Choose pattern based on threat model, compliance, operational maturity, and integration complexity.

HardTechnical

74 practiced

Design a back-pressure management strategy for a pipeline that begins at an API gateway, passes through a worker pool, and calls third-party endpoints. Include queue sizing, admission control (dropping vs throttling), circuit breakers, dynamic scaling, and observability to detect and respond to overload. Explain how you'd avoid cascading failures.

Sample Answer

Situation overview: we have an ingress API gateway → internal worker pool (async queue) → third‑party endpoints. Goal: keep system responsive under overload, protect downstream and third parties, and avoid cascading failures.

Design (high level):- Edge first: enforce admission control at API gateway to stop overload before internal resources are consumed.- Bulkheads: partition worker pool and downstream clients (per-tenant / per-caller) so one noisy tenant can’t exhaust the whole pool.- Back-pressure chain: gateway → queue (bounded) → worker pool; when queue saturates, gateway applies shedding/throttling policy.

Queue sizing (practical):- Use Little’s Law: Q = λ * W where λ = expected arrival rate, W = target max wait. Pick W based on SLO (e.g., 200ms). Add headroom for bursts (x1.5–2).- Bounded queues per priority class and per tenant; total queue capacity = sum + reserve for critical traffic.- Example: if λ=200 rps, acceptable wait W=0.05s → Q≈10; set per-priority Q=10, global Q=200 (configurable).

Admission control: dropping vs throttling- Prefer graded approach: - Step 1: polite throttling (HTTP 429 + Retry‑After) at gateway using token bucket for fair smoothing. - Step 2: if sustained overload or critical queue fill, degrade non‑critical features (circuit degrade) and switch to shedding: drop low-priority requests immediately (return 503/429).- Use priority + SLAs: always accept high-priority/health-checks, shed background/bulk tasks.

Circuit breakers & resiliency to downstream- Per‑downstream circuit breakers (concurrent failures, error rate, latency). States: closed → open → half‑open with exponential backoff and test probes.- Use adaptive thresholds: if downstream latency > threshold or error-rate spikes, open circuit early to protect worker threads and queues.- Fast fail in worker when downstream CB open: return controlled error to caller (or enqueue for retry if idempotent).- Retries: bounded, with exponential backoff + jitter and retry budget tracking to avoid retry storms.

Dynamic scaling- Horizontal autoscaling of worker pool based on multiple signals: queue length, request latency, error rate, CPU. Use multi-metric rules (scale-up when queue > Q_high and CPU>50%; scale-down conservatively).- Warm pools / pre-warming to meet sudden bursts; scale faster up than down.- Use graceful drain on scale-in; respect in-flight limits.

Observability & detection- Metrics: per-tenant and global request rate, queue occupancy, worker concurrency, downstream latency/error-rate, circuit breaker state, 429/503 counts, retry counts, SLA latency percentiles.- Distributed tracing to see slow segments (gateway → queue wait → worker → third party).- Alerts: queue occupancy > threshold, sustained 5xx increase, CB open-rate > X, scaling failures.- Dashboards: heatmap of per-tenant queue usage and error injection points.- Playbooks: automated actions (auto-shed low priority traffic, spin up new workers), and human escalation steps.

Avoiding cascading failures- Edge shedding prevents overload propagation.- Bulkheads + per-tenant quotas limit blast radius.- Circuit breakers stop slow/failed downstreams draining resources.- Backpressure propagation: gateway reduces ingress upon internal queues saturated (closed-loop control).- Limit retries and make them client-aware (Retry-After).- Maintain a small reserve of capacity for health-checks and control messages.- Use graceful degradation: serve cached or reduced functionality rather than full failure.

Trade-offs and notes- Aggressive shedding improves stability but hurts availability; calibrate per business priority.- Autoscaling reduces manual intervention but has latency — combine with proactive limits.- Observability and automated runbooks are essential; simulate failures (chaos testing) to validate thresholds.

Example technologies- API gateway: Envoy/NGINX with rate limiting and token buckets.- Queue/workers: Kafka/RabbitMQ/Redis streams or in‑process bounded queues.- Circuit breakers: Hystrix-like or resilience4j/OpenTelemetry for CB state emit.- Orchestration: Kubernetes HPA with custom metrics.

This strategy gives layered protection: prevent overload at ingress, isolate failures with bulkheads and CBs, respond dynamically with scaling, and maintain visibility and automation to detect and recover—minimizing cascading failures while preserving critical availability.

HardTechnical

89 practiced

A customer requires transmission and storage of PII via integrations for 7 years fulfilling GDPR and other privacy constraints. Design an end-to-end architecture covering encryption in transit and at rest, key management, access control, audit logging, data minimization, retention policies, right-to-erasure, and how to keep aggregated historical financial records intact when deletion is requested.

Sample Answer

Requirements (clarify): store/transmit PII for 7 years, satisfy GDPR (data subject rights incl. erasure), maintain aggregated historical financial records when individuals request deletion, strong encryption, auditability, scalable key management and retention controls.

High-level architecture:- Clients ↔ API Gateway (mTLS) → Ingress services → Validation & Pseudonymization layer → Processing/Storage- Persistent stores: Encrypted Object Store (S3-like) for raw files, Encrypted DB for records, Data Warehouse for aggregated analytics- KMS/HSM for keys, IAM & ABAC for access, SIEM for logs, Workflow engine for DSARs (Data Subject Access/Erasure Requests)

Encryption- In transit: TLS1.3, mutual TLS for partner integrations, certificate pinning, strict cipher suites, HSTS.- At rest: Envelope encryption. Data encrypted with data keys (DEKs) using AES-256-GCM; DEKs encrypted by Customer Master Keys (CMKs) in a FIPS 140-2/3 HSM-backed KMS (cloud or on-prem).- Per-tenant/per-record DEKs for high-sensitivity PII; deterministic encryption only when needed for search (with caution).

Key management- HSM-backed KMS (e.g., AWS KMS CloudHSM, Azure Key Vault HSM, Vault with HSM). CMKs with key policies, separation of duties.- Envelope encryption implemented in services. Rotate CMKs annually, rotate DEKs per-record or per-batch. Maintain key versioning.- Key lifecycle: create → use → rotate → retire → delete. Support scheduled key rotation and automated re-encryption or maintain DEK wrapping to avoid full re-encryption.- Crypto-shredding: to irreversibly delete data, securely delete/wipe DEKs and/or CMKs (subject to legal/retention hold).

Access control- Principle of least privilege, role-based IAM + attribute-based policies for data access (e.g., by environment, role, purpose).- Network segmentation, bastion hosts, private endpoints. Services authenticate via mTLS + short-lived tokens (OIDC), mutual authentication for inter-service calls.- Data access APIs enforce purpose/consent checks and attribute-level filtering (only return minimized fields).

Audit logging & monitoring- Immutable, tamper-evident logging (WORM, append-only) for all access, key operations, key rotations, DSAR workflow actions. Logs pushed to SIEM and retained per compliance (encrypted, integrity-checked).- Real-time alerts for anomalous access (SIEM, UEBA). Periodic audit reports and access reviews.

Data minimization & design- Ingest only required PII. Use pseudonymization: replace direct identifiers with irreversible pseudonyms or tokens; store mapping in a separate, highly protected token vault with stricter access controls.- Use field-level hashing/salted tokens for lookup where necessary. Prefer reversible tokenization only when business-critical.

Retention & Right-to-Erasure (DSAR)- Retention engine manages retention policies per data type and legal/regulatory rules (e.g., 7-year retention for financial PII). Policies describe retention start, expiry, legal holds.- DSAR flow: 1. Verify identity/authorization. 2. Check applicable retention/legal hold. If legal hold exists, deny erasure with legal reason; log decision. 3. If no hold and within retention, enforce deletion/pseudonymization per policy. 4. Actions: remove PII from primary stores, delete mapping in token vault, crypto-shred DEKs or zero-out fields and replace with anonymized marker. Record audit trail of deletions.- Implement automated background jobs to purge data when retention expires; ensure backups and replicas honor deletions (retention-aware backup lifecycle, use backup-level encryption keys and deletion synchronized).

Keeping aggregated historical financial records intact after deletion- Store analytics-ready aggregates using irreversible aggregation/anonymization before or at ingestion. Two approaches: 1. Derive and store aggregates that do not contain PII (e.g., sums, counts, averages) and ensure aggregation is at a granularity that prevents re-identification (k-anonymity thresholds). 2. Use provable unlinkability: keep aggregates computed from pseudonymized identifiers; when DSAR erases PII, delete token mapping but retain aggregates since they cannot be re-associated to individuals. For extra safety, apply differential privacy/noise or ensure group sizes > threshold.- If raw transaction-level data is needed for accounting and must remain, replace PII fields with irreversible hashes or tokens and keep transaction attributes necessary for auditing. If law requires preserving transactional evidence (e.g., for tax), maintain irreversibly pseudonymized records that preserve amounts/timestamps but not identity.- Document and get legal sign-off for retention of aggregated/anonymized records as permissible under GDPR (scientific/statistical/archival or legal obligations). Ensure Data Protection Impact Assessment (DPIA) records rationale.

Operational considerations & trade-offs- Crypto-shredding is fast but must coordinate with backups and replicas; recommend key destruction combined with metadata deletion and backup lifecycle management.- Deterministic encryption enables search but increases linkage risk—use only when necessary and restrict access.- Balance re-encryption cost vs. wrapping keys: using per-record DEKs wrapped by CMKs reduces need to re-encrypt large datasets on CMK rotation.- Regular penetration tests, periodic key compromise drills, and documented incident response.

Example technologies (illustrative)- Ingress/API: Kong/Envoy with mTLS- KMS/HSM: AWS KMS + CloudHSM, HashiCorp Vault with HSM- Storage: S3 with SSE-C/SSE-KMS, PostgreSQL Transparent Data Encryption or field encryption libraries- Tokenization: Vault Transit + dedicated token vault- Monitoring: Splunk/Elastic SIEM, AWS CloudTrail, immutable storage- Workflow: Argo Workflows or BPM engine for DSARs

Summary: enforce layered protections—strong transport and at-rest encryption with HSM-backed KMS and envelope encryption, strict IAM/ABAC and tokenization for minimization, robust immutable audit trails, retention engine honoring legal holds, and a designed approach to keep aggregated/anonymized financial records by removing linkability while preserving necessary business/audit data.

Unlock Full Question Bank

Get access to hundreds of API Integration and System Connectivity interview questions and detailed answers.

Join thousands of developers preparing for their dream job.