Backend-for-Frontend and Client-Specific API Design Questions

Overview
Design a resumable, secure browser → S3 upload flow using S3 multipart presigned URLs, server-side coordination, and post-upload virus scanning. Key goals: show progress, allow retries, prevent cross-user overwrite, and only mark files available after scanning.

Endpoints

• POST /upload/initiate
  • Inputs: user_id, filename, size, mime
  • Server: create DB record (upload_id, owner, status=initiated), generate S3 multipart createMultipartUpload on server to get uploadId, return uploadId + per-part presigned PUT URLs (or endpoint to request per-part presign)
• GET /upload/presign-part?upload_id=&part_number=
  • Short-lived presigned PUT for each part
• POST /upload/complete
  • Inputs: upload_id, parts ETags
  • Server: calls S3 CompleteMultipartUpload, verifies parts, updates DB status=uploaded, enqueues virus-scan job
• GET /upload/status?upload_id=
  • Returns progress, scan status, final object URL (signed or public) when available
• POST /upload/cancel
  • Aborts multipart upload and cleans DB

S3 configuration & keys

• Bucket: private, block public access
• Object key: uploads/{user_id}/{uuidv4()}/{original_filename}
  • Use UUID per upload -> prevents overwrite
• Store temporary uploads under prefix "quarantine/" or tag metadata scanned=false

Presign lifetime strategy

• Very short TTL for each presigned PUT (e.g., 5–15 minutes)
• If long uploads, obtain presigns per-part on demand from GET /upload/presign-part; server validates user ownership and upload_id
• Server enforces max part size & total parts

Frontend chunk coordination & progress

• Split file into S3 multipart-compliant parts (5 MB min except final)
• Upload parts in parallel (limit concurrency 3–6), retry transient failures with exponential backoff
• Track bytes uploaded per part; compute overall progress = sum(uploaded bytes)/total
• If a presign expires, request a new presign for that part and retry
• On network loss, resume by querying completed parts from server/S3 and uploading remaining parts

Finalize & verify

• After client sends parts ETags, server CompleteMultipartUpload
• Move object to quarantine prefix (or keep and tag scanned=false)
• Enqueue virus scan (SQS -> Lambda/EC2 with ClamAV or third-party scanning service)
• On scan pass: update DB status=available, move object to final prefix or set scanned=true and create final signed URL
• On scan fail: mark status=infected, remove object, notify user

Security & ownership

• DB maps upload_id → owner; server checks this on every presign/complete request
• Use immutable UUID keys so users can’t overwrite others’ objects
• All presigns validated server-side; short TTLs and per-part on-demand reduce exposure

Monitoring & retries

• Log multipart progress, failed parts, scan results
• Background job to abort stale uploads after TTL (cleanup) and reclaim storage

This flow provides resumability, progress visibility, strong ownership guarantees, and ensures files are only published after server-side virus scanning.

Clarify requirements & assumptions

• Order creation calls Payment Service (charge) and Inventory Service (reserve) as two-step saga. Both services are separate microservices with HTTP + async events. Compensation: refund payment, release inventory.

API endpoints & flow (orchestrator-based)

• POST /orders -> returns 202 + orderId, sagaToken (idempotency)
• GET /orders/{id}/status -> returns state: pending, succeeded, failed, compensated
• Internals: Orchestrator calls:
  • POST /payments/charge { orderId, amount, paymentMethod, sagaToken }
  • POST /inventory/reserve { orderId, items, sagaToken }
• Compensation endpoints:
  • POST /payments/refund { orderId, chargeId, sagaToken }
  • POST /inventory/release { orderId, items, sagaToken }

Message / event contracts

• Command messages: { type, orderId, sagaToken, payload }
• Events published: OrderCreated, PaymentSuccess {chargeId}, PaymentFailed {reason}, InventoryReserved, InventoryFailed, PaymentRefunded, InventoryReleased, OrderCompleted, OrderCompensated
• Use compact json schema and version field.

Idempotency

• Use sagaToken (UUID) passed to all downstream calls; services persist processed tokens for deduplication.
• For payment, also use provider idempotency key (passed to gateway).

Saga state storage

• Store saga state in durable store (Postgres / DynamoDB) table: orderId, sagaToken, currentStep, attempts, events[], lastUpdated, status.
• Use optimistic locking or single-writer orchestrator instance (leader election) to avoid races.
• Retain event log for audit and retries.

Frontend UX for intermediate & final states

• On POST /orders return 202 with orderId & initial state = pending.
• Poll or use WebSocket/Server-Sent Events to update:
  • pending (waiting for operations)
  • succeeded/completed (green, show chargeId, reserved items)
  • failed (show failure reason and options)
  • compensated (show that payment was refunded and items released; show timeline and contact support)
• Show progress steps (Payment → Inventory → Final), with timestamps and retry guidance.

Error handling & retries

• Exponential backoff, bounded retries, poison queue to alert humans.
• Circuit breakers for downstream services.

Trade-offs: Orchestration vs Choreography

• Orchestration (recommended here for order): centralized control, easier to reason about compensations, simpler to present deterministic user-facing status. Downside: single orchestrator complexity and potential bottleneck.
• Choreography: services emit events and react; more decoupled and scalable, fewer central dependencies. Downside: harder to guarantee ordering, implement multi-step compensations, and show unified frontend status.
• Hybrid: orchestrator for critical flows, choreography for non-critical eventual consistency events.

Reasoning: as a full-stack dev, orchestration gives clearer UX/state management and simpler idempotency/compensation coordination; choose choreography when teams require loose coupling and eventual consistency tolerance.

Approach summary
Design an incremental, operation-log + checkpoints protocol using CRDTs (preferred) or OT, with Last-Writer-Wins (LWW) only for simple meta fields. Clients keep local op-log + checkpoint (vector clock / causal-dot); server stores authoritative op-log shards and periodic checkpoints for fast catch-up.

Conflict-resolution comparison

• CRDTs (recommended): convergent, no central transform, works well offline, handles concurrent inserts/deletes in rich-text (use sequence CRDT like RGA or Yjs/Yjs-like structures). Pro: deterministic, simple client merging. Con: larger metadata overhead.
• Operational Transform (OT): lower metadata, good for central server coordination. Pro: smaller ops; Con: complex transformation logic, hard offline-first without central operation ordering.
• Last-Writer-Wins: simple for simple fields (title, tags). Not safe for collaborative rich-text — data loss risk.

Server endpoints
Provide REST/HTTP+JSON (or gRPC) for incremental sync:

• POST /sync/push
  • body: { clientId, baseCheckpoint, ops: [ {id, type, payload, causalDot, timestamp} ] }
  • returns: { acceptedOps: [...], conflictHints }
• GET /sync/pull?clientId=&sinceCheckpoint=
  • returns: { ops: [...], checkpoint: <serverCheckpoint>, snapshotURI? }
• POST /sync/checkpoint
  • body: { checkpointId, rootHash, tombstonesSummary }
  • returns: 200
• POST /attachments/init
  • body: { noteId, filename, size, contentType, checksum }
  • returns: { uploadUrl, attachmentId, chunkSize }
• POST /attachments/complete
  • body: { attachmentId, checksum, encryptionMeta }

Example token/checkpoint format (compact JSON Web Token-like):

• checkpoint: base64url( { nodeId, seq: 1245, vectorClock: {clientA:1245,clientB:900}, rootHash } )

Store ops as append-only immutable records with:

• op id (uuid + clientId)
• causal dot/vector
• type (insert/delete/format/attach-ref)
• payload (small, base64 for binary metadata)
• signature/HMAC for integrity

Storage & token formats

• Server DB: append-only table (Postgres/Wal-based) + periodic materialized snapshots in object store (S3). Use indexes by noteId and sequence.
• Client checkpoint tokens: compact base64 JSON including vectorClock and rootHash; sign with HMAC to prevent tampering.
• Ops encoded as newline-delimited JSON or protobuf for compactness.

Attachments handling

• Store attachments outside op-log in object storage (S3) with content-addressed storage (SHA-256). In op-log store only attachment references (attachmentId = sha256 + size).
• Upload flow:
  • Client POST /attachments/init → server returns presigned upload URL (multipart/chunked) and per-chunk signed upload tokens.
  • Client uploads encrypted chunks client-side (AES-GCM with key per attachment). Send attachment metadata to server with attachmentId and encrypted-key wrapped with server public key.
  • On complete, server verifies checksum and stores metadata.
• Security: encrypt attachments client-side; server never has plaintext key unless user chooses server-side key sharing. Use per-attachment keys stored encrypted by user keys or KMS for share scenarios.
• Efficiency: dedupe by content-hash, range-chunk resumable uploads, lazy download (only fetch when rendering or requested), CDN for delivery.

Edge cases & operational notes

• GC tombstones: compact CRDT periodically; produce new checkpoint snapshot and truncate op-log up to checkpoint.
• Large offline gaps: client sends baseCheckpoint; server computes delta via seq/vector diffs; send snapshot if delta too large.
• Auth/Integrity: JWT for auth, HMAC-signed checkpoints, per-op signatures optional for audit.
• Testing: simulate high-concurrency merges, network partitions, attachment corruption.

This design favors CRDTs for correctness offline, uses compact checkpoint tokens + append-only ops for incremental sync, and treats attachments as encrypted, deduplicated external objects referenced in the op-log for secure, efficient handling.

Approach (brief)

• Use fetch with an external AbortController so caller can cancel.
• Retry up to 3 times on network errors or 5xx responses, with exponential backoff 500ms, 1000ms, 2000ms.
• Do not retry on 4xx; surface the response error to caller.
• Throw clear errors including attempt count and original response when applicable.

Code

// fetchWithRetries.js
async function fetchWithRetries(url, options = {}, maxRetries = 3) {
  const baseDelay = 500; // ms
  const controller = options.signal ? null : new AbortController();
  if (controller) options.signal = controller.signal;

  let attempt = 0;
  let lastError = null;

  while (attempt <= maxRetries) {
    try {
      attempt++;
      const res = await fetch(url, options);

      // Do not retry on client errors (4xx)
      if (res.status >= 400 && res.status < 500) {
        const body = await res.text();
        const err = new Error(`Request failed ${res.status}`);
        err.status = res.status;
        err.body = body;
        throw err;
      }

      // Retry for 5xx or return successful response
      if (res.ok) return res;
      lastError = new Error(`Server error ${res.status}`);
      lastError.status = res.status;
      // fall through to retry logic
    } catch (err) {
      if (err.name === 'AbortError') throw err; // surface cancellation immediately
      lastError = err;
    }

    if (attempt > maxRetries) break;

    // exponential backoff: 500, 1000, 2000
    const delay = baseDelay * Math.pow(2, attempt - 1);
    await new Promise((resolve, reject) => {
      const timeout = setTimeout(resolve, delay);
      // If caller provided an external signal, abort the wait when signaled
      const signal = options.signal;
      if (signal) signal.addEventListener('abort', () => {
        clearTimeout(timeout);
        reject(new DOMException('Aborted during backoff', 'AbortError'));
      }, { once: true });
    });
  }

  // After retries exhausted
  const finalErr = new Error(`Failed after ${attempt} attempt(s)`);
  finalErr.cause = lastError;
  throw finalErr;
}

// Usage:
const ac = new AbortController();
fetchWithRetries('/api/resource', { method: 'GET', signal: ac.signal })
  .then(res => res.json())
  .then(data => console.log(data))
  .catch(err => {
    if (err.name === 'AbortError') console.log('Request canceled');
    else console.error('Fetch failed:', err);
  });

// To cancel:
// ac.abort();

Notes

• Caller can pass their AbortController.signal to cancel immediately.
• 4xx responses are thrown immediately with status/body for caller handling.
• Errors include cause/attempt info so callers can log or surface appropriate UI messages.

Overview & requirements clarification
Design for 50k RPS peak, public API, low latency (<100ms p95), strong auth, high resilience and observable.

High-level architecture

• Client -> CDN/Edge (CloudFront/Cloudflare) -> API Gateway (AWS ALB + API GW / Kong/Envoy) -> Authentication layer -> Traffic router/load balancer -> Microservice cluster (K8s) -> Datastore (read replicas + write master / sharded DB) and caches (Redis/Clustered Memcached).

Component responsibilities

• Edge/CDN: terminate TLS, WAF, geo-routing, aggressive caching of public GETs, rate limiting.
• API Gateway: auth token validation (JWT verification + introspection for opaque tokens), routing, request/response transforms, circuit-breaker hooks, rate limiting per API key.
• Auth at scale: validate JWTs at gateway (local public key cache via JWKS with rotation). For opaque tokens, use a lightweight introspection cache (Redis) to avoid auth-server hot spots.
• Microservices: stateless containers, expose health checks, implement retries with exponential backoff, local L1 cache for hot reads.
• Databases: OLTP RDS with read replicas + write scaling via partitioning; use DynamoDB/Cassandra for massive scale public endpoints.
• Caching: CDN for public GETs (cache key = URL + headers), API Gateway/edge cache for semi-public, Redis as centralized low-latency cache with TTLs and cache-aside pattern.
• Resilience: circuit breakers at gateway and client libs (Hystrix-like), bulkheads per downstream, graceful degradation endpoints.
• Autoscaling: HPA in K8s based on CPU/RPS/latency; cluster autoscaler; scale DB read replicas and Redis cluster shards.
• Observability: distributed tracing (OpenTelemetry), structured logs, metrics (Prometheus + Grafana), alerts on SLO breaches.
• Failover: multi-AZ + multi-region deployment, cross-region failover with active-passive data replication and DNS failover (health checks), fallback responses for non-critical features.

Why key choices

• JWT validation at edge reduces latency and load on auth server; introspection cache for opaque tokens balances security and scale.
• CDN offloads read-heavy public endpoints, reducing origin traffic dramatically.
• Circuit breakers + retries prevent cascading failures and allow graceful degradation.

This design balances throughput, security, and resilience suitable for a full-stack engineer to implement end-to-end.