HTTP Protocol and Web Fundamentals Questions

Cross-Origin Resource Sharing (CORS) is a browser-enforced security mechanism that controls how a web page from origin A (scheme+host+port) can request resources from origin B. Browsers enforce it to protect users from malicious cross-site actions (e.g., a malicious site making authenticated requests to your bank) by restricting which cross-origin requests are allowed.

Key pieces:

• Access-Control-Allow-Origin: Response header that tells the browser which origin(s) may read the response. It can be a specific origin (https://app.example.com) or "" (allow any origin). For credentials (cookies/Authorization) it must be a specific origin and not "".
• Access-Control-Allow-Headers: Response header listing which custom request headers the server allows (e.g., Authorization, X-Requested-With). Used when the client sends non-simple headers.
• OPTIONS preflight: For "non-simple" requests (methods other than GET/POST with simple content-types or when custom headers/credentials are used), the browser sends an OPTIONS request first to ask the server what is permitted. The server must respond with CORS headers (Allow-Origin, Allow-Methods, Allow-Headers, optionally Allow-Credentials and Max-Age).

Safe CORS configurations:

• Public API used by browsers: Allow only the specific client origins (explicit list), return Access-Control-Allow-Origin with that origin, set Access-Control-Allow-Credentials if cookies/tokens are needed (true) and ensure server-side auth/CSRF protection; keep Access-Control-Allow-Methods tight (GET, POST, etc.) and enumerate needed headers. Example: only https://app.example.com and https://admin.example.com.
• Strictly internal API (non-browser clients): Avoid broad browser CORS allowances. Do not set Access-Control-Allow-Origin to "*" unless truly public and safe. Prefer no CORS headers if only used by server-to-server calls. If some internal web dashboards require browser access, whitelist internal origins and require mTLS or VPN/IP restrictions.

Principle: CORS is a browser convenience for enforcing same-origin policy — servers must still authenticate and validate requests; CORS should be an additional, minimal access-control layer, not the primary defense.

Start by clarifying goals: allow clients to begin processing as data is produced, avoid OOM on server or client, handle slow consumers (backpressure) and allow clients to resume after disconnects.

Approach comparison

• Chunked Transfer Encoding (HTTP/1.1): server sends a sequence of byte chunks with no Content-Length. Pros: universal, simple to implement, low latency. Cons: no built-in framing/semantic events, no multiplexing, hard to resume in middle except by application-level offsets.
• Server-Sent Events (SSE): text/event-stream over HTTP. Pros: simple event framing, reconnection & Last-Event-ID support for resuming; browser-friendly. Cons: unidirectional, limited binary support, single stream per connection.
• HTTP/2 Streaming: multiplexed frames with flow control per stream. Pros: built-in flow control/backpressure via WINDOW_UPDATE, multiplex many streams on one connection, binary-safe. Cons: requires HTTP/2 stack, more complex server logic.
• Resumable Downloads (range requests / checkpointing): use HTTP Range header or application-level chunk IDs. Pros: precise resume, efficient for large files. Cons: requires client coordination and server support.

Implementation details

• Backpressure:
  • HTTP/1.1 chunked: implement server write throttling tied to socket write readiness (non-blocking I/O). Use reactive streams or write buffers; drop or pause data production when output buffer high.
  • SSE: similar to chunked; browsers auto-reconnect but server must avoid unbounded buffering.
  • HTTP/2: leverage per-stream flow control (WINDOW_UPDATE). Use server libraries that expose window sizes so producer pauses until window opens.
• Resuming:
  • Use durable checkpoints: include sequence numbers, byte offsets or chunk IDs in each chunk/event.
  • For files, support Range requests; for generated streams, accept a resume token (e.g., checkpoint ID) in query/header and start producing from that offset.
  • For SSE, use Last-Event-ID header; for custom protocols include token in reconnect URL.
• Practical pattern:
  • Stream data as framed records (length-prefixed or newline-delimited JSON) so consumers can re-sync.
  • Persist mapping from checkpoints to offsets (in DB or object store).
  • Offer short-lived resume tokens (signed) to prevent replay and control cost.
• Error & disconnect handling:
  • Make streams idempotent or support replay; on disconnect, server can keep temporary state for a grace period.
  • Log metrics for slow consumers and consider dropping connections that exceed buffering limits.

Example stack choices

• For low-latency & many streams: HTTP/2 with gRPC or raw HTTP/2 frames.
• For browser clients: SSE with Last-Event-ID or chunked with newline-delimited JSON.
• For resumable large file transfers: Range headers or application-level chunk endpoints + signed checkpoint tokens.

Trade-offs summary

• Use HTTP/2 when you need robust built-in flow control and multiplexing.
• Use chunked or SSE for simplicity/broad compatibility.
• Always add framing, checkpoints, and server-side flow control hooks to implement backpressure and resumability robustly.

Content negotiation lets an API serve multiple representation formats (JSON, Protobuf) by using HTTP headers to choose the best response for the client.

Basic flow:

• Client sends Accept: application/json or Accept: application/x-protobuf (or application/protobuf).
• Server inspects Accept and returns the matching Content-Type: header with the payload encoded accordingly. If no match, return 406 or a sensible default (e.g., JSON).

Caching and CDNs:

• Cache keys must include the negotiated format. Use Vary: Accept (or more specific Vary: Accept-Encoding, Accept) so caches and CDNs store separate entries per format. Without Vary, a CDN could serve protobuf bytes to a JSON client.
• Prefer explicit content-type-based routes or file extensions for long-lived CDN cached assets when possible (e.g., /v1/resource.json vs /v1/resource.protobuf) to avoid Vary complexity.
• Ensure CDNs respect Vary and forward Accept header or configure edge rules to normalize/route by format.

Backward compatibility when adding a new format:

• Keep JSON as default for older clients; only serve new format when client requests it.
• Maintain the same semantic API contract (field names/semantics) across formats and evolve with versioning or additive fields. For Protobuf, add optional fields with stable numeric tags.
• Provide clear media types and documentation, and a feature-detection header (e.g., Accept) or explicit query param for clients that can't set Accept.
• Test caches and clients to ensure intermediaries don’t strip or ignore Vary/Accept.

Examples:

• Response header for protobuf: Content-Type: application/x-protobuf; charset=binary
• Response header for JSON: Content-Type: application/json; charset=utf-8
• Ensure server sets Vary: Accept on responses to allow safe caching per format.

This approach minimizes breaking changes, keeps CDNs correct, and lets clients opt into more efficient formats like Protobuf.

HTTP caching for REST APIs improves performance by letting clients or intermediaries reuse responses instead of re-fetching every time. Key mechanisms:

Cache-Control (response header)

• public: response may be cached by any cache (CDN, browser). Use for static, shareable resources.
• private: only the end-user’s browser may cache; intermediaries must not store it. Use for user-specific data.
• max-age=<seconds>: how long the response is fresh. Clients/Proxies serve from cache without revalidation during this window.
• no-cache: cache may store the response but must revalidate with origin before serving (not “don’t store”).
• no-store: do not store at all (sensitive data).

Validation: conditional requests to avoid full payloads

• ETag / If-None-Match: server returns a strong/weak ETag header (hash/version). Client sends If-None-Match with the ETag; if unchanged, server replies 304 Not Modified with empty body.
• Last-Modified / If-Modified-Since: server provides last-modified timestamp; client sends If-Modified-Since. If not modified, server responds 304.

When to return 304: when validation shows the resource hasn’t changed (ETag matches or last-modified ≤ since date). 304 keeps headers (updated caching headers allowed) but omits body to save bandwidth.

Practical guidance:

• Use Cache-Control + ETag for best control (ETag precise, works across partial changes).
• Prefer public + long max-age for immutable assets (versioned URLs). For user-specific endpoints return private or no-store.
• For APIs that must reflect fresh auth/permissions, avoid public caching and ensure authenticated responses include appropriate cache headers.
• Ensure ETag generation is deterministic (e.g., hash of payload or version) and lightweight.