Real-Time APIs with WebSockets and SSE Questions

Requirements & constraints:

• Keep connections open (no forced reconnects)
• Older clients must continue to get usable events
• New clients can receive richer fields
• Safe, gradual rollout and observability

High-level strategy

1. Explicit envelope + semantic versioning

   • Each SSE data payload is a JSON envelope:
     {
     "schema":"stream.events.v1",
     "version":"1.2.0",
     "event":"order.updated",
     "ts":"2025-12-06T...Z",
     "data":{ ... }
     }
   • Use MAJOR.MINOR.PATCH. MAJOR bumps are incompatible.

2. Server capability negotiation (handshake)

   • Clients include Accept-Schema or Accept-Version header at SSE subscribe GET (or ?accept=stream.events.v1,>=1.1.0).
   • Server sends an initial "server.hello" event immediately over SSE listing supported schemas/versions and feature flags:
     {
     "event":"server.hello",
     "supported":["stream.events.v1:>=1.0.0","stream.events.v2:>=2.0.0"],
     "defaults":{...}
     }
   • If client provides no header, server assumes safe baseline (lowest supported MAJOR).

3. Versioning policy & compatibility rules

   • Additive changes (new optional fields) => bump MINOR: safe for old clients (they ignore unknown fields).
   • Required/structural changes => bump MAJOR: treated as incompatible; server will not send MAJOR+1 to clients that negotiated MAJOR N unless using a migration gateway.
   • Deprecate fields with at least 2 MINOR cycles and inform via schema registry and server.hello.

4. Runtime transformation & migration path

   • Implement a transformation layer on the server that can:
     • Emit events in multiple schema shapes on demand (based on client negotiation).
     • For MAJOR bumps, provide a compatibility transformation or flattened view to keep old clients working.
   • Support per-connection adapters: when a client negotiates v1.x, transform v2 events to v1 before sending.
   • Gradual rollout: start sending new fields only to clients that advertise capability; feature-flag rollout by client version.

5. Tooling & governance

   • Maintain a schema registry (JSON Schema/Protobuf/Avro) with compatibility checks (backward/forward).
   • Automated CI that validates new schemas against compatibility rules and generates transforms.
   • Telemetry: track client versions, transformation errors, event sizes, and feature adoption.
   • Announce deprecations via server.hello and separate "schema.notice" events; provide migration docs and SDK updates.

Example flow

• Client A (old) connects without Accept-Version. Server.hello advertises v1 and v2; server defaults to v1 and transforms events into v1 envelope.
• Client B (new) connects with Accept-Version: stream.events.v2. Server sends v2 events including new fields; older fields remain present when possible.

Edge cases & trade-offs

• Performance: per-connection transformation adds CPU; mitigate with cacheable transforms, route high-volume clients to dedicated transformers.
• Large MAJOR changes: may require client SDK updates and a migration window where server supports both MAJOR versions in parallel.
• Partial compatibility: prefer additive design to minimize MAJOR bumps.

Why this works

• Explicit schema/version in envelope + negotiated capabilities ensures clients know what to expect without disconnects.
• Server-side adapters enable backward compatibility without forcing immediate client upgrades.
• Schema registry + CI enforces safe evolution while telemetry guides gradual migration.

Requirements & trade-offs:

• Real-time delivery to mobile + server clients, low latency, fanout to many subscribers, intermittent connectivity for mobile, strong delivery guarantees optional.

High-level conclusion:
gRPC streaming (HTTP/2 bidirectional or server-streaming) is a strong option for low-latency, type-safe real-time notifications, but it requires careful handling for firewalls/NAT, long-lived connection load-balancing, language ergonomics, and offline delivery. Hybrid design with fallbacks (push, polling, push notifications) is recommended.

Wire compatibility:

• gRPC uses HTTP/2 and protobufs: compact, version-tolerant (field tags/optional fields), forward/backward compatible if you follow protobuf rules.
• Not trivial to interoperate with non-gRPC clients without an HTTP/JSON gateway; consider gRPC-Web or a REST shim for browsers/legacy clients.

Client ergonomics & codegen:

• Strong across major languages (Java, Go, Python, C++). Generated stubs handle stream APIs but ergonomics vary: Java/Kotlin coroutines or reactive wrappers improve usability; Python async/generator APIs are okay.
• Beware of large codegen surface on mobile (Android .aar size) and Swift interop quirks; invest in thin client libraries wrapping generated code to present consistent APIs.

Firewall/NAT/proxies:

• HTTP/2 helps traverse most proxies, but many corporate proxies block long-lived HTTP/2 or require HTTP/1.1. gRPC-Web over HTTP/1.1+WebSocket can help for proxies and browsers.
• Mobile networks may drop idle TCP; use keepalive pings (tune interval and timeout) and exponential reconnect with jitter to avoid thundering herds.

Load balancing & long-lived connections:

• Long-lived streams mean connection count ~ client count. Use connection multiplexing per edge instance and L4/L7 load balancers that support sticky routing or proxying (Envoy with HTTP/2 proxying).
• Prefer architecture: clients connect to a fleet of gateways (stateless wrt subscriptions) which fan out to backend pub/sub services (Redis Streams, Apache Kafka, or NATS) — gateways maintain streams while subscription state can be in a central store (Redis) or pushed from the pub/sub.
• Use consistent hashing or affinity only if you need per-connection state local to gateway; otherwise keep gateways stateless and use a fan-out message router to reduce rebalancing complexity.

Scaling to millions:

• Scale horizontally: Gateway pool behind an autoscaling layer; partition topics across backend pub/sub clusters; use efficient binary framing to keep per-connection memory small.
• Offload fanout to specialized systems (Cloud Pub/Sub, Kafka with stream processors, or purpose-built push services) rather than trying to deliver from a single monolith.
• CPU/memory: reduce per-connection allocations, use event-driven servers (libuv, epoll) and connection multiplexing. Monitor descriptors and tune OS limits.

Fallback strategies:

• Mobile: integrate platform push (APNs/FCM) when app backgrounded or connection unavailable; use silent pushes to wake app for real-time sync.
• Web/legacy: gRPC-Web, SSE, or WebSocket.
• If gRPC unavailable, fall back to HTTPS poll or long-poll.

Offline / queued delivery guarantees:

• Decide guarantees up front: at-most-once vs at-least-once vs exactly-once semantics.
• For at-least-once, persist messages in durable store backend with per-client offsets; on reconnect, client sends last-received-id to resume. Implement idempotency keys for downstream handling.
• For mobile offline buffering, combine server-side durable queue + push notification to indicate pending items. Consider TTLs and storage costs.
• If ordering matters, partition per-client or per-topic key and track offsets.

Operational concerns & best practices:

• Tune keepalive and retry/backoff; implement client jitter.
• Monitor connection churn, latency, error rates, queue lengths, memory per connection.
• Secure: TLS, auth (JWT/mTLS), per-call authorization.
• Use feature flags and canary rollouts for any protocol changes; maintain backward-compatible protobuf evolution.
• Provide lightweight SDK wrappers to hide complexity, handle reconnection, resume, and fallback selection.

Bottom line: gRPC streaming is a high-performance, type-safe choice for real-time notifications if you invest in gateway architecture, pub/sub backplane, robust client libraries, and multi-protocol fallbacks (APNs/FCM, gRPC-Web, polling) to cover firewalls, mobile backgrounding, and offline guarantees.

High-level approach: compare each push mechanism from the server-side perspective across scalability, firewall/proxy behavior, ordering, reconnection, and integration with a conventional HTTP backend.

WebSockets

• Scalability: Long-lived bidirectional TCP connections; requires connection-per-client. Scale with connection-aware infrastructure (horizontal app servers + connection balancers like AWS ALB/Nginx or dedicated TCP proxies) and state sharing (sticky sessions, shared pub/sub like Redis, Kafka, or a message broker).
• Firewall/proxy: Uses an HTTP Upgrade, then raw frames over TCP. Generally allowed but some corporate proxies block or terminate them; falls back needed.
• Ordering guarantees: Server-to-client messages over a single WebSocket are ordered; if you have multiple server instances, ordering across re-routes depends on how you route messages (centralized publisher preserves order).
• Reconnection: Client must detect close and reconnect; use exponential backoff + jitter; include last-seen message id/sequence for resume or fetch missed events.
• HTTP integration: Use existing HTTP stack to authenticate and upgrade; back-end publishes via pub/sub to whichever instance holds the socket.

Server-Sent Events (SSE)

• Scalability: Also long-lived but uni-directional (server → client). Easier to implement on existing HTTP servers (EventSource). Scale similarly — connection-per-client and pub/sub needed.
• Firewall/proxy: Works over plain HTTP (text/event-stream) and is often proxy-friendly; proxies sometimes buffer; use proper headers (Cache-Control: no-cache, Connection: keep-alive) and chunked transfer encoding.
• Ordering guarantees: Ordered per connection; server must sequence events if clients reconnect and request missed events.
• Reconnection: Built-in automatic reconnect in EventSource; include an "id" field so client can set Last-Event-ID to catch up after reconnect.
• HTTP integration: Simple — treat as a long GET that streams chunks; backend can push via pub/sub to the front-end process streaming the response.

Long-polling

• Scalability: Simulates push by holding HTTP requests for short durations. Lighter server resources per connection (requests time out or return quickly) but increases request rate and latency; scales via stateless servers + pub/sub; higher CPU/HTTP connection overhead at large scale.
• Firewall/proxy: Works with any HTTP proxy/firewall; most robust.
• Ordering guarantees: Ordered per client if you sequence messages and ensure next poll fetches in-order items using offsets/timestamps.
• Reconnection: Each poll is a new request; client immediately issues next request on response or timeout; use heartbeat/poll interval tuning and include last-seen id.
• HTTP integration: Very natural; backend exposes endpoints to fetch events since last id; simpler to integrate with existing request/response handlers.

Webhooks

• Scalability: Pushes from your server to third-party endpoints (not browser). Scales by batching, retry queues, and worker pools; durable queue (e.g., Kafka/DB) for reliability.
• Firewall/proxy: Outbound HTTP POSTs — usually permitted; target endpoints may be behind proxies/firewalls and reject or be slow.
• Ordering guarantees: Typically at-most-once or at-least-once depending on retries; ordering across endpoints must be handled by the consumer or by sequencing payloads and using idempotency keys.
• Reconnection/retries: Server must implement retry/backoff, dead-letter queues, exponential backoff, and idempotency to handle duplicates.
• HTTP integration: Treated as standard outbound HTTP calls; your backend produces events into a queue and worker processes deliver webhooks asynchronously.

Practical recommendations

• For browser dashboards: prefer WebSockets for full-duplex needs (interactivity) or SSE for server-to-client streaming (simpler). Fall back to long-polling where WebSockets/SSE are blocked.
• Always use a durable pub/sub (Redis streams, Kafka, or message broker) between business logic and connection workers so any worker can serve any client and preserve ordering/ replay via sequence ids.
• Include message ids, timestamps, and idempotency tokens. Implement resume logic (last-seen id) and exponential backoff with jitter. Monitor connection counts and use autoscaling and connection-aware proxies (or managed realtime services) to handle spikes.