Clarify requirements: push real-time notifications to thousands of concurrent clients, low latency, best-effort ordering, heterogenous clients (browsers, mobile, services), operate through LB/firewalls, support scale and backpressure.
Comparison (concise):
- Connection scale:
- SSE: simple HTTP long-polling over one TCP per client; scales well but per-connection memory on servers; ideal for read-only streams.
- WebSockets: full-duplex TCP; efficient for high message rates and bi-directional needs; similar per-connection overhead but more flexible.
- gRPC streaming: uses HTTP/2 multiplexing (single connection per client) with lower TCP connection counts; great for many logical streams per connection.
- Load-balancer behavior:
- SSE/WebSocket: LB must support sticky TCP/TLS connections and long-lived idle timeouts (tune timeouts, health checks).
- gRPC: requires HTTP/2-aware LBs (NGINX with http2, Envoy) and proper connection pooling.
- HTTP/2 support:
- SSE: HTTP/1.1 only (workarounds exist).
- WebSocket: originally HTTP/1.1 upgrade; WebSocket over HTTP/2 (RFC 8441) limited support.
- gRPC: native HTTP/2; benefits multiplexing and head-of-line avoidance per stream.
- Reconnection semantics:
- SSE: built-in EventSource reconnection with last-event-id header; simple.
- WebSocket: manual keepalive and reconnection logic.
- gRPC: client libraries often handle reconnect/backoff; need idempotency semantics.
- Firewall compatibility:
- SSE/WebSocket: broadly compatible (uses standard ports 80/443); some strict proxies may block WebSocket upgrades.
- gRPC: uses HTTP/2 over 443—usually allowed but some middleboxes mishandle HTTP/2.
- Operational complexity:
- SSE: lowest complexity server-side.
- WebSocket: moderate (session management, scaling).
- gRPC: higher (HTTP/2 LBs, observability, language/runtime support).
Recommendation (architecture):
- Use hybrid: gRPC streaming for first-class native apps and services; WebSocket for browsers when bi-directional or lower latency needed; SSE as fallback for simple one-way browser notifications.
- Front tier: edge LB (Cloud LB/Envoy) that routes by client capability; terminate TLS, forward to stateless ingress fleet.
- Connection management: run a horizontally scalable fleet of connection servers that keep in-memory connection maps and publish client registrations to a distributed pub/sub (Kafka, Redis Streams, or cloud pub/sub). Use consistent hashing or sticky sessions to route a user's connections to same server (reduces fan-out).
- Message fan-out: central message broker (topic per notification type) -> connection servers subscribe and push to clients.
- Backpressure strategies:
- Per-connection send buffers with bounded queues; drop/merge policy for low-value notifications.
- Use TCP readable/writable signals: pause reading from broker when connection queues exceed threshold (flow-control at app level).
- For gRPC/HTTP2, leverage stream-level flow-control and window updates.
- Implement priority tiers and rate-limiting per user and per tenant; apply exponential backoff and circuit-breakers for overloaded clients.
- Observability & ops:
- Metrics (connections, queue sizes, message latency), distributed tracing, and automated autoscaling based on open connections and CPU.
- Graceful drain: signal ingress to stop accepting new connections and migrate subscriptions.
Trade-offs: gRPC reduces TCP connections but increases LB/operator complexity; WebSocket maximizes browser compatibility and bi-directional flows. Hybrid approach gives best coverage while central pub/sub and per-ingress bounded queues keep system resilient.
