Requirements:
- Functional: per-tenant routing (tenant-specific backends or versions), schema-based request validation, propagate observability headers for tracing.
- Non-functional: low latency (<100ms added), high throughput, horizontal scalability, multi-tenant isolation, pluggable validation rules.
High-level architecture:
API Gateway (stateless fleet) -> Auth & Tenant Resolver -> Routing Engine -> Schema Validator -> Upstream Data Services (per-tenant or shared)
Observability: Incoming trace headers (traceparent / b3) forwarded; gateway emits spans/metrics to tracing system and metrics to Prometheus.
Key components & responsibilities:
- Edge load balancer (Cloud LB / ELB): TLS termination, health checks.
- Gateway instances (Kubernetes / auto-scaling group): stateless, handle tenant resolution, header propagation, validation, routing.
- Tenant Config Store (etcd/Redis/Consul + versioned configs): holds routing rules, schema URIs, rate limits; pushed via config watch.
- Schema Validation Service (embedded or sidecar using JSON Schema/Avro/Protobuf): validates payloads, returns 4xx on failure.
- Control Plane UI / API: manage tenant rules, schemas, rollout.
- Observability: OpenTelemetry SDK on gateway, export to Jaeger/OTel Collector; metrics to Prometheus/Grafana.
Data flow:
- LB -> Gateway: authenticate, resolve tenant from header/cert/path -> fetch tenant routing & schema from local cache -> validate request -> forward with trace headers to selected upstream -> collect span + metrics.
Scaling horizontally:
- Make gateway stateless; cache tenant configs locally with watch/consul sync to avoid central bottleneck.
- Autoscale gateways by CPU/QPS/memory. Scale schema validator horizontally or run as sidecar per gateway for CPU isolation.
- Use sharded Redis/Consul for config store and CDN for larger static schemas.
- For high fan-out, use a routing layer that supports subset routing (canary, blue/green) stored in config store.
- Use circuit breakers and bulkheads to prevent slow upstreams from affecting others.
Trade-offs:
- Inline validation reduces downstream load but increases gateway CPU — mitigate via sidecars or Lua/wasm fast-paths.
- Consistent low-latency requires local caches and eventual config consistency; accept small propagation delay on config updates.
Security & multi-tenant isolation:
- Per-tenant RBAC in control plane, rate limits per tenant, mTLS to upstreams when needed.
