Cloud Networking and VPC Design Questions

A CDN (Content Delivery Network) is a distributed network of edge servers that cache and serve content from locations closer to users to reduce latency, offload origin servers, and increase availability. Edge caching stores copies at POPs so requests hit the edge instead of the origin.

Resource types that benefit most:

• Static assets: HTML/CSS/JS, images, fonts — high cacheability and high request volume
• Media files: video/audio, large binaries, software updates — reduce bandwidth and improve throughput
• CDN-friendly API responses: read-heavy, mostly-identical responses (public blobs, product catalogs)
• Immutable artifacts: versioned assets and container images

Cache invalidation strategies:

• TTL-based caching (Cache-Control max-age) for automatic expiry
• Versioned URLs / cache-busting (content fingerprinting) for instant invalidation
• Explicit PURGE/INVALIDATE API for emergency removals
• Stale-while-revalidate / stale-if-error to balance freshness and availability
• Surrogate keys to invalidate related groups atomically
• Use short TTLs for dynamic resources and longer for immutable ones

When a CDN won't help:

• Highly personalized or per-request dynamic content (user dashboards, SSE, websockets)
• Low-latency origin-dependant operations where edge doesn’t reduce critical path (complex auth requiring origin roundtrips)
• Small user base concentrated near origin (no geographic distribution gains)
• Upload-heavy workloads (CDN helps distribution but not origin ingestion)

SRE considerations: monitor cache hit ratio, purge latency, origin request rate, and TLS/HTTP handshake overhead. Use origin shielding to reduce origin load, and balance TTLs vs. freshness to meet SLOs.

VPC peering: a direct, private L3 connection between two VPCs in the same (or supported) region—point-to-point, non-transitive (peering A<>B and B<>C does NOT route A<>C) and low-latency with no single managed-hop. Good for simple, high-performance app-to-app links.

VPN / Direct Connect / Dedicated Interconnect:

• VPN: encrypted IPsec over Internet. Quick, low-cost, higher latency/variable jitter, good as backup or for low-throughput.
• Direct Connect (AWS) / Dedicated Interconnect (GCP): private physical connections between your on‑prem and cloud network. Much lower latency, higher bandwidth, SLA-backed, predictable performance. Requires colo, circuit setup and recurring port/month + data transfer costs.

Transit Gateway: managed hub that enables scalable, transitive routing between many VPCs and on‑prem connections. Simplifies routing (spoke-hub model), supports high throughput, and integrates with Direct Connect/VPN.

Recommendation for hybrid-cloud needing low latency + transitive routing:

• Primary: use a transit gateway paired with Direct Connect / Dedicated Interconnect for predictable low-latency, high-bandwidth, transitive routing across many VPCs and on‑prem. Transit gateway handles central routing and security controls.
• Secondary/DR: IPsec VPN on top for failover.

Cost & operational trade-offs:

• Cost: transit gateway and Direct Connect incur fixed monthly charges (port, attachment, data processing); peering is cheaper per-pair but scales poorly (N^2). VPN is cheap but can have data egress costs and performance limits.
• Ops: Direct Connect requires coordination with carriers/colo and longer lead time. Transit gateway reduces routing complexity and operational overhead vs managing many peering connections or manual route propagation. Peering is simplest for two-VPC cases and minimal ops.
• Security: centralizing routing via transit gateway eases policy enforcement but creates a single control plane to secure. Ensure least-privilege route tables, segmentation, and monitoring.

As an SRE I’d prioritize Direct Connect + Transit Gateway for performance and manageability, keep VPN as backup, and budget for circuit/attachment costs and runbooks for failover and monitoring.

VPC components — brief definitions:

• Subnet: IP CIDR slice within a VPC; can be public (has route to Internet Gateway) or private (no direct IGW route).
• Route table: Maps destination CIDRs to targets (IGW, NAT, peering, TGW, ENI); associated per-subnet.
• Internet Gateway (IGW): Horizontally scaled gateway that enables inbound/outbound traffic between VPC and Internet for public subnets.
• NAT Gateway (or NAT instance): Allows resources in private subnets to initiate outbound Internet connections (patching, image pulls) while preventing inbound initiation.
• Security Group (SG): Stateful virtual firewall attached to ENIs; rules allow inbound/outbound by IP/port; return traffic auto-allowed.
• Network ACL (NACL): Stateless subnet-level firewall; explicit allow/deny rules evaluated by rule number; must allow return traffic explicitly.

Recommended 3-tier layout (text sketch):

• Public subnets (across AZs): Load balancers, bastion/jump hosts. Route table → IGW.
• Private app subnets (across AZs): Application servers. Route table → NAT Gateway in public subnet.
• Private DB subnets (across AZs, further restricted): Databases, caches. Route table → no NAT/IGW (optional minimal NAT for backups).

Traffic flows:

• Client → Internet → IGW → Public subnet ELB → SG allows 80/443 from 0.0.0.0/0 → ELB forwards to app instances in private subnet (SG allows only ELB SG on necessary ports).
• App instances → DB in DB subnet (SG allows only app SG on DB port). DB has no IGW access.
• App instances outbound to Internet (OS/patches) go through NAT Gateway in public subnet; NAT’s elastic IP handles responses.
• Admin SSH: Bastion in public subnet (SG restricts SSH to trusted IPs); from bastion to app/db via internal SG rules.

Why this supports security and operations:

• Principle of least privilege: SGs restrict traffic by role (ELB→app, app→db). Databases never directly reachable from Internet.
• Defense-in-depth: SGs (stateful) + NACLs (stateless, optional deny rules for broad threats) provide layered controls.
• High availability: Subnets and NAT/IGW deployed per-AZ; multi-AZ ELB and DB replicas ensure failover.
• Operational needs: NAT enables outbound updates while preventing inbound exposure; bastion + IAM + session logging/audit support secure admin access. Route tables and tagging simplify routing, automation, and IaC provisioning.
• Monitoring/incident readiness: Flow logs on VPC/subnets + CloudWatch/Prometheus metrics help troubleshoot traffic anomalies and enforce SLOs.

A load balancer health check is a probe the LB periodically performs against backend instances to determine whether they can accept traffic. Proper health checks let the LB route traffic only to healthy hosts and trigger failover when needed.

Typical parameters

• Protocol: HTTP(S), TCP, TLS, or ICMP — choose based on what “serving traffic” actually means.
• Path/port: HTTP path (e.g., /healthz) or TCP port to hit.
• Interval: how often probes run (e.g., 10s).
• Timeout: how long to wait for a response before considering the probe failed (e.g., 2–3s).
• Healthy threshold: consecutive successes required to mark healthy (e.g., 3).
• Unhealthy threshold: consecutive failures required to mark unhealthy (e.g., 2).

How incorrect checks cause problems

• Traffic blackhole: overly strict checks (e.g., requiring complex DB-backed response) can mark all backends unhealthy during transient downstream failures, causing the LB to send traffic nowhere.
• Failover flapping: too-short intervals, low thresholds, or tight timeouts make instances flip healthy/unhealthy rapidly (flapping), causing traffic churn and cache misses. Network jitter or GC pauses can trigger false failures.

Safe defaults (good starting point)

• Protocol: HTTP(S) GET to a lightweight endpoint (e.g., /healthz)
• Path handler: should be fast, read-only, not dependent on slow downstreams
• Interval: 10s
• Timeout: 2–3s
• Healthy threshold: 3
• Unhealthy threshold: 2–3
• Use staggered start and probe jitter to avoid thundering herds

Advanced checks

• TCP/TLS connect to validate layer 4/handshake without app logic
• HTTP with JSON that includes app-specific readiness flags
• Header or authentication checks for proxies
• Scripted or agent-based health checks that run local diagnostics (disk, threadpool, DB connections) and expose a simple 200/500
• Canary weighting & gradual ramp-up combined with health-check windows for deploys

Best practices

• Keep checks lightweight and representative of traffic path.
• Separate readiness (controls routing) from liveness (restarts container) where supported.
• Use conservative thresholds and add exponential backoff or damping to prevent flapping.
• Monitor health-check metrics (failure rates, probe latency) and alert if many hosts fail simultaneously.

CIDR notation

• CIDR (Classless Inter-Domain Routing) expresses an IP block as A.B.C.D/N where N = number of fixed prefix bits. Example: 10.0.0.0/16 gives 65,536 addresses; splitting to /24 yields 256-addres subnets like 10.0.1.0/24. CIDR lets you allocate and aggregate address space flexibly.

Public vs private subnets (purpose)

• Public subnet: route table has a default route (0.0.0.0/0) to an Internet Gateway (IGW). Resources with public IPs (load balancers, bastion hosts) live here.
• Private subnet: no direct 0.0.0.0/0 → IGW. Instances are not reachable from internet; used for app servers and databases for security.

NAT gateway usage

• NAT gateway (or NAT instance) placed in a public subnet allows outbound internet access for instances in private subnets (software updates, external APIs) while preserving their lack of inbound internet reachability. Route table for private subnets points 0.0.0.0/0 to NAT.

Route tables

• Public RT: subnet → IGW for 0.0.0.0/0.
• Private RT: subnet → NAT for 0.0.0.0/0 and route to other internal CIDR ranges as needed.
• DB subnet: more restrictive RT (no internet route), only internal routes + VPC peering/VPN as required.

Minimal HA VPC sketch (2 AZs)

• VPC: 10.0.0.0/16
• AZ-A:
  • Public subnet A (10.0.1.0/24): ALB (internet-facing), NAT gateway
  • Private app subnet A (10.0.2.0/24): Auto-scaling app instances (no public IP)
  • DB subnet A (10.0.3.0/28): Primary managed DB (private)
• AZ-B:
  • Public subnet B (10.0.4.0/24): ALB targets + NAT gateway (optional for cross-AZ HA)
  • Private app subnet B (10.0.5.0/24): ASG instances
  • DB subnet B (10.0.6.0/28): Read replica or multi-AZ managed DB
• Security groups:
  • ALB SG: allow 80/443 from 0.0.0.0/0
  • App SG: allow 80/443 from ALB SG; allow outbound to NAT
  • DB SG: allow DB port only from App SG
• High availability considerations: place NAT gateways and app subnets in both AZs; use multi-AZ managed DB and cross-AZ load balancing. This design isolates public surface area, enables secure outbound access, and keeps the database non-internet-facing—matching SRE goals for reliability and security.