Network Security: Firewalls, NAT, and VPN Questions

High-level idea:

• Attach an XDP (eXpress Data Path) program at NIC ingress to run BPF code in kernel context. Use XDP to drop, pass, or redirect packets before kernel network stack (L2/L3) — e.g., redirect suspicious flows to a userspace collector via AF_XDP or to a hardware/TC offload via XDP_REDIRECT. This allows line-rate mitigation of volumetric DDoS close to the wire.

Performance benefits:

• Executes in kernel at NIC driver RX path, avoiding socket/stack cost → sub-microsecond decisions and very high packets-per-second handling.
• Can drop malicious packets before interrupts/CPUs do heavy work, drastically reducing CPU and memory pressure.
• Zero-copy redirect to AF_XDP or NIC using XDP_REDIRECT preserves throughput and low latency.

Example (conceptual XDP action):

// pseudo-C snippet for XDP
SEC("xdp") int ddos_filter(struct xdp_md *ctx) {
  // lookup 5-tuple in BPF hashmap of blacklisted prefixes/ratelimits
  if (is_blacklisted(ctx)) return XDP_DROP;
  if (needs_redirect(ctx)) return bpf_redirect_map(&tx_port_map, port, 0);
  return XDP_PASS;
}

Limitations:

• Statefulness: XDP runs per-packet in kernel; maintaining large, complex state (per-connection tracking, long time windows) is costly. BPF maps add state but limited by memory and LRU eviction semantics.
• Complexity: Writing safe, verified BPF requires kernel knowledge, verifier constraints (bounded loops, stack limits), and careful testing to avoid performance regressions.
• Portability: XDP behavior depends on NIC/drivers and kernel versions; some features (hardware offload) vary. AF_XDP/zero-copy availability and driver support differ across platforms.
• Debugging & observability: Kernel context limits printf; need maps, perf events, userspace consumers.

Safe deployment strategy:

1. Requirements & test harness: Build synthetic traffic generator (pktgen, tcpreplay) reproducing attack profiles.
2. Unit/in-kernel testing: Validate with small maps and BPF self-tests on staging kernels matching prod.
3. Gradual rollout:
   • Start in monitor-only mode: Count suspicious packets in maps (XDP_PASS) and export metrics to observe rates and false positives.
   • Move to redirect-to-scrubber (AF_XDP/userspace or dedicated scrubbing appliance) for suspect traffic while still passing others.
   • Finally enable in-kernel drop for verified signatures/prefixes.
4. Rate-limiting & safety nets:
   • Implement conservative thresholds, per-prefix rate-limits, and whitelist critical control-plane IPs.
   • Use fail-open semantics by default: if XDP program fails to load or maps unreachable, allow traffic rather than block.
5. Fallbacks & automation:
   • Keep automated rollback (systemd unit or orchestration playbook) to unload BPF quickly.
   • Integrate health checks and alerts: if p99 latency/packet drops spike, automatically revert to monitor-only.
6. Observability:
   • Export BPF map stats, eBPF perf events, and AF_XDP counters to Prometheus/ELK.
   • Baseline normal traffic to tune thresholds and avoid collateral damage.

Key operational tips:

• Keep programs minimal and hotspot-optimized; push heavy analysis to userspace collectors.
• Use map TTLs and LRU maps to bound memory.
• Test across kernel/NIC combos; maintain compatibility matrix.
• Coordinate with network team and scrubbing providers; document rollback runbooks.

This approach provides near-line-rate mitigation with low overhead while minimizing blast radius through staged testing, monitoring, and conservative fail-open defaults.

Requirements (clarify): least-privilege, three tiers (web, app, db) in a VPC across AZs, internal traffic allowed only as needed, centralized auditing and automated enforcement.

Example security-group rules (direction, protocol, port, source):

• Web SG (public-facing)
  • Inbound:
    • TCP 80, 443 from 0.0.0.0/0 (or specific CDN/IP ranges) — only for LB or edge IPs
    • TCP 22 from corporate bastion SG (for admin) — not 0.0.0.0/0
  • Outbound:
    • TCP 443 to App SG (allow HTTPS API calls)
• App SG (internal)
  • Inbound:
    • TCP 443 from Web SG (reference by SG id) — only from load balancer/web instances
    • TCP 10250 (or app-specific) from monitoring SG
  • Outbound:
    • TCP 5432 to DB SG (Postgres) — only DB SG
• DB SG (private, no public access)
  • Inbound:
    • TCP 5432 from App SG only
  • Outbound:
    • TCP 443 to backups endpoint (VPC endpoint IP range) and DNS

Key principles:

• Use security-group references (SG as source) instead of CIDR where possible to bind intent to role.
• Restrict admin ports to jump/bastion SG or corporate IPs.
• Deny all else (implicit deny).

Cross-AZ traffic management:

• Place instances in each AZ and attach same SGs — SGs are region-wide, not AZ-bound.
• Use internal (zone-aware) load balancers to route traffic AZ-local when possible (reduce cross-AZ egress cost).
• For stateful services, use AZ-local replicas and prefer AZ-local read/serve patterns; ensure health-checks/target groups limited to same AZ if needed.
• If sensitive traffic must cross AZ, ensure encryption in transit (TLS) and monitor inter-AZ latency/throughput.

Auditing and enforcing at scale:

• IaC + Modules: Define SGs as reusable Terraform/CloudFormation modules with descriptive tags and enforced inputs (ports, allowed SGs). Keep canonical modules in a central registry.
• Policy-as-code: Enforce rules in CI with tools like Open Policy Agent, Sentinel, or Terraform Cloud policies to reject nonconforming changes.
• Drift detection & automated remediation: Run periodic scans (AWS Config rules, Azure Policy, GCP Forseti) and automated remediation lambdas to revert or notify.
• Logging & monitoring: Enable VPC Flow Logs to a central log lake (S3/BigQuery) and analyze with SIEM (Splunk/Elastic/Datadog). Alert on unexpected sources/ports, spikes, or cross-AZ abnormal flows.
• Change governance: Require pull requests, automated plan diffs, and approvers for SG changes; maintain audit trails with CloudTrail.
• Inventory & tagging: Maintain an authoritative map of SG-to-service (service tags, owner, expiration).
• Regular penetration testing and periodic rule minimization reviews (remove unused rules using flow-log analysis).

This approach ensures least privilege by using SG references, tight ports, IaC-driven rules, cross-AZ performance consideration, and automated auditing/enforcement for scale and reliability.

Clarify goal: keep legitimate clients served while minimizing resource exhaustion on the host.

conntrack tuning and limits

• Problem: SYN floods fill conntrack table and consume memory/CPU.
• Mitigation: raise nf_conntrack_max cautiously, reduce conntrack timeout for SYN_RECV, and use hashsize appropriately.
• Example:

sysctl -w net.netfilter.nf_conntrack_max=262144
sysctl -w net.netfilter.nf_conntrack_tcp_timeout_syn_recv=10
sysctl -w net.netfilter.nf_conntrack_buckets=65536

• Reasoning: shorten SYN_RECV timeout frees slots faster; but increasing max uses more RAM.

iptables rate limiting

• -m limit: global token-bucket style, cheap, good for low-rate control.

iptables -A INPUT -p tcp --dport 80 --syn -m limit --limit 100/s --limit-burst 200 -j ACCEPT
iptables -A INPUT -p tcp --dport 80 --syn -j DROP

• -m recent: per-IP tracking, can block repeat offenders:

iptables -A INPUT -p tcp --dport 80 --syn -m recent --name synflood --set
iptables -A INPUT -p tcp --dport 80 --syn -m recent --name synflood --update --seconds 60 --hitcount 20 -j DROP

• Tradeoffs: recent consumes conntrack-like state and CPU; limit is stateless and cheaper.

tcp_syncookies

• Enable to avoid allocation for half-open connections:

sysctl -w net.ipv4.tcp_syncookies=1

• Pros: protects TCP stack memory; cons: disables some TCP options (large windows, SACK) during cookie mode and may not help non-TCP SYN attacks.

tc (ingress policing)

• Use hierarchical policing or u32 filters to rate-limit traffic at kernel ingress, useful for shaping before netfilter.

tc qdisc add dev eth0 handle ffff: ingress
tc filter add dev eth0 parent ffff: protocol ip u32 match ip dport 80 0xffff police rate 100mbit burst 10k drop

• Pros: acts earlier in RX path (before expensive processing); cons: complex rules and may drop legitimate bursts.

Host vs upstream filtering

• Upstream (ISP / cloud provider) filtering (blackholing, scrubbing) reduces bandwidth on your link and lowers host CPU/memory usage — preferred for large volumetric attacks.
• Host-level mitigations are faster to deploy and can protect control-plane behavior, but are limited by your link capacity and host resources.
• Hybrid: use host-level filtering for fine-grained per-IP/behavioral mitigations and request quick upstream scrubbing for volumetric floods.

Operational notes

• Monitor conntrack table, SYN_RECV counts, and interface drops. Alert on thresholds.
• Test rules in non-blocking mode, apply gradually, and keep allowlists for known peers.
• Maintain runbooks with provider contacts for escalation.