Networking, VPC, and Connectivity Questions

Deep understanding of AWS VPC architecture including subnets (public and private), route tables, Network Address Translation (NAT), internet gateways, and VPC endpoints. Knowledge of security groups and network ACLs. Understanding of VPN and AWS Direct Connect for hybrid connectivity. DNS and Route 53 routing policies.

MediumTechnical

24 practiced

Design an outbound egress strategy for thousands of EC2 instances that will make a large number of simultaneous outbound connections creating high SNAT demand. Discuss NAT Gateway scaling limits, ephemeral port exhaustion, alternatives such as multiple NAT Gateways per AZ, proxying, autoscaling NAT instances, and monitoring approaches to detect and mitigate port exhaustion.

Sample Answer

Clarify goals & constraints:- Thousands of EC2s will open many simultaneous outbound TCP connections -> high SNAT (ephemeral port) pressure.- Requirements: high concurrency, low latency, high availability across AZs, predictable scaling, cost/operational trade-offs.

Key problem:- SNAT maps many ephemeral source ports to a limited set of public IP addresses. Each public IP has ~64k ephemeral ports (practical usable ≈ ~55–60k after OS/AWS reservations). So total concurrent outbound flows ≈ (#EIPs) * ~55k. Exceeding that causes port exhaustion and failed connections.

Design options and trade-offs

1) Multiple NAT Gateways per AZ (recommended baseline)- Put at least one NAT Gateway per AZ to avoid cross-AZ data charges and single-AZ failures.- Provision multiple NAT Gateways per AZ (each with its own Elastic IP) to multiply available ephemeral ports and throughput.- Pros: fully managed, HA, low ops. Cons: cost scales with number of NATs/EIPs.

2) Add more EIPs per NAT (via additional NAT Gateways)- Each additional NAT Gateway + EIP gives another ~55k ports. For capacity planning, compute required EIPs: Example: 10,000 EC2s * 20 concurrent connections = 200k concurrent flows -> ~4 NAT Gateways/EIPs needed (200k/55k ≈ 3.6 → 4).- Also validate NAT Gateway throughput limits (per-GW baseline throughput ~5 Gbps; scale by adding NAT Gateways).

3) Autoscaling NAT instances (self-managed)- Use an autoscaling group of NAT instances behind an internal ALB or route table trick with NAT instance fleet and a proxy/load balancing layer.- Pros: full control (more EIPs, conn tracking tuning), potentially lower cost at scale.- Cons: operational complexity, HA and maintenance, instance connection tracking limits, must design for failover.

4) Dedicated proxy fleet (recommended for heavy HTTP traffic)- Deploy horizontally scaled HTTP/HTTPS proxy (e.g., Envoy, Squid, HAProxy) in each AZ. Use connection pooling, Keep-Alive, HTTP/2 multiplexing to drastically reduce client-side concurrent ports.- Front proxies can have multiple EIPs or sit behind NAT Gateways; for scaling, use ASG with auto-attach EIPs or put proxies in private subnets and route via NATs.- Pros: reduces ephemeral ports by reusing backend connections; adds caching, TLS termination, observability. Cons: operational cost and complexity.

5) Avoid SNAT where possible- Use VPC Endpoints (Interface/PrivateLink) for AWS services to remove public SNAT for those APIs.- Use Direct Connect or Transit Gateway for external private peering where applicable.

Connection-efficiency mitigations (must implement)- Use connection pooling, HTTP Keep-Alive, HTTP/2 or gRPC multiplexing.- Increase client-side reuse and reduce unnecessary short-lived connections.- If using sockets directly, tune ephemeral port ranges and TIME_WAIT reuse carefully (but only as last resort).- Implement client-side backoff and retry jitter.

Monitoring and detection- CloudWatch NAT Gateway metrics: ActiveConnections (per NAT), BytesIn/Out, PacketsDrop. Watch ActiveConnections / NAT port capacity.- VPC Flow Logs: aggregate SYN/SYN-ACK patterns, failed TCP attempts, retransmits.- OS/Dashboard metrics on EC2s: ephemeral port usage (ss/netstat), TIME_WAIT counts, socket creation rates.- Application metrics: connection failures, latencies, 5xx rates to external services.- Synthetic checks to external endpoints to detect partial failure because of SNAT exhaustion.- Alerts: - ActiveConnections per NAT > 70% of estimated ports. - Rapid rise in TIME_WAIT on instances. - Increase in outbound TCP errors or connection timeouts.

Runbook and mitigation steps- Immediate: shift some subnets to alternate NAT Gateways/EIPs (route table change per-subnet), throttle clients, enable client backoff.- Mid-term: add NAT Gateways/EIPs or scale proxy fleet; enable per-AZ routing to distribute.- Long-term: refactor to use connection pooling/proxy, VPC endpoints, or private connectivity to reduce public SNAT.

Example architecture (practical)- Per-AZ: 2 NAT Gateways (each with EIP) + a proxy ASG (Envoy) for HTTP workloads.- Route private subnets to proxy (local) which forwards pooled connections through NAT Gateways.- Use Auto-scaling for proxy based on active upstream connections; add NAT Gateways when aggregate ActiveConnections approaches capacity.- Use CloudWatch + VPC Flow Logs + instance-level metrics with dashboards/alerts.

Summary- Combine multiple NAT Gateways per AZ (scale EIPs) with a proxy/pooling layer to minimize port consumption. Use VPC endpoints where possible. Monitor NAT ActiveConnections, VPC Flow Logs and host socket states; implement automated scaling/alerts and a clear runbook to add NAT capacity or throttle workloads when exhaustion nears.

EasyTechnical

21 practiced

Explain AWS VPC endpoints and the differences between Interface Endpoints (PrivateLink) and Gateway Endpoints. Provide concrete examples: how would you grant private access to S3 and to a private API in your VPC without exposing traffic to the public internet?

Sample Answer

VPC endpoints let resources in your VPC privately communicate with AWS services or your own/private services without using the public internet. Two main types:

- Gateway Endpoint - Supported services: S3 and DynamoDB. - Implemented as a route table target (no ENIs). - Low-latency, cost-effective for high-throughput storage access. - Use case: EC2/EMR instances access S3 privately.

- Interface Endpoint (AWS PrivateLink) - Creates elastic network interfaces (ENIs) in your subnets with private IPs that forward traffic to the service. - Works for most AWS services and third-party or your own NLB-backed services. - Fine-grained access via security groups and endpoint policies. - Use case: Access a private API hosted by a service owner or your own service behind an NLB.

Example: Private access to S3 (Gateway Endpoint)1. Create a VPC endpoint type gateway for com.amazonaws.<region>.s3.2. Add the endpoint to the route tables of subnets that need access.3. Optionally attach an endpoint policy to restrict buckets.

Endpoint policy example (allow access only to my-bucket):

json

{
  "Statement":[
    {
      "Principal":"*",
      "Action":"s3:*",
      "Effect":"Allow",
      "Resource":["arn:aws:s3:::my-bucket","arn:aws:s3:::my-bucket/*"]
    }
  ]
}

Example: Private access to a private API (Interface Endpoint / PrivateLink)1. If API is your own: front your service with an NLB in the service VPC; create a VPC Endpoint Service.2. In your consumer VPC, create an Interface Endpoint for the service (com.amazonaws.vpce.<id> or the service name).3. This creates ENIs in chosen subnets; secure with security groups allowing only your clients.4. Use endpoint policy to allow only specific principals or hostnames.

CLI snippet to create an interface endpoint for a service (consumer VPC):

bash

aws ec2 create-vpc-endpoint \
  --vpc-id vpc-123abc \
  --vpc-endpoint-type Interface \
  --service-name com.amazonaws.us-east-1.execute-api \
  --subnet-ids subnet-1 subnet-2 \
  --security-group-ids sg-12345

Key differences & trade-offs:- Gateway endpoints: cheaper, no SG control, limited to S3/DynamoDB.- Interface endpoints: broad service support, security-group control, per-ENI cost.- For S3 choose Gateway Endpoint; for private APIs or services choose Interface Endpoint (PrivateLink).

MediumTechnical

23 practiced

Explain how Route 53 Private Hosted Zones enable split-horizon DNS for hybrid applications. Describe cross-account private hosted zone sharing, conditional forwarding to on-prem DNS via Route 53 Resolver rules, resolving internal names from Lambda/EKS, and common pitfalls such as overlapping domain names.

Sample Answer

Split‑horizon DNS with Route 53 Private Hosted Zones (PHZ) lets you expose different DNS answers to clients based on network location: a public hosted zone serves Internet clients, while a PHZ with the same name returns internal records to resources in associated VPCs. This is ideal for hybrid apps where internal endpoints (RDS, private ALBs, service names) must resolve differently inside the VPC than from the Internet.

Cross‑account PHZ sharing and VPC association:- Create the PHZ in one account (often the networking/account). Use Resource Access Manager (RAM) to share the PHZ with other AWS accounts or directly associate VPCs across accounts (you must accept RAM share and then associate the VPC). Only VPCs explicitly associated with the PHZ will receive its answers.- Best practice: centralize PHZ ownership and control access via IAM/RAM to simplify auditing and updates.

Conditional forwarding to on‑prem DNS:- Use Route 53 Resolver endpoints + rules: create an outbound resolver endpoint in your VPC and a rule that forwards queries for on‑prem zones (e.g., corp.example.com) to your on‑prem DNS IPs over the AWS-managed VPC‑to‑on‑prem connectivity (Direct Connect / VPN). For on‑prem clients to resolve AWS private names, create an inbound endpoint and configure your on‑prem DNS to forward to that inbound endpoint.- Use rule priorities and test with dig/nslookup. Monitor with Resolver query logs.

Resolving internal names from Lambda / EKS:- Lambda: configure the function to run in the target VPC and subnets with NAT or routing as needed. VPC-enabled Lambdas use the VPC’s DNS resolver — they’ll resolve PHZ names if the VPC is associated.- EKS: CoreDNS forwards through the VPC resolver; ensure kube-dns/CoreDNS config isn’t blocking or overriding lookups. For cluster services, consider split horizon for service discovery (externalName, internal DNS suffix).- Ensure VPC attribute enableDnsHostnames and enableDnsSupport are true.

Common pitfalls:- Overlapping domains: having identical names in public hosted zone and PHZ is intended, but overlapping PHZs (two PHZs with same name) associated to the same VPC cause ambiguous resolution; AWS will choose the hosted zone with the most specific matching record or may produce unexpected results — avoid multiple PHZs for same domain in same VPC.- Missing VPC association: resources won’t see PHZ unless the VPC is associated (cross‑account association must be explicitly granted).- Lambda/EKS networking: forgetting to run Lambdas in the VPC or misconfigured subnet/NAT breaks resolver access.- Resolver rule order/priority: incorrect priorities can route queries incorrectly.- Security and auditing: open resolver endpoints can be abused; restrict security groups and monitor logs. Be mindful of costs for endpoints and query logging.- DNS caching & TTLs: updates in PHZ may not propagate immediately due to client caches; lower TTL for rapid cutovers.- Limits: VPC association limits, resolver endpoints per region — plan architecture accordingly.

Summary practical steps:1. Centralize PHZ ownership; share via RAM and associate required VPCs.2. Deploy Resolver inbound/outbound endpoints and create forwarding rules for on‑prem zones.3. Ensure Lambdas run in VPC and EKS CoreDNS is configured to use cluster/VPC resolver.4. Avoid multiple PHZs with same name per VPC; control access and monitor with Resolver logs.

MediumTechnical

21 practiced

A customer requires that S3 buckets only be accessed from within their VPC via VPC Gateway Endpoints (no public internet access). Explain how you would configure the VPC endpoint and S3 bucket policies to enforce this, describe pitfalls such as cross-account access and console access, and state how you would verify enforcement. Provide policy-level guidance (no code required).

Sample Answer

Approach — goals: ensure S3 is reachable only from resources inside the customer VPC using an S3 Gateway Endpoint (no public internet), prevent console/browser access via internet, and handle cross-account access securely.

Configure the VPC endpoint- Create an S3 Gateway Endpoint in the VPC, attach it to the route tables for all subnets that should access S3.- Do not enable “Policy: Full” unless needed; use a restrictive endpoint policy to allow only desired principals, actions, and buckets from this VPC. Use the endpoint policy to limit usage to specific bucket ARNs and optionally to specific VPC IDs and source principals.

S3 bucket policy guidance (policy-level)- Require requests to originate from the gateway endpoint by checking aws:SourceVpce (or aws:SourceVpc and aws:SourceIp for additional assurance). Example condition keys: "aws:SourceVpce": "<vpce-id>".- Deny any requests that do not satisfy the above condition (explicit Deny with NotPrincipal or NotCondition) so public or internet-originating requests fail even if credentials exist.- If cross-account access is required, grant explicit Allow to the AWS Account or role ARNs but retain the SourceVpce condition so access is only effective when coming via the endpoint.- Include least-privilege actions (s3:GetObject, s3:PutObject) rather than wildcard s3:* where possible.- Add MFA or encryption conditions as needed for extra controls.

Pitfalls and considerations- Console access: AWS Management Console S3 operations from a browser still make API calls from the client IP — if a user in the customer’s on-premises network or internet attempts console access, aws:SourceVpce will not be set, so the bucket policy will block it. However, IAM users/roles with S3 permissions could still access via the internet if not blocked; ensure the bucket policy explicitly denies requests not via the endpoint.- Cross-account principals: An Allow to another account must still include the SourceVpce condition; otherwise those principals can access over the public internet. Also consider role assumption flows and service principals.- Shared VPCs, NAT, and proxy: Ensure route tables don’t accidentally direct S3 traffic via NAT or internet gateway; S3 gateway endpoints use prefix list route targets—verify no conflicting routes.- VPC Endpoint Policy vs Bucket Policy: Endpoint policies are additive with bucket policies. Do not rely solely on endpoint policy—bucket policy should enforce origin restriction (defense in depth).- VPC Endpoint idempotency: If you recreate an endpoint, its ID changes — update bucket policies accordingly.- Privatelink vs Gateway Endpoint: S3 uses Gateway endpoints; Interface endpoints (PrivateLink) are not used for S3 data plane and would not satisfy aws:SourceVpce for S3.

Verification- Positive test: From an EC2 instance in the VPC, confirm s3:GetObject/s3:PutObject succeed and that requests use the VPC endpoint (check VPC flow logs or verify that traffic to S3 IP prefixes does not go via IGW).- Negative tests: From a host outside the VPC (e.g., laptop on internet or another VPC without endpoint), attempt the same S3 operations; expect explicit AccessDenied due to policy.- Console test: From a browser outside the VPC, attempt S3 operations — should be denied.- Audit: Enable S3 server access logs and CloudTrail data events for S3 to confirm request source, and review for any requests that lack the SourceVpce attribute. Use AWS Config rule (s3-bucket-public-read-prohibited and custom rules) to detect non-compliant buckets.- Automation: Add CI/CD checks that validate bucket policies include the SourceVpce condition and that endpoint IDs in policies match live VPC endpoints.

Summary- Enforce via explicit Deny in bucket policy for any request not coming from the vpce-id, use endpoint policy to restrict who/what can use the endpoint, handle cross-account by combining explicit Allow with source conditions, and verify with in-VPC and external negative tests plus logging/CloudTrail audits.

MediumSystem Design

20 practiced

Design a secure and scalable multi-account network architecture for 50 AWS accounts where shared services (logging, firewall, audit) are centralized. Explain how you would use AWS Transit Gateway (or alternatives) for hub-and-spoke connectivity, segmentation via route tables, enforce security controls, and onboard new accounts quickly and safely.

Sample Answer

Requirements & constraints:- Centralize shared services (logging, firewall, audit) for 50 accounts, isolate workloads, enforce security controls, fast safe onboarding, scalable & least-privilege.

High-level architecture:- AWS Organizations with a networking account (hub), security account (shared services), logging/audit account, and 50 workload accounts (spokes).- Central Transit Gateway (TGW) owned by networking account. Use AWS Resource Access Manager (RAM) if you need to share resources or create attachments per-account from networking account. Each workload account creates a VPC and TGW attachment (or networking account creates attachments via cross-account IAM role).- Create an inspection VPC in security account with Network Firewall or third-party NGFW behind Gateway Load Balancer (GWLB). Attach inspection VPC to TGW.- Logging/audit account receives VPC Flow Logs, CloudWatch Logs, CloudTrail, and centralized S3/Glue for retention and analysis.

Segmentation via TGW route tables:- Use multiple TGW route tables to enforce segmentation (e.g., prod, non-prod, management, internet egress).- Associate each spoke attachment to one route table (association) and control propagated routes from attachments (propagation). This gives per-spoke isolation and selective connectivity to shared services/inspection/Internet.- Example: Prod spokes -> TGW-RT-prod which propagates to inspection VPC and logging VPC; Dev spokes -> TGW-RT-dev with no internet route.

Security controls & enforcement:- East-west/ north-south inspection: Route all traffic to/from spokes through the inspection VPC using TGW route tables and GWLB/NW Firewall. Use policy-based rules in NGFW or AWS Network Firewall for segmentation controls.- Centralized egress with NAT + proxy or third-party WAF. Use AWS WAF + CloudFront for public apps.- Encryption: Enable IPSec/TLS for on-prem links (Site-to-Site VPN or Direct Connect + Transit Gateway attachments).- Monitoring & detection: Central GuardDuty, Security Hub aggregation, AWS Config aggregator, and centralized CloudWatch dashboards fed from spoke accounts using cross-account roles or CloudWatch cross-account observability.- IAM & governance: Enforce Service Control Policies (SCPs) for network API restrictions, use AWS SSO with least privilege roles, and restrict who can create TGW attachments.- Logging: VPC Flow Logs at ENI/TGW/Load Balancer level shipped to central logging account; ensure log integrity (S3 Object Lock, MFA delete if needed).

Onboarding process (fast & safe):1. Bootstrap pipeline: Use AWS Control Tower or a custom landing zone built with AWS Organizations + CloudFormation/CDK/ Terraform modules.2. Automated account provisioning: An onboarding pipeline (CodePipeline/Step Functions) runs IaC templates to create baseline VPC, subnets, security groups, TGW attachment, route table association, CloudWatch/Flow Log configuration, and IAM roles.3. Approval gates: Integrate guardrails and manual approvers for adding prod accounts; run security scans (CIS AWS Foundations checks) automatically.4. Post-provision validations: Automated tests (reachability, NGFW policy hit, log flow) and send reports to owners.5. Update process: Use StackSets / Service Catalog to roll out changes; restrict who can modify network or attach to TGW via SCP and IAM.

Scalability & alternatives:- TGW scales well for tens of thousands of routes; segment using multiple TGW route tables to avoid route explosion. Use route limits consideration (advertised attachments and prefixes).- Alternatives: VPC Peering (not scalable for 50 accounts), Transit VPC (older pattern), or AWS PrivateLink for service-specific connectivity. Use TGW + PrivateLink for high-security per-service exposure.- For cross-region: Use TGW peering or use a global transit architecture with TGW attachments per region and replication of inspection/logging.

Operational and cost considerations:- Use cost allocation tags and per-account billing. Monitor TGW data processing and egress costs; centralize egress to reduce duplicated NAT costs.- Test failover: Multi-AZ for inspection and redundancy for TGW (regional), and blueprint for emergency isolation (detach a spoke, blackhole route).

This design gives centralized control, per-account isolation via TGW route tables, enforced inspection & logging, and automated safe onboarding through IaC and guardrails.

Unlock Full Question Bank

Get access to hundreds of Networking, VPC, and Connectivity interview questions and detailed answers.

Join thousands of developers preparing for their dream job.