Cloud Networking and VPC Design Questions

Direct Answer

Most avoidable cloud network latency comes from a hop the architecture did not need to take: an extra Availability Zone (AZ) crossing, a NAT (Network Address Translation) gateway sitting on the path, a DNS lookup that could have been cached, or a fresh connection negotiated when an existing one could have been reused. The fix pattern repeats across all four: keep the call path inside one zone when a healthy option exists, cache what does not need to be re-resolved or re-negotiated on every request, and reserve genuinely cross-zone or cross-region hops for where availability actually requires them.

Hops Between Availability Zones

Every cross-zone hop adds a round trip a same-zone call avoids. The fix is zone affinity: prefer a request's full call chain, application tier, cache, and where possible the database read path, staying in the same zone as the caller when a healthy same-zone target exists, only crossing zones when the closer target is unhealthy or does not exist. This is the same principle behind Kubernetes topology-aware routing and a per-zone read-replica layout.

NAT Gateways

A NAT gateway on the egress path is itself an extra hop, and if it is a single, shared NAT gateway in one zone rather than one per zone, reaching it from another zone forces a cross-zone hop just to leave the VPC at all. Placing one NAT gateway per Availability Zone, with each zone's subnets routing to their own zone's gateway, removes that specific added hop.

DNS Resolution

An uncached DNS lookup is a full round trip, sometimes several, through a chain of resolvers, before the actual request even starts. Two things matter here: setting an appropriately long time-to-live (TTL) on records that do not change often, so clients and resolvers are not forced to re-look-up on every connection, and confirming the client's own resolution behavior actually respects that TTL. Some HTTP client libraries or language-runtime DNS caches ignore the record's TTL entirely and re-resolve on their own fixed short interval, which is a common, hidden latency source that looks like "network latency" but is really a client-side misconfiguration.

Connection Reuse

A fresh TCP handshake plus a fresh TLS handshake, including certificate exchange, on every single request adds several round trips before any application data moves. HTTP keep-alive, reusing an already-open connection for multiple requests, and TLS session resumption, using session tickets or session IDs to skip the full handshake's asymmetric-cryptography negotiation on reconnect, both remove those round trips for repeat calls to the same backend. This matters most for high-request-rate, short-payload traffic, where handshake overhead is a large fraction of the total time per request.

And Similar: Other Common Contributors

Cross-region hops are categorically larger than cross-zone hops, since they cover real geographic distance rather than a different data center in the same metro area. A load balancer or proxy that opens a new connection to its own backend on every client request also just relocates the handshake cost rather than removing it, so connection reuse needs to be checked at every hop in the chain, not only at the client's edge.

Worked Example

Rather than asserting a specific millisecond figure, which depends on region, instance type, and payload size and would be an unverifiable claim here, compare the shape of the path before and after applying these mitigations:

Before: client performs an uncached DNS lookup on every request, traffic crosses a shared NAT gateway in a different zone, the application tier in one zone calls a cache instance in another zone, and every call negotiates a fresh TLS handshake.

After: the client honors DNS caching per the record's TTL, each zone routes through its own NAT gateway with no forced cross-zone hop, the application tier prefers the same-zone cache replica when it is healthy, and repeat calls reuse a persistent, already-negotiated connection.

Each arrow removed between "before" and "after" is one full round trip eliminated per request. The actual millisecond value should come from load-testing the specific workload in its real environment, which is the correct place to produce that number, not an assumption made here.

Trade-offs and Pitfalls

Chasing same-zone speed by removing all cross-zone redundancy trades away availability; keep zone affinity as a preference that fails over across zones when needed, not a hard same-zone-only requirement, or a single zone's failure becomes a full outage.

Setting DNS TTL very long to minimize lookups also slows failover: a changed address behind a load balancer, DNS-based failover, or a blue-green cutover takes up to the full TTL to propagate to caching clients. Tune TTL against how fast the system actually needs to fail over, not purely to minimize lookups.

TLS session resumption reduces handshake cost, but the session state, tickets or an encrypted blob, has to be managed by the terminating layer; at large fleet scale this needs a shared ticket key or a sufficiently sticky load balancer, an operational detail worth surfacing rather than assuming it works automatically.

Never present a specific "typically adds N milliseconds" figure as fact; only a measurement from an actual load test against the real environment earns that claim.

Direct answer

At 500-plus tenants, the network design has to answer three separate questions with three separate mechanisms: how tenants are isolated from each other (a VPC-per-tenant pattern attached to a shared Transit Gateway, TGW), how customers who need their own private connectivity get it without the provider hand-building a one-off integration each time (AWS Resource Access Manager, or RAM, and PrivateLink), and how usage gets metered and onboarding stays fast at hundreds of tenants (automation, not a human provisioning each tenant). Trying to solve all three with one mechanism is what breaks down first at this scale.

Structured elaboration

VPC-per-tenant with Transit Gateway. Each tenant gets a dedicated VPC, attached to a shared TGW in the provider's account, giving strong network-level isolation (a tenant's VPC route table simply has no route to another tenant's VPC) without the cost and operational overhead of fully separate AWS accounts per tenant. This is the right default for tenants with meaningfully different resource needs, security postures, or compliance requirements from each other.

Shared VPC pattern as the lighter-weight alternative. For a large number of smaller, more homogeneous tenants where per-tenant network isolation is overkill, a shared VPC (using RAM to share subnets across accounts, or a single VPC with tenant workloads separated at the application layer instead) reduces the per-tenant fixed cost of a dedicated VPC and TGW attachment; the trade-off is weaker network-level isolation, which needs to be compensated for with strong application-level tenant isolation (per-tenant IAM, or Identity and Access Management, scoping and data partitioning) since the network boundary alone is no longer doing that job.

Route table scaling. At 500-plus tenant VPC attachments, a single flat TGW route table becomes both a scaling concern (approaching the routes-per-route-table ceiling) and a segmentation concern (every tenant's routes visible in one place); segment by tenant tier or by product line into multiple TGW route tables, each holding only the routes relevant to that segment, both because it scales better and because it limits the blast radius, how far the damage from a single mistake can spread, of a route table misconfiguration to one segment rather than the whole tenant base.

AWS Resource Access Manager (RAM) for customer-managed connectivity. When a specific customer needs to connect their own AWS environment into the SaaS (Software as a Service) provider's network (rather than accessing it purely over the public internet), RAM lets you share the TGW itself (or a subnet, in the shared-VPC pattern) with the customer's own AWS account under a controlled resource share, so the customer's traffic reaches the provider's network without the provider having to build a bespoke peering or VPN integration per customer request.

PrivateLink for customer connectivity that shouldn't require sharing your TGW at all. For customers who want private connectivity but shouldn't be handed a share into your core network fabric, exposing specific services via PrivateLink (an Endpoint Service backed by a Network Load Balancer, consumed by the customer's own Interface VPC Endpoint) gives them the "never touches the public internet" property without extending your TGW's trust boundary into their account.

Tenant isolation enforcement. Security groups and NACLs (network access control lists) scoped per tenant VPC provide the baseline; the TGW route tables described above provide the network-level segmentation between tenants; and, for the shared-VPC pattern specifically, IAM policies and resource tagging enforce isolation at the application and data layer where the network layer alone can't.

Metering network usage per tenant. VPC Flow Logs, tagged and attributed per tenant VPC (or per tenant-tagged resource in the shared-VPC pattern), feed a usage-metering pipeline; because each tenant VPC is a distinct, taggable resource in the VPC-per-tenant pattern, attributing NAT gateway data-processing charges, TGW data-processing charges, and cross-AZ (Availability Zone) transfer per tenant is mostly a tagging and cost-allocation exercise rather than a novel metering problem.

Tenant onboarding at scale. Manually provisioning a new tenant's VPC, TGW attachment, route table association, and security baseline does not scale past a small number of tenants; a self-service pipeline (an infrastructure-as-code template triggered by a signup or provisioning event) that allocates the tenant's CIDR (Classless Inter-Domain Routing) block from a governed IPAM (IP address management) pool, applies the standard security baseline, and attaches the new VPC to the correct TGW route table for that tenant's tier is what makes onboarding tenant 500 take the same few minutes as onboarding tenant 5.

The identity and logging angle: cross-account IAM roles and centralized logging via PrivateLink. For tenants (or the provider's own internal teams) that need cross-account access into the provider's environment, for example a support engineer troubleshooting a specific tenant's workload, scope access through cross-account IAM roles with session tags identifying which tenant's resources the session is allowed to touch, backed by federation (SSO or OIDC, OpenID Connect) rather than long-lived per-tenant credentials, so access is auditable and time-bounded per session rather than standing. For centralized logging specifically, rather than relying on cross-account S3 (Simple Storage Service) bucket policies (which get unwieldy to audit correctly at hundreds of tenant accounts, since a single overly-broad bucket policy statement can expose more than intended), route each tenant VPC's log export through a PrivateLink endpoint service into a central logging account, which keeps the trust boundary explicit and per-tenant (each tenant's endpoint connection is individually visible and revocable) rather than implicit in a shared bucket policy's conditions.

flowchart TB
    subgraph SaaSAcct[SaaS provider account]
        TGW[Transit Gateway]
        CORE[Core services VPC]
        MON[Central monitoring and egress inspection]
    end
    TGW --> CORE
    TGW --> MON
    subgraph T1[Tenant VPC 1]
        A1[Tenant workload]
    end
    subgraph T2[Tenant VPC 2]
        A2[Tenant workload]
    end
    subgraph T3[Tenant VPC N, via RAM-shared subnet]
        A3[Tenant workload]
    end
    T1 --> TGW
    T2 --> TGW
    T3 --> TGW
    Cust[Customer-managed connectivity via PrivateLink] --> CORE

Worked example

A SaaS provider onboards a new enterprise tenant that requires private connectivity from their own AWS account. The onboarding pipeline provisions a dedicated tenant VPC, drawing its CIDR block automatically from the IPAM pool, attaches it to the shared TGW under the "enterprise tier" route table (segmented from the "self-serve tier" route table used by smaller tenants), and applies the standard security baseline. Because this specific tenant wants private connectivity into a specific reporting service rather than a full network share, the provider exposes that one service via a PrivateLink Endpoint Service instead of sharing the TGW itself with the tenant's account, keeping the trust boundary narrow. The tenant's VPC Flow Logs, tagged with the tenant's ID, feed the same metering pipeline used for every other tenant, attributing that tenant's NAT and TGW data-processing costs automatically.

Trade-offs and pitfalls

VPC-per-tenant gives strong isolation but means the provider's TGW attachment count, route table complexity, and per-tenant fixed cost all scale linearly with tenant count, which is the right trade for tenants that need or pay for that isolation but becomes wasteful for a long tail of small, low-usage tenants better served by the shared-VPC pattern; picking the wrong pattern for a given tenant's tier is more common than picking the wrong pattern architecture overall. On the identity side, the biggest real risk is a support or operations role that starts with a properly scoped, tenant-specific session-tagged permission and, over time, accumulates broader standing access "for convenience," which quietly erodes the exact isolation guarantee the rest of this design was built to provide.

Direct answer

You cannot route two networks that use the same address range to each other directly, so the migration has to introduce a translation layer that lets both estates keep functioning under their original addressing while you migrate services across incrementally, rather than attempting a single flag-day renumbering of either company's entire network. The practical approach is a temporary NAT (Network Address Translation) or proxy bridge using non-overlapping address space, service-by-service cutover through it, and a rollback plan that assumes at least one cutover attempt will need to be reversed.

Structured elaboration

Confirming the conflict and scoping the bridge. If both companies' AWS estates use 10.0.0.0/16, they overlap entirely, confirmed programmatically as True when you check whether the two ranges overlap. Rather than renumbering either side (a large, risky, and slow undertaking across every application, firewall rule, and piece of documentation that hard-codes the old range), introduce a bridge using AWS's shared address space, 100.64.0.0/10 (RFC 6598, historically reserved for carrier-grade NAT but explicitly permitted by AWS as a secondary VPC CIDR precisely for scenarios like this one), which by construction does not overlap with either company's RFC 1918 (private IPv4) range.

Translation options. A NAT/proxy tier sits in a dedicated "bridge" VPC or account: for company A reaching a specific company B service, a NAT device rewrites the destination from a bridge-range address to the real overlapping 10.0.0.0/16 address inside company B's network, and vice versa for the return path. In practice this is usually implemented as a small number of purpose-built translation targets (one per service being bridged, not a blanket network-wide NAT), because translating an entire overlapping /16 transparently is far more complex than translating the handful of services that actually need to talk to each other during the transition window.

Service discovery during the transition. DNS resolution has to point at the bridge addresses for cross-company traffic while continuing to resolve normally for traffic that stays within one company's own network, which typically means split-horizon DNS (the same name resolving to a different address depending on which network is asking) implemented through conditional forwarding rules scoped specifically to the services being bridged, rather than a network-wide DNS change.

Cross-account connectivity and Transit Gateway (TGW) routing patterns. Each company's estate attaches to a shared TGW in the bridge account. The bridge VPC's route table, not the companies' own VPC route tables, holds the translation logic, so neither company's existing routing has to change at all during the transition; only the bridge account's routes need to know about both sides.

Incremental cutover. Migrate one service or one business unit at a time: stand up the bridge path for that service, validate it under real traffic, then either leave the bridge in place long-term for that pairing or, once feasible, renumber the smaller or newer of the two overlapping ranges (often the acquired company's estate, since it's usually smaller and more recently stood up) so the bridge for that specific service can eventually be retired.

Automation and rollback. Drive the bridge's NAT/proxy configuration and the DNS forwarding rules through infrastructure as code so a specific service's bridge configuration can be stood up, and torn down, as a single atomic change. Keep the pre-migration state (the original DNS records, the original route tables) intact and untouched until the new path has been validated in production for a defined burn-in period, so reverting a single problematic service's cutover means restoring one small, well-understood change rather than unwinding a network-wide renumbering.

Worked example

Company A's payments-api.a.internal (in company A's 10.0.5.0/24) needs to be reachable from company B's order-processing service (in company B's overlapping 10.0.7.0/24). The bridge VPC allocates 100.64.1.0/24 and creates a NAT target there that maps 100.64.1.10 to 10.0.5.10 (the real payments API address) on company A's side. Company B's DNS gets a forwarding rule so payments-api.a.internal resolves to 100.64.1.10 instead of the real, colliding 10.0.5.10. Company B's order-processing service, still addressed entirely within its own unchanged 10.0.7.0/24, reaches the bridge address over the shared TGW, and the bridge NAT rewrites the destination to the real address inside company A's network before it enters company A's VPC. Neither company's existing route tables or internal addressing changed; only the bridge account and company B's DNS forwarding rule for this one name did.

Trade-offs and pitfalls

A NAT/proxy bridge adds a hop and a piece of infrastructure that has to be operated and monitored for as long as the overlap persists, and every additional service bridged this way adds its own translation rule and DNS entry to keep track of, so this approach scales to a handful to a few dozen cross-company integration points, not to "just NAT the whole network" as a permanent architecture. The temptation to treat the bridge as a permanent fix rather than a transition tool is the main long-term risk: without a concrete plan and timeline to eventually renumber the smaller estate, the bridge's complexity becomes permanent, ad hoc technical debt that the next integration project inherits.

Direct answer

For connecting an on-premises data center to the cloud, VPN over the internet is the cheapest and fastest to stand up but inherits the public internet's variable latency and has no cloud-provider service-level agreement (SLA) for the path outside the cloud; a dedicated private link (AWS Direct Connect or Azure ExpressRoute) gives consistent, high-bandwidth, low-latency connectivity backed by an SLA but costs more and takes longer to provision; third-party SD-WAN (software-defined wide area network) sits above either underlying transport and adds centralized policy, application-aware routing, and often automatic failover across multiple links, at the cost of another vendor and management layer. The right choice depends on what the workload actually needs: sporadic, low-volume, latency-tolerant traffic fits VPN; sustained, latency-sensitive, or compliance-bound traffic justifies a dedicated connection; and a multi-site organization managing many locations' connectivity centrally often layers SD-WAN over one or both.

Structured elaboration

Worked example

Migration framing. A retailer doing a lift-and-shift of its on-premises data center to the cloud initially connects over a Site-to-Site VPN to start moving non-critical workloads immediately, since the VPN can be live the same day and the migration timeline cannot wait weeks for a physical cross-connect. In parallel, they order a Direct Connect circuit at a colocation facility, which takes on the order of weeks to provision. Once Direct Connect is live, they configure Border Gateway Protocol (BGP) so the dedicated circuit is preferred for all cloud-bound traffic (achieved by advertising a shorter AS-path or a better local-preference value on the Direct Connect side) with the VPN kept active as an automatic failover path, giving the migration both a fast start and, once ready, a durable dual-path design rather than a one-time VPN that later has to be torn out and replaced. For cross-region resilience during the migration window, they also plan bandwidth headroom on both paths so a regional failover does not saturate whichever link absorbs the shifted traffic.

Cost-driven recommendation for a scheduled bulk-transfer workload. A different team runs a nightly batch job transferring a large, predictable volume of data (say, tens of terabytes) from an on-premises system to cloud storage during a fixed overnight window. Because the volume is large and the timing is predictable and recurring (not sporadic), the data-transfer economics favor Direct Connect as the primary path, sized to comfortably clear the nightly volume inside the batch window, with an IPsec VPN kept purely as a failover for the rare case the dedicated circuit is down during that window. Sizing the VPN failover path only needs to cover the batch's minimum acceptable throughput, not match Direct Connect's full capacity, since it exists to avoid a missed run, not to be the steady-state path.

Reliability comparison, concretely. An internet-path VPN carries no AWS or Azure commitment for the portion of the trip between the customer's router and the cloud provider's edge; if a transit provider somewhere on the public internet degrades, there is no SLA to invoke against the cloud provider for that. A Direct Connect connection, deployed as multiple connections terminating on separate devices across more than one physical location (the resiliency model AWS recommends for production), carries an SLA and a defined mean-time-to-repair (MTTR) commitment for the dedicated portion of the path; a single, non-redundant Direct Connect connection, by contrast, carries no SLA at all, which is a common and costly misunderstanding.

Trade-offs and pitfalls

The most common mistake is ordering a single Direct Connect connection and treating it as production-ready simply because it is "dedicated": without a second connection on a separate device, ideally at a second location, there is no SLA and no protection against a single device or facility failure, which defeats much of the point of choosing Direct Connect over VPN in the first place. A second pitfall is under-provisioning the VPN failover path's bandwidth on the theory that it will "rarely" be used, then discovering during an actual Direct Connect outage that the VPN cannot carry production traffic at an acceptable level either. Finally, teams evaluating SD-WAN sometimes expect it to fix underlying transport problems it cannot fix. SD-WAN can steer traffic intelligently across the links you give it and add centralized policy, but it does not manufacture bandwidth or eliminate the latency characteristics of a genuinely poor underlying link.

Direct Answer

Cross-AZ (Availability Zone) data transfer and NAT gateway data processing are two separate charges with two separate fixes; measure each one before touching anything, then apply cheap short-term changes immediately (route AWS-native traffic off NAT entirely) and pursue architectural long-term changes (per-AZ NAT placement, egress proxies, Direct Connect) for whatever traffic genuinely still needs to leave.

Step-by-Step Diagnosis

1. Pull Cost Explorer or the Cost and Usage Report filtered by usage type, the regional data-transfer line for cross-AZ bytes and the NAT gateway line for processed bytes, to size each driver in dollars rather than guessing which one dominates.
2. Query VPC Flow Logs, already a good general practice to have enabled, grouped by source and destination subnet or Availability Zone, and separately by whether the destination ENI is a NAT gateway, to find which services or subnets actually generate the volume.
3. Cross-reference against the deployment topology: is a chatty service pinned unevenly across zones, does a distributed data store's replication protocol inherently generate cross-zone traffic, is a batch job pulling large images or datasets through a single shared NAT gateway.

Reducing Cross-AZ Transfer

Prefer AZ-aware routing so a service calls a same-zone backend when one is healthy, cutting east-west cross-zone hops. But distinguish cross-zone traffic that is buying you something, quorum writes and multi-zone replication that provide the durability guarantee, from cross-zone traffic that is just an accident of default load-balancer or scheduler behavior. Only the second kind is waste worth removing.

Reducing NAT Gateway Costs

The most commonly overlooked driver is traffic to AWS-native services, object storage, a managed database, a container registry, that never needed to leave the AWS network in the first place.

Short term. Add Gateway VPC endpoints for object storage and the managed database service (no hourly or per-gigabyte charge) immediately; this alone commonly removes a large share of NAT-billed traffic as a same-day, low-risk change. Add Interface VPC endpoints (AWS PrivateLink) for other heavily used AWS APIs; these carry their own hourly and per-gigabyte cost but are still typically far cheaper than routing the same traffic through NAT, and they remove a public-internet hop for API calls.

Long term. Deploy one NAT gateway per Availability Zone, with each zone's private subnets routing to their own zone's NAT gateway rather than all zones sharing a single NAT gateway in one zone. A shared NAT gateway is itself a generator of the OTHER cost driver in this question, because reaching it from another zone is a cross-AZ transfer. Fixing NAT placement per zone addresses both drivers at once. For very high, predictable outbound volume, an egress proxy fleet you operate yourself can undercut NAT Gateway's linear per-gigabyte charge at scale, worth it only above a volume threshold you should size against your own numbers first. For predictable, large, recurring egress to one external partner, AWS Direct Connect's per-gigabyte egress via a dedicated circuit is typically cheaper than internet egress through NAT at sustained high volume.

None of this trades away security or availability: VPC endpoints remove a public-internet path rather than weakening anything, and per-AZ NAT preserves the standard high-availability pattern AWS itself recommends, avoiding a single NAT gateway becoming a single point of failure for an entire region's egress.

Worked Example

Assume a service moves 10,000 GB per month through a single, shared NAT gateway in one zone, while half its callers run in a different zone, so 5,000 GB of that traffic first crosses zones before it is even NAT-processed. Using AWS's published per-region rates as of this writing (verify current rates for your own region before relying on this), NAT Gateway data processing at $0.045 per GB and cross-AZ data transfer at $0.01 per GB in each direction:

If the flow-log analysis shows 40% of that 10,000 GB is object-storage or managed-database traffic that a Gateway Endpoint removes from NAT entirely:

Adding per-AZ NAT removes the remaining 5,000 GB cross-AZ leg entirely, since each zone's traffic now reaches its own local NAT gateway:

That is a 46% reduction, from $500 to $270 per month, using only endpoint offload and per-AZ NAT placement, before considering the higher-effort egress-proxy or Direct Connect options. These are illustrative inputs to show the arithmetic, not a claim about any specific account's real traffic; rerun this against the account's actual Cost Explorer and flow-log numbers.

Trade-offs and Pitfalls

Deploying a NAT gateway per zone raises the fixed hourly charge, since each NAT gateway bills independently; for a genuinely low-traffic, occasional-egress workload, consolidating to fewer NAT gateways can sometimes cost less in absolute dollars than three times the hourly charge. Run the numbers for the real traffic profile rather than applying "one NAT per zone" as a rule with no math behind it.

Treating all cross-AZ traffic as waste is a mistake; a large share of it is the price of durability and availability guarantees, and removing it means giving up those guarantees, not optimizing.

An egress proxy fleet shifts cost from a fully managed AWS service to something the team operates and patches; only justified once the operating cost is clearly smaller than the NAT savings at the account's actual volume.

Never present a specific dollar savings figure to stakeholders without redoing this math against the account's real Cost Explorer numbers first; the worked example above is a template for the calculation, not a promise.