Cloud Networking and VPC Design Questions
Designing networks inside a cloud provider: VPC/VNet topology, subnets, route tables, gateways, NAT, and peering, plus private connectivity through VPC endpoints and cloud load balancers. Covers segmentation, security groups and network ACLs, hybrid connectivity to on-premises data centers over VPN or dedicated links like Direct Connect and ExpressRoute, IP address planning across many VPCs and accounts, and how cloud network design differs from traditional data-center networking.
Explain when to use PrivateLink (interface endpoints) versus gateway endpoints for cloud services. Discuss data plane scaling, cross-account access, security posture, cost model (per-ENI, data charges), and limitations such as AZ-level endpoints, throughput bottlenecks, and supported services. Provide concrete examples where each is preferable.
Sample Answer
Direct answer
Use a Gateway Endpoint whenever the target is one of the small set of services it supports (Amazon S3 and DynamoDB) and you only need reachability from within a single Virtual Private Cloud (VPC)'s route tables; it costs nothing extra and adds no infrastructure to manage. Use an Interface Endpoint, built on AWS PrivateLink, for everything else, or when you specifically need reachability from on-premises, from a peered VPC, or across accounts, since it places an actual network interface with a private IP address in your subnet and can be reached from anywhere with network access to that interface, at the cost of a per-Availability-Zone (AZ) hourly charge plus data processed.
Structured elaboration
Data plane scaling. A Gateway Endpoint's data plane is just a routing rule; it does not itself have a throughput ceiling separate from the underlying network path. An Interface Endpoint's data plane is an actual elastic network interface (ENI), which by default supports up to 10 Gbps of bandwidth per AZ and automatically scales up to 100 Gbps per AZ; a workload needing more than that in a single AZ needs to spread load across multiple AZs or contact the provider, but for the overwhelming majority of workloads this ceiling is far above what they will ever push through one endpoint.
Cross-account access. A Gateway Endpoint is scoped to the route tables of VPCs you explicitly associate it with, all typically within reach of your own account's networking; extending it cleanly to another account's VPC is awkward. An Interface Endpoint is the natural fit for cross-account access: the service owner publishes an endpoint service, and any other account can create its own interface endpoint against it (subject to the service owner's allow-list), with each side managing its own resources independently and no shared route-table coordination required.
Security posture. A Gateway Endpoint's access control is an endpoint policy plus the target service's own resource policy (for example, an S3 bucket policy). An Interface Endpoint adds a genuine extra layer beyond that: because it is a real ENI sitting in your subnet, you can also attach a security group directly to it, controlling at the network layer which of your own resources may even attempt to reach it, independent of whatever policy the service itself enforces.
Cost model. A Gateway Endpoint has no separate charge. An Interface Endpoint is billed per AZ-hour it is deployed in, plus a per-gigabyte data-processing charge, on top of whatever cross-AZ or cross-region data-transfer charges would otherwise apply; for a service that only supports Gateway Endpoints (S3, DynamoDB), this makes the Gateway Endpoint strictly cheaper for equivalent same-VPC access, which is the main reason to prefer it there whenever it is available and sufficient.
Limitations to know cold:
- AZ-level endpoints. An Interface Endpoint's network interface is deployed per AZ; if you only deploy it in one AZ while your workload spans three, resources in the other two AZs either fail to reach it or incur a cross-AZ data path and added latency, depending on configuration. The standard practice is to deploy the interface endpoint in every AZ the consuming workload actually runs in.
- Throughput bottlenecks. Even though the per-endpoint ceiling scales automatically up to 100 Gbps per AZ, a workload concentrated in a single AZ is still bounded by that one AZ's endpoint capacity; spreading the workload (and the endpoint) across AZs is the mitigation, not raising a limit that mostly manages itself.
- Supported services. Not every AWS service, and not every third-party service, publishes an endpoint service at all; before designing around PrivateLink for a specific dependency, confirm it actually supports Interface Endpoints rather than assuming universal coverage.
Worked example
Where a Gateway Endpoint is clearly preferable. A batch analytics job running entirely within one VPC reads and writes large volumes of data to S3. Because the target is S3, reachability is needed only from within that one VPC, and the workload is data-volume-sensitive (meaning the per-gigabyte Interface Endpoint charge would add real cost at scale), a Gateway Endpoint is the clear choice: same private-network benefit, no additional hourly or per-gigabyte charge.
Where an Interface Endpoint is clearly preferable. A different team needs to call AWS's Key Management Service (KMS) to decrypt data from an on-premises system connected over Direct Connect, not just from within the VPC. KMS does not support Gateway Endpoints at all (only a small set of services do), and even if it did, a Gateway Endpoint's route-table-based reachability would not extend cleanly to an on-premises network the way an Interface Endpoint's private IP, reachable over the same Direct Connect connection, does. Here an Interface Endpoint, deployed in each AZ the on-premises traffic might land in, is the only workable option, and its cost is easily justified by the sensitivity and volume characteristics of KMS calls, which are typically small, frequent requests rather than large data transfers, so the per-gigabyte charge is not the dominant cost concern the way it would be for a bulk S3 workload.
Trade-offs and pitfalls
The most common mistake is deploying an Interface Endpoint in only one AZ "to save cost" for a workload that actually spans three, which either creates a hidden cross-AZ dependency (extra latency and data-transfer cost that often outweighs the endpoint savings) or, in some configurations, breaks reachability from the other AZs outright; the fix is deploying the endpoint in every AZ the workload runs in and accepting that as the real cost of using PrivateLink there. A second pitfall is assuming a service supports Gateway Endpoints because a sibling service does (S3 and DynamoDB are genuinely the only two, a narrower list than many people expect), and only discovering the gap when trying to associate a Gateway Endpoint with a service that has never offered one. Finally, for data-volume-heavy workloads, defaulting to Interface Endpoints out of convenience when a Gateway Endpoint would have sufficed quietly adds a recurring per-gigabyte cost that compounds significantly at scale, which is worth checking explicitly rather than assuming the endpoint type is a purely mechanical choice.
For a compliance-heavy environment, describe how you would restrict and monitor outbound egress traffic from private subnets, including the use of NAT gateway, centralized proxy, firewall rules, and logging. Explain pros/cons of forcing egress through a single inspection point.
Sample Answer
Direct answer
For a compliance-heavy environment, the requirement is usually not just "block bad outbound traffic" but "prove every outbound connection was inspected and can be reconstructed later," which means forcing all egress from private subnets through one controlled, logged inspection point rather than letting each subnet or account exit independently. The trade-off you're explicitly accepting is a single choke point that adds latency, cost, and a scaling bottleneck, in exchange for the auditability a compliance program actually needs.
Structured elaboration
The building blocks. A NAT (Network Address Translation) gateway still handles the address translation itself, but instead of sitting directly in each workload's own path to the internet, it sits behind (or alongside) a centralized proxy or firewall tier that every subnet's outbound traffic is routed through, typically via a shared inspection VPC (Virtual Private Cloud) reached over a Transit Gateway (TGW). A centralized forward proxy (or a firewall appliance like AWS Network Firewall or a third-party equivalent, fronted by a Gateway Load Balancer for scale) applies domain and IP allowlisting or denylisting, TLS (Transport Layer Security) inspection where policy requires it, and logs every connection attempt, allowed or blocked. Flow logs and, where applicable, proxy access logs are shipped to a retained, tamper-evident log store, since "we had a firewall" is a much weaker compliance answer than "here is the log of every connection this workload made in the last 12 months."
Pros of forcing all egress through one inspection point. A single, well-audited enforcement point is dramatically easier to certify and reason about than dozens of independent NAT gateways each with their own, potentially drifting, rule set; a compliance auditor can review one policy and one log stream rather than reconciling configuration across every account. It also makes detection easier: a single place to watch for anomalous destinations or volumes across the entire estate, rather than needing to aggregate signals from many independent egress points after the fact.
Cons of forcing all egress through one inspection point. Every workload's outbound traffic now takes an extra hop, adding latency that's real even if often small, and concentrating both the traffic volume and the cost (data-processing charges accrue at the centralized NAT/firewall tier, at a scale proportional to the whole estate's egress, not any one workload's). It also creates a single point that must be sized and made highly available correctly, because an outage there doesn't just degrade one workload's egress, it degrades every workload's egress across the estate simultaneously; and it becomes an organizational bottleneck if every new outbound destination a team needs requires a change request against a shared, centrally-owned firewall rule set, which can slow legitimate work if the change process isn't kept fast.
What a compliance program typically requires beyond the mechanism itself. Documented change control on the firewall/proxy rule set (who approved adding a new allowed destination, and why), retained logs meeting the specific regulation's retention window, and periodic access review confirming the rule set still reflects only what's currently needed, since egress rules tend to accumulate permissive exceptions over time if nobody prunes them.
Worked example
A financial services company subject to a regulatory requirement to log and control all outbound network connections routes every workload VPC's 0.0.0.0/0 traffic through a shared inspection VPC via TGW. The inspection VPC runs a proxy that allowlists specific external domains (a payment processor, a small number of SaaS (Software as a Service) vendors, and nothing else by default) and logs every connection attempt with the requesting workload's identity, destination, and outcome. When an application team needs a new external dependency, they submit a change request to add the domain to the allowlist rather than being able to reach arbitrary internet destinations by default; the resulting log, retained for the regulation's required period, becomes the artifact the compliance team hands to an auditor as evidence that outbound connectivity is both controlled and observable.
Trade-offs and pitfalls
The single inspection point is the correct design for the stated compliance goal, but treating it as a "set it once" control invites rule-set rot: allowlist entries added under time pressure during an incident and never revisited accumulate into a broad, effectively-uncontrolled allowlist that technically satisfies "we have an egress control" while defeating its purpose. Sizing the shared inspection tier for peak aggregate load across the whole estate, not just its current load, matters too: because every workload depends on the same choke point, under-provisioning it turns a routine traffic spike in one team's workload into a shared outage for every team behind it.
Your NAT Gateway costs have grown dramatically due to large volumes of container image pulls and OS updates across many accounts. Propose a combination of architecture and operational changes to reduce NAT and egress costs while maintaining security isolation and reliability for development and production workloads.
Sample Answer
Direct Answer
Stop paying to fetch the same bytes through NAT repeatedly: cache container images close to the workload with a pull-through cache or a regional registry mirror, mirror the OS package repositories the same way, and route whatever still must reach AWS APIs through VPC endpoints instead of NAT, all rolled out as shared infrastructure across accounts rather than as a per-account fix.
Diagnose First, Per Account
Attribute the volume with the same VPC Flow Log approach as any NAT cost investigation, but segmented per account since this is explicitly a many-accounts problem. Container image pulls are typically dominated by a small number of large base images pulled repeatedly, on every pod start, every CI (continuous integration) run, every node bootstrap, rather than by genuinely unique data.
Fixing Container Image Pull Costs
Amazon Elastic Container Registry (ECR) Interface VPC endpoints (for the registry API and the Docker-compatible data endpoint) remove the NAT hop for pulls from ECR, but they are not sufficient by themselves: image layers are actually stored in and served from Amazon S3, so an S3 Gateway endpoint is also required, a gotcha that leaves pulls silently falling back to NAT when teams add only the ECR endpoints.
ECR's pull-through cache feature is the single biggest lever when many accounts are pulling the same public base images repeatedly: configure ECR as a local cache in front of a public registry so the first pull of a given image and tag in the region fetches it once, and every subsequent pull, across every account attached to the shared cache, is served from the regional cache over PrivateLink and S3 rather than re-fetched from the public internet through NAT each time.
For the multi-account structure specifically, centralize the pull-through cache and the shared VPC endpoints in a shared-services or network account, reachable from spoke accounts via Transit Gateway (a central hub connecting many VPCs and accounts without pairing each one individually) or Resource Access Manager (RAM)-shared endpoints, so this is configured and patched once rather than per account.
Fixing OS Update Costs
Route package-manager traffic to a regional mirror where the distribution provides one, or stand up an internal package proxy or mirror refreshed on a schedule, so instances pull updates from inside the region instead of the public internet on every patch cycle. The equivalent for Windows fleets is a local update-management server rather than every instance reaching a public update endpoint directly through NAT.
Security Isolation and Reliability for Dev vs Production
VPC endpoints and a private image cache do not reduce isolation, they remove a public-internet path that existed before, which is a security improvement rather than a trade-off against one. Per-account NAT gateways can stay in place for whatever residual, genuinely external traffic remains, sized down now that the highest-volume, noisiest traffic, images and packages, is gone.
Development and test accounts are usually the highest-volume, lowest-value NAT traffic, a CI pipeline pulling the same base image hundreds of times a day. Route that traffic through the shared cache aggressively, since a cache miss there is cheap to recover from. Production should get its own dedicated cache or registry replica rather than sharing a rate limit or availability fate with a noisy development environment, so a development CI storm cannot affect a production node's ability to pull an image during an incident.
Worked Example
Assume 50 AWS accounts, each running CI that pulls the same 0.5 GB base image an average of 20 times per day, currently all routed through NAT to the public registry:
Total pulls/month=50×20×30=30,000 pulls Total NAT-processed GB/month (no cache)=30,000×0.5 GB=15,000 GB NAT processing cost (no cache)=15,000×$0.045/GB=$675/monthWith a shared ECR pull-through cache across the 50 accounts, the only traffic left touching NAT is the cache's own periodic refresh from the public registry, not 30,000 individual pulls. If the cache refreshes once per day:
Cache-refresh GB/month=1×0.5 GB×30=15 GB NAT processing cost (with cache)=15×$0.045=$0.68/monthThat is roughly a 99% cut in NAT-billed data for this one image alone, since the remaining 29,970 pulls now flow over PrivateLink and S3 at that path's own, much smaller endpoint cost rather than the NAT per-gigabyte charge, and the saving scales with account count because the fix is shared infrastructure rather than a per-account change. As with any cost projection, treat these inputs as illustrative and re-derive the real numbers from each account's own CI logs.
Trade-offs and Pitfalls
A shared pull-through cache becomes a single dependency for every account's CI. If it is rate-limited by the upstream public registry or has an outage, every account's builds can stall at once, which is exactly why production needs its own replica rather than depending on the same instance as development.
Forgetting the S3 Gateway endpoint alongside the ECR interface endpoints is the most common implementation miss: pulls keep falling back to NAT, and the team does not realize the actual image bytes are an S3 fetch, not just an ECR API call.
Centralizing the cache in a shared-services account adds cross-account IAM (Identity and Access Management, the permissions system controlling who and what can access a resource) and networking complexity, resource policies on the registry, shared endpoints, versus the simplicity of every account doing its own thing, but the cost and consistency payoff usually justifies it once there are more than a handful of accounts.
A self-hosted OS package mirror is one more system to patch and secure; only worth running once the NAT savings or reliability benefit clearly outweighs that operational cost.
Describe the roles and behaviors of an Internet Gateway (IGW), NAT Gateway/instance, and Egress-Only Internet Gateway for IPv6 in a cloud VPC. Explain the differences in IPv4 vs IPv6 outbound/inbound behavior, and propose a pattern for a mixed IPv4/IPv6 VPC hosting public and internal services.
Sample Answer
Direct answer
An Internet Gateway (IGW) provides bidirectional internet access for IPv4 traffic: resources with a public IPv4 address can both initiate outbound connections and receive unsolicited inbound ones. A NAT Gateway (or NAT instance) gives IPv4 resources in a private subnet outbound-only internet access without a public IP of their own, and no path exists for anything on the internet to initiate a connection back to them. For IPv6, there is no equivalent to NAT (every IPv6 address in a subnet with an IGW attached is globally routable by design, so there is nothing to translate), so the Egress-Only Internet Gateway exists specifically to give IPv6 resources the same outbound-only behavior a NAT Gateway gives IPv4 resources, without translating any addresses at all.
Structured elaboration
IPv4 outbound and inbound behavior. IPv4 addresses inside a Virtual Private Cloud (VPC) are private by default (drawn from RFC 1918 ranges) and not directly reachable from the internet at all until you attach an IGW and either assign a public IP directly to a resource in a subnet whose route table points at the IGW, or route through a NAT Gateway for outbound-only access. This is why IPv4 needs two distinct gateway concepts: one for full bidirectional reachability (IGW, paired with a public IP) and a separate one purely for outbound-only reachability (NAT Gateway), because IPv4's limited address space made private addressing plus translation the default architecture from the start.
IPv6 outbound and inbound behavior, and why it is structurally different. IPv6 addresses assigned within a VPC are globally unique and, once an IGW is attached to the VPC, are directly routable to and from the internet; there is no private/public IPv6 address distinction the way there is for IPv4, and no address-translation step happens at all. That means the IGW alone gives an IPv6 resource full bidirectional internet reachability the moment its subnet's route table points at it, with nothing analogous to "assign a public IP" required. The consequence for anyone wanting outbound-only IPv6 behavior (an application server that needs to call external application programming interfaces (APIs) but should never accept unsolicited inbound connections) is that you cannot simply reuse the NAT Gateway concept, since there is no address to translate; that gap is exactly what the Egress-Only Internet Gateway fills; it is stateful and permits only outbound-initiated IPv6 traffic and its return responses, blocking any connection attempt that originates from the internet.
Summary table:
| Gateway | Protocol | Direction | Address handling |
|---|---|---|---|
| Internet Gateway (IGW) | IPv4 and IPv6 | Bidirectional | IPv4: requires a public IP mapped via one-to-one address translation. IPv6: no translation, the address is already globally routable |
| NAT Gateway / NAT instance | IPv4 only | Outbound-initiated only | Translates many private IPv4 addresses to one (or a small pool of) public IPv4 address |
| Egress-Only Internet Gateway | IPv6 only | Outbound-initiated only | No translation at all; simply blocks internet-initiated inbound connections while allowing outbound-initiated traffic and its replies |
Worked example
A pattern for a mixed IPv4/IPv6 VPC hosting both public and internal services. The VPC is configured dual-stack (both an IPv4 and an IPv6 CIDR block, a defined range of IP addresses, assigned) with both an IGW and an Egress-Only Internet Gateway attached, plus an IPv4 NAT Gateway. The public web tier's subnet route table routes both IPv4 and IPv6 default traffic to the IGW, and the web tier's resources have both a public IPv4 address (or sit behind a dual-stack load balancer) and a globally routable IPv6 address, giving external clients on either protocol a working path in. The private application tier's subnet route table routes IPv4 default traffic to the NAT Gateway (outbound-only, translated) and IPv6 default traffic to the Egress-Only Internet Gateway (outbound-only, untranslated), so the tier can call external APIs over either protocol without being reachable from the internet on either one. The database tier typically gets no default route on either protocol at all, since it has no legitimate reason to reach the internet in either direction.
graph TD
INTERNET((Internet)) --> IGW[Internet Gateway - IPv4 and IPv6]
INTERNET -.->|blocked inbound| EIGW[Egress-Only IGW - IPv6 outbound only]
IGW --> WEB[Web tier: dual-stack, public]
WEB --> APP[App tier: private]
APP -->|IPv4 outbound| NAT[NAT Gateway]
APP -->|IPv6 outbound| EIGW
NAT --> IGW
EIGW -.-> INTERNET
Dual-stack service compatibility. Enabling IPv6 on a VPC does not automatically make every dependency reachable over IPv6: managed services, third-party APIs, monitoring agents, and even some internal tooling may only support IPv4 endpoints, so a dual-stack rollout has to explicitly verify each dependency's IPv6 support rather than assuming parity, and applications typically need "happy eyeballs" style dual-stack connection logic (trying both address families and using whichever connects first or successfully) to avoid a slow or failed connection attempt against a dependency that silently does not support IPv6 yet.
Trade-offs and pitfalls
The most common mistake is assuming the Egress-Only Internet Gateway behaves like a NAT Gateway for IPv6 (translating addresses) rather than understanding it does no translation at all, it simply enforces directionality; this matters because it means every IPv6 resource behind it still has its real, globally unique address visible to whatever it connects out to, which is a meaningfully different privacy and identifiability property than IPv4 NAT's address-hiding side effect. A second pitfall is enabling dual-stack on a VPC and forgetting that security groups and NACLs need their own explicit IPv6 rules; an IPv4-only security group rule set does not implicitly cover IPv6 traffic, so a resource can end up with your intended IPv4 restrictions in place while its IPv6 address is wide open, or vice versa, simply because the second protocol's rules were never added. Finally, teams sometimes roll out dual-stack broadly before checking whether every downstream dependency actually supports IPv6, discovering compatibility gaps only when a specific integration mysteriously times out on one protocol while working fine on the other.
Design a scalable, secure multi-tenant network architecture for a SaaS application on AWS supporting 500+ tenants. Tenants require isolation, optional customer-managed connectivity, and centralized monitoring/egress inspection. Discuss VPC-per-tenant with Transit Gateway, shared VPC patterns, route table scaling, use of Resource Access Manager (RAM), and how you'd enforce tenant isolation, meter network usage per tenant, and support tenant on-boarding.
Sample Answer
Direct answer
At 500-plus tenants, the network design has to answer three separate questions with three separate mechanisms: how tenants are isolated from each other (a VPC-per-tenant pattern attached to a shared Transit Gateway, TGW), how customers who need their own private connectivity get it without the provider hand-building a one-off integration each time (AWS Resource Access Manager, or RAM, and PrivateLink), and how usage gets metered and onboarding stays fast at hundreds of tenants (automation, not a human provisioning each tenant). Trying to solve all three with one mechanism is what breaks down first at this scale.
Structured elaboration
VPC-per-tenant with Transit Gateway. Each tenant gets a dedicated VPC, attached to a shared TGW in the provider's account, giving strong network-level isolation (a tenant's VPC route table simply has no route to another tenant's VPC) without the cost and operational overhead of fully separate AWS accounts per tenant. This is the right default for tenants with meaningfully different resource needs, security postures, or compliance requirements from each other.
Shared VPC pattern as the lighter-weight alternative. For a large number of smaller, more homogeneous tenants where per-tenant network isolation is overkill, a shared VPC (using RAM to share subnets across accounts, or a single VPC with tenant workloads separated at the application layer instead) reduces the per-tenant fixed cost of a dedicated VPC and TGW attachment; the trade-off is weaker network-level isolation, which needs to be compensated for with strong application-level tenant isolation (per-tenant IAM, or Identity and Access Management, scoping and data partitioning) since the network boundary alone is no longer doing that job.
Route table scaling. At 500-plus tenant VPC attachments, a single flat TGW route table becomes both a scaling concern (approaching the routes-per-route-table ceiling) and a segmentation concern (every tenant's routes visible in one place); segment by tenant tier or by product line into multiple TGW route tables, each holding only the routes relevant to that segment, both because it scales better and because it limits the blast radius, how far the damage from a single mistake can spread, of a route table misconfiguration to one segment rather than the whole tenant base.
AWS Resource Access Manager (RAM) for customer-managed connectivity. When a specific customer needs to connect their own AWS environment into the SaaS (Software as a Service) provider's network (rather than accessing it purely over the public internet), RAM lets you share the TGW itself (or a subnet, in the shared-VPC pattern) with the customer's own AWS account under a controlled resource share, so the customer's traffic reaches the provider's network without the provider having to build a bespoke peering or VPN integration per customer request.
PrivateLink for customer connectivity that shouldn't require sharing your TGW at all. For customers who want private connectivity but shouldn't be handed a share into your core network fabric, exposing specific services via PrivateLink (an Endpoint Service backed by a Network Load Balancer, consumed by the customer's own Interface VPC Endpoint) gives them the "never touches the public internet" property without extending your TGW's trust boundary into their account.
Tenant isolation enforcement. Security groups and NACLs (network access control lists) scoped per tenant VPC provide the baseline; the TGW route tables described above provide the network-level segmentation between tenants; and, for the shared-VPC pattern specifically, IAM policies and resource tagging enforce isolation at the application and data layer where the network layer alone can't.
Metering network usage per tenant. VPC Flow Logs, tagged and attributed per tenant VPC (or per tenant-tagged resource in the shared-VPC pattern), feed a usage-metering pipeline; because each tenant VPC is a distinct, taggable resource in the VPC-per-tenant pattern, attributing NAT gateway data-processing charges, TGW data-processing charges, and cross-AZ (Availability Zone) transfer per tenant is mostly a tagging and cost-allocation exercise rather than a novel metering problem.
Tenant onboarding at scale. Manually provisioning a new tenant's VPC, TGW attachment, route table association, and security baseline does not scale past a small number of tenants; a self-service pipeline (an infrastructure-as-code template triggered by a signup or provisioning event) that allocates the tenant's CIDR (Classless Inter-Domain Routing) block from a governed IPAM (IP address management) pool, applies the standard security baseline, and attaches the new VPC to the correct TGW route table for that tenant's tier is what makes onboarding tenant 500 take the same few minutes as onboarding tenant 5.
The identity and logging angle: cross-account IAM roles and centralized logging via PrivateLink. For tenants (or the provider's own internal teams) that need cross-account access into the provider's environment, for example a support engineer troubleshooting a specific tenant's workload, scope access through cross-account IAM roles with session tags identifying which tenant's resources the session is allowed to touch, backed by federation (SSO or OIDC, OpenID Connect) rather than long-lived per-tenant credentials, so access is auditable and time-bounded per session rather than standing. For centralized logging specifically, rather than relying on cross-account S3 (Simple Storage Service) bucket policies (which get unwieldy to audit correctly at hundreds of tenant accounts, since a single overly-broad bucket policy statement can expose more than intended), route each tenant VPC's log export through a PrivateLink endpoint service into a central logging account, which keeps the trust boundary explicit and per-tenant (each tenant's endpoint connection is individually visible and revocable) rather than implicit in a shared bucket policy's conditions.
flowchart TB
subgraph SaaSAcct[SaaS provider account]
TGW[Transit Gateway]
CORE[Core services VPC]
MON[Central monitoring and egress inspection]
end
TGW --> CORE
TGW --> MON
subgraph T1[Tenant VPC 1]
A1[Tenant workload]
end
subgraph T2[Tenant VPC 2]
A2[Tenant workload]
end
subgraph T3[Tenant VPC N, via RAM-shared subnet]
A3[Tenant workload]
end
T1 --> TGW
T2 --> TGW
T3 --> TGW
Cust[Customer-managed connectivity via PrivateLink] --> CORE
Worked example
A SaaS provider onboards a new enterprise tenant that requires private connectivity from their own AWS account. The onboarding pipeline provisions a dedicated tenant VPC, drawing its CIDR block automatically from the IPAM pool, attaches it to the shared TGW under the "enterprise tier" route table (segmented from the "self-serve tier" route table used by smaller tenants), and applies the standard security baseline. Because this specific tenant wants private connectivity into a specific reporting service rather than a full network share, the provider exposes that one service via a PrivateLink Endpoint Service instead of sharing the TGW itself with the tenant's account, keeping the trust boundary narrow. The tenant's VPC Flow Logs, tagged with the tenant's ID, feed the same metering pipeline used for every other tenant, attributing that tenant's NAT and TGW data-processing costs automatically.
Trade-offs and pitfalls
VPC-per-tenant gives strong isolation but means the provider's TGW attachment count, route table complexity, and per-tenant fixed cost all scale linearly with tenant count, which is the right trade for tenants that need or pay for that isolation but becomes wasteful for a long tail of small, low-usage tenants better served by the shared-VPC pattern; picking the wrong pattern for a given tenant's tier is more common than picking the wrong pattern architecture overall. On the identity side, the biggest real risk is a support or operations role that starts with a properly scoped, tenant-specific session-tagged permission and, over time, accumulates broader standing access "for convenience," which quietly erodes the exact isolation guarantee the rest of this design was built to provide.
Unlock Full Question Bank
Get access to all 36 Cloud Networking and VPC Design interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.