Cloud Networking and VPC Design Questions
Designing networks inside a cloud provider: VPC/VNet topology, subnets, route tables, gateways, NAT, and peering, plus private connectivity through VPC endpoints and cloud load balancers. Covers segmentation, security groups and network ACLs, hybrid connectivity to on-premises data centers over VPN or dedicated links like Direct Connect and ExpressRoute, IP address planning across many VPCs and accounts, and how cloud network design differs from traditional data-center networking.
Compare traditional bastion-host (jump box) patterns with agent-based approaches such as AWS Systems Manager Session Manager or Azure Bastion. Discuss differences in attack surface area, auditing and session recording, patching responsibilities, least privilege access, and recommend a secure, operationally maintainable pattern for remote administrator access.
Sample Answer
Direct answer
A traditional bastion host is a hardened jump box sitting in a public subnet with an open inbound port (typically SSH or RDP) that administrators connect to before hopping onward to private resources; it is itself a standing, internet-facing attack surface that has to be patched, monitored, and key-managed forever. An agent-based approach, such as AWS Systems Manager Session Manager or Azure Bastion, replaces that open inbound port with an agent on the target instance that opens an outbound-only connection to the cloud provider's session-broker service, so administrators authenticate through the same identity and access management (IAM) system as everything else and no inbound port needs to exist at all. For most organizations today, the agent-based approach is the right default: it is not just more convenient, it structurally removes an entire class of exposure the bastion pattern cannot avoid.
Structured elaboration
| Dimension | Traditional bastion host | Agent-based (Session Manager / Azure Bastion) |
|---|---|---|
| Attack surface | An always-on, internet-facing host with an open inbound port; a single compromised bastion is a pivot point into every private resource it can reach | No inbound port on any target instance; the connection is always initiated outbound from the instance to the provider's service, so there is no listening port for an attacker to find or exploit |
| Auditing and session recording | Depends entirely on what you build yourself: shell history, a session-recording tool bolted on, or nothing at all if not deliberately configured | Session logging and, optionally, full session recording, are built into the service and centrally stored, tied to the authenticated identity that started the session, with no extra tooling required |
| Patching responsibility | You own the bastion host's operating system, SSH/RDP daemon, and any hardening, indefinitely, as a standing piece of infrastructure | The provider manages the underlying service; you are only responsible for keeping the lightweight agent on each target instance up to date, which is a much smaller and more automatable surface |
| Least privilege | Access control is whatever the bastion's own user accounts and network rules enforce, often coarser than the cloud provider's IAM model and managed separately from it | Access is granted through the same fine-grained IAM policies as everything else in the account, so a specific engineer can be scoped to specific instances, for a specific time window, without a separate credential system to maintain |
| Credential model | Typically SSH keys or shared accounts, which have to be distributed, rotated, and revoked, often manually | Tied to the identity provider already in use; revoking someone's IAM access revokes their session access at the same time, with no separate key to rotate or revoke |
| Operational cost over time | Grows over time: patch cycles, key rotation, monitoring the bastion itself for compromise, capacity planning for a host that is really just a means to an end | Low and mostly fixed: the service itself is managed, and per-instance overhead is a small agent rather than an entire dedicated host |
Worked example
Recommended pattern. For a company running a fleet of private-subnet instances that administrators need occasional shell access to, the recommended pattern is to remove the bastion host entirely and rely on Session Manager (or the equivalent service on other clouds) for all administrative access, layered with the following hardening controls:
- Multi-factor authentication (MFA) enforced at the identity provider, not just a password, before a session can be started at all, since the session broker is only as strong as the identity check gating it.
- Session recording enabled and shipped to a separate, access-controlled log store (not just left in the default retention), so a compromised administrative identity's actions are still reviewable after the fact, and so the recording itself cannot be tampered with by whoever was using the compromised identity.
- Source restricted at the IAM policy level to only the specific instances and only the specific engineers who need them, scoped by tag or resource identifier rather than a blanket "any engineer can reach any instance" policy, which is the least-privilege principle applied to session access specifically rather than assumed to be handled elsewhere.
- No inbound security-group rule for SSH or RDP on any target instance at all, which is the concrete, verifiable proof the bastion's attack surface has actually been removed, not just supplemented; a lingering "just in case" SSH rule left open alongside Session Manager defeats much of the point.
A legacy environment still running a jump box during a transition can apply the same hardening principles to it in the meantime (restrict its source IP range as tightly as possible, require MFA for its own login, ship session logs somewhere durable), but this is explicitly a stopgap, not the target state, since none of that hardening removes the fundamental issue that the bastion is still an always-on, internet-facing host with an open inbound port.
Trade-offs and pitfalls
The most common mistake during a transition is standing up the agent-based approach alongside the existing bastion "just in case" and never actually decommissioning the bastion, which leaves the original attack surface fully intact while adding a second access path to manage and audit; the security benefit only materializes once the old inbound port is actually closed. A second pitfall is treating the agent-based service as a complete solution without also tightening the identity side of it: session-broker access inherits whatever IAM policy grants it, so an overly broad policy ("any authenticated user in the account may start a session to any instance") reproduces much of the bastion's own weak, coarse-grained access model, just without the open port. Finally, a legitimate reason organizations sometimes keep a hardened bastion around longer than expected is a genuine offline or air-gapped requirement, or a compliance framework that has not yet been updated to recognize agent-based session brokering as an equivalent control; in those cases, apply every hardening control above to the bastion itself rather than treating "we still have a bastion" as an excuse to skip them.
Describe operational uses for VPC Flow Logs. Explain how you'd configure destinations (CloudWatch Logs vs S3), sampling vs full logs, retention policies, parsing challenges, and integration with SIEM or analytics pipelines for use cases like security detections and cost analysis.
Sample Answer
Direct answer
Treat Flow Logs as an operational data pipeline you have to design, not a checkbox you enable once: pick a destination that matches how you'll actually query the data, decide up front whether you need every flow or a representative slice, set a retention policy that matches your compliance and cost tolerance, and plan for the parsing effort before the first incident forces you to do it under pressure.
Structured elaboration
Destinations: CloudWatch Logs versus S3. CloudWatch Logs is the right choice when you need near-real-time metric filters and alarms (for example, alerting the moment REJECT records spike on a sensitive subnet), because CloudWatch Logs Insights can query recent data with low latency and integrate directly with CloudWatch Alarms. Amazon S3 (Simple Storage Service) is the right choice for high-volume, long-retention, or batch-analytical use, because it's dramatically cheaper per gigabyte at scale and pairs naturally with Amazon Athena for SQL-style ad hoc querying or with a SIEM (Security Information and Event Management) ingestion pipeline. Many mature setups use both: a short CloudWatch Logs retention window for live alerting, with the same logs (or the S3-destined copy) retained far longer in S3 for forensics and compliance.
Sampling versus full logs. Full logging (every flow, every network interface) gives complete visibility but generates real volume, especially in a busy production VPC (Virtual Private Cloud), translating directly into ingestion and storage cost. A narrower approach, logging REJECT-only traffic, or only specific subnets carrying sensitive workloads, cuts cost significantly while preserving the highest-value signal (most day-to-day debugging and security monitoring cares more about what got blocked than about the sea of normal ACCEPT traffic), at the cost of losing full traffic-volume visibility for cost or capacity-planning use cases that specifically need the ACCEPT records too.
Retention policies. Set retention deliberately at both layers: the CloudWatch Logs group's retention setting (which otherwise defaults to "never expire" and silently grows forever) and the S3 bucket's lifecycle policy (transitioning older data to cheaper storage classes, or expiring it, on a schedule matched to your actual compliance requirement, commonly 90 days to a year for general operational use and longer for regulated environments).
Parsing challenges. The default record format is a whitespace-delimited flat string, not JSON, which means most ingestion pipelines need an explicit parser or a custom log format with fields you've pre-selected; get the field order and format right at flow-log creation time, because you cannot change an existing flow log's format after the fact, only delete it and create a new one, which loses continuity in your historical data.
Integration with SIEM and analytics pipelines. Streaming through Amazon Data Firehose into a SIEM lets security teams correlate network flow data with other signals (authentication logs, application logs) in one place, which is where flow logs earn their keep for detections like lateral movement (an internal host suddenly talking to hosts it's never talked to before) or data exfiltration (an unusual, sustained high-byte-count flow to an external destination). For cost analysis, the same data, queried via Athena against the S3 destination, lets you attribute cross-AZ or NAT gateway data-processing charges to the specific flows generating them, which billing data alone cannot do.
Worked example
A security team wants to detect a specific pattern: an internal host suddenly initiating outbound connections to a large number of distinct external IPs in a short window, a common exfiltration or scanning signature. They configure flow logs on the relevant VPCs to stream to Data Firehose into their SIEM, with a custom format that includes pkt-dstaddr in addition to the version-2 default fields (needed because the destination might be behind a NAT gateway, whose flow log entry shows the NAT's own address in the standard dstaddr field rather than the packet's true destination). The SIEM rule counts distinct dstaddr values per srcaddr per 5-minute window and alerts above a threshold; the same underlying data, retained in S3 for a year under a lifecycle policy, later gets queried through Athena during a retrospective audit to confirm no prior instance of the same pattern had been missed.
Trade-offs and pitfalls
Enabling full logging everywhere without a sampling or scoping decision is the most common way this gets expensive fast, particularly in a high-throughput production VPC. The second common mistake is choosing a destination based on what's easiest to set up rather than how the data will actually be used, landing everything in CloudWatch Logs and only later discovering that months of forensic data are prohibitively costly to store there long-term compared to S3. Finally, remember that a flow log's format is fixed at creation, so retrofitting a needed field (like pkt-dstaddr for NAT gateway visibility) means starting a new flow log going forward, not recovering the field from history you already collected.
Evaluate an architecture that uses Network Load Balancers with proxy appliances versus Gateway Load Balancer (GWLB) with managed firewall appliances for centralized egress inspection and NAT. Discuss transparency, scaling characteristics, fault tolerance, cost, and operational patching/management considerations.
Sample Answer
Direct Answer
Default to Gateway Load Balancer (GWLB) with managed firewall appliances for centralized egress inspection, because it was purpose-built for exactly this: a transparent, layer-3 bump-in-the-wire with native flow-aware scaling and failover of the appliance fleet. Fall back to a Network Load Balancer (NLB) with a proxy appliance fleet only when the chosen appliance vendor does not support GWLB's integration protocol, or when the design specifically needs load-balancer features (like TLS termination at the LB) that a pure layer-3 GWLB does not provide.
How the Two Architectures Actually Differ
| Dimension | NLB + proxy appliance fleet | GWLB + managed firewall appliance |
|---|---|---|
| Transparency | NLB can preserve source IP, but the proxy appliance itself typically has to NAT or rewrite it unless explicitly configured for transparent mode | Native transparency: GWLB uses GENEVE (Generic Network Virtualization Encapsulation, a protocol that wraps the original packet so it reaches the appliance unchanged) encapsulation on UDP port 6081 to hand the appliance the original packet and get it back, so the appliance sees the real 5-tuple (the five values, source IP, destination IP, source port, destination port, and protocol, that together identify one traffic flow) without any NAT trick |
| Scaling | Standard Auto Scaling Group (ASG) behind an NLB, using layer-4 hashing | Also ASG-backed, but GWLB adds flow stickiness (5-tuple, 3-tuple, or 2-tuple) that pins a given flow to the same appliance instance for the life of the connection, which stateful appliances like an intrusion detection system need |
| Fault tolerance | Target health checks drive failover; new connections reroute, but in-flight flows on a failed target generally drop, standard layer-4 behavior | Same target-health-check mechanism, plus GWLB endpoints let multiple consumer VPCs share one resilient inspection fleet without each needing its own appliance set |
| Cost | NLB hourly charge plus load balancer capacity units, plus the appliance's own EC2 cost; the proxy software licensing model is usually simpler since it's generic infrastructure | GWLB hourly plus its own capacity units, plus the appliance cost, which for a named next-generation firewall vendor is often licensed by throughput and can dominate either architecture's bill |
| Operational patching | You own the full proxy stack end to end, full control, full patching burden | The appliance vendor (an Elastic Load Balancing partner) maintains compatibility with the GENEVE integration, and centralizing the fleet behind one GWLB means one place to roll out a patch instead of a bespoke proxy fleet you maintain yourself |
Worked Example
flowchart LR
subgraph Spoke[Spoke VPC]
App[Application subnet]
GWLBe[GWLB endpoint, dedicated subnet]
end
subgraph Inspection[Inspection VPC]
GWLB[Gateway Load Balancer]
FW1[Firewall appliance A]
FW2[Firewall appliance B]
NAT[NAT gateway / IGW]
end
App -->|route table next hop| GWLBe
GWLBe -->|GENEVE, UDP 6081| GWLB
GWLB --> FW1
GWLB --> FW2
FW1 --> NAT
FW2 --> NAT
Each spoke VPC's application subnet routes egress traffic to a GWLB endpoint in a dedicated endpoint subnet within the same VPC. The GWLB endpoint forwards over GENEVE to the Gateway Load Balancer in a central inspection VPC, which distributes flows across the firewall appliance fleet with 5-tuple stickiness, and the inspected traffic exits through a shared NAT gateway or internet gateway. The equivalent NLB-based design instead routes spoke VPC traffic across a Transit Gateway (a central hub that connects many VPCs and on-premises networks together instead of pairing each one individually) attachment to an inspection VPC where the appliances are registered directly as NLB targets, terminating and re-originating the connection at the proxy rather than passing the original packet through.
Trade-offs and Pitfalls
GWLB does not give TLS inspection for free. It is a layer-3 service that hands the appliance a raw IP packet; decrypting and inspecting TLS is still the appliance's own job, including managing the certificates needed to do it.
Choosing NLB with a proxy fleet out of familiarity, without first checking whether the specific appliance can run in pure layer-4 proxy mode without breaking an application's own IP-based access-control assumptions, is a common and avoidable mistake.
GWLB endpoints need a dedicated subnet per consuming VPC and Availability Zone specifically for the endpoint, on top of the application subnet routing through it, an extra piece of subnet design the Transit-Gateway-based NLB pattern does not require, and one many network teams do not expect the first time.
What would flip the recommendation entirely: if the chosen firewall vendor does not support the GENEVE integration at all, check the Elastic Load Balancing partner list, then NLB with a proxy fleet is the only option regardless of every other consideration above.
Design a hardened bastion/access solution that eliminates inbound SSH from the internet, supports audit and session recording, and allows emergency access for on-call engineers. Compare options: AWS Systems Manager Session Manager, Azure Bastion, traditional bastion hosts with just-in-time (JIT) access, and third-party jump hosts. Describe IAM policies, ephemeral credentials, MFA, session logging, and a migration plan to roll out the safest option.
Sample Answer
Direct answer
Eliminate inbound SSH (Secure Shell) or RDP (Remote Desktop Protocol) entirely rather than trying to secure a bastion host that still has an open inbound port: use an agent-based, outbound-only connection broker (AWS Systems Manager Session Manager, Azure Bastion, or a comparable managed service) so the engineer authenticates through identity and access management (IAM) rather than a network path, and the target instance never has a listening port exposed to anything, including the "bastion" itself in the traditional sense.
Structured elaboration
| Option | Inbound port required | Session recording | Credential model | Emergency/on-call fit |
|---|---|---|---|---|
| AWS Systems Manager Session Manager | None (agent makes an outbound connection to the Systems Manager service) | Native: sessions can be logged to CloudWatch Logs or S3 | IAM policy grants session start on specific instances; no SSH key or password ever exists on the instance | Good: access is granted by adding an IAM policy, revocable instantly, no key distribution |
| Azure Bastion | None from the internet; Bastion is a managed PaaS (platform as a service) reachable only via the Azure portal or CLI over HTTPS | Native session logging available | Azure AD (Microsoft's identity and access management service, recently renamed Entra ID) authentication and role-based access control | Good: similarly identity-driven, no VM-level open port |
| Traditional bastion host with just-in-time (JIT) access | Yes, but only opened for a short, approved window per request | Depends on tooling layered on top (session recording usually bolted on via script or a proxy) | Often SSH keys or short-lived certificates issued per request | Workable, but the JIT approval step itself becomes a dependency during an incident |
| Third-party jump host / VPN appliance | Yes, generally always-open to authorized networks | Varies by vendor | Often a mix of shared credentials and MFA (multi-factor authentication), harder to keep fully per-user | Weakest fit here: usually the most operational overhead to keep current and audited |
IAM policies. Access is granted as an IAM policy statement scoping which instances (by tag or resource ARN, or Amazon Resource Name) a given role or user may start a session on, which replaces "who has the SSH key" with "who has the IAM permission," a model that's centrally auditable and instantly revocable by removing the policy, with no key rotation or distribution problem.
Ephemeral credentials. No long-lived SSH key ever needs to exist on the instance at all; the session broker (Systems Manager) authenticates the human via IAM and establishes the connection without a static credential ever being provisioned. This directly closes the most common real-world bastion failure mode: a leaked or never-rotated SSH private key granting standing access indefinitely.
Multi-factor authentication (MFA). Because access is gated by IAM sign-in rather than possession of a key, requiring MFA on the underlying identity (via an IAM policy condition or the organization's identity provider) applies uniformly to every session-broker connection, without needing a separate MFA integration bolted onto the bastion host itself.
Session logging. Every keystroke and output of a session can be streamed to CloudWatch Logs or an S3 bucket, giving a full audit trail per session, tied to the IAM identity that started it, which is materially stronger than a traditional bastion's SSH access log, which typically only shows that a connection happened, not what was done inside it.
Migration plan. Roll out in phases rather than a single cutover: install and validate the Systems Manager agent on a subset of instances first, grant IAM session-start permissions to a pilot group of engineers, and run the new path in parallel with the existing bastion for a defined period; once the pilot group confirms full workflow coverage (including any tooling that assumed direct SSH, like certain deployment scripts), remove inbound SSH/RDP security group rules for the migrated instances, and only then decommission the legacy bastion host itself, since removing it too early, before every workflow has an equivalent, forces engineers back to workarounds that reopen inbound access informally.
flowchart LR
Eng[On-call engineer] -->|IAM auth plus MFA| SSM[Session broker: Systems Manager Session Manager]
SSM -->|outbound-only agent connection, no open inbound port| Instance[Private EC2 instance]
SSM -->|session transcript| Log[CloudWatch Logs or S3]
SSM -->|session start and stop events| Audit[CloudTrail audit trail]
Worked example
An on-call engineer needs emergency access to a production database host at 2 a.m. With the session-broker model, they authenticate to the AWS console or CLI with their existing IAM identity (already requiring MFA), which already carries a break-glass IAM policy scoped to session-start on production instances tagged role=oncall-emergency, and start a session directly; no key to locate, no bastion IP to remember, no separate VPN client. The full session transcript lands in CloudWatch Logs automatically, and the security team's next-morning review shows exactly which commands were run, by whom, and for how long, without needing to correlate a bastion's SSH log against a separate change ticket.
Trade-offs and pitfalls
The session broker becomes a critical dependency itself: if the managed service or its agent has an outage, emergency access through that path is unavailable, so a genuine break-glass fallback (a tightly controlled, alarmed, rarely-used traditional path) is still worth keeping for that narrow case, rather than assuming the managed service is infallible. A second pitfall in migration: cutting over the humans but forgetting the automation, deployment scripts, configuration management tools, or monitoring agents that were quietly relying on direct SSH access, which breaks in ways that look unrelated to the bastion project until someone traces the failure back to the removed inbound rule.
Given a three-tier application (web, app, database) inside one VPC, propose the specific security group and network ACL rules that implement least privilege between the tiers. For each tier, specify the ports and traffic direction, and say whether you'd enforce it with a stateful security group or a stateless NACL and why.
Sample Answer
Direct Answer
Enforce least privilege between the three tiers with security groups that reference each other by security-group ID rather than IP range, since that survives autoscaling and re-addressing; keep network ACLs (NACLs) as a coarse, mostly-default backstop at the subnet boundary rather than trying to replicate the same fine-grained per-tier logic in a stateless, rule-numbered format.
Per-Tier Rules
| From (source) | To (destination) | Port / protocol | Direction | Enforced by | Why |
|---|---|---|---|---|---|
| Load balancer or internet | Web tier | 443/TCP (plus 80 for redirect only) | Inbound to web | Security group | Only the entry-point tier needs public or load-balancer-facing exposure |
| Web tier | App tier | Application's listening port, for example 8080/TCP | Outbound from web, inbound to app | Security group | Referencing the app tier's security-group ID, not a CIDR block (a range of IP addresses written like 10.0.0.0/16), means scaling web instances never requires a rule change |
| App tier | Database tier | Database port, for example 5432/TCP for PostgreSQL or 3306/TCP for MySQL | Outbound from app, inbound to database | Security group | Only the app tier, never the web tier and never the internet, may reach the database |
| Database tier | (none) | Default-deny outbound | Outbound | Security group | A database in this design never initiates outbound connections, so there is nothing legitimate for an open outbound rule to permit |
| Whole subnet | Explicitly known-bad CIDR ranges | All | Inbound | Network ACL | A coarse, subnet-wide blocklist backstop, not the primary access control |
Security Groups vs Network ACLs, and Why
Security groups are stateful: allowing an inbound request automatically allows its reply, and they attach per instance (technically per elastic network interface), which is exactly the granularity a three-tier, per-service rule set needs. They are the right layer for the tier-to-tier rules above.
Network ACLs are stateless: a reply to an allowed request needs its own explicit rule, and they attach per subnet, applying to everyone in it regardless of which specific instance sent or received the traffic. Trying to mirror the exact tier-to-tier port matrix in NACLs duplicates the security-group logic in a harder-to-maintain, rule-numbered format, and is a common source of a self-inflicted outage when someone inserts a new rule at the wrong priority and silently blocks a needed reply. NACLs are best reserved for what they are uniquely good at, a subnet-wide deny on a known-bad range, with a default allow for everything else, leaving the actual least-privilege enforcement in the stateful security groups.
If a NACL rule set is used for something finer anyway, for example to satisfy a compliance requirement for an explicit second layer, remember that a client-initiated TCP connection's return traffic arrives on an ephemeral port, typically in the 1024 to 65535 range, and must be explicitly allowed both inbound and outbound, unlike a stateful security group where allowing the request automatically allows the reply.
Trade-offs and Pitfalls
Writing NACL rules that duplicate the security-group matrix doubles the maintenance surface for no additional protection and is a real, recurring source of self-inflicted outages.
Referencing tiers by CIDR block instead of security-group ID is the most common mistake in a rule set like this: a CIDR-based rule breaks the moment the subnet is resized or an instance lands in a different Availability Zone, while a security-group-ID reference keeps working automatically as the fleet scales.
Security groups default to allowing all outbound traffic unless explicitly restricted; it is easy to forget this and leave a tier able to reach anywhere outbound, exactly the kind of gap a compromised instance would exploit.
A stricter default-deny-egress (blocking all outbound traffic by default, allowing only named exceptions) posture on every tier is more secure but adds ongoing maintenance, every new legitimate destination needs an explicit rule. Most teams accept that cost for the database tier, which rarely changes, and relax it somewhat for the app tier if it calls several external services, provided each destination is named explicitly rather than left open to everywhere.
Unlock Full Question Bank
Get access to all 14 Cloud Networking and VPC Design interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.