Cloud Networking and VPC Design Questions
Designing networks inside a cloud provider: VPC/VNet topology, subnets, route tables, gateways, NAT, and peering, plus private connectivity through VPC endpoints and cloud load balancers. Covers segmentation, security groups and network ACLs, hybrid connectivity to on-premises data centers over VPN or dedicated links like Direct Connect and ExpressRoute, IP address planning across many VPCs and accounts, and how cloud network design differs from traditional data-center networking.
Design a hardened bastion/access solution that eliminates inbound SSH from the internet, supports audit and session recording, and allows emergency access for on-call engineers. Compare options: AWS Systems Manager Session Manager, Azure Bastion, traditional bastion hosts with just-in-time (JIT) access, and third-party jump hosts. Describe IAM policies, ephemeral credentials, MFA, session logging, and a migration plan to roll out the safest option.
Sample Answer
Direct answer
Eliminate inbound SSH (Secure Shell) or RDP (Remote Desktop Protocol) entirely rather than trying to secure a bastion host that still has an open inbound port: use an agent-based, outbound-only connection broker (AWS Systems Manager Session Manager, Azure Bastion, or a comparable managed service) so the engineer authenticates through identity and access management (IAM) rather than a network path, and the target instance never has a listening port exposed to anything, including the "bastion" itself in the traditional sense.
Structured elaboration
| Option | Inbound port required | Session recording | Credential model | Emergency/on-call fit |
|---|---|---|---|---|
| AWS Systems Manager Session Manager | None (agent makes an outbound connection to the Systems Manager service) | Native: sessions can be logged to CloudWatch Logs or S3 | IAM policy grants session start on specific instances; no SSH key or password ever exists on the instance | Good: access is granted by adding an IAM policy, revocable instantly, no key distribution |
| Azure Bastion | None from the internet; Bastion is a managed PaaS (platform as a service) reachable only via the Azure portal or CLI over HTTPS | Native session logging available | Azure AD (Microsoft's identity and access management service, recently renamed Entra ID) authentication and role-based access control | Good: similarly identity-driven, no VM-level open port |
| Traditional bastion host with just-in-time (JIT) access | Yes, but only opened for a short, approved window per request | Depends on tooling layered on top (session recording usually bolted on via script or a proxy) | Often SSH keys or short-lived certificates issued per request | Workable, but the JIT approval step itself becomes a dependency during an incident |
| Third-party jump host / VPN appliance | Yes, generally always-open to authorized networks | Varies by vendor | Often a mix of shared credentials and MFA (multi-factor authentication), harder to keep fully per-user | Weakest fit here: usually the most operational overhead to keep current and audited |
IAM policies. Access is granted as an IAM policy statement scoping which instances (by tag or resource ARN, or Amazon Resource Name) a given role or user may start a session on, which replaces "who has the SSH key" with "who has the IAM permission," a model that's centrally auditable and instantly revocable by removing the policy, with no key rotation or distribution problem.
Ephemeral credentials. No long-lived SSH key ever needs to exist on the instance at all; the session broker (Systems Manager) authenticates the human via IAM and establishes the connection without a static credential ever being provisioned. This directly closes the most common real-world bastion failure mode: a leaked or never-rotated SSH private key granting standing access indefinitely.
Multi-factor authentication (MFA). Because access is gated by IAM sign-in rather than possession of a key, requiring MFA on the underlying identity (via an IAM policy condition or the organization's identity provider) applies uniformly to every session-broker connection, without needing a separate MFA integration bolted onto the bastion host itself.
Session logging. Every keystroke and output of a session can be streamed to CloudWatch Logs or an S3 bucket, giving a full audit trail per session, tied to the IAM identity that started it, which is materially stronger than a traditional bastion's SSH access log, which typically only shows that a connection happened, not what was done inside it.
Migration plan. Roll out in phases rather than a single cutover: install and validate the Systems Manager agent on a subset of instances first, grant IAM session-start permissions to a pilot group of engineers, and run the new path in parallel with the existing bastion for a defined period; once the pilot group confirms full workflow coverage (including any tooling that assumed direct SSH, like certain deployment scripts), remove inbound SSH/RDP security group rules for the migrated instances, and only then decommission the legacy bastion host itself, since removing it too early, before every workflow has an equivalent, forces engineers back to workarounds that reopen inbound access informally.
flowchart LR
Eng[On-call engineer] -->|IAM auth plus MFA| SSM[Session broker: Systems Manager Session Manager]
SSM -->|outbound-only agent connection, no open inbound port| Instance[Private EC2 instance]
SSM -->|session transcript| Log[CloudWatch Logs or S3]
SSM -->|session start and stop events| Audit[CloudTrail audit trail]
Worked example
An on-call engineer needs emergency access to a production database host at 2 a.m. With the session-broker model, they authenticate to the AWS console or CLI with their existing IAM identity (already requiring MFA), which already carries a break-glass IAM policy scoped to session-start on production instances tagged role=oncall-emergency, and start a session directly; no key to locate, no bastion IP to remember, no separate VPN client. The full session transcript lands in CloudWatch Logs automatically, and the security team's next-morning review shows exactly which commands were run, by whom, and for how long, without needing to correlate a bastion's SSH log against a separate change ticket.
Trade-offs and pitfalls
The session broker becomes a critical dependency itself: if the managed service or its agent has an outage, emergency access through that path is unavailable, so a genuine break-glass fallback (a tightly controlled, alarmed, rarely-used traditional path) is still worth keeping for that narrow case, rather than assuming the managed service is infallible. A second pitfall in migration: cutting over the humans but forgetting the automation, deployment scripts, configuration management tools, or monitoring agents that were quietly relying on direct SSH access, which breaks in ways that look unrelated to the bastion project until someone traces the failure back to the removed inbound rule.
Describe operational uses for VPC Flow Logs. Explain how you'd configure destinations (CloudWatch Logs vs S3), sampling vs full logs, retention policies, parsing challenges, and integration with SIEM or analytics pipelines for use cases like security detections and cost analysis.
Sample Answer
Direct answer
Treat Flow Logs as an operational data pipeline you have to design, not a checkbox you enable once: pick a destination that matches how you'll actually query the data, decide up front whether you need every flow or a representative slice, set a retention policy that matches your compliance and cost tolerance, and plan for the parsing effort before the first incident forces you to do it under pressure.
Structured elaboration
Destinations: CloudWatch Logs versus S3. CloudWatch Logs is the right choice when you need near-real-time metric filters and alarms (for example, alerting the moment REJECT records spike on a sensitive subnet), because CloudWatch Logs Insights can query recent data with low latency and integrate directly with CloudWatch Alarms. Amazon S3 (Simple Storage Service) is the right choice for high-volume, long-retention, or batch-analytical use, because it's dramatically cheaper per gigabyte at scale and pairs naturally with Amazon Athena for SQL-style ad hoc querying or with a SIEM (Security Information and Event Management) ingestion pipeline. Many mature setups use both: a short CloudWatch Logs retention window for live alerting, with the same logs (or the S3-destined copy) retained far longer in S3 for forensics and compliance.
Sampling versus full logs. Full logging (every flow, every network interface) gives complete visibility but generates real volume, especially in a busy production VPC (Virtual Private Cloud), translating directly into ingestion and storage cost. A narrower approach, logging REJECT-only traffic, or only specific subnets carrying sensitive workloads, cuts cost significantly while preserving the highest-value signal (most day-to-day debugging and security monitoring cares more about what got blocked than about the sea of normal ACCEPT traffic), at the cost of losing full traffic-volume visibility for cost or capacity-planning use cases that specifically need the ACCEPT records too.
Retention policies. Set retention deliberately at both layers: the CloudWatch Logs group's retention setting (which otherwise defaults to "never expire" and silently grows forever) and the S3 bucket's lifecycle policy (transitioning older data to cheaper storage classes, or expiring it, on a schedule matched to your actual compliance requirement, commonly 90 days to a year for general operational use and longer for regulated environments).
Parsing challenges. The default record format is a whitespace-delimited flat string, not JSON, which means most ingestion pipelines need an explicit parser or a custom log format with fields you've pre-selected; get the field order and format right at flow-log creation time, because you cannot change an existing flow log's format after the fact, only delete it and create a new one, which loses continuity in your historical data.
Integration with SIEM and analytics pipelines. Streaming through Amazon Data Firehose into a SIEM lets security teams correlate network flow data with other signals (authentication logs, application logs) in one place, which is where flow logs earn their keep for detections like lateral movement (an internal host suddenly talking to hosts it's never talked to before) or data exfiltration (an unusual, sustained high-byte-count flow to an external destination). For cost analysis, the same data, queried via Athena against the S3 destination, lets you attribute cross-AZ or NAT gateway data-processing charges to the specific flows generating them, which billing data alone cannot do.
Worked example
A security team wants to detect a specific pattern: an internal host suddenly initiating outbound connections to a large number of distinct external IPs in a short window, a common exfiltration or scanning signature. They configure flow logs on the relevant VPCs to stream to Data Firehose into their SIEM, with a custom format that includes pkt-dstaddr in addition to the version-2 default fields (needed because the destination might be behind a NAT gateway, whose flow log entry shows the NAT's own address in the standard dstaddr field rather than the packet's true destination). The SIEM rule counts distinct dstaddr values per srcaddr per 5-minute window and alerts above a threshold; the same underlying data, retained in S3 for a year under a lifecycle policy, later gets queried through Athena during a retrospective audit to confirm no prior instance of the same pattern had been missed.
Trade-offs and pitfalls
Enabling full logging everywhere without a sampling or scoping decision is the most common way this gets expensive fast, particularly in a high-throughput production VPC. The second common mistake is choosing a destination based on what's easiest to set up rather than how the data will actually be used, landing everything in CloudWatch Logs and only later discovering that months of forensic data are prohibitively costly to store there long-term compared to S3. Finally, remember that a flow log's format is fixed at creation, so retrofitting a needed field (like pkt-dstaddr for NAT gateway visibility) means starting a new flow log going forward, not recovering the field from history you already collected.
A fintech customer requires microsegmentation to limit lateral movement between services. Describe how you would implement microsegmentation in a cloud environment using a combination of security groups, NACLs, host-based firewalls, service mesh policies, and cloud firewall appliances. Explain enforcement points, policy lifecycle (authoring, testing, rollout), performance impact, and a migration strategy from a flat network.
Sample Answer
Direct Answer
Layer four distinct enforcement points, security groups, network ACLs, host-based firewalls, and service mesh policy, plus a centralized cloud firewall appliance for anything crossing a trust boundary, so a compromised service is stopped by more than one control. Treat policy authoring, testing, and rollout as a reviewed, staged pipeline with automated drift detection rather than a one-time firewall change, because in a fintech environment the rules that get hand-added during an incident are exactly the ones that quietly outlive the incident.
Enforcement Points and What Each One Actually Stops
- Security groups (SGs): stateful, attached per elastic network interface (ENI). The primary AWS-native microsegmentation primitive between service tiers; reference other tiers by security-group ID, not IP range, so autoscaling doesn't break the rule.
- Network ACLs (NACLs): stateless, attached per subnet. Good for a coarse, subnet-wide blocklist; not fine-grained enough alone for service-to-service policy.
- Host-based firewalls (iptables, nftables, or a fleet-managed agent): enforce policy even if the workload itself is compromised and tries to open an unexpected listener, and are the only layer here that can see the actual process or user making the connection, not just the packet.
- Service mesh policies (mutual TLS plus layer-7 authorization, Istio- or Linkerd-style): enforce identity-based access rather than IP-based access, which matters once workloads are autoscaled or containerized and IP addresses churn faster than SG rules can safely be automated.
- Cloud firewall appliances (for example a Gateway Load Balancer fronting a next-generation firewall): the centralized chokepoint for anything crossing a VPC, account, or internet boundary that the layers above do not cover.
The layers compose rather than duplicate: SG and NACL narrow the network path, the host firewall narrows what a compromised host can originate even off that path, mesh mTLS narrows who is allowed to call what regardless of network path, and the appliance is the checkpoint at the actual trust boundary.
Policy Lifecycle: Authoring, Testing, Rollout, and Drift
Authoring. Define policy as code (Terraform for SG and NACL rules, an authorization-policy definition for the mesh), reviewed by pull request, mapped against a real service-dependency graph rather than a guess at who calls whom.
Testing. Apply a new rule in an audit or log-only mode first (most service meshes and next-generation firewalls support this) against real production-shaped traffic before it can deny anything, and canary a new deny rule against a single instance or a single availability zone before a fleet-wide rollout.
Rollout. Stage the rollout (canary, then a percentage, then everywhere) with an automated rollback if an error-rate or health-check service-level objective (SLO) regresses.
Rule drift and cross-account audit. Rules drift because an emergency exception, "just open this port for the incident", gets added and never removed. Three controls close this: policy-as-code as the only allowed change path, enforced by denying direct console-level security-group edits outside the CI role; a scheduled drift-detection job that diffs the live SG and NACL state against the git-declared state and pages on divergence; and a recurring cross-account audit (an AWS Config aggregator, or a scheduled reachability check across every account in the Organization) so an exception opened in one account does not silently persist for months unnoticed by anyone outside that account. Every manual exception should carry an automatic expiry.
Performance Impact
Security groups and NACLs are enforced in the hypervisor's fast path, so their overhead is effectively negligible. Host-based firewalls add a small, fixed per-packet cost that is usually dwarfed by TLS termination cost. A service mesh sidecar proxy is the layer that adds the most measurable overhead: an extra userspace hop per call, whose dominant cost is proxy processing and serialization rather than the mutual-TLS handshake itself, since sessions are reused across calls. Mitigate with sidecar resource sizing, connection pooling, and meshing only the boundaries that genuinely need identity-based policy rather than every internal call. Avoid asserting a specific millisecond number here; it depends on payload size, proxy implementation, and instance type, and should be validated with a load test against the real environment before it goes into an SLA (service-level agreement, a contractual performance commitment made to customers).
Migration Strategy from a Flat Network
- Map the real dependency graph first, from VPC Flow Log analysis or existing mesh telemetry. Do not guess the call graph.
- Turn on monitor-only (log, don't enforce) mode across every layer at once, to build confidence in the mapped graph without breaking production.
- Segment by blast-radius priority, not alphabetically: isolate the service handling the most sensitive data, in a fintech context typically anything touching payment or customer PII (personally identifiable information), first.
- Introduce one enforcement layer at a time in production, starting with SG default-deny (cheapest and most reversible), verify, then add mesh mTLS, then host firewalls, then the appliance.
- Keep a documented, time-boxed break-glass path, an emergency access rule that requires an approval workflow to enable, so a migration incident does not turn into "we opened the network to everything for a week and forgot."
Worked Example
A payments call graph: web calls orders-api, which calls ledger-service, which calls both the database and a separate fraud-service. Under the flat-network starting point, all four sit in one subnet with one broad security group. The migration puts each service behind its own security group referencing the caller's SG by ID (orders-api's SG allows inbound only from web's SG on its listening port; ledger-service's SG allows inbound only from orders-api's SG), adds a mesh identity policy so fraud-service only accepts calls whose mesh-issued certificate identifies them as ledger-service, and puts a GWLB-fronted firewall at the VPC boundary so nothing in this graph can be called from outside the VPC at all except through that one inspected path. Each of these was rolled out in log-only mode first against the real, already-flowing traffic, then flipped to enforce, tier by tier, over the migration.
Trade-offs and Pitfalls
Relying on security groups alone in an autoscaled or containerized environment, where pod IP addresses are ephemeral, forces a choice between rules too broad (allowing an entire subnet CIDR) or unmaintainable (constantly editing IP-based rules). Identity-based mesh policy is what actually solves this; IP-based rules alone do not.
Log-only mode that never gets flipped to enforce is a common failure. Without an owner and a deadline for the flip, monitor-only policy is a dashboard, not a control.
A service mesh adds real operational weight: a control plane, certificate rotation, sidecar upgrades, versus the near-zero maintenance of security groups and NACLs. Take it on when ephemeral-IP, identity-based segmentation is a genuine requirement, not because it is fashionable.
A centralized firewall appliance concentrates policy and inspection but also becomes a scaling and failure-domain dependency; reserve it for the actual trust-boundary chokepoints rather than east-west traffic the mesh already covers.
A customer requires that S3 buckets only be accessed from within their VPC via VPC Gateway Endpoints (no public internet access). Explain how you would configure the VPC endpoint and S3 bucket policies to enforce this, describe pitfalls such as cross-account access and console access, and state how you would verify enforcement. Provide policy-level guidance (no code required).
Sample Answer
Direct answer
Enforcing "S3 only from inside our VPC, never the public internet" is a two-sided control: a Gateway Endpoint routes the VPC's S3 traffic privately, and an S3 bucket policy with an explicit deny condition makes that the only way in, regardless of what any individual identity's permissions would otherwise allow. Neither side alone is sufficient: the endpoint alone just gives you a private path that requests can still bypass by going out over the internet with valid credentials, and the bucket policy alone has nothing to point at without the endpoint existing first.
Structured elaboration
Step 1: create the Gateway Endpoint and associate it with the private subnets' route tables. Once associated, requests to S3's published address ranges from resources in those subnets route through the endpoint rather than a NAT gateway (the device that swaps a private IP address for a public one to reach the internet) or the internet gateway.
Step 2: enforce it with a bucket policy condition, not just the endpoint's existence. The endpoint being present does not stop someone with valid AWS credentials and network access to the public internet from calling S3 from outside the VPC entirely (their laptop, a different account's compute, the AWS Command Line Interface (CLI) from home). The enforcement has to live on the bucket itself: an explicit Deny statement in the bucket policy that denies all S3 actions unless the request's aws:sourceVpce condition key matches your specific endpoint ID (or aws:sourceVpc if you want to allow any endpoint within a given VPC, not just one specific endpoint). An explicit deny in a bucket policy overrides any Allow granted elsewhere, including a user's own broad S3 permissions, which is exactly the property you need: no identity-based permission, however permissive, can route around this specific bucket's requirement.
Pitfall: cross-account access. If another AWS account legitimately needs access to this bucket (a common real case, not just an edge case to guard against), their requests will originate from their own VPC's endpoint, with a different endpoint ID and potentially a different VPC ID than yours. A condition scoped only to your specific aws:sourceVpce will silently break that legitimate cross-account access rather than erroring clearly, so the policy needs to explicitly enumerate every endpoint ID (or VPC ID) that is allowed to be the source, across every account that has a real need, and that list has to be kept current as accounts and endpoints change.
Pitfall: console access. The AWS Management Console itself does not originate S3 requests from inside your VPC. An administrator trying to browse the bucket's contents through the console UI is making the request from AWS's console infrastructure, not from an endpoint inside your VPC, so a policy that strictly requires aws:sourceVpce will also block console-based access to that bucket for everyone, including legitimate administrators, which is often the first thing that breaks and gets reported as "the policy doesn't work." If console access needs to remain possible, the policy has to carve out an explicit exception (for example, allowing a specific administrative role regardless of source, ideally still gated by other conditions like requiring multi-factor authentication) rather than trying to force all access through the VPC endpoint uniformly.
Verifying enforcement. Do not treat "the policy was written" as "the control works." Verify from at least three angles: (1) from inside the VPC, through the endpoint, confirm access still succeeds; (2) from outside the VPC entirely, using valid credentials that would otherwise have permission (a request from a personal machine, or from a different VPC without the endpoint condition satisfied), confirm access is now denied, and confirm the denial reason in the request's logs actually cites the condition, not some unrelated permission gap that would have denied it anyway; (3) review the account's access logs or the cloud provider's equivalent (for example, enabling S3 server access logging or the account-wide API activity log) specifically filtered to this bucket, looking for any successful request whose source does not match an expected endpoint, since a successful non-endpoint request is direct evidence the control has a gap somewhere.
Worked example
A healthcare analytics company stores de-identified patient datasets in an S3 bucket that must never be reachable except from the VPC running its analytics pipeline. The bucket policy denies all actions on the bucket and its objects unless the request's aws:sourceVpce matches the analytics VPC's specific S3 gateway endpoint ID, with one explicit exception: a narrowly scoped compliance-audit role, gated additionally by a requirement for multi-factor authentication, is allowed regardless of source, so an auditor can still review the bucket's configuration and access logs through the console without punching a hole in the control for everyone else. A second account belonging to a partner organization that legitimately needs read access to a specific prefix in the bucket is granted access through a second, explicitly listed endpoint condition referencing that partner account's own VPC endpoint, rather than being blocked outright or, worse, "solved" by weakening the policy to allow any source. Verification runs as a scheduled check: a script from outside the VPC attempts a read and confirms it is denied, a script from inside the VPC confirms it succeeds, and the bucket's access logs are reviewed weekly for any successful request whose source VPC endpoint is not on the explicitly approved list.
Trade-offs and pitfalls
The most common mistake is writing the deny condition, confirming that access from inside the VPC still works, and stopping there, without ever testing that access from outside the VPC is actually denied; a typo in the condition key or a policy Allow statement elsewhere in the account that unintentionally overrides the intended restriction (extremely rare for an explicit Deny, but a real risk if the policy is structured incorrectly) can leave the control silently ineffective while looking correct on the happy path. A second pitfall is treating this as a one-time setup: every new legitimate cross-account consumer, every new administrative tool that needs console-based access, and every account boundary change (an account merger, a new environment) needs the policy revisited, or legitimate access breaks in a way that gets "fixed" by loosening the condition rather than by adding the new source explicitly. Finally, remember what this control does and does not cover: it restricts the network path by which S3 can be reached, it does not replace identity-based least privilege; a compromised, over-permissioned role reachable from inside the approved VPC can still do damage to the bucket the policy does not prevent, since it satisfies the source condition legitimately.
Recommend an approach for organizing Security Groups at scale: per-application, per-tier, or per-environment. For each approach, explain pros/cons regarding manageability, least privilege, rule explosion, automation, and operational impacts in a large organization.
Sample Answer
Direct answer
Organize security groups (SGs) around the application tier they protect, not around the environment or the individual application, and layer host-based firewalls and a Web Application Firewall (WAF) in front of the public tier as defense in depth rather than relying on security groups alone; a per-tier scheme is the one that scales in a large organization without either drowning teams in rule sprawl or collapsing least privilege into one giant permissive group.
Structured elaboration
| Approach | Manageability | Least privilege | Rule explosion risk | Automation fit | Operational impact at scale |
|---|---|---|---|---|---|
| Per-application | Clear ownership, easy to reason about one app at a time | Strong, since each SG only allows what that specific app needs | High: hundreds of applications means hundreds of near-duplicate SGs, and shared infrastructure (a common database tier, a shared cache) ends up referenced by every one of them | Works well if SG creation is fully templated per app, but the sheer count strains tooling and review at scale | Every new application is a new SG to review and approve, and a security audit has to walk hundreds of near-identical rule sets one at a time to confirm none has quietly drifted |
| Per-tier (web, application, data, management) | Moderate: a smaller, stable number of SGs that map to a mental model everyone shares | Strong, if each tier's SG only allows the specific ports needed from the tier immediately in front of it | Low to moderate: the number of SGs stays roughly constant as the number of applications grows, since new apps reuse the existing tier structure | Best fit for automation: a new application in the "application tier" just gets attached to the existing application-tier SG rather than needing a bespoke one | Onboarding a new application is a one-line attach, not a new review, and a security audit only has to walk a handful of stable, well-understood rule sets regardless of how many applications sit behind them |
| Per-environment (dev, staging, prod) | Simple at a glance, but conflates unrelated concerns | Weak on its own: an environment-wide SG tends toward "anything in this environment can talk to anything else in this environment" | Low SG count, but each SG becomes broad and permissive, which is the opposite of least privilege | Easy to automate, but automating a bad access-control shape doesn't make it a good one | Low day-to-day overhead, but that low overhead is a symptom of the SG doing almost no real access control; any incident inside one environment has a wide blast radius across every application sharing that environment's SG |
Recommendation and reasoning. Per-tier is the right default at scale because it decouples the number of security groups from the number of applications, which is the dimension that actually grows uncontrolled in a large organization; per-application least-privilege intentions are good but the SG count grows linearly with every new service, and per-environment collapses tiers together in a way that violates least privilege as soon as you ask "should the web tier really be able to reach the data tier's admin port." In practice, combine per-tier SGs as the backbone (web, application, data, management/bastion) with narrow, per-application SGs layered on top only where a specific application genuinely needs an exception the tier-level rule doesn't cover, rather than choosing one pattern exclusively.
Host-based firewalls as defense in depth. Security groups operate at the ENI (elastic network interface) level and only see IP, port, and protocol; a host-based firewall (iptables/nftables on Linux, Windows Firewall) running on the instance itself adds a second, independent enforcement layer that still protects the instance even if a security group is ever misconfigured too permissively, and can enforce finer-grained policy (per-process rules, for instance) that security groups have no visibility into at all. This isn't redundant work, it's the standard defense-in-depth principle: two independent layers fail independently, so a single mistake in one layer doesn't fully expose the instance.
WAF in front of the public tier. For any tier actually facing the internet, a Web Application Firewall sitting in front of the load balancer adds a layer that security groups structurally cannot provide: security groups only see IP, port, and protocol, never the content of a request, while a WAF inspects HTTP-layer payloads for attack patterns (SQL injection, cross-site scripting) and can rate-limit or block based on request content and behavior. The public-tier security group still does its job (restricting which ports are even reachable), and the WAF does a job no security group configuration could ever do, inspecting what's actually inside the allowed traffic.
Worked example
A large organization with 200 microservices organizes security groups into four tier-level groups: sg-web (allows 443 inbound from the internet or from a WAF-fronted load balancer only), sg-app (allows the application port inbound only from sg-web, referenced by security group ID rather than a CIDR range), sg-data (allows the database port inbound only from sg-app), and sg-mgmt (allows administrative access only from the session-broker service, not from the internet). A new microservice being onboarded is simply attached to sg-app and, if it talks to the shared data tier, inherits access through sg-data's existing rule referencing sg-app, with no new security group needed for the common case; the team only creates a bespoke, narrower SG on top if this particular service needs an exception (say, a webhook receiver that needs a different inbound port than the rest of the application tier).
Trade-offs and pitfalls
The per-tier model's main risk is over-trusting the tier boundary itself: referencing sg-app as an allowed source in sg-data's rule is much better than allowing a broad CIDR range, but it still means every application in the tier can reach the data tier's port, so a compromised low-value service in the application tier has a path toward the data tier that a stricter, per-application rule would have prevented. The mitigation is exactly the "layer per-application exceptions on top" and "add host-based firewalls and, on the public edge, a WAF" guidance above: no single layer, including a well-organized security group scheme, is sufficient on its own at this scale.
Unlock Full Question Bank
Get access to all 16 Cloud Networking and VPC Design interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.