AWS Core Services and Architecture Questions

Work from the outside in: start with Application Load Balancer (ALB) metrics since they show whether the problem sits between the client and your service or inside it, then use AWS X-Ray to find which downstream call within the request is actually slow, then drop into logs at that specific span to find the root cause. Amazon CloudWatch instance-level metrics answer one narrow question: is the underlying compute starved of resources. The pattern that separates infrastructure from application problems is whether the slowness correlates with resource saturation (CPU, memory, disk queue) or with a specific downstream call showing up consistently in X-Ray traces.

Diagnostic order and what each layer tells you

1. ALB metrics first: TargetResponseTime (time the backend took to respond), HTTPCode_Target_5XX_Count, HealthyHostCount, and RequestCountPerTarget (average request load per target in the target group; ALB has no client-facing queue-depth metric the way Classic Load Balancers do). Rising TargetResponseTime with a steady HealthyHostCount points at the application; a rising RequestCountPerTarget alongside a dropping HealthyHostCount points at capacity, the same targets absorbing more load because fewer of them are healthy.
2. CloudWatch instance and service metrics: CPU utilization, memory (if the CloudWatch agent publishes it), disk queue length, and status check failures. Sustained CPU above roughly 70 to 80 percent, a growing disk queue, or a failed status check is a real infrastructure signal; normal-looking infrastructure metrics next to a slow ALB response time is a strong sign the problem lives in application code or a downstream dependency, not the host.
3. X-Ray traces: the service map and per-trace latency breakdown show which segment of the request is actually slow, an internal computation, a database call, or a third-party API. A long span on a downstream call, a slow SQL query, a saturated connection pool, is an application or dependency problem even though it shows up as backend latency at the ALB.
4. Logs: once X-Ray points at a specific segment, application logs, database slow-query logs, and container standard output around that timestamp and trace ID confirm the actual cause, a stack trace, a lock wait, a garbage-collection pause, or a connection pool timeout.

Patterns that separate infrastructure from application

• Infrastructure: CPU, memory, or disk saturation across many hosts, failing status checks, or RequestCountPerTarget rising alongside a drop in HealthyHostCount, all independent of what any individual trace shows.
• Application: normal host-level metrics, but TargetResponseTime and X-Ray both point at a specific span (a database call, a downstream API, a lock), and logs at that timestamp show exceptions, long garbage-collection pauses, or connection pool exhaustion.
• Mixed, and easy to misdiagnose: Auto Scaling Group cooldown delays, or a database hitting its own connection or throughput limit, look infrastructure-shaped on a dashboard but are actually caused by an upstream traffic pattern or an application-side connection leak.

Tying this to service-level objectives and alerting

Set alarm thresholds on the same metrics used above, tied to an actual service-level objective (SLO), for example 99.9 percent of requests under 500 milliseconds, rather than an arbitrary round number, so an alarm firing means the error budget (the allowed amount of failure or excess latency for the period) is actually at risk. Track the error budget's burn rate rather than paging on every threshold breach, since a two-minute blip that recovers on its own should not wake anyone up, while a sustained burn that will exhaust the monthly budget in a few hours should.

Worked example

Checkout latency creeps from a 99th-percentile of 200 milliseconds to 1.2 seconds over 20 minutes. ALB metrics show TargetResponseTime climbing while HealthyHostCount stays flat and RequestCountPerTarget stays flat too, ruling out a capacity problem. CloudWatch shows CPU at 25 percent across all targets, ruling out host saturation. X-Ray's service map shows the slow span is consistently a call to the inventory service, with its own latency climbing in the trace timeline. Logs on the inventory service around that window show a specific slow-query pattern: a full table scan on an unindexed column that only becomes slow once the table crossed a size threshold. That is a database and application problem end to end, not infrastructure, and the fix, adding an index, has nothing to do with scaling anything.

Trade-offs and pitfalls

• Jumping straight to logs before narrowing scope with ALB and CloudWatch metrics wastes time searching a haystack. The metrics exist specifically to tell you where to look before you start reading logs.
• X-Ray requires sampling and instrumentation to already be in place before the incident. If tracing is not enabled, or the sampling rate is too low to capture the slow requests, you lose the fastest path to the root cause and fall back to correlating logs by timestamp, which is slower and noisier.
• Alarming on raw metric thresholds instead of an SLO-tied burn rate produces either too many pages (threshold too tight) or missed real degradations (threshold too loose). Revisit thresholds against actual historical percentiles, not a guess.
• A downstream dependency showing up as "your" latency in X-Ray is still your incident to manage even though the root cause sits in a service you do not own. Know the escalation path to that team before you need it.

Direct answer
Operational Excellence means running operations with the same engineering discipline you would apply to the system itself, so operational quality improves continuously instead of depending on any one person's vigilance. AWS's current Well-Architected Framework expresses that idea as eight design principles, listed just below. A real interview conversation rewards going deep on one principle rather than spreading equally thin across all eight, so this answer carries "make frequent, small, reversible changes" all the way through people, process, and automation.

The eight Operational Excellence design principles

• Organize teams around business outcomes: the operating model, ownership, and incentives are structured around the business results a team is accountable for, not around technology silos.
• Implement observability for actionable insights: instrument workloads for performance, reliability, cost, and health, and use that telemetry to act, not just to populate a dashboard.
• Safely automate where possible: define workloads and their operations as code and automate them in response to events, with guardrails such as rate limits, error thresholds, and approvals bounding what the automation can do (this is where the older "perform operations as code" principle now lives).
• Make frequent, small, reversible changes: ship changes at a size and cadence where, if something breaks, reverting is a viable first response instead of a multi-hour rollback exercise. This is the principle this answer goes deep on below.
• Refine operations procedures frequently: treat runbooks as living documents, updated from every incident, not written once at launch.
• Anticipate failure: design and test for component failure before it happens, via chaos testing or game days, instead of discovering failure modes in production.
• Learn from all operational events and metrics: every operational event, not only incidents, feeds back into architecture and process decisions.
• Use managed services: prefer AWS-managed services to reduce the undifferentiated operational burden of running the underlying infrastructure yourself.

(Earlier versions of the framework listed five; AWS has since split and added principles, most visibly folding the older "perform operations as code" principle into the broader "safely automate where possible.")

Structured elaboration: make frequent, small, reversible changes, from people to automation

• Why it's a design principle, not just a deploy-cadence preference: a large, infrequent change bundles many independent risks into one deploy, so when something breaks, both the blast radius and the diagnosis space are large. A small, frequent, reversible change isolates risk to roughly one thing at a time and makes "revert" a viable first response instead of a multi-hour rollback exercise.
• People: engineers own shipping the smallest change that delivers value, not the largest change that's convenient to bundle; release ownership sits with the team making the change, not a separate release-management function, so the person best positioned to revert quickly is the person who shipped it.
• Process: every change ships behind a feature flag or a canary release policy (shift a small percentage of traffic, watch error-rate and latency service level objectives (SLOs) for a fixed window, then proceed or auto-rollback); change-approval overhead scales with blast radius, a config-flag flip needs less ceremony than a schema migration, rather than one uniform change-advisory-board gate for everything.
• Automation: a CI/CD pipeline runs the canary policy automatically; an automated rollback triggers off a monitoring alarm crossing the SLO threshold during the canary window; infrastructure changes are expressed as versioned infrastructure as code, so "revert" means "redeploy the previous known-good version," not "someone remembers what they clicked."

Worked example
A team changes how a checkout service calculates shipping cost. Old way: one large release bundling the new calculation, a UI change, and a database migration, deployed to all traffic at once; when shipping costs come out wrong for a subset of orders, working out which of the three bundled changes caused it, and rolling all three back, takes hours. New way, applying the principle: the database migration ships first and alone, with no user-visible change; the calculation change ships next behind a feature flag, to a small percentage of traffic for a fixed window while a monitoring alarm watches the "shipping cost equals expected value" business metric; only after that canary is clean does it roll out fully; the UI change ships last, independently. If the calculation canary trips the alarm, the automated rollback disables the flag for everyone within the alarm's evaluation window, isolating the failure to exactly the one change that caused it.

Trade-offs & pitfalls

• Small, frequent changes require release infrastructure (feature flagging, canary tooling, fast rollback) up front; a team without that tooling feels the "frequent" part as pure overhead until it's built.
• "Reversible" isn't automatic for schema changes or data migrations: design the migration to be additive and backward-compatible in the same change, so the earlier release remains loadable, rather than assuming reversibility comes for free.
• Not every change decomposes cleanly: an atomic external-contract change, like a hard-cutover API version deprecation, doesn't split into small reversible steps, and the principle's job there is to minimize how often such changes happen, not force-fit them into a canary flow that doesn't fit.

Direct answer

Split by traffic shape: put Amazon CloudFront in front of everything, serve the 80% static slice straight from an S3 origin with long cache lifetimes, and route the 20% dynamic slice through CloudFront to an Application Load Balancer (ALB) in front of Amazon Elastic Container Service (ECS) on Fargate, backed by ElastiCache for Redis for hot reads and Amazon Aurora for durable relational data. CloudFront and S3 handle the volume cheaply; the compute and database layers only need to be sized for the dynamic fraction.

Sizing the traffic

200,000 requests per minute is $200{,}000 / 60 \approx 3{,}333\text{ RPS}$. At an 80/20 static/dynamic split:

$\text{static RPS} = 3{,}333 \times 0.8 \approx 2{,}667 \qquad \text{dynamic RPS} = 3{,}333 \times 0.2 \approx 667$

That reframes the problem: the CDN and object store need to comfortably absorb ~2,667 RPS of cacheable traffic (well within normal CloudFront/S3 capability), and the compute/database tier only needs to be provisioned for ~667 RPS of application logic, roughly a quarter of the naive full-traffic number.

Architecture

flowchart LR
    Client["Clients"]
    CF["CloudFront\n(cache + WAF)"]
    S3["S3\n(static origin)"]
    ALB["ALB"]
    ECS["ECS on Fargate\n(API service)"]
    Redis["ElastiCache\nfor Redis"]
    Aurora["Aurora primary"]
    Replica["Aurora read replica"]

    Client --> CF
    CF -->|"cache miss, static path"| S3
    CF -->|"dynamic path"| ALB --> ECS
    ECS --> Redis
    ECS --> Aurora
    ECS --> Replica

Layer-by-layer choices

Worked example: cache hit rate and origin load

If the CDN cache hit rate on the static path is 95%, only 5% of the 2,667 static RPS reaches S3 as an actual origin fetch:

$2{,}667 \times 0.05 \approx 133\text{ RPS to the S3 origin}$

That is comfortably inside S3's automatic per-prefix scaling behavior (AWS documents at least 3,500 PUT/COPY/POST/DELETE and 5,500 GET/HEAD requests per second per partitioned prefix, scaling further as request rate sustains), so no special prefix-sharding is needed at this traffic level. If the hit rate instead dropped to 70% due to poor cache-control headers, origin load would jump to about 800 RPS, still fine for S3 alone but a sign the CDN configuration, not the storage layer, is the thing to fix first.

Trade-offs and pitfalls

• The static/dynamic split is the whole design. Treating all 3,333 RPS as needing the same compute/database investment massively over-provisions the expensive tier; the entire cost and latency win comes from recognizing 80% of the load never needs to reach an application server.
• Cache invalidation for near-static content (a product page that changes occasionally) needs a deliberate TTL and invalidation strategy; too aggressive a TTL serves stale data, too short a TTL erodes the origin-offload benefit that makes this design cheap.
• Aurora Serverless v2 is a reasonable alternative to fixed-size provisioned instances if the dynamic 667 RPS is spiky rather than steady, at some cost premium versus right-sized provisioned capacity with autoscaling read replicas at steady state.
• Fargate versus EC2 ASG for the dynamic tier is an operational-capacity trade-off, not a correctness one: Fargate removes host management at a higher per-vCPU cost; an ASG is cheaper at sustained high utilization if the team already carries EC2 operational overhead.

A hub-and-spoke security visibility architecture designates one AWS account, the hub (often called the security or log archive account), as the delegated administrator for Amazon GuardDuty and AWS Security Hub, and as the destination for every other account's (the spokes') AWS CloudTrail logs, so one team gets a unified view without operating separately in every account. On a brand-new account, before anything else, the three services to enable first are CloudTrail (so an audit trail exists from minute one), GuardDuty (so threat detection starts baselining normal behavior from minute one), and IAM Access Analyzer (so accidental external access grants are caught immediately), because everything else in a security program depends on those three signals already flowing.

Architecture

flowchart TB
    subgraph Org["AWS Organizations"]
        Mgmt["Management account"]
    end
    subgraph Spokes["Spoke / member accounts"]
        SpokeA["Account A workloads"]
        SpokeB["Account B workloads"]
        SpokeC["Account C workloads"]
    end
    subgraph Hub["Hub / security account"]
        GD["GuardDuty\ndelegated administrator"]
        SH["Security Hub\ndelegated administrator"]
        S3["Central S3 log bucket\nSSE-KMS, Object Lock"]
        EB["EventBridge"]
        SIEM["SIEM / ticketing"]
    end
    Mgmt -- designates --> GD
    Mgmt -- designates --> SH
    SpokeA -- CloudTrail org trail --> S3
    SpokeB -- CloudTrail org trail --> S3
    SpokeC -- CloudTrail org trail --> S3
    SpokeA -- findings --> GD
    SpokeB -- findings --> GD
    SpokeC -- findings --> GD
    GD -- findings --> SH
    S3 -- queried by --> SH
    SH -- normalized findings --> EB
    EB -- routes --> SIEM

Why hub-and-spoke, mechanically

• CloudTrail: enable a single AWS Organizations trail from the management account (or a delegated administrator), which automatically applies to every current and future member account and delivers all logs to one Amazon Simple Storage Service (S3) bucket in the hub account. This removes the "did someone forget to enable logging in the new account" failure mode entirely.
• GuardDuty: the management account designates one member account as the GuardDuty delegated administrator. That account can then auto-enable GuardDuty for every existing and future member account, and every member's findings roll up into the delegated administrator's view. GuardDuty is a regional service, so the delegated administrator must be the same account in every Region you operate in, not just one.
• Security Hub: designate a delegated administrator for Security Hub too, and use its central configuration feature to push standards and controls, for example the AWS Foundational Security Best Practices standard, to every account and organizational unit from one place, instead of configuring each account separately. Security Hub also ingests and normalizes GuardDuty findings alongside its own checks, so the hub ends up with one unified findings view rather than two dashboards.
• EventBridge and downstream routing: Security Hub findings can trigger Amazon EventBridge rules in the hub account to notify a security information and event management (SIEM) tool, open a ticket, or, for well-understood finding types, run an automated remediation through AWS Systems Manager Automation.

The first three services on a brand-new account, and why that order

1. CloudTrail: with no audit trail, nothing else is investigatable after the fact. This has to exist before any other service produces something worth investigating.
2. GuardDuty: a passive, low-effort detector with no infrastructure to run, that starts baselining normal network and API behavior immediately. The earlier it is on, the sooner its anomaly detection becomes reliable.
3. IAM Access Analyzer: catches the most common real-world incident category, a storage bucket, IAM role, or encryption key policy that unintentionally grants access to an external account or the public, through static policy analysis rather than waiting to observe an actual attack.

Worked example

A company with a management account and fifteen workload accounts under one AWS Organization stands up a dedicated security account as the hub. Step one: enable an Organizations CloudTrail trail from the management account, delivering to a central S3 bucket in the security account, with S3 Object Lock in governance mode and server-side encryption using a customer-managed AWS Key Management Service (KMS) key that only the security account administers, so even a compromised workload account cannot delete its own audit history. Step two: designate the security account as the GuardDuty delegated administrator and set auto-enable to cover all accounts, so all fifteen existing accounts and any future account get GuardDuty on day one. Step three: designate the same security account as the Security Hub delegated administrator, turn on central configuration, and push the AWS Foundational Security Best Practices standard to every account from one configuration policy. Within a day, the security account's Security Hub console shows a single, deduplicated findings list spanning all fifteen accounts instead of fifteen separate consoles nobody has time to check individually.

Trade-offs and pitfalls

• Using one account as the delegated administrator for both GuardDuty and Security Hub is the common pattern, but it concentrates real blast radius in that account. Protect it with its own strict IAM boundary and multi-factor authentication, since it can see, and for some findings act, across the entire Organization.
• GuardDuty's regional nature is easy to get wrong: designating the delegated administrator in one Region does not cover another Region automatically, and AWS requires the same account to be the delegated administrator everywhere GuardDuty is enabled.
• Security Hub's central configuration model requires choosing, per account or organizational unit, whether it is centrally managed (only the delegated administrator can change its settings) or self-managed (the account manages its own). Mixing these without a clear policy leads to accounts drifting out of the baseline.
• A single central logging bucket is simple to reason about but becomes a single point of both cost and access-control complexity at scale. Partition by account and date, and separate read access for the security team from the write access granted to the log delivery role.
• Enabling every possible security service on day one sounds appealing but produces alert fatigue before anyone has tuned suppression rules. The three named above are the minimum viable baseline, not the complete target state.

A layered AWS network security posture applies controls at increasingly broad scopes: Security Groups filter traffic per resource, Network Access Control Lists (NACLs) add a coarse subnet-level backstop, AWS Web Application Firewall (WAF) filters application-layer requests at the edge, AWS Shield absorbs Distributed Denial of Service (DDoS) attacks, and AWS Firewall Manager centrally enforces all of the above across every account in an AWS Organization. No single control is sufficient alone: Security Groups stop most unwanted traffic, but WAF and Shield exist specifically because Security Groups cannot see inside a Hypertext Transfer Protocol (HTTP) request or absorb a flood, and Firewall Manager exists because per-account discipline stops scaling past a handful of accounts.

Where each control belongs

flowchart TB
    Internet["Internet traffic"] --> Shield["AWS Shield Standard/Advanced\nedge DDoS absorption"]
    Shield --> Edge["CloudFront / Application Load Balancer"]
    Edge --> WAF["AWS WAF\napplication-layer filtering, rate-based rules"]
    WAF --> NACL["Subnet NACL\nstateless, coarse allow/deny"]
    NACL --> SG["Security Group\nstateful, per-resource least privilege"]
    SG --> App["Application tier: EC2 / ECS / Lambda"]
    App --> DBSG["Database-tier Security Group"]
    DBSG --> DB["RDS / database"]
    FM["AWS Firewall Manager"] -.enforces policy on.-> WAF
    FM -.enforces policy on.-> Shield
    FM -.enforces policy on.-> SG
    Org["AWS Organizations + Service Control Policies"] -.governs.-> FM

Control-by-control role

• Security Groups (host/resource): a stateful firewall attached to Elastic Network Interfaces (ENIs), Amazon Elastic Compute Cloud (EC2) instances, and load balancers. Reference other security groups by ID instead of Classless Inter-Domain Routing (CIDR) ranges wherever possible, so a rule survives an IP address change.
• NACLs (subnet): stateless, evaluated in rule-number order, and applied to everything in the subnet. Use them as a blunt secondary backstop, for example blocking a known-bad CIDR range, rather than the primary control, since being stateless means you must explicitly allow the return traffic too.
• AWS WAF (application layer): sits in front of CloudFront or an Application Load Balancer and inspects the actual HTTP request. Use managed rule groups for common attack patterns plus rate-based rules to throttle a client hammering an endpoint. This layer also matters for serverless front doors: an API Gateway or Lambda function URL behind WAF gets the same rate-based throttling against a flood of malformed or maliciously injected events trying to drive up invocation cost, protection a Security Group cannot provide since Lambda has no traditional network ingress to filter.
• AWS Shield: Shield Standard is free and automatically protects every CloudFront distribution and Application Load Balancer against common network and transport-layer DDoS traffic. Shield Advanced adds continuous access to the DDoS Response Team, cost protection against scaling charges caused by an attack, and broader detection, worth the added cost for internet-facing production endpoints where downtime or an attack-driven bill spike is unacceptable.
• AWS Firewall Manager: centralizes WAF rule sets, Shield Advanced protections, security group policies, and DNS Firewall rules across every account in an AWS Organization from one security admin account, so a new account automatically inherits the baseline instead of relying on someone to configure it by hand.

Common misconfigurations

• Security Groups open to the entire internet on administrative or database ports, usually left over from initial setup and never tightened.
• Treating the NACL as the primary control: because NACLs are stateless, forgetting the ephemeral-port return-traffic rule silently breaks legitimate connections, and teams then "fix" it by opening the NACL wide, defeating its purpose.
• Applying WAF to the Application Load Balancer but exposing a second entry point, like a CloudFront distribution serving a static site or a direct storage-service website endpoint, that bypasses WAF entirely.
• Enabling Shield Advanced generally but not on the specific resources that need it, then discovering during an attack that the DDoS Response Team engagement and cost protection only apply to protected resources.
• Rolling out Firewall Manager policies without first mapping which accounts and organizational units are in scope, which either misses new accounts or breaks a legitimate account that needed an exception.

Worked example

A public checkout API sits behind CloudFront and an Application Load Balancer, backed by ECS tasks and an RDS database, across twenty AWS accounts in one Organization. Shield Standard covers CloudFront and the load balancer automatically. Shield Advanced is added because a DDoS-driven CloudFront bill spike during a past sale directly cost the business money. WAF sits on the CloudFront distribution with a managed rule group plus a rate-based rule capping any single IP address at a fixed request rate over a five-minute window. The load balancer's security group only allows inbound traffic from CloudFront's managed prefix list, not the open internet. The ECS task security group only allows inbound traffic from the load balancer's security group by reference, and the RDS security group only allows inbound traffic from the ECS task security group. Firewall Manager, run from the security account, pushes the same WAF rule set and Shield Advanced protection to every account tagged "production" in the Organization, so the twentieth account gets the same baseline as the first without a manual step.

Trade-offs and pitfalls

• Defense in depth adds real operational complexity: every layer is one more place a legitimate request can be blocked and one more thing to debug. Automate the rollout, through Firewall Manager, CloudFormation StackSets, or an infrastructure-as-code module, rather than hand-configuring each account, or the layers drift out of sync.
• WAF on CloudFront gives global, edge-level protection and lower origin latency; WAF on the Application Load Balancer is regional and sees traffic after it has already passed the content delivery network. Decide deliberately rather than bolting WAF onto whichever resource is convenient.
• Shield Advanced's cost protection only applies to resources it is actively protecting, so scoping it correctly matters as much as enabling it.
• Centralizing enforcement through Firewall Manager and Organizations Service Control Policies is powerful, but an overly broad policy can block legitimate exceptions, like a development account intentionally testing without WAF. Pair centralization with a documented, auditable exception process.