Amazon Senior Network Engineer Interview Preparation Guide

Answer (concise diagnosis and remediation)

Problem — why hosts in VLAN 30 can’t talk across switches

• The trunk Gi1/0/1 does not allow VLAN 30 (allowed list 1-100,200 actually includes 30? — but given output shows Gi1/0/1 allowed 1-100,200 and Gi1/0/2 allowed 1-20,30,40; however hosts and MACs are on Gi1/0/3 and Gi2/0/4 and MACs are learned on access ports but traffic won’t traverse because one trunk between the switches is pruning/filtered for VLAN 30). Most likely cause: VLAN 30 is not being carried across the inter-switch trunk due to an allowed-vlan mismatch or VLAN pruning on the trunk (Gi1/0/1’s effective allowed list or a pruned state excludes 30).

Safe remediation plan (steps to fix and validate)

1. Confirm current trunk allowed VLANs and VLAN status:

show interfaces trunk
show running-config interface Gi1/0/1
show vlan id 30

2. If Gi1/0/1 truly lacks VLAN 30 in its allowed list, add it (non-disruptive — does not shut the trunk):

configure terminal
interface Gi1/0/1
 switchport trunk allowed vlan add 30
end

3. Verify:

show interfaces trunk
show mac address-table vlan 30
ping from a host in VLAN 30 to a remote host in VLAN 30

4. Rollback/cleanup: if you used "add" it’s safe; to revert:

configure terminal
interface Gi1/0/1
 switchport trunk allowed vlan remove 30
end

Notes and cautions

• Check native VLAN consistency (both trunks show native vlan 1 here so OK).
• If VTP or dynamic pruning (DTP/auto) is in use, validate VTP mode and prune settings.
• Make changes during a maintenance window if this trunk carries many VLANs—though adding a VLAN to allowed is low risk.
• After fix, confirm MACs move to the remote trunk and hosts can ping.

Overview / short comparison
SNMP polling is pull-based, low-bandwidth but high-latency for many metrics and relies on MIBs; streaming telemetry (gNMI/gRPC + OpenConfig/YANG) is push-based, efficient, supports structured data and near-real-time insight.

Operational benefits (10,000 devices)

• Lower polling overhead and reduced SNMP poll storm risk.
• Near real-time alerts and finer fidelity (event-driven, sampled, or periodic streams).
• Standardized, vendor-neutral models (OpenConfig) improve cross-vendor correlation and automation.
• Better scalability: collectors ingest streams rather than polling each device on schedule.

Tooling & collector changes

• Replace/augment SNMP NMS with gNMI-capable collectors (e.g., Telegraf, Prometheus remote write adapters, OpenConfig-capable collectors like OpenTelemetry agents, gNMI-gRPC proxies).
• Implement message bus (Kafka) for scale, buffering, and replay.
• ETL/normalization layer to map vendor YANG to canonical OpenConfig schema.
• Migration scripts, config management (Ansible/Crosswork) to enable telemetry on devices.

Vendor compatibility risks

• Older devices may lack gNMI or OpenConfig support; require software upgrades, replacements, or SNMP-to-gNMI gateways.
• Partial YANG coverage across vendors necessitates per-vendor mapping and testing.
• Feature parity gaps (some telemetry counters/configs only available via vendor-specific models).

Security implications

• gRPC with TLS + mutual authentication recommended (mTLS).
• Proper cert lifecycle, PKI, and key management at scale.
• Secure collectors (RBAC, hardened endpoints), network-level isolation (VPNs or dedicated management fabric).
• Monitor for data exfiltration; apply rate limits and encryption in transit and at rest.

Observability / telemetry model improvements

• Strongly-typed, hierarchical data (YANG/OpenConfig) enables consistent dashboards, deterministic alerts, and automated remediation.
• Metadata (timestamps, sequence numbers, sensor paths) improves correlation and root-cause analysis.
• Ability to subscribe to state changes reduces blind spots versus periodic SNMP polls.

Migration approach (high level)

• Inventory device capabilities → pilot 100 devices across vendors → deploy collectors and Kafka → run gNMI in parallel with SNMP for 3 months → validate parity, automate onboarding → roll out in waves, retire SNMP where appropriate.

Approach summary
Quantify debt by classifying items, measuring technical severity and probability, translating to customer impact (availability, performance, security), then scoring and prioritizing into a remediation backlog with estimated effort and ROI.

Categories & metrics

• Outdated device OS / unsupported firmware
  • Metrics: % devices on unsupported versions, CVE exposure count, time-since-EOL.
  • Customer risk: map CVE severity + exploitability to outage probability; estimate cost = expected downtime hours * $/hour SLA penalty or business loss.
• Manual runbooks / operational toil
  • Metrics: time-to-recover (MTTR) with manual steps, number of manual steps, frequency of incidents requiring runbook.
  • Customer risk: model additional MTTR * incident frequency -> added downtime cost and error-rate cost.
• Shadow configs / undocumented changes
  • Metrics: config drift %, undocumented changes per month, number of services affected.
  • Customer risk: probability of misconfiguration-induced outage * outage cost.

Scoring & translation

• Compute normalized scores and a risk score:

Risk Score = Likelihood (0-10) * Impact (0-10)

• Estimate expected cost:

Expected Cost = Risk Score/100 * Max Outage Cost

• Prioritize by (Expected Cost / Remediation Effort hours) — highest ROI first.

Backlog & governance

• For each item: title, risk score, expected cost, remediation steps, effort estimate, owner, SLA for fix.
• Track reduction in risk and re-evaluate quarterly.

This yields a data-driven prioritized plan linking technical fixes to business value.

Situation / Task
Three months ago I led a cross-functional response when our edge routers lost reachability to multiple ISPs and an IXP, causing service degradation for ~30% of customers over ~48 hours. My goal: restore traffic quickly, stabilize routing, and prevent recurrence.

Timeline & Actions

• Hour 0–2: Declared Sev1, assembled network, ops, cloud, and customer success leads. Opened incident bridge and Slack channel; assigned scribe.
• Hour 2–8: Triage — I ran BGP dump and traceroute comparison, confirmed prefix withdrawals and inconsistent AS_PATHs. Directed temporary mitigations: prepended routes to a secondary transit, activated emergency anycast announce for critical prefixes, and applied inbound prefix-lists to block flapping routes.
• Day 1: Escalated to ISP contacts and IXP NOC; coordinated synchronized counter-announcements and community tag changes. Shared packet captures and BGP logs. Kept customers updated with hourly status and a technical incident summary.
• Day 2: ISPs identified a misconfigured route-reflector cluster causing route leaks between ASes. We tested a configuration rollback in maintenance window, validated convergence, then fully restored normal announcements.

Root Cause & RCA
Root cause: improperly staged RR config pushed by a peer ISP that removed prefix filtering, causing route leaks and path oscillation. Contributing factors: insufficient eBGP session monitoring and no automated sanity checks for sudden AS_PATH changes.

Post-Incident Changes

• Implemented automated BGP anomaly detection (AS_PATH anomalies, withdrawal spikes) with pager alerts.
• Added stricter prefix- and AS-path filters and route-object validation from IRR/RPKI.
• Introduced cross-org runbooks for multi-ISP incidents and scheduled quarterly joint drills with key ISPs/IXPs.
• Upgraded customer comms process: templated status pages and SLA-impact indicators.

Result & Learnings
Services fully restored within 48 hours; customer SLA credits minimized through proactive communication. I learned the value of rapid ISP coordination, defensible temporary mitigations, and investing early in automated routing sanity checks.

Direct answer

I'd treat this as a portfolio of tactics, not one big fix: content delivery network (CDN, a network of edge servers that caches content closer to users) caching, compression, and delta-based transfer for large recurring payloads together plausibly reach the 30-50% target, and the business case is strong because the required engineering investment is small relative to a $2 million annual bill, typically paying back in a handful of months even under a pessimistic estimate of the savings.

Structured elaboration

The tactics, roughly ordered by effort-to-impact ratio:

• CDN adoption and caching. Serve static and cacheable content from edge locations instead of the origin. Impact scales with what fraction of total bytes are cacheable and how well cache headers are tuned; typically the single largest lever for a content- or API-heavy service.
• Compression. Enable modern compression (Brotli or gzip for text and JSON, modern image codecs like WebP or AVIF for images) on anything not already compressed. Cheap to implement, moderate impact, and the main cost is a small increase in origin CPU load.
• Delta or diff-based transfer. For large payloads that change incrementally between requests (client sync flows, binary updates), send only the changed portion instead of the full object. High impact where applicable, but only applies to specific traffic patterns, not general-purpose traffic.
• Direct network peering or a committed connectivity arrangement. For high-volume, predictable flows (bulk data transfer to a known partner, or a specific customer with heavy traffic), a direct connection or negotiated committed-volume pricing can cut the per-gigabyte rate materially, though it requires predictable volume to justify the setup cost.
• Reducing cross-region and cross-cloud replication for hot paths. Prefer region-local processing and asynchronous, lower-frequency replication over synchronous cross-region traffic on the hot path, since egress between regions or providers is often the most expensive category per gigabyte.

Modeling the ROI and payback:

1. Classify current traffic by content type and destination to establish a baseline: how many bytes, at what cost, going where.
2. Estimate each tactic's savings as a percentage of the baseline it actually applies to (compression only affects compressible content, delta transfer only affects flows with incremental change), not as a percentage of total spend.
3. Sum the tactics' savings for a total expected reduction.
4. Estimate the engineering cost as engineer-months at a fully-loaded cost rate.
5. Payback period is the engineering investment divided by the annual savings, expressed in months.

Worked example

Baseline: $2,000,000 a year in egress. A conservative, tactic-by-tactic estimate: CDN and caching cuts 20% of total egress ($400,000), compression cuts 8% ($160,000), and delta transfer on the highest-volume sync flows cuts 12% ($240,000), for a combined 40% reduction, $800,000 a year in savings.

Engineering cost: a first phase (CDN rollout and enabling compression) at 2 engineers for 2 months is 4 engineer-months, and a second phase (delta transfer implementation and peering setup) at 3 engineers for 4 months is 12 engineer-months, for 16 engineer-months total. At a fully-loaded cost of $15,000 per engineer-month, that's $240,000 in engineering investment.

Sensitivity: if the tactics only achieve the low end of the target range, 30% instead of 40% ($600,000 a year), payback stretches to 4.8 months. At the high end, 50% ($1,000,000 a year), payback shortens to about 2.9 months. Even the conservative end of the range clears payback well inside a year, which is the strength of this specific business case: the engineering cost is small and mostly one-time, while the savings are large and recur every year afterward.

Trade-offs and pitfalls

• Estimating a tactic's savings as a percentage of total egress instead of the fraction of traffic it actually applies to is the most common way this kind of estimate goes wrong; compression does nothing for already-compressed video, and delta transfer does nothing for traffic that doesn't have a stable base to diff against.
• CDN caching introduces its own operational cost: cache invalidation complexity and the risk of serving stale content if cache lifetimes (time-to-live, TTL) aren't tuned correctly. That's a real, ongoing cost, not just a one-time setup cost.
• Compression trades bandwidth for CPU time; if origin servers are already CPU-constrained, the "savings" partially show up as a new compute cost instead of a pure win, and that trade-off needs to be measured, not assumed away.
• Multi-cloud or multi-region replication done for redundancy can quietly increase egress if it's not designed with cost in mind; prefer region-local processing and asynchronous, lower-frequency replication over synchronous cross-region traffic wherever the workload can tolerate it.
• The engineer-month cost estimate is only as good as the fully-loaded rate used; using a rate that's too low (ignoring benefits, overhead, and management time) makes every proposed initiative look more attractive than it actually is.

Direct answer
A mentoring relationship going badly is rarely one big failure; it's usually a slow accumulation of choices, like taking on too much of the work yourself to protect the outcome, that quietly undercut the mentee's growth. The honest answer names a specific relationship, is candid about what you did (not just what the mentee did), and shows what changed in how you mentor afterward.

What "went badly" usually looks like

• Common patterns: being too directive and doing the hard parts yourself to protect delivery; giving feedback too infrequently or too late to be actionable; misjudging the mentee's actual gap (treating a confidence problem as a skill problem, or the reverse); or disengaging when the relationship got effortful.
• A strong answer picks one specific pattern and owns your part in it, rather than a vague "they weren't a good fit."

What separates a senior answer from a junior one

• Junior answers blame the mentee ("they just weren't receptive") or stay abstract ("communication could have been better"). Senior answers identify a decision you made and trace its actual effect: what you did, what it produced, and why it made sense to you at the time even though it was wrong.
• Senior answers also show what changed structurally afterward, not just an apology or a resolution to "communicate better." Concrete changes: an explicit mentoring agreement up front, checkpoints instead of open-ended availability, deliberately handing over ownership even when it's slower.

How to close it out

• End on what you'd do differently now, stated specifically enough that it's clear you'd actually behave differently in the next relationship, not just that you feel bad about the last one.

Worked example
During a stretch project with a hard deadline, I mentored a junior engineer by taking over the riskiest parts myself rather than coaching them through it, to keep the timeline safe. That worked in the short term, but it meant they never built confidence handling ambiguity or incidents on their own, and toward the end of the project they told me directly that they felt sidelined rather than developed. That was the moment it became clear the relationship hadn't done what I'd intended, even though the project itself shipped fine.

What I changed afterward: instead of stepping in when something got risky, I started requiring myself to narrate my reasoning out loud and have the mentee drive, only taking over if there was a genuine, immediate risk. I also set an explicit checkpoint (a short regular sync, not just "come find me") so growth stalls would surface early instead of only becoming visible at the end of a project. The relationship after that wasn't measured by how smoothly the project went; it was measured by whether the mentee could handle the next similar situation without me in the room, which is a slower thing to build but the actual point of mentoring.

Trade-offs and pitfalls

• The tempting failure mode is optimizing for the deliverable (visible and rewarded) at the expense of the mentee's growth (slower and less visible), especially under deadline pressure.
• Being self-critical is necessary but insufficient; an answer that's all remorse with no concrete process change reads as unreflective in a different way.
• Watch for over-correcting into never stepping in, which just replaces one failure mode (too directive) with another (abandoning someone to a mistake they can't yet recover from alone).

Clarify requirements

• 10k RPS baseline, 99.99% SLA, withstand up to 400 Gbps attack, global audience → multi-region, anycast + scrubbing hybrid.

High-level components

• Edge: Anycasted DDoS-capable edge routers (BGP with remote-triggered blackhole (RTBH) and flow-spec support).
• Global traffic steering: Anycast DNS + Geo-aware load balancers (GSLB) to direct to nearest PoP.
• Scrubbing: Regional scrubbing centers (cloud provider + third-party scrubbing partners) sized to absorb 400 Gbps aggregate; auto-scaleable scrubbing in cloud for spikes.
• LB & app-plane: Regional LBs (L3/L4) feeding regional WAFs (perimeter WAF + app WAF) then app servers behind internal LBs.
• Control plane: Central orchestration for BGP announcements, route filtering, and flow-spec policies.
• Observability: Netflow/IPFIX, BGP monitoring, synthetic health checks, latency telemetry, SIEM.

Traffic steering & mitigation

• Default: Anycast to nearest PoP for normal traffic.
• Detect: Anomaly detection triggers diversion to scrubbing (BGP community or AS-path prepend to steer to scrubbing AS).
• Active steering: Use BGP community tags and preconfigured upstream peers to redirect specific prefixes to scrubbing centers; fallback to RTBH for volumetric scrub.
• Selective steering: Flow-spec or inline scrubbing for targeted 5‑tuple filtering to avoid full reroute.

Health checks & failover

• Multi-level: TCP/HTTP synthetic checks, active probes from multiple regions, LB health checks (fast-failover ~5s), and application heartbeats.
• Failover: GSLB route removed if region fails health checks; BGP withdraw + GSLB fallback to next region.

Latency & placement

• Place PoPs near user population; keep scrubbing path optimized to avoid extra RTT for legitimate traffic (only steer suspicious prefixes).
• Use inline WAF at PoP for low-latency filtering; deep inspection in scrubbing centers.

Cost vs. resilience trade-offs

• Full-cloud scrubbing autoscale: lower capital, higher OPEX.
• Dedicated scrubbing appliances: higher CAPEX, lower long-term OPEX, predictable performance.
• Hybrid: keep small always-on scrubbing + cloud burst for 400 Gbps cap.

Deployment & rollback

• Staged rollout: lab → single PoP → multi-PoP canary → global.
• Preflight: test BGP announcements, flow-spec rules in dev, simulate failover and traffic steering with test prefixes.
• Automation: IaC + playbooks for BGP changes, with automatic rollback on health/latency regressions.
• Rollback steps: withdraw engineered BGP communities, revert to baseline anycast prefix, re-enable removed routes; each step automated and instrumented.
• Runbooks & on-call: documented procedures, escalation, and scheduled DR rehearsals.

This design provides multi-layer defense, BGP-driven steering for large-volume attacks, selective steering to minimize latency for legitimate users, and clear deployment/rollback paths.

Direct answer
The TCP three-way handshake establishes a reliable connection before any data flows: the client sends a SYN, the server replies with a combined SYN-ACK, and the client finishes with an ACK. Its job is to let both sides agree on starting sequence numbers and confirm that both directions of the path actually work before committing application data to the wire.

Structured elaboration

1. SYN: the client picks an initial sequence number (ISN, essentially a large pseudo-random 32-bit number) and sends a segment with the SYN flag set and that sequence number. The client moves to the SYN-SENT state.
2. SYN-ACK: the server, if it's listening on that port, picks its OWN initial sequence number, and replies with a segment that has both the SYN flag set (announcing the server's own sequence number) AND the ACK flag set (acknowledging the client's sequence number + 1). The server moves to the SYN-RECEIVED state.
3. ACK: the client acknowledges the server's sequence number + 1 with a plain ACK segment. Both sides now move to ESTABLISHED, and either side may now send data.

Why three steps rather than two: TCP needs BOTH sides' sequence numbers acknowledged, since TCP is full-duplex (both directions need independent sequence tracking). A two-way handshake could confirm only one direction; the third message is what confirms the client's original SYN actually arrived, closing the loop for the client's own sequence space.

Worked example
Suppose a client opens a TCP connection to a web server on port 443. The client sends SYN, seq=1000. The server responds SYN, ACK, seq=5000, ack=1001 (acknowledging the client's SYN by number+1). The client responds ACK, seq=1001, ack=5001. From this point, the client's next data byte will carry sequence number 1001, and the server's next data byte will carry sequence number 5001; each side tracks the OTHER side's sequence space independently via the ACK field of every following segment.

Trade-offs & pitfalls
A frequent mistake is describing the handshake as three round trips; it's actually one and a half round trips of latency, because the SYN-ACK piggybacks the server's SYN onto its ACK of the client's SYN. This is also exactly why TCP always incurs at least one round trip of setup latency before any data can flow, which is the whole motivation behind newer mechanisms like TCP Fast Open that try to send data alongside the very first SYN.

Answer (Network Engineer perspective)

Brief stance

• For a hyperscale DC I recommend GENEVE as the long-term overlay due to TLV extensibility and modern offload support; VXLAN remains a pragmatic interim choice where hardware limits exist; GRE/MPLS is useful where MPLS service functions or legacy control-plane is required.

Extensibility (TLVs)

• GENEVE: designed for arbitrary TLVs (variable options header) so you can add telemetry, security context, SFC metadata without protocol changes.
• VXLAN: limited (VXLAN-GPE extends VXLAN but less standardized for arbitrary TLVs).
• GRE/MPLS: flexible payload but no standardized TLV framework comparable to GENEVE — metadata must be shoehorned into labels or separate headers.

NIC / switch offload

• Modern NICs (SmartNICs, Broadcom, Intel Ether) now support VXLAN and GENEVE hardware offload (encap/decap, checksum, RSS, TSO). Verify vendor-specific firmware and driver support.
• GRE/MPLS offload is less consistently supported; MPLS label push/pop offload exists on some ASICs but often poorer than VXLAN/GENEVE.
• Action: validate offload matrix per NIC/switch model; test CPU savings and throughput.

EVPN control-plane integration

• EVPN works with VXLAN (EVPN-VXLAN), GENEVE (EVPN-GENEVE) and MPLS (EVPN-MPLS). EVPN carries Type-5/2/3 routes and supports encapsulation negotiation via route targets/extended communities.
• GENEVE + EVPN is increasingly supported in SW/hw stacks; ensure your controller/orchestrator and vendor OS support VNI/tenant mapping and route type translations.

MTU / fragmentation implications

• Overhead examples (outer Ethernet+IP+UDP+encap):
  • VXLAN (IPv4+UDP+Vxlan): ~50 bytes overhead
  • GENEVE: base ~38 bytes + TLVs (variable; plan extra 20–100+ bytes)
  • GRE/MPLS: variable (GRE ~24B+; MPLS labels 4B each)
• Hyperscale must standardize jumbo MTU (e.g., 9000–9216) across fabric to avoid fragmentation.
• Don’t rely on fragmentation; fragmented packets break NIC offload and SFC. Validate PMTU, enable DF handling in orchestration, and enforce MTU for VMs/containers.

Operating / tooling considerations

• Observability: GENEVE TLVs enable richer telemetry; however tooling must parse them. Ensure collectors (sFlow/IPFIX, VTEP telemetry, P4 counters) understand chosen encap.
• Troubleshooting: VXLAN has richer ecosystem (debug tools), while GENEVE may need newer tooling/plugins.
• Orchestration: cloud controller, SR-IOV, SRv6 interactions, and firewall rules must be updated for outer headers.
• Security: update ACLs, microsegmentation policies to account for outer IPs and ports.

Recommendation & migration path

• Greenfield: GENEVE + EVPN control-plane, mandatory jumbo MTU, validate NIC offload; deploy telemetry TLVs incrementally.
• Brownfield (existing VXLAN fleet): 1) Inventory hardware offload + firmware capability matrix. 2) Enable EVPN as common control-plane if not present to centralize reachability. 3) Dual-encapsulation strategy: allow coexistence (some leafs speak VXLAN, new ones GENEVE) with EVPN route-type translation or VTEP-level decap/encap translation. 4) Upgrade NIC firmware/drivers and orchestrator to support GENEVE. 5) Roll out MTU uniformly, test with perf, flow visibility, and failure modes. 6) Cutover tenants incrementally; fall back to VXLAN until stable.

Trade-offs summary

• GENEVE: best extensibility and future-proofing; requires newer hw/tooling.
• VXLAN: broad existing support, mature tooling; limited metadata.
• GRE/MPLS: use when MPLS features or interop with legacy MPLS DC spine required.

Concrete next steps for interview follow-up

• Request your current hardware model list and controller versions so I can produce an offload/feature matrix and a phased migration plan with test cases and rollback points.

These path characteristics matter a lot for resilience, especially for stateful applications.

Common issues

• Latency increases session setup time and makes synchronous cross-region calls expensive.
• Jitter hurts any real-time or timeout-sensitive traffic and can trigger false failovers.
• Packet loss causes retransmissions, which can make TCP look slow or unstable.
• MTU mismatch can create fragmentation, blackholes, or intermittent failures if PMTUD is blocked.

Mitigations

• Keep state local when possible; avoid synchronous cross-region dependencies in the request path.
• Use replication that matches the workload: async for most services, synchronous only when the business really needs it.
• Standardize MTU across VPNs, tunnels, and interconnects, and use MSS clamping where needed.
• Tune retries and timeouts carefully so the application does not amplify transient network issues.
• For stateful sessions, use affinity, session replication, or graceful failover so users do not restart from scratch.

For me, the key is to design stateful systems so the network is not part of the critical path more than necessary. The more a service depends on every packet arriving perfectly, the more expensive WAN issues become.