Amazon Network Engineer (Mid-Level) Interview Preparation Guide

Clarify requirements

• Preserve consistency and recovery SLAs (RPO/RTO), 10x growth in bandwidth demand, minimal changes to app where possible, secure DC interconnects.

Network-level optimizations

• WAN capacity & path: upgrade links or add parallel dark-fiber/MPLS/EVPN-VXLAN overlays where justified; prefer 100G/400G optics with ROADM for cost-effective scaling.
• Traffic engineering: implement policy-based routing and QoS to prioritize replication traffic with rate limits and class-based shaping to avoid congestion with production traffic.
• WAN acceleration: deploy TCP acceleration, selective ACK tuning, and window scaling; use dedicated WAN accelerators (protocol-aware proxies) to reduce RTT penalties.
• Dedupe-aware flow control: terminate dedupe/compression appliances at each DC (inline or virtual) to collapse redundant bytes before transmission.
• Encryption offload: use inline IPsec/GRE with hardware crypto offload to avoid CPU bottlenecks.

Application-level optimizations

• Block-level deduplication and delta replication: convert full-file syncs to incremental, fingerprint blocks (rolling hashes), send only changed blocks.
• Compression + content-defined chunking: use gzip/snappy with chunking that maximizes cross-dc similarity.
• Change capture: integrate with app’s CDC/log-based replication to stream logical changes rather than bulk data.
• Scheduling & throttling: perform heavy syncs in off-peak windows; implement dynamic throttling tied to link utilization.
• Consistency-preserving techniques: use write-ahead logs, sequence numbers, or vector clocks to ensure idempotent replays; maintain checkpoints and snapshot metadata to meet RPO/RTO.

Metrics & trade-offs

• Track bytes saved, throughput, latency, and recovery times. Dedupe/compression reduces bandwidth but adds CPU and storage; WAN accel reduces RTT effects but is additional appliance cost. Prioritize solutions that reduce transmitted bytes first (dedupe/delta/CDC), then optimize transport.

Implementation plan

• Phase 1: deploy monitoring, baseline traffic, enable incremental replication and compression.
• Phase 2: add dedupe/acceleration appliances and QoS policies.
• Phase 3: capacity upgrade and automation for failover testing to validate SLAs.

Approach (brief)
Create an SVI on the L2 switch for VLAN 20 so clients have gateway, trunk to router carrying VLAN 20, and configure router-on-a-stick subinterface with ip helper-address pointing to the DHCP server.

Switch SVI / VLAN config

interface Vlan20
 description Clients-VLAN-20
 ip address 10.20.20.1 255.255.255.0
 no shutdown

interface GigabitEthernet1/0/1
 switchport trunk encapsulation dot1q
 switchport mode trunk

Router (router-on-a-stick) subinterface

interface GigabitEthernet0/0.20
 encapsulation dot1Q 20
 ip address 10.20.20.254 255.255.255.0
 ip helper-address 10.0.0.10   ! forwards DHCP & other UDP services
 no shutdown

Why ip helper-address / behavior to check when no lease

• DHCP uses UDP: client -> server uses source UDP 68 dest UDP 67; server replies src 67 dst 68.
• Router relay receives client broadcast (DHCPDISCOVER) and unicasts to 10.0.0.10 with giaddr = 10.20.20.254; server uses giaddr to allocate correct subnet.
• If clients get no address check:
  • Is trunk up and VLAN 20 allowed between switch and router?
  • Is SVI up and correct IP/mask?
  • ACLs or firewall blocking UDP ports 67/68 on path to 10.0.0.10.
  • Confirm helper configured on correct subinterface and that server responds to giaddr subnet.
  • Use packet captures / debug:
    • On router: debug ip dhcp server packets / debug ip packet detail to see forwarded UDP 67.
    • On server: check received packet source and giaddr.
  • Check ARP: router must ARP for client when replying (if relay, server reply returns to router which forwards to client).
• Optional: consider DHCP Relay Agent Information (option 82) if server expects it; can disable or configure appropriately.

Design (weeks 1–3)

• Inventory 200 routers (model, IOS, interfaces, current RIP config).
• Logical design: plan OSPF area layout (backbone Area 0, local areas), EIGRP AS for specific legacy clusters if needed. Define address summarization, LSDB size limits, authentication (MD5/sha), MTU/HELLO timers, and route tagging for redistribution.
• Decide dual-run strategy: run RIP and target protocol in parallel with controlled redistribution + route tags to prevent loops.

Staging (weeks 2–4)

• Build lab replica of representative topologies (core, distribution, remote).
• Pre-load configs, scripts, automated push templates (Ansible/NETCONF).
• Validate IOS versions, enable BFD and graceful-restart capabilities.

Pilot (week 4)

• Select 8–12 routers (diverse sites).
• Execute cutover in maintenance window: enable OSPF/EIGRP, implement route-maps to prefer new protocol (lower admin distance), keep RIP as backup.
• Validate convergence, CPU/memory, prefix counts.

Cutover (weeks 5–8, phased by region)

• Phased rollouts (10–20 routers/day). Automated deploy + pre-checks and post-checks.
• Gradually switch redistribution off and remove RIP after stability.

Risk mitigation & rollback

• Dual-run with route tags prevents loops; use admin-distance manipulation to control preferred paths.
• Keep configuration backups and one-touch rollback playbooks to re-enable RIP preference or revert interfaces.
• Maintain escalation runbook and spare routers/config-ready replacements.

Testing & validation

• Pre/post route tables, traceroutes, ping, path MTU tests, BFD timers, SLA probes.
• Convergence tests: planned interface flaps and link failures; measure converge time (target < X sec).
• Monitoring: SNMP, NetFlow, syslog, and automated alerting for route flaps or metric shifts.

Minimizing downtime & ensuring consistency

• Parallel protocols (no immediate RIP removal), preference via admin-distance, route tags, and summarization to limit LSDB growth.
• Use maintenance windows, traffic engineering (temporary static routes or policy-based routing) for critical flows.
• Stagger cutovers, validate each batch before next.

Stakeholder communication

• Weekly roadmap + pre-cutover notification 72/24/2 hours.
• Runbook shared with NOC, security, application owners.
• Post-cutover report with KPIs (convergence time, packet loss, incident list) and rollback incidents.

Example rollback snippet (Ansible task):

- name: rollback to RIP preference
  ios_config:
    lines:
      - no router ospf 1
      - router rip
      - version 2

Clarify scope & success criteria

• Define exactly what competitor’s feature does, which customers request it, and measurable success (usage %, retention lift, ARR impact, performance/SLA targets).

Demand & market validation

• Quantify: number of customers, ARR at stake, NPS/CSAT signals, RFEs. Run short surveys, sales interviews, and A/B landing page to estimate conversion before engineering.

Technical assessment

• Map required control-plane changes, touchpoints (auth, state, orchestration), and rollback complexity.
• Identify technical risks: data migration, backward compatibility, operational burden, security, and failure modes. Score likelihood × impact.

Opportunity-cost & prioritization

• Compare effort (engineering FTE-months), infra costs, and roadmap alternatives using RICE (Reach, Impact, Confidence, Effort). Include maintenance cost delta.

Prototype / validate cheaply

• Options:
  • Canary shim: implement feature as an overlay service (east-west proxy or controller extension) to emulate behavior without full re-architecture.
  • Config/feature-flag hack in staging for a pilot customer.
  • Emulator in lab to gather telemetry and failure scenarios.
• Run a short customer pilot (1–3 customers) behind NDA, measure key metrics and ops burden.

Decision criteria (Go/No‑Go)

• Go if: validated demand meets ARR threshold, technical risk reduced to acceptable levels by mitigations, RICE score beats alternatives, and pilot metrics meet SLAs.
• No‑Go / Revisit if: high unmitigable risk, negative pilot ops impact, or better ROI elsewhere.

Governance & timeline

• Set a 3-phase plan: Prototype (4–8 weeks), Pilot (8–12 weeks), Full build with checkpoint gates and rollback plan. Assign owner and success metrics for each gate.

Direct answer

In the room, you say plainly that you do not know, avoid guessing, and commit to a specific person, channel, and deadline for the answer rather than a vague "I'll get back to you." Afterward, you turn the gap into a fast, self-directed catch-up: you go straight to the fastest reliable source and verify it yourself, so that what you deliver at the follow-up is not just the fact but evidence that you now actually understand the area, which is what rebuilds credibility rather than just closing the ticket.

Structured elaboration

The live response and what it commits to. Name the gap precisely instead of deflecting ("I do not have the exact number for that specific configuration" beats a vague dodge), and commit to something concrete: who you will check with, how you will follow up, and by when. That commitment becomes the deadline that forces the catch-up that follows; a soft "I'll look into it" gives you nothing to be held to and no real urgency to close the gap fast.

The fast self-directed catch-up. Between the meeting and the follow-up, go to the fastest reliable source rather than the slowest thorough one: the colleague who actually owns that part of the product, the real system or configuration instead of a general document, a past support case that already answered something similar. Do not just collect the answer, verify or test it yourself if you can, so you are not repeating something secondhand you cannot defend if the customer asks a natural next question.

Rebuilding credibility rather than just delivering the answer. The customer is not only tracking whether you got the fact right; they are recalibrating how much they trust you going forward. Showing up with the answer plus a sign that you actually understand the mechanism behind it, so you can field a follow-up question live, closes the gap in a way that a bare, correct fact does not.

Worked example

A customer asks about an edge-case rate-limit behavior the presenter does not know off the top of their head. In the room: "I don't know that specific limit, let me confirm with the engineer who owns that service and get back to you by end of day tomorrow." Afterward, instead of searching general docs first, they message that engineer directly, get the real number and how it behaves at the edge, and then reproduce the behavior themselves in a test environment rather than just repeating what they were told. They follow up the next morning, ahead of the committed deadline, with the answer and one related edge case the customer had not even asked about, which is what actually shifts how the customer sees their competence.

Trade-offs & pitfalls

The single most damaging alternative is guessing or bluffing to avoid an awkward pause; a wrong answer delivered confidently costs far more credibility than an honest gap does. There is a real trade-off between speed and verification: going to the fastest source is right, but repeating an unverified answer just to hit your deadline can turn one gap into two. And following up late, or with less specificity than you promised, reopens the exact doubt the honest "I don't know" was supposed to contain.

I would address it privately and early, before frustration turns into teamwide resentment. I would start with a direct but nonjudgmental conversation: "I've noticed a few commitments have slipped, and it's affecting our planning. Is something blocking you?" The point is to understand the cause, not accuse them.

If they are overloaded or unclear on priorities, I would help clarify scope and agree on one realistic next step. I would also ask for smaller, more frequent check-ins so issues surface sooner. If the pattern is about skill or confidence, I would offer support or pair on the hardest part.

At the same time, I would keep the rest of the team informed only at a necessary level, without gossiping or blaming. If the misses continue after a clear conversation, I would bring the facts to the manager in a neutral way: dates, commitments, and impact. That protects the relationship while still protecting the team. The goal is accountability with respect, not public pressure.

For example, say the teammate is Priya, and the missed commitment is the payments-service integration tests: she has said she would finish them by Friday three sprints in a row and hasn't. I would message her directly: "I've noticed the payments-service tests have slipped the last three Fridays. Is something blocking you, or is the estimate off?" Priya explains she has also been pulled into unplanned support tickets and didn't want to flag it. We agree on one realistic next step: she owns just the critical-path test cases by Wednesday and hands the rest to me, and we add a five-minute check-in every Monday and Thursday so a slip surfaces mid-sprint instead of at the deadline. Two sprints later, one of those check-ins catches a new blocker early and the deadline holds.

Roles: forward proxy, reverse proxy, API gateway (short)

• Forward proxy: client-side intermediary that enforces outbound segmentation, content filtering, egress TLS inspection and DLP; used to restrict which external services internal hosts can reach and to apply per-user policy.
• Reverse proxy: server-side entry point that terminates incoming connections, enforces ingress ACLs, load‑balances and offloads TLS; provides central place for WAF and routing to backend zones.
• API gateway: domain-aware reverse proxy that adds auth, rate limiting, request transformation, and API-level policies; often integrates with identity and telemetry.

Deployment pattern (reverse proxy + WAF fronting microservices in internal zones)

• Perimeter: Internet → CDN / global load balancer → WAF+Reverse Proxy cluster (in DMZ). WAF enforces OWASP rules and blocks malicious payloads before routing.
• TLS termination: TLS is terminated at the reverse proxy/WAF for inbound traffic (mutual TLS optional). After inspection, the proxy can re‑encrypt to internal services (TLS re‑establish) to preserve confidentiality across zones.
• Internal zones: Microservices split into zones (e.g., public API zone, business-logic zone, data zone) separated by internal subnets and NSGs. Reverse proxy routes requests to zone-specific ingress proxies or service mesh ingress gateways.
• Routing & service discovery: Reverse proxy uses service discovery (DNS SRV, Consul, or control-plane API) to resolve backend endpoints and apply zone-aware routing (sticky sessions, blue/green). For complex microservices, the reverse proxy delegates intra-cluster routing to the service mesh (Envoy) which handles mTLS, retry, and circuit breaking.
• Segmentation & security controls: DMZ reverse proxy enforces coarse-grain controls; internal proxies/mesh enforce fine-grain RBAC, mTLS, and telemetry. Egress forward proxies control outbound flows from internal zones.

Operational notes / trade-offs

• Terminating TLS at WAF simplifies inspection but requires rigorous key management and re‑encryption to trust internal hops.
• Placing service discovery behind the reverse proxy prevents backend endpoint exposure; use authenticated APIs for discovery.
• Combine WAF + reverse proxy for defense-in-depth; use service mesh for east-west security and observability.

High-level approach
Build a multi-tier, distributed ingestion pipeline: edge collectors at PoPs → regional Kafka clusters for buffering → stream processing (Flink) → long-term stores (hot/cold) + search/indexing. Aim for loss-tolerant, scalable, and privacy-safe telemetry for 100k endpoints across 200 PoPs.

Collectors & Placement

• Lightweight collectors (Go/agent) per PoP to receive NetFlow/IPFIX, sampled pcap, device state (SNMP/syslog), and gNMI/gRPC streaming telemetry.
• Each PoP runs redundant collectors behind load balancers; collectors batch + forward to local Kafka cluster (or proxy to regional Kafka) over TLS.

Scaling ingestion

• Use partitioned Kafka topics per data type and PoP (e.g., netflow.poP.<id>), sized for peak throughput.
• Autoscale collectors and Kafka partitions based on throughput; use Flink consumers to normalize, enrich (geo, asset tags), and downsample before storage.

Sampling strategy & where to capture

• Flow records: ingest all sampled flows from devices (device-level flow sampling 1:100–1:1000 based on link).
• Packet capture: do sampling at PoP edge (sFlow/tail-based) for general visibility; enable full-capture only on-demand or for forensic windows (triggered via flow anomaly), stored in short-term hot store.
• Streaming telemetry: full granular state pushed continuously (telemetry is low-bandwidth structured); aggregate and store metrics, and keep raw streams in Kafka for a short TTL.

Retention tiers & indexing

• Hot tier (Elasticsearch/Opensearch + S3 warm): last 7–30 days for quick search/alerts.
• Warm tier (S3 + query layer like Athena/OpenSearch frozen): 30–365 days.
• Cold/Archive (object store, compressed Parquet): >1 year for compliance.
• Index flows and metadata in OpenSearch with time-based indices, asset and tag fields for fast queries; store heavy payloads (pcap) in object store with pointers in indices.

Privacy & compliance

• PII masking at ingestion: parsers redact payloads, hash identifiers (with salt) where needed.
• Encryption in transit and at rest (TLS, KMS).
• RBAC + attribute-based access control for search and raw capture retrieval.
• Automated retention enforcement, audit logs, and data access monitoring; support legal hold overrides.

Trade-offs

• Sampling reduces storage and CPU but can miss rare flows — mitigate with anomaly-triggered full captures.
• Kafka + Flink adds operational overhead but ensures resilience and realtime enrichment.

This design balances scalability, searchability, and privacy for a 100k-endpoint, 200-PoP deployment.

nftables snippet (recommended)

table inet filter {
  set http_rate_limit {
    type ipv4_addr
    flags timeout
    timeout 10s
    elements = { }
  }

  chain input {
    type filter hook input priority 0;
    tcp dport 80 ct state new meter @http_rate_limit { ip saddr limit rate 200/second burst 400 packets } accept
    # drop everything else (or continue other rules)
    drop
  }
}

iptables (conntrack + hashlimit)

# requires xt_hashlimit
iptables -N HTTP_LIMIT
iptables -A INPUT -p tcp --dport 80 --syn -m conntrack --ctstate NEW -j HTTP_LIMIT
iptables -A HTTP_LIMIT -m hashlimit --hashlimit-name http_per_src --hashlimit-above 200/sec --hashlimit-burst 400 --hashlimit-mode srcip --hashlimit-htable-expire 10000 -j DROP
iptables -A HTTP_LIMIT -j ACCEPT

Why this works

• Limits NEW TCP connects per source IP to 200/sec with 400 burst (packets treated as connection attempts).
• nftables' meter is efficient and in-kernel; iptables uses hashlimit module.

Limitations & operational impacts

• High-cardinality distributed attacks (many source IPs) bypass per-src limits because each IP stays under the threshold — aggregate load still saturates CPU/NIC/firewall.
• Conntrack table growth: tracking many NEW connections (even dropped after SYN) consumes conntrack entries and memory; risk of table exhaustion and kernel dropping legitimate flows.
• CPU overhead: per-packet accounting (hashlimit/nft meter) increases cycle cost under high PPS.
• Spoofing and ephemeral port tricks can reduce effectiveness.

Mitigations

• Combine with SYNPROXY or TCP SYN cookies to avoid conntrack state for half-open connections.
• Use upstream DDoS scrubbing, network ACLs, rate-limit by /24 aggregates, BGP blackholing, or eBPF/XDP for very high PPS.
• Monitor conntrack size and tune nf_conntrack_max, timeouts, and hashing.

Configuration (commands to enter in global config):

interface GigabitEthernet0/1
 description WAN - public IP
 ip address 198.51.100.2 255.255.255.0
 ip nat outside
!
interface GigabitEthernet0/0
 description LAN - inside
 ip address 192.168.10.1 255.255.255.0
 ip nat inside
!
access-list 10 permit 192.168.10.0 0.0.0.255
ip nat inside source list 10 interface GigabitEthernet0/1 overload

• The ACL matches the inside local network. The last command configures NAT overload (PAT) using the interface IP 198.51.100.2.

Interface directionality

• ip nat inside: applied to interfaces on the private/LAN side.
• ip nat outside: applied to the interface facing the public/Internet side.

Static NAT vs PAT/Overload

• Static NAT: 1:1 mapping (ip nat inside source static <local> <public>) — preserves a fixed public IP for an internal host; required for inbound reachability.
• PAT/Overload: many-to-one mapping using ports; multiple internal hosts share one public IP with different source ports — ideal for outbound Internet access.

Verification commands and what they show

• show ip nat translations
  • Lists active NAT translations (inside local/inside global and protocol/ports).
• show ip nat statistics
  • Shows NAT engine counters, overload pool info, and translation counts.
• show running-config | section ip nat
  • Confirms NAT config (ACL, pool/interface, overload).
• show access-lists 10
  • Verifies ACL matches the intended inside subnet.
• debug ip nat
  • Real-time NAT translation events (use carefully in production).

Use these to confirm translations, troubleshoot port exhaustion, and validate ACL matching.