Amazon Network Engineer (Mid-Level) Interview Preparation Guide

Direct answer

RSVP-TE (Resource Reservation Protocol with Traffic Engineering extensions, RFC 3209) builds an MPLS label switched path (LSP, a one-way tunnel through the network) in a fixed sequence, set out step by step below. The ingress router computes a route that satisfies the constraints (CSPF, constrained shortest path first, run over a database of link attributes learned from the IGP). It signals that route hop by hop with a PATH message. The egress answers with a RESV message that travels back along the same path, and as it passes each router that router reserves the bandwidth and assigns the label. Fast reroute (FRR) then protects the LSP locally, and re-optimization later moves it to a better path without dropping traffic.

Step by step

The sequence to give first: the IGP floods link data, CSPF picks the route, PATH carries it, RESV commits bandwidth and labels, and refresh keeps the state alive. Protection and re-optimization come after the tunnel is up.

1. Link attributes are flooded. The IGP carries traffic engineering data to every router in the area. In OSPF this is an Opaque LSA of type 10 (an OSPF message type for extra data, flooded only within one area) carrying sub-TLVs (small labelled fields inside it) for maximum bandwidth, maximum reservable bandwidth, unreserved bandwidth at each of eight priority levels (0 to 7) and an administrative group (a 32-bit set of colours, RFC 3630). The eight priority levels work like this: 0 is the strongest and 7 the weakest, and the figure for level p is the bandwidth still reservable by an LSP whose setup priority is p, so bandwidth held by weaker LSPs counts as available to a stronger one. IS-IS carries the same attributes in its own extensions. Each router collects this into the traffic engineering database (TED).
2. Path computation at the head end (CSPF). The ingress removes every link that fails the constraints (not enough unreserved bandwidth at the LSP's priority, excluded colour), then runs shortest path on what is left. For example, on a path A-C-D-B, link C-D has 10 Gbps reservable and LSP-X holds 7 Gbps there at hold priority 5, so C-D advertises 10 Gbps unreserved at priorities 0 to 4 and 3 Gbps at priorities 5 to 7. A new 6 Gbps LSP with setup priority 7 sees 3 Gbps and CSPF removes C-D; the same request with setup priority 2 sees 10 Gbps, keeps C-D, and when signalled it preempts LSP-X (setup priority 2 is stronger than X's hold priority 5), which must then find another path. The result is an explicit route.
3. PATH message, downstream. The ingress sends PATH hop by hop along the explicit route. Its objects include the EXPLICIT_ROUTE (ERO, the list of hops), the LABEL_REQUEST (asks each hop for a label and names the payload type), the SESSION_ATTRIBUTE (setup and hold priorities, where a new LSP can preempt an existing one only if its setup priority is numerically lower than the other's hold priority, and flags such as "node protection desired") and optionally the RECORD_ROUTE (RRO, which collects the path actually taken). PATH installs path state at each hop and carries the traffic description; it reserves nothing.
4. RESV message, upstream. Reservations in RSVP are receiver-driven (RFC 2205): the egress sends RESV back, and each router, as it processes RESV, checks admission (is there room on the outgoing link for this request), reserves the bandwidth and allocates a label, putting it in the LABEL object sent to its upstream neighbour. This is downstream-on-demand label distribution (labels are requested by the ingress and handed back toward it), and RFC 3209 treats labels received on different interfaces as different even when the numbers match.
5. Soft state. PATH and RESV state expires unless refreshed; RFC 2205 suggests a default refresh period of 30 seconds, configurable per interface. A lost refresh does not drop the tunnel at once because state survives several missed refreshes.
6. Forwarding. The ingress pushes the label it got, each transit router swaps, and the egress (or penultimate hop) pops.

The reservation is a control-plane accounting entry. Nothing in the forwarding path polices the LSP to its reserved rate unless an operator adds policing, so reservations only help if the ingress admits traffic honestly.

Message trace on a four-router path

Tunnel from A to B along A-C-D-B, 6 Gbps, label values illustrative.

1. PATH goes A to C to D to B, each hop carrying the explicit route C, D, B and the 6 Gbps request. Each router records path state; no bandwidth is reserved yet.
2. B, the egress, answers with RESV toward D and asks for label 3 (implicit null, so the penultimate router pops).
3. D receives RESV, checks it has 6 Gbps free on D-B, reserves it, allocates label 300200 for this LSP and sends RESV to C carrying 300200.
4. C reserves 6 Gbps on C-D, allocates 300100 and sends RESV to A carrying 300100.
5. A reserves 6 Gbps on A-C and now knows the label to use.
6. Data: A pushes 300100; C swaps 300100 for 300200; D pops (label 3 asked for) and B receives the plain packet.

Worked example: bandwidth-constrained path computation

Two routes from A to B: A-C-D-B with metric 30 and A-E-F-B with metric 45; each link has 10 Gbps reservable. Five LSPs request 6, 6, 3, 3 and 3 Gbps in sequence.

import heapq

# (node, node): IGP/TE metric; links are bidirectional with
# independent per-direction bookkeeping, so only the A->B direction is tracked here.
LINKS = {("A", "C"): 10, ("C", "D"): 10, ("D", "B"): 10,
         ("A", "E"): 15, ("E", "F"): 15, ("F", "B"): 15}
RESERVABLE = 10  # Gbps reservable per link (= 10G link, one direction)
free = {k: RESERVABLE for k in LINKS}

def cspf(src, dst, bw):
    """Prune links with less than bw unreserved, then run Dijkstra on TE metric."""
    adj = {}
    for (a, b), m in LINKS.items():
        if free[(a, b)] >= bw:
            adj.setdefault(a, []).append((b, m))
    dist, prev, pq = {src: 0}, {}, [(0, src)]
    while pq:
        d, u = heapq.heappop(pq)
        if d > dist[u]: continue
        for v, m in adj.get(u, []):
            if d + m < dist.get(v, 1e9):
                dist[v], prev[v] = d + m, u
                heapq.heappush(pq, (d + m, v))
    if dst not in dist: return None
    path = [dst]
    while path[-1] != src: path.append(prev[path[-1]])
    return path[::-1], dist[dst]

for name, bw in [("lsp-1", 6), ("lsp-2", 6), ("lsp-3", 3), ("lsp-4", 3), ("lsp-5", 3)]:
    r = cspf("A", "B", bw)
    if r is None:
        print(f"{name} {bw}G: no path with {bw}G free -> nothing to signal, tunnel stays down")
        continue
    path, cost = r
    for a, b in zip(path, path[1:]): free[(a, b)] -= bw
    print(f"{name} {bw}G: {'-'.join(path)} (metric {cost}); free now {sorted(set(free.values()))}")
print({f"{a}-{b}": f"{RESERVABLE - v}/{RESERVABLE}G reserved" for (a, b), v in free.items()})

lsp-1 6G: A-C-D-B (metric 30); free now [4, 10]
lsp-2 6G: A-E-F-B (metric 45); free now [4]
lsp-3 3G: A-C-D-B (metric 30); free now [1, 4]
lsp-4 3G: A-E-F-B (metric 45); free now [1]
lsp-5 3G: no path with 3G free -> nothing to signal, tunnel stays down
{'A-C': '9/10G reserved', 'C-D': '9/10G reserved', 'D-B': '9/10G reserved', 'A-E': '9/10G reserved', 'E-F': '9/10G reserved', 'F-B': '9/10G reserved'}

The first LSP takes the shorter route; the second does not fit (6 + 6 = 12 > 10) so CSPF prunes the shorter route and takes the longer one; the third and fourth fit in the 4 Gbps left on each route; the fifth finds no path and the tunnel stays down (the head end finds no route, so it sends no PATH message at all; a PathErr appears only when a router further along refuses a route that the head end believed was free, for example because its database was stale). Both routes end at 9 of 10 Gbps reserved, 90 percent. The example sets the reservable bandwidth equal to the link rate for simplicity; in production set it lower to leave headroom for traffic that is not reserved. The growth path for the failed fifth LSP is more capacity on one route, a higher setup priority so it can preempt a lower-priority LSP, or a smaller or split bandwidth request.

Protection with fast reroute (RFC 4090)

Terms: the PLR (point of local repair) is the router next to the failure; the merge point (MP) is where the backup rejoins the original path. Two methods:

• One-to-one backup: a separate detour LSP is created for each protected LSP at each PLR.
• Facility backup: one bypass tunnel protects a whole set of LSPs through label stacking: the PLR swaps the LSP label for the one the merge point expects and pushes the bypass label on top, so a single bypass protects many LSPs and per-LSP state does not grow with the number of protected LSPs. Example on the trace above, protecting link C-D with a bypass C-G-D (router G is an illustrative addition): normally C swaps 300100 for 300200 and sends to D. When C-D fails, C still swaps 300100 for 300200 (the label D expects for this LSP) and then pushes the bypass label, say 400, so the stack is [400, 300200] and the packet goes to G. G is the last router before D on the bypass, so it pops 400, and D receives the packet with label 300200 exactly as if nothing had failed (RFC 4090 describes this case, with penultimate hop popping on the bypass).

Link protection and node protection answer what is avoided, and one-to-one and facility answer how the backup is built; they are separate choices. Link protection avoids only the failed link; node protection avoids the next router as well, and the "node protection desired" flag in SESSION_ATTRIBUTE asks for it. RFC 4090 targets restoration in tens of milliseconds, since the PLR acts locally.

Re-optimization

After repair the LSP sits on a backup path. Re-optimization recomputes the best route and moves the LSP make-before-break (the new path is built before the old one is torn down): the ingress signals the new path under the same session using the Shared Explicit (SE) reservation style (one reservation shared by the old and new paths of the same tunnel on links they have in common), so the new and old paths do not double-count bandwidth on links they share, shifts traffic, then tears down the old path. Triggers include a timer, a link coming back up and a manual command. Re-optimization too often causes churn; too rarely leaves LSPs on long paths.

Pitfalls

Reservations drift from real utilisation (an LSP reserved at 6 Gbps may carry 2 Gbps), refresh load grows with the number of LSPs a router carries (a full mesh of N edge routers has N x (N-1) one-way tunnels), and a bypass tunnel that is not bandwidth-aware can congest the very links it uses.

Direct answer

Size each scope from peak concurrent clients (not headcount) so the dynamic pool is at most about 70% used at peak, keep printers and other fixed devices outside the dynamic pool with DHCP reservations, set the lease time to match how fast devices come and go, and alert on pool utilisation at 80% (warn) and 90% (critical) plus on any allocation failure. The 70% target and the 80/90 thresholds are my engineering choices, not standards: they leave room for a surge and give the on-call person time to act before users see an error.

DHCP (Dynamic Host Configuration Protocol) gives a device an address on a lease, a time-limited right to use it. A scope is the address range a server may lease for one subnet. Three parts of a scope are used together below. The pool is the block the server hands out dynamically. A reservation ties one device's hardware address (its MAC address, the unique identifier built into its network card, written like 02:42:ac:1f:01:aa) to one fixed IP address that the server always gives that device. An excluded range is a block inside the scope that the server never leases, kept for devices configured by hand.

Step 1: size each scope from peak, with the arithmetic

Rule: pool needed = peak dynamic clients / 0.70, then add the fixed and infrastructure addresses, then choose the smallest subnet that holds the total (usable hosts = 2^(32 - prefix) - 2).

Layout for the office /23, 10.30.0.0/23: infrastructure 10.30.0.1 to .4, reservations 10.30.0.10 to .31 (22 addresses), dynamic pool 10.30.0.100 to 10.30.1.254 (156 + 255 = 411 addresses). The table's 272 is the smallest pool that meets the 70% rule, while this layout builds 411 addresses: a /23 cannot be smaller than its 510 usable addresses, so the pool is stretched to fill the space left after the 4 infrastructure and 22 reserved addresses. Peak use is 190 / 411 = 46%, well under the 70% ceiling because the /23 boundary gave extra room. Warehouse /25, 10.30.8.0/25: infrastructure .1 to .4, reservations .10 to .21 (12), pool .30 to .126 (97 addresses), peak use 60 / 97 = 62%. A tempting /24 for the office (254 usable) would leave 254 - 26 = 228 addresses for the pool after the 22 fixed devices and 4 infrastructure addresses, so 190 clients would run at 83%, over the target and into the warning band.

Guest Wi-Fi is sized differently, by turnover. Devices there arrive, leave without releasing their address, and every lease occupies an address until it expires. In the steady state, addresses held = arrival rate x lease time. With 60 new devices an hour at peak:

So guest networks need a short lease (1 hour here) more than a large subnet. Office networks with stable laptops can use 8 hours, which keeps renewal traffic low: a client renews at T1, half the lease by default (RFC 2131), so 8 hours means T1 = 4 hours. The short guest lease costs more renewals but those are cheap compared with an exhausted pool. Wi-Fi privacy features (MAC randomisation) make a phone or laptop present a random hardware address per network and sometimes change it over time, so one device can look like several clients to the server. Measure arrivals rather than guess.

Step 2: fixed devices, reservation versus static address

Prefer DHCP reservations (the server always gives this device's hardware address the same address) over typing a static address on the device: the address, gateway, DNS and other options change in one place, nothing on the device needs a visit, and the address sits in the server's records. Keep reservations outside the dynamic pool, as in the layout above, so they never consume pool capacity and are never handed to somebody else. Use true static addresses only for devices that must work without DHCP (the DHCP servers, routers, switches), and put those in an excluded range so the server never leases them.

A monitoring detail measured in a lab with Kea, ISC's DHCP server: configured with "reservations-out-of-pool": true and one reservation, the subnet-wide counter assigned-addresses counts the reserved device, but the pool counters do not. One reserved device and two ordinary clients over a 4-address pool:

subnet 1: pool 2/4 = 50%
subnet-wide assigned (includes reservations): 3
allocation failures: 0

Dividing the subnet-wide figure by pool size would report 75% and overstate utilisation, so alert on the pool counters. This script produced the output. It asks the server for its counters and prints three of them:

import json, socket

s = socket.socket(socket.AF_UNIX)
s.connect("/run/kea/ctrl.sock")
s.sendall(json.dumps({"command": "statistic-get-all"}).encode())
buf = b""
while True:
    buf += s.recv(65536)
    try:
        stats = json.loads(buf)["arguments"]
        break
    except ValueError:
        pass

for subnet in (1,):
    total = stats[f"subnet[{subnet}].pool[0].total-addresses"][0][0]
    used = stats[f"subnet[{subnet}].pool[0].assigned-addresses"][0][0]
    print(f"subnet {subnet}: pool {used}/{total} = {100 * used / total:.0f}%")
    print("subnet-wide assigned (includes reservations):",
          stats[f"subnet[{subnet}].assigned-addresses"][0][0])
    print("allocation failures:", stats["v4-allocation-fail-subnet"][0][0])

Reading the script: Kea listens for JSON commands on a control socket, a local file-like channel (/run/kea/ctrl.sock) that a program on the same machine can connect to. statistic-get-all returns every counter in one reply, which can arrive in several chunks, so the while loop keeps reading until the text parses as JSON. Each counter is a list of [value, timestamp] pairs with the newest first, so [0][0] means "the first pair, its value": the current reading. The names are built from the subnet id and pool position: subnet[1].pool[0] is subnet 1, first pool. The same request printed raw in the lab, with one reservation and two ordinary clients leased (timestamps are from the run):

subnet[1].pool[0].total-addresses [[4, '2026-10-06 08:29:56.897500']]
subnet[1].pool[0].assigned-addresses [[2, '2026-10-06 08:30:00.698243'], [1, '2026-10-06 08:30:00.414831'], [0, '2026-10-06 08:29:56.897508']]
subnet[1].assigned-addresses [[3, '2026-10-06 08:30:00.698242'], [2, '2026-10-06 08:30:00.414829'], [1, '2026-10-06 08:30:00.098931'], [0, '2026-10-06 08:29:56.897505']]

The pool counter has gone 0, 1, 2 over the three timestamps, the subnet-wide counter 0, 1, 2, 3 because it also counts the reservation, and [0][0] picks the 2 and the 3.

The lab config was one subnet, 172.31.1.0/24, a pool of 172.31.1.100 to 172.31.1.103, a control socket at /run/kea/ctrl.sock, and the reservation {"hw-address": "02:42:ac:1f:01:aa", "ip-address": "172.31.1.50"}.

Step 3: detect exhaustion before users do

Continuing the lab, three more clients arrive against the 4-address pool (each udhcpc was run as udhcpc -n -q -f -t 2, so it sends two DISCOVER packets before giving up; BusyBox's default is three, which makes the counter read 3 instead of 2):

udhcpc: lease of 172.31.1.102 obtained from 172.31.1.5, lease time 3600
udhcpc: lease of 172.31.1.103 obtained from 172.31.1.5, lease time 3600
udhcpc: no lease, failing
subnet 1: pool 4/4 = 100%
subnet-wide assigned (includes reservations): 5
allocation failures: 2

The third client got nothing, and the failure counter reads 2 because that client sent its discovery twice before giving up: it counts failed discoveries, not people. Use it as a paging signal (any increase in 5 minutes), and use utilisation for the early warning:

• Warn at 80% and critical at 90% of the pool, per scope, evaluated for the busiest hour you expect.
• On Windows, this lists scopes over the warn level and labels them (-Failover includes failover-related scope statistics; PercentageInUse is the property Microsoft's own example filters on). It was checked for syntax with the PowerShell parser and the cmdlet and property names against Microsoft Learn, not run, because the DhcpServer module needs Windows:

$warn = 80; $crit = 90
Get-DhcpServerv4ScopeStatistics -Failover |
    Where-Object { $_.PercentageInUse -ge $warn } |
    Sort-Object PercentageInUse -Descending |
    Select-Object ScopeId, PercentageInUse,
        @{ n = 'Level'; e = { if ($_.PercentageInUse -ge $crit) { 'CRITICAL' } else { 'WARN' } } }

• Also watch the server log for "no address available" style lines and for repeated DISCOVER from the same hardware address with no OFFER.
• Add a trend alert: utilisation rising across days, because a slow leak is easier to fix than an outage.

Look-alikes and high availability

Real exhaustion is not the only reason clients stop getting addresses, and the first look-alike is the one that can hide behind a healthy dashboard. A rogue DHCP server (an unauthorised device answering requests, such as a home router plugged in) does not use up your pool: clients take its offers instead, so your utilisation can look low while users fail. A DHCP starvation attack (one host requesting addresses with many fake hardware addresses) really does fill the pool. Switch-level DHCP snooping (the switch watches DHCP traffic and lets only ports you mark trusted send server replies) and a per-port limit on requests defend against both. Also look for devices that never release, and for duplicate-address declines: before using an offered address a client checks with ARP (a local-network "who has this address?" query) whether anyone else already holds it, and if so it sends a DHCPDECLINE message to the server (RFC 2131, section 4.4.1), so a stream of declines means something outside the server's records is using addresses the server thinks are free.

If you run a failover pair, the two servers share one pool, so size the scope for the whole peak, not per server. A split scope is different: each server holds only its slice, so an 80/20 split of the 411-address office pool above gives the small server about 82 addresses against a peak of 190, which cannot cover a failure.

Pitfalls

Sizing from headcount instead of peak concurrent devices; counting reservations as pool use; the same long lease on every VLAN; alerting only after the first user complaint; and forgetting that exclusions and reservations shrink the usable range below what the subnet mask suggests.

Direct answer

Frequent BGP route flaps causing blackholes and high router CPU point at instability being amplified rather than absorbed, so the diagnosis has two parallel tracks: find what's actually flapping (a physical link, a peer's own instability, or a design issue causing your routers to reprocess the same change repeatedly), and separately assess whether your dampening and timer settings are making the impact worse rather than better.

Structured elaboration

1. Identify the specific flapping route(s) and their source: route-flap statistics (many platforms track flap counts per prefix) point at exactly which prefixes are unstable; correlate this against BGP debug/log output and syslogs to see whether the flaps originate from a specific neighbor's own instability, a locally flapping link, or an upstream instability being propagated through your network.
2. Distinguish genuine instability from an internal design amplifying it: an iBGP full mesh without route reflectors, or excessive de-aggregation (announcing many small prefixes instead of aggregating), both increase the number of routers that must reprocess every flap and the number of individual updates each flap generates, amplifying a single real event into much larger CPU and update-processing load than the underlying instability actually warrants.
3. Tune route dampening carefully, understanding its real trade-off: dampening suppresses a route after it exceeds a flap-frequency threshold, protecting the network from the CPU and update-processing cost of constant reprocessing, but it also means a route that's SUPPRESSED is unreachable even after it stabilizes, until the penalty decays; overly aggressive dampening can turn a brief, real flap into an extended, self-inflicted blackhole lasting far longer than the original instability.
4. Review BGP timers: excessively short hold/keepalive timers make sessions more sensitive to transient congestion or brief link issues, triggering session resets (and associated full-table reprocessing) that a slightly more tolerant timer would have absorbed without a full session bounce.
5. Consider architectural changes for durable prevention: route reflectors reduce the number of BGP speakers that must reprocess every update in an iBGP mesh; sensible aggregation reduces the total number of distinct prefixes that can independently flap; both reduce the AMPLIFICATION of any single instability event, rather than addressing the instability's original source.

Worked example

Route-flap statistics show one specific /24 flapping roughly every 90 seconds, sourced from a specific eBGP neighbor. Syslogs on that neighbor's session show repeated brief link-down events on their side. Locally, this network runs a full iBGP mesh of 40 routers with no route reflectors, so each flap generates update processing on all 40 routers simultaneously, which is what's actually driving the high CPU, not the flap itself (a single flap on a single-router network would barely register). Applying moderate route dampening to this specific prefix while the peer investigates their link reduces the immediate CPU impact; deploying route reflectors is flagged as the durable architectural fix, since it would reduce the reprocessing amplification for ANY future flap, not just this one.

Trade-offs & pitfalls

Aggressive dampening is a real, double-edged tool: it protects against reprocessing cost but can extend a legitimate brief outage into a much longer self-inflicted one if tuned too tightly, since a stabilized route stays suppressed until its penalty decays below the reuse threshold. Address the AMPLIFICATION (full mesh, over-de-aggregation) as a separate, architectural problem from the underlying instability's source; fixing one without the other leaves you exposed to the next unrelated flap causing the same disproportionate impact.

Direct answer. For a data center with dozens of routers and thousands of hosts, run a single OSPF (Open Shortest Path First) area, area 0, for the fabric and carry only router loopbacks (a virtual interface that never goes down, used as the router's stable address), point-to-point links and a bounded set of server-subnet prefixes in it. Create extra areas only at the edges where you need to hide detail: a totally stubby area for a block that only needs a default, or an NSSA (not-so-stubby area, one that blocks outside routes yet lets a router inside it inject some) for an external attachment whose router injects routes, because a plain stub cannot hold such a router, with the border routers summarizing. Keep SPF (shortest path first, the route computation each router reruns when the topology changes) cheap by keeping the link-state database small, using /31 point-to-point links (a two-address subnet, one address for each end), summarizing at area borders, and throttling SPF with timers, rather than by splitting a fabric that does not need splitting.

Why hosts are not the scaling problem

OSPF does not carry individual hosts, it carries routers, links and prefixes. A host added to a subnet that is already advertised adds nothing to the database: 100 hosts or 1,000 hosts on one subnet cost the same single prefix. The size of the link-state database (LSDB, the table of link-state advertisements, LSAs, that every router in the area holds) follows router count, link count and prefix (subnet) count, which is why the worked sizing below counts subnets and not hosts.

Worked sizing (computed)

Assume a leaf-spine fabric: 4 spine routers and 40 leaf (top-of-rack) routers, every leaf connected to every spine, 100 hosts per leaf, 5 server VLANs (subnets) per leaf.

A 44-router, 404-prefix LSDB is small for a modern router: it is a fraction of the several-hundred-router areas described below as the point where splitting starts to pay (a design rule of thumb, not a protocol limit). Splitting it into areas would add border routers, summaries to maintain and new failure modes (a partitioned area 0, a missing summary) in return for saving almost nothing. That is why I would not split this fabric.

Where areas and area types go

Per Cisco documentation, a stub area blocks external (type 5) LSAs, the advertisements for routes redistributed from outside OSPF, and uses a default route, a totally stubby area also blocks inter-area summary LSAs, and an NSSA allows type 7 LSAs for redistribution. For a stub area, area stub must be configured on every router in the area, while no-summary goes only on the ABR. Per RFC 2328, virtual links cannot cross a stub area and an ASBR (autonomous system boundary router, a router that redistributes external routes) cannot sit inside a plain stub area.

Keeping SPF cost down

1. Fewer, more meaningful prefixes. Advertise loopbacks and /31 links, summarize server subnets at ABRs with area N range prefix mask, and use passive-interface on host-facing interfaces: it suppresses hellos, so no adjacency can form toward hosts.
2. Point-to-point network type on fabric links (ip ospf network point-to-point). Cisco states that router priority (used for designated router election) applies only to multiaccess networks, so a point-to-point link has no DR election to run. The designated router (DR) is the one router on a shared segment such as Ethernet that all the others form adjacencies with; electing it is an extra step that a link with just two routers does not need.
3. Cost that matches the links. The default reference bandwidth is 100 Mbps, so with that default every link of 100 Mbps or faster gets the same cost of 1 and OSPF cannot tell 10G from 100G. Set auto-cost reference-bandwidth to the same value on every router. With 100,000 Mbps, cost is reference divided by bandwidth: a 10G link costs 10, a 25G link costs 4 and a 100G link costs 1. A mismatched reference between routers makes the two ends of a path disagree about its cost.
4. Throttle SPF with timers throttle spf spf-start spf-hold spf-max-wait (milliseconds), so a burst of changes triggers a few computations instead of one per LSA. The three values are the initial delay before the first computation after a change, the minimum hold time between two consecutive computations, and the maximum wait time between two consecutive computations. Cisco's defaults are 5000, 10000 and 10000 ms; fabrics typically lower the start delay (the example below uses 50, 200 and 5000).
5. Use BFD (Bidirectional Forwarding Detection) for failure detection instead of shrinking hello timers: bfd all-interfaces under the OSPF process plus bfd interval 50 min_rx 50 multiplier 3 on the interface (Cisco syntax; the multiplier is the number of missed BFD packets before the neighbor is declared down) detects a failure in 3 x 50 = 150 ms of silence without loading the OSPF control plane.
6. Contain churn. Keep flapping edge links out of area 0, because every flap is flooded and triggers SPF in every router of that area. RFC 2328 also limits how often one LSA can be originated (MinLSInterval, 5 s) and accepted (MinLSArrival, 1 s).

router ospf 1
 auto-cost reference-bandwidth 100000
 timers throttle spf 50 200 5000
 bfd all-interfaces
 network 10.255.0.0 0.0.255.255 area 0
 passive-interface Vlan100
interface Ethernet1/1
 ip ospf network point-to-point
 bfd interval 50 min_rx 50 multiplier 3

Reading the configuration: auto-cost reference-bandwidth 100000 and timers throttle spf 50 200 5000 are items 3 and 4 above; bfd all-interfaces turns on BFD for the OSPF interfaces; network 10.255.0.0 0.0.255.255 area 0 enables OSPF on every interface whose address falls inside 10.255.0.0/16 and puts it in area 0 (the second value is a wildcard mask, the inverse of a subnet mask: 0 bits must match, 255 bits may be anything); passive-interface Vlan100 stops hellos on that host-facing interface while its subnet is still advertised, provided the subnet is also covered by a network statement (the name is illustrative, and here it is assumed to sit inside 10.255.0.0/16); on the fabric interface, ip ospf network point-to-point and the bfd interval line are items 2 and 5.

What flips this design

Past a few hundred routers, or where operators want failures contained per session, many fabrics use BGP (Border Gateway Protocol) as the underlay instead of OSPF. At dozens of routers, a single OSPF area is simpler to operate than that or than a multi-area split.

Pitfalls

• Splitting by habit: a data center of this size gains nothing from an area per rack.
• Summarizing at an ABR hides a failed subnet inside the summary, so traffic can black-hole until the whole summary disappears.
• Leaving the default reference bandwidth on a 10G/100G fabric.
• A stub flag set on some routers of an area and not others: they do not form adjacencies.

I would spend the first week comparing three things: what exists, what is measured, and what the team actually learns from it. Order of work:

Day 1 to 2: coverage. Export the device inventory (the team's source of truth) and the list of devices currently polled, and diff them. Compute coverage overall and for critical devices, list devices monitored but missing from inventory (orphans, which nobody owns), and devices in both whose polls are failing (known but silent). Example, run in a Python container:

inventory = {"core1": "critical", "core2": "critical", "dist1": "critical",
             "edge1": "critical", "acc1": "standard", "acc2": "standard",
             "acc3": "standard", "wan-cpe7": "standard"}
polled_ok_last_15m = {"core1", "core2", "dist1", "acc1", "acc2"}
in_monitoring_db = polled_ok_last_15m | {"old-sw9", "edge1"}

missing = sorted(set(inventory) - in_monitoring_db)
orphans = sorted(in_monitoring_db - set(inventory))
failing = sorted(set(inventory) & (in_monitoring_db - polled_ok_last_15m))
covered = len(set(inventory) & polled_ok_last_15m)
print(f"covered {covered}/{len(inventory)} = {covered/len(inventory):.1%}")
crit = [d for d, c in inventory.items() if c == "critical"]
crit_cov = [d for d in crit if d in polled_ok_last_15m]
print(f"critical covered {len(crit_cov)}/{len(crit)} = {len(crit_cov)/len(crit):.1%}")
print("not monitored:", missing)
print("monitored but not in inventory:", orphans)
print("known but not polled successfully:", failing)

Output (run in a Python 3.12 container): covered 5/8 = 62.5%; critical covered 3/4 = 75.0%; not monitored: ['acc3', 'wan-cpe7']; monitored but not in inventory: ['old-sw9']; known but not polled successfully: ['edge1']. In the failing line the parentheses are for readability, not correctness: Python's set - (difference) already binds tighter than & (intersection), so set(inventory) & in_monitoring_db - polled_ok_last_15m also gives ['edge1']. The line means "inventory devices that are in the monitoring database but were not polled successfully". Here in_monitoring_db means the monitoring system's list of configured targets. The finding is that a critical edge device is blind, which outranks everything else.

Day 2 to 3: data quality. For each poll target ask: does it succeed (in Prometheus-style tools, up is 1 when a target's last poll worked and 0 when it failed, so up == 0 for 10 minutes finds targets that keep failing), does the scrape finish within its interval, and are the right counters used? 32-bit octet (byte) counters on fast links wrap quickly: a 32-bit counter holds 2^32 = 4,294,967,296 bytes, which is 34,359,738,368 bits, so a link running flat out at 1 Gbit/s (1,000,000,000 bits a second) wraps it in 34.4 seconds (RFC 2863 gives 34 seconds) and at 10 Gbit/s in 3.4 seconds (computed). Polls slower than that miss whole wraps, so check for the 64-bit counters (ifHCInOctets, the high-capacity count of bytes received, defined in RFC 2863, the standard that defines SNMP interface counters). Check SNMP version: v1 and v2c community strings travel unencrypted, so note v3 as a security gap. Check time sync (NTP, Network Time Protocol): without consistent clocks, logs and events cannot be correlated.

Day 3 to 4: alert quality. Pull the last 90 days of alerts. Count alerts per rule, find the top 5 noisy rules, and for each ask whether it led to action. Look for alerts with no owner and critical alerts nobody has ever seen fire. Then take the last 5 to 10 incidents and record how each was first noticed: monitoring, a user report, or luck. If most were first reported by users, monitoring is not doing its job whatever the dashboards show.

Day 4 to 5: blind spots and retention. List what is not measured at all: provider circuits (leased lines from carriers; the SLA, the service level agreement the provider promises, is the provider's number, not yours), wireless, DNS and DHCP, VPN, cloud networks, and application paths between sites (synthetic probes). Check retention: can you chart this month against last year? Check whether monitoring itself is monitored (a heartbeat alert is a rule built to fire permanently; a separate check pages someone if that always-on alert ever stops arriving, which proves the alerting path itself is broken).

Gaps I would expect to find, in rough order of cost: unmonitored or silently failing critical devices; alerts nobody trusts; no synthetic path tests; weak SNMP security; short retention.

What I would fix first, and why: (1) get every critical-path device polled and alerting, because an unmeasured outage is the costliest gap and the fix is small; (2) delete or demote the noisiest alert rules so that pages mean something, because trust in alerts decays quickly once people start ignoring them; (3) add a few synthetic probes between key sites; (4) migrate to SNMP v3 and 64-bit counters. I would hold off on new dashboards and tools until the first two are done, since a prettier view of incomplete data adds little.

I would end the week by writing the findings as a short, ranked list with the coverage numbers, and agree the first two fixes with the team before changing anything.

I would address it privately and early, before frustration turns into teamwide resentment. I would start with a direct but nonjudgmental conversation: "I've noticed a few commitments have slipped, and it's affecting our planning. Is something blocking you?" The point is to understand the cause, not accuse them.

If they are overloaded or unclear on priorities, I would help clarify scope and agree on one realistic next step. I would also ask for smaller, more frequent check-ins so issues surface sooner. If the pattern is about skill or confidence, I would offer support or pair on the hardest part.

At the same time, I would keep the rest of the team informed only at a necessary level, without gossiping or blaming. If the misses continue after a clear conversation, I would bring the facts to the manager in a neutral way: dates, commitments, and impact. That protects the relationship while still protecting the team. The goal is accountability with respect, not public pressure.

For example, say the teammate is Priya, and the missed commitment is the payments-service integration tests: she has said she would finish them by Friday three sprints in a row and hasn't. I would message her directly: "I've noticed the payments-service tests have slipped the last three Fridays. Is something blocking you, or is the estimate off?" Priya explains she has also been pulled into unplanned support tickets and didn't want to flag it. We agree on one realistic next step: she owns just the critical-path test cases by Wednesday and hands the rest to me, and we add a five-minute check-in every Monday and Thursday so a slip surfaces mid-sprint instead of at the deadline. Two sprints later, one of those check-ins catches a new blocker early and the deadline holds.

Define what each tier is allowed to reach, then default-deny everything else, enforced with whichever combination of Virtual Private Clouds (VPCs), subnets, security groups, and network policies fits the platform:

• Ingestion tier: only needs to WRITE new data into the central data store. It should never read back from the store, never talk to the training or analytics cluster, and never receive inbound connections from anything except its external data sources.
• Transformation and worker tier: needs both read and write access to the central data store, reading raw data in and writing transformed data back, but has no legitimate reason to talk directly to the analytics cluster or to be reachable from the ingestion tier.
• Central data store: the shared resource. It accepts writes from ingestion and the worker tier, and reads from the worker tier and the analytics cluster, but should never itself initiate connections outward to any tier; a data store that only answers requests and never calls out closes off one lateral-movement path.
• Training and analytics cluster: only needs READ access to the central data store. It should never be able to write back (protecting data integrity, so a compromised analytics job can't corrupt the source data) and never needs to reach the ingestion tier at all.

flowchart LR
    ING[Ingestion Tier] -->|write only| STORE[Central Data Store]
    WORKER[Transformation and Worker Tier] -->|read and write| STORE
    STORE -->|read only| ANALYTICS[Training and Analytics Cluster]
    ING -.blocked.-> ANALYTICS
    WORKER -.blocked.-> ANALYTICS
    ANALYTICS -.blocked.-> ING

Enforcement mechanisms

Inside a single cloud VPC, place each tier in its own subnet and use security groups scoped per tier, so each tier's group only allows inbound from the specific tier(s) it should accept traffic from, on the specific ports it serves. Across separate VPCs or accounts, express the same intent as VPC-level routing and firewall rules. Inside a Kubernetes cluster, use a default-deny NetworkPolicy per namespace (a rule object that controls which pods may talk to which) with explicit allow rules matching exactly the tier-to-tier edges above.

Worked example

If the data store listens on its database port, the policy reads: "ingestion tier as source, data store as destination, that port, write-path only," with the write restriction enforced at the application or credential layer since network policy alone can't distinguish a read from a write on the same open port; and separately, "analytics cluster as source, data store as destination, that port, read-only," enforced through a read replica or a database role restricted to read-only queries. Network-layer controls can restrict WHO connects and on WHAT port; restricting read versus write on an already-open connection is a job for database-level permissions layered on top, not for network segmentation alone.

Trade-offs and pitfalls

It's tempting to give the analytics cluster direct read access to the SAME live data store the worker tier writes to, since replicating data out is more work. The trade-off is that analytics workloads, often bursty and exploratory, then share load and blast radius with the production pipeline; many designs instead give analytics a separate read replica or materialized extract, trading some data freshness for isolation. Also remember the granularity mismatch: security groups and NetworkPolicies work at the network layer and cannot express "this connection may only read," so read/write restriction always needs a second layer on top of the network segmentation described here.

Direct answer

I weigh the security benefit against the friction a customer will actually feel, using numbers where I can. Then I look for the lowest-friction way to get most of the protection, and I roll it out in stages with measurements and an exit. Security wins when the risk is serious and not otherwise reducible, but how it ships is a customer-experience decision.

How I weigh it

1. Risk removed. What attack or loss does the change prevent, how likely, and for whom? Ask security for a specific scenario, not "best practice."
2. Friction added. Which customers feel it, how often, at what moment (signing in daily versus a rare admin action), and what do they do when annoyed (abandon, call support, work around it)?
3. Cheaper routes to the same protection. Apply the strict control only where risk is high (risk-based or step-up checks: ask for an extra proof only when something looks unusual or the action is sensitive), remember trusted devices, or give a smoother factor.
4. Who carries the cost. Friction on a signup flow costs conversion; friction in an enterprise admin flow costs support calls.

Three related cases

Worked example (illustrative): rolling out MFA to 100,000 accounts

• Stage 1: 5% of accounts (5,000), chosen to include small and large customers. Track setup completion, lockout-related support contacts, and sign-in success.
• Gate: advance only if the thresholds agreed beforehand with security and support are met. Illustrative thresholds: at least 80% of prompted accounts finish setup (4,000 of the 5,000), no more than 10 lockout-related support contacts a week (2 per 1,000 accounts), and sign-in success falls by no more than 1 percentage point. Miss any one and I pause, fix the cause, and rerun the stage.
• Stage 2 and 3: widen, with in-app explanation of why, then move from encouraged to required on a published date. Keep a rollback and an exception path for customers with real blockers.

Pitfalls

Do not frame it as security versus customers. Do not roll out to everyone at once. Do not set a "will not hurt conversion" bar with no measurement behind it.

Several widely-deployed legacy protocols transmit everything, including credentials, in plaintext, and each has a secure, encrypted successor that should replace it. The right approach to migration is a short-term compensating control paired with a tracked long-term replacement, not an instant rip-and-replace that a legacy environment often cannot support on day one.

The pairs

Migration strategy per pair

• Telnet to SSH: short-term, restrict Telnet to an isolated management network with strict access control and close monitoring if some legacy device firmware cannot be upgraded immediately; long-term, disable the Telnet daemon entirely and enforce SSH-only management.
• FTP to SFTP/SCP: short-term, confine any remaining FTP service to internal-only networks and never expose it to the internet; long-term, migrate transfer scripts and workflows to SFTP and decommission the FTP service.
• HTTP to HTTPS: short-term, add an HTTP-to-HTTPS redirect and obtain a valid TLS certificate; long-term, enforce HTTPS-only with HTTP Strict Transport Security (HSTS) so browsers refuse to fall back to plaintext even if an attacker tries to intercept the initial redirect.

Worked example

An analyst runs a network scan and finds 40 internal hosts still answering on port 23 (Telnet) and 12 legacy file servers on port 21 (FTP), while the customer-facing website already redirects HTTP to HTTPS. Short-term mitigation applied immediately: a firewall rule restricts ports 23 and 21 to the management subnet only, cutting exposure to the general LAN and internet to zero without waiting for a change window. Long-term: a tracked migration ticket per host group targets a move to SSH and SFTP with a decommission date, and any device whose firmware genuinely cannot be updated in time gets its own documented compensating control (isolation plus logging) rather than being quietly left exposed.

Trade-offs and pitfalls

Some embedded or legacy hardware genuinely cannot support SSH or TLS, so isolation is a legitimate interim step, not an excuse to defer the migration indefinitely. And enabling HTTPS without also enforcing a redirect and HSTS still leaves the plaintext path reachable, which defeats much of the purpose of migrating in the first place.

Direct answer
TCP's retransmission timeout (RTO) is computed from a smoothed estimate of round-trip time (SRTT) plus a term for how MUCH the round-trip time has been varying (RTTVAR), not from RTT alone, so the timer stays reasonably tight on a stable path and automatically widens on a noisy one.

Structured elaboration
The standard algorithm (RFC 6298) updates two running estimates on every new RTT sample:
$\text{RTTVAR} = (1-\beta)\cdot \text{RTTVAR} + \beta \cdot |\text{SRTT} - \text{RTT}_{\text{sample}}|$
$\text{SRTT} = (1-\alpha)\cdot \text{SRTT} + \alpha \cdot \text{RTT}_{\text{sample}}$
with the standard constants $\alpha = 1/8$ and $\beta = 1/4$. The retransmission timeout itself is then:
$\text{RTO} = \text{SRTT} + \max(\text{clock granularity}, 4 \cdot \text{RTTVAR})$
The intuition: if round-trip times are steady (RTTVAR is small), RTO sits close to the smoothed RTT, so genuine loss is detected quickly. If round-trip times are jittery (RTTVAR is large, common on congested or highly variable paths), RTO widens automatically, so ordinary jitter isn't mistaken for loss and doesn't trigger a storm of unnecessary retransmissions.

Retransmissions triggered purely by the RTO timer firing (as opposed to fast retransmit via duplicate ACKs) are treated as a MUCH stronger signal of serious congestion, since it means not even a single later segment's ACK arrived to trigger a duplicate-ACK-based fast retransmit; the response is to collapse all the way back to slow start, not the gentler fast-recovery halving.

Worked example
Suppose a connection's SRTT has settled around 50ms with RTTVAR around 10ms. RTO would be approximately $50 + 4\times10 = 90$ ms. If a burst of network jitter pushes several samples up to 120ms, RTTVAR grows to reflect that variability (say to 25ms), and RTO widens to roughly $70 + 4\times25 = 170$ ms (SRTT itself also shifts upward, more slowly, toward the new samples). This is why an environment full of many short-lived connections (each starting from scratch with no RTT history) behaves differently from one with few long-lived connections (which have had time to build a stable, well-calibrated SRTT/RTTVAR estimate): short-lived connections are stuck using a generic initial RTO (commonly 1 second per RFC 6298) until they've collected enough samples to calibrate, making them systematically slower to detect a REAL loss early in their life.

Trade-offs & pitfalls
A common mistake is assuming a single fixed RTO value would be simpler and just as effective; a fixed timeout either fires too eagerly on a naturally variable path (spurious retransmissions that waste bandwidth and can trigger unnecessary congestion-window collapses) or too slowly on a stable path (wasted time before a real loss is detected), the adaptive SRTT/RTTVAR scheme exists specifically to avoid both failure modes at once.