Networking Fundamentals and Protocols Questions

Direct answer
The bandwidth-delay product (BDP) is the amount of data that can be "in flight" on a link at any instant, bandwidth multiplied by round-trip time, and it's the minimum window size TCP needs to keep the link fully utilized. On a 10 Gbps link with 150ms round-trip time, the BDP is large enough that the original 16-bit TCP window field (max 65,535 bytes) can represent only a tiny fraction of it, which is exactly why window scaling exists.

Structured elaboration
The BDP calculation is:
$\text{BDP (bits)} = \text{bandwidth (bits/s)} \times \text{RTT (s)}$
For a 10 Gbps link ($10 \times 10^9$ bits/s) with a 150ms ($0.150$ s) round-trip time:
$\text{BDP} = 10 \times 10^9 \times 0.150 = 1.5 \times 10^9 \text{ bits} = 187{,}500{,}000 \text{ bytes} \approx 187.5\text{ MB}$
The plain TCP window field is 16 bits, so the largest window it can express without scaling is $2^{16}-1 = 65{,}535$ bytes, about 0.035% of the 187.5 MB needed. Without more window than that, the sender would have to stop and wait for an ACK every 64KB, and at this RTT that caps throughput far below the link's actual 10 Gbps capacity, no matter how fast the link itself is.

The window scale option (negotiated ONLY during the handshake, in the SYN and SYN-ACK) adds a scale factor $S$ (0 to 14) that the receiver applies to its advertised window: the real window becomes $\text{advertised value} \times 2^S$. To cover a 187.5 MB requirement, we need the smallest $S$ such that $65{,}535 \times 2^S \geq 187{,}500{,}000$, which comes out to $S=12$ (giving a maximum representable window of $65{,}535 \times 4096 = 268{,}431{,}360$ bytes, comfortably above the 187.5 MB requirement).

Worked example
For a smaller, more common case, a 100 Mbps link with 100ms RTT, the BDP is:
$100 \times 10^6 \times 0.100 = 1 \times 10^7 \text{ bits} = 1{,}250{,}000 \text{ bytes} \approx 1.25\text{ MB}$
Here the minimum scale factor needed is $S=5$ (giving a max window of $65{,}535 \times 32 = 2{,}097{,}120$ bytes), a much smaller ask than the 10 Gbps case, illustrating that window scaling matters more the higher the bandwidth-delay product climbs, not RTT or bandwidth alone. Beyond window sizing, actually reaching close to line rate on a link like this also typically needs the OS socket buffers (net.ipv4.tcp_rmem/tcp_wmem on Linux) raised to match the negotiated window (a window the OS hasn't allocated buffer space for is wasted), and Selective Acknowledgment (SACK) enabled so a single lost segment somewhere in a large in-flight window doesn't force retransmission of everything after it.

Trade-offs & pitfalls
Window scaling is negotiated ONLY at connection setup; if either endpoint doesn't advertise the option in its SYN, the connection falls back to the un-scaled 64KB ceiling for its entire lifetime, a frequent, hard-to-spot cause of "high-bandwidth link, mysteriously capped throughput" that shows up when an old middlebox strips the option or a misconfigured host has scaling disabled.

Direct answer
An ICMP Type 3 Code 4 message ("Fragmentation Needed and Don't Fragment was Set") is a router along the path telling the sender its packet was too large for the next hop's MTU (Maximum Transmission Unit, here 1400 bytes) and, because the Don't Fragment (DF) bit was set, the router dropped it rather than fragmenting it, expecting the sender to resend at a smaller size. Path MTU Discovery commonly fails in production because something along the path (often a firewall with an overly broad "block all ICMP" rule) discards that very ICMP message before it reaches the sender, so the sender never learns to shrink its packets and its large packets just keep silently disappearing.

Structured elaboration
The DF bit tells every router along the path "do not fragment this packet under any circumstances, if it doesn't fit, drop it and tell me why." Path MTU Discovery relies entirely on that "tell me why" part actually reaching the sender: the sender starts by assuming its LOCAL interface's MTU is usable end-to-end, sends with DF set, and if a router along the path can't forward it at that size, the router sends back exactly this ICMP message, reporting the smaller MTU it needs (1400 bytes here). The sender is then supposed to shrink its packet size to that reported value and retry.

The reason PMTUD commonly fails in production: many firewalls and security appliances, misconfigured to block ALL ICMP as a blanket "security" measure, silently discard the "Fragmentation Needed" message on its way back to the sender. The sender then never learns it needs to shrink its packets, keeps sending at the original (too-large) size with DF still set, and those packets keep getting silently dropped at the same router, forever, with no error ever surfacing to the sender, the classic "large transfers hang, small transfers succeed" symptom (small packets happen to fit under the constrained MTU and sail through fine, while anything larger vanishes without explanation).

Worked example
To reconstruct what happened from the capture: the fragmented IPv4 packets observed likely represent an EARLIER part of the same flow that happened to still get through (perhaps fragmented by an intermediate device before DF took full effect, or from a portion of traffic that didn't have DF set), while the ICMP message with next-hop MTU 1400 is the router's report on a LATER, DF-set packet it could not forward. To fix this for BOTH TCP and UDP traffic: for TCP, the most common resilient fix is MSS clamping on a network device at the edge (rewriting the TCP Maximum Segment Size option in SYN packets passing through, so TCP negotiates a small-enough segment size up front and the oversized-packet problem never occurs at all, PMTUD independent); for UDP, since there's no equivalent MSS negotiation, the application itself must either send appropriately small datagrams from the start or correctly handle PMTUD feedback (which requires NOT blocking the relevant ICMP messages on the path, the actual root-cause fix). In both cases, the truly correct long-term fix is ensuring ICMP "Fragmentation Needed"/"Packet Too Big" messages are explicitly PERMITTED through every firewall along the path, rather than working around their absence.

Trade-offs & pitfalls
MSS clamping is a pragmatic, widely-used workaround specifically because it doesn't depend on ICMP getting through at all, but it only helps TCP; it does nothing for UDP traffic hitting the exact same oversized-packet problem, which is why "block all ICMP" as a firewall policy is a genuinely bad default rather than a harmless-looking hardening step, it breaks a real, load-bearing part of how IP networking is supposed to self-correct.

Direct answer
When a client keeps retransmitting its SYN and never sends the final ACK (while the server's SYN-ACK is only retransmitted once before the connection times out), the most likely causes are that the client's connect timeout hasn't fired yet, that something between the two hosts is dropping the ACK specifically (an asymmetric path or a stateful device confused about direction), or that the client-side application itself never actually attempted the ACK due to a bug. A different failure shape, the server sends SYN-ACK and the client immediately sends RST, points to a different family of causes entirely: the client rejecting the connection outright.

Structured elaboration
For the "client retransmits SYN, no final ACK" pattern, work through causes in order of likelihood:

1. Asymmetric routing dropping only the return-to-forward-direction ACK path. If the SYN-ACK reaches the client (we know it does, since the client keeps retransmitting new SYNs rather than giving up, meaning it IS getting a response of some kind) but the client's ACK can't get back to the server along some other path, a stateful firewall or NAT device on that asymmetric path may be dropping the ACK because it doesn't recognize the connection's state in that direction.
2. A middlebox is rewriting or dropping specific TCP options in the SYN-ACK that the client's stack doesn't handle gracefully, causing it to silently discard the SYN-ACK and retry instead of ACKing it.
3. Client-side firewall or security policy is specifically blocking outbound ACKs to that destination while permitting the outbound SYNs, an unusual but real misconfiguration.
4. MTU (Maximum Transmission Unit)-related silent packet loss on the SYN-ACK's return path if it happens to be an unusually large segment (rare for a SYN-ACK specifically, since it typically carries little payload, but worth ruling out if other symptoms point that way).

For the different shape (SYN-ACK followed by an IMMEDIATE client RST): this usually means the client-side application decided, upon establishing the connection, that it doesn't actually want it, for example an application-level timeout that already expired while the handshake was in flight, a client-side connection pool that raced two connection attempts and is aborting the loser, or a security tool on the client actively resetting connections that don't match an expected certificate or policy.

Worked example
To distinguish these hypotheses in practice, compare timestamps and evidence on BOTH ends: if the server's capture shows the SYN-ACK leaving on time but the client's capture never shows it arriving, the problem is in the path (asymmetric routing, a device eating it). If the client's capture shows the SYN-ACK arriving cleanly but no ACK is ever generated by the client's own stack, the bug is on the client host itself (application logic, local firewall) rather than the network path.

Trade-offs & pitfalls
A common mistake is assuming a stuck handshake is always a network problem; a client-side timeout race (the application gives up right as the handshake completes) produces an outwardly identical-looking symptom to a network drop and is only distinguishable by comparing what each side's own capture actually shows, not by reasoning about the network path alone.

Direct answer
TCP is connection-oriented and guarantees reliable, in-order delivery with built-in flow and congestion control, at the cost of handshake setup latency and head-of-line blocking; UDP is connectionless, with no delivery guarantees, ordering, or congestion control, trading reliability for minimal overhead and lower latency. The right choice depends on whether the application can tolerate loss and reordering itself, or needs the transport layer to handle it.

Structured elaboration

Two examples per protocol: TCP is the right choice for a database connection or a file transfer, where losing or reordering even one byte silently would corrupt the result, and the application has no interest in reimplementing reliability itself. UDP is the right choice for live video/voice calls or DNS queries, where a single lost or late packet is better DISCARDED and moved past (an old, out-of-order audio frame is useless once its playback moment has passed) than retransmitted at the cost of added latency that would make the whole stream feel laggy.

Worked example
A scenario where building custom reliability ON TOP of UDP beats plain TCP: a real-time multiplayer game sending frequent position updates. TCP's strict in-order delivery means a single lost packet blocks EVERY later packet from being delivered to the application until the lost one is retransmitted and received (head-of-line blocking), even if those later packets contain fresher, more relevant position data. Building a thin reliability layer over UDP lets the application decide per-message whether it's worth retransmitting (a player's current position, three updates old, usually isn't worth retransmitting, a newer update has probably already superseded it) rather than being forced into strict in-order delivery for data where "newest wins" matters more than "nothing lost."

Trade-offs & pitfalls
A common mistake is treating "TCP is reliable, UDP isn't" as the END of the analysis; the real question is whether your application's OWN definition of correctness matches TCP's specific guarantees (strict ordering, full reliability) or would be better served by a custom scheme that's more permissive in exactly the ways TCP is rigid. Building your own reliability on UDP is real engineering work (implementing retransmission, sequencing, and congestion awareness yourself), not a shortcut, it's justified specifically when TCP's guarantees don't match what the application actually needs.

Direct answer
IPv4 fragmentation splits a packet too large for a link into smaller pieces, each carrying enough header information (Identification, Flags, Fragment Offset) for the destination to reassemble the original packet, and it can fail in several distinct ways, a missing fragment, overlapping fragments, or a reassembly timeout, any of which prevents the original packet from ever being reconstructed.

Structured elaboration
Three IPv4 header fields work together to make fragmentation and reassembly possible:

• Identification: a 16-bit value the ORIGINAL packet is stamped with; every fragment of that same original packet carries the SAME Identification value, so the receiver knows which fragments belong together.
• Flags: includes the "Don't Fragment" (DF) bit, which tells routers along the path NOT to fragment this packet (instead, if it's too large for the next link, they must drop it and report the problem back via Internet Control Message Protocol, ICMP), and the "More Fragments" (MF) bit, set on every fragment except the LAST one, telling the receiver more pieces are still coming.
• Fragment Offset: a 13-bit field (in units of 8 bytes) indicating WHERE in the original packet's payload this particular fragment's data belongs, letting the receiver reassemble fragments that might arrive out of order.

The receiver buffers incoming fragments sharing the same Identification (and same source/destination address pair and protocol) until either all fragments have arrived (the last one identified by MF=0) and can be stitched back together in Fragment-Offset order, or a reassembly timer expires first.

Worked example
If a fragment carrying offset 0-1000 and another carrying offset 1000-2000 arrive, but the fragment that should have covered 2000-2500 (the actual final piece, with MF=0) never arrives, perhaps dropped somewhere along the path, the receiver has no way to know reassembly is even complete; it will hold the two partial fragments in its reassembly buffer until its reassembly timeout (commonly around 30 to 60 seconds depending on the OS) expires, then discard them and the entire original packet is lost, even though 2 of its 3 pieces successfully arrived. Overlapping fragments (two fragments both claiming to cover the SAME offset range, but with different content) are treated with suspicion by modern stacks specifically because that pattern was historically exploited in fragmentation-based attacks (evading firewall inspection by presenting different content to the firewall's re-assembly logic than to the actual destination host's).

Trade-offs & pitfalls
Fragmentation is something you generally want to AVOID rather than rely on: it multiplies the chance of a single original packet failing to arrive (since ALL its fragments must arrive for it to be usable at all, a single lost fragment dooms the whole packet even if every other fragment made it), it's more CPU-expensive for receivers to reassemble than to simply process one appropriately-sized packet, and many network devices (especially older or security-focused ones) handle fragments inconsistently or drop them outright, treating fragmentation itself as suspicious traffic. This is exactly why Path MTU (Maximum Transmission Unit) Discovery exists: to let a sender learn the right packet size UP FRONT and avoid needing fragmentation at all.