Networking Fundamentals and Protocols Questions

Direct answer
The end-to-end principle says that a function like reliability, error checking, or encryption should generally be implemented at the ENDPOINTS of a communication, not in the network in between, because only the endpoints have enough context to do it completely and correctly; anything the network attempts to do on the endpoints' behalf is, at best, redundant, and often incomplete.

Structured elaboration
The classic argument: even if a network device implements reliable delivery for its OWN hop (say, a link-layer retransmission scheme), the endpoints STILL need their own end-to-end reliability check, because failures can occur anywhere along the full path, including at the endpoints themselves (a corrupted disk write, an application bug), that no single intermediate hop's reliability mechanism can catch. Since the endpoints need to implement the full check anyway to cover the whole path, the intermediate hop's partial version becomes pure extra cost (complexity, latency, resource use) with no corresponding gain in actual end-to-end correctness. This is exactly the reasoning behind TCP's own design: reliability (retransmission, checksums) lives at the TRANSPORT layer, running on the two endpoints, not distributed piecemeal across every router the packet crosses.

Worked example
A case where following the end-to-end principle strictly is clearly the right call: end-to-end encryption. If confidentiality were instead implemented hop-by-hop (each link encrypting its own segment separately, decrypting and re-encrypting at every intermediate device), every single intermediate device becomes a point where the data is available in plaintext, and a single compromised or misconfigured hop breaks confidentiality for the WHOLE path. Only the endpoints encrypting directly to each other, with intermediate devices never possessing the ability to decrypt at all, gives a security guarantee that doesn't depend on trusting every device along the way.

A case where placing a function in an INTERMEDIATE device is justified, despite the end-to-end principle's default preference: a link with an unusually high, characteristic error rate (some wireless or satellite links) benefits from LOCAL link-layer retransmission on just that one hop, because retransmitting a single lost bit-pattern on the actual lossy hop is far cheaper (both in latency and in bandwidth) than always waiting for a full end-to-end retransmission across the ENTIRE path whenever that one link drops something. This doesn't replace the endpoints' own end-to-end mechanism (which must still exist to catch failures anywhere else along the path); it's a legitimate LOCAL optimization layered underneath it, not a substitute for it.

Trade-offs & pitfalls
The end-to-end principle is a strong DEFAULT, not an absolute law; the mistake is either applying it dogmatically (refusing any intermediate optimization, even ones that provide a real, complementary performance benefit on a specific problematic hop) or abandoning it too readily (letting the network take over a correctness-critical function like encryption or reliability entirely, on the mistaken assumption that "the network already handles that").

Direct answer
The UDP header is deliberately minimal, just four fields: source port, destination port, length, and checksum, and it provides no reliability, no ordering, and no flow or congestion control at all. The checksum is optional over IPv4 (it can be all-zeros to mean "not computed") but MANDATORY over IPv6, since IPv6 dropped the network-layer checksum that IPv4 had, leaving UDP's checksum as the only integrity check left covering the payload for that traffic.

Structured elaboration

• Source port (16 bits): the sending application's port, allowing a reply to be addressed back to the right process; can legitimately be zero if no reply is expected.
• Destination port (16 bits): identifies which application on the receiving host should get the datagram.
• Length (16 bits): the total length of the UDP header plus payload, in bytes, this is how a receiver knows where the datagram actually ends (UDP has no separate "end of message" marker otherwise).
• Checksum (16 bits): a checksum computed over a pseudo-header (which includes the source/destination IP addresses, borrowed conceptually from the IP layer to catch certain misdelivery errors) plus the UDP header and payload.

Over IPv4, the sender is technically permitted to skip computing the checksum entirely, since IPv4 packets already carry a header checksum which catches SOME corruption, though notably NOT payload corruption. Over IPv6, sending a UDP checksum is mandatory, precisely because IPv6 has no header checksum of its own at all, so UDP's checksum became the last line of defense for detecting corruption anywhere in the packet.

Worked example
If corrupted UDP payloads are reaching an application in production despite the checksum being enabled, that's actually a meaningful signal about WHERE the corruption is happening: a valid checksum plus corrupted payload data can only mean the corruption happened AFTER the checksum was computed and BEFORE the packet was actually transmitted onto the wire (for instance, in host memory, in a buggy driver, or in hardware), or that checksum offloading to the NIC is misconfigured or buggy (many NICs compute the checksum in hardware rather than the OS, and a broken offload implementation can silently produce or accept bad checksums). It would NOT typically indicate ordinary in-transit bit-flip corruption, since that's exactly the class of error the checksum exists to catch and reject.

Trade-offs & pitfalls
The 16-bit checksum, while better than nothing, is not cryptographically strong and won't catch every possible corruption pattern, especially certain kinds of systematic bit errors; applications with strict data-integrity requirements over UDP (like some real-time media or gaming protocols) often layer their own additional integrity or authentication checks on top rather than relying on the UDP checksum alone.

Direct answer
Over the life of a connection, TCP's congestion window grows exponentially in slow start, switches to growing linearly in congestion avoidance once it approaches a known safe ceiling, and reacts to loss with fast retransmit and fast recovery rather than always restarting from scratch.

Structured elaboration

1. Slow start: the connection begins with a small congestion window (historically 1 segment; modern stacks start higher, commonly around 10 segments per RFC 6928) and roughly DOUBLES the window every round trip, since each of the ACKs for the previous batch triggers sending two new segments. This continues until either loss occurs, or the window reaches a threshold called ssthresh (slow start threshold), at which point the sender switches strategies.
2. Congestion avoidance: once at or above ssthresh, growth switches from exponential to roughly linear (classically, additive increase of about one segment per round trip), a much more cautious probe for additional capacity.
3. Fast retransmit: if the sender sees three duplicate ACKs (the receiver repeatedly acknowledging the same byte, implying a specific segment is missing but LATER data did arrive), it retransmits the missing segment immediately, without waiting for the retransmission timer to expire, since three duplicate ACKs is strong, specific evidence of loss rather than simple reordering.
4. Fast recovery: after a fast retransmit, rather than collapsing all the way back to slow start, ssthresh is set to about half the current window, and the window itself is set near that halved value, so the sender doesn't have to re-earn all its previous progress from a window of one segment; it resumes near where it estimates the path can actually sustain.

Worked example
Picture a connection whose window has grown to 64 segments in flight when a single segment is lost and detected via three duplicate ACKs (not a full timeout). Fast retransmit resends the missing segment immediately. Fast recovery sets ssthresh to roughly 32 (half of 64) and the window to near that value, then resumes congestion avoidance's linear growth from there, rather than collapsing to slow start's small initial window and doubling all the way back up. Contrast this with a RETRANSMISSION TIMEOUT (no duplicate ACKs arrived at all, meaning the loss was severe enough that the whole flight of data went missing): that's a much stronger loss signal, and the sender resets ssthresh to half the current window but drops the actual window all the way back to slow start's minimum, since a timeout implies the path may be far more broken than a few duplicate ACKs would suggest.

Trade-offs & pitfalls
It's a common mistake to say TCP always halves its window on any loss and moves on; a full retransmission timeout is treated far more conservatively (full reset to slow start) than a fast-retransmit-detected loss (a much gentler recovery), because the ABSENCE of any duplicate ACKs at all is itself informative: it suggests either a much larger loss event or a badly congested/broken path, not just one unlucky dropped segment.

Direct answer
An ICMP Type 3 Code 4 message ("Fragmentation Needed and Don't Fragment was Set") is a router along the path telling the sender its packet was too large for the next hop's MTU (Maximum Transmission Unit, here 1400 bytes) and, because the Don't Fragment (DF) bit was set, the router dropped it rather than fragmenting it, expecting the sender to resend at a smaller size. Path MTU Discovery commonly fails in production because something along the path (often a firewall with an overly broad "block all ICMP" rule) discards that very ICMP message before it reaches the sender, so the sender never learns to shrink its packets and its large packets just keep silently disappearing.

Structured elaboration
The DF bit tells every router along the path "do not fragment this packet under any circumstances, if it doesn't fit, drop it and tell me why." Path MTU Discovery relies entirely on that "tell me why" part actually reaching the sender: the sender starts by assuming its LOCAL interface's MTU is usable end-to-end, sends with DF set, and if a router along the path can't forward it at that size, the router sends back exactly this ICMP message, reporting the smaller MTU it needs (1400 bytes here). The sender is then supposed to shrink its packet size to that reported value and retry.

The reason PMTUD commonly fails in production: many firewalls and security appliances, misconfigured to block ALL ICMP as a blanket "security" measure, silently discard the "Fragmentation Needed" message on its way back to the sender. The sender then never learns it needs to shrink its packets, keeps sending at the original (too-large) size with DF still set, and those packets keep getting silently dropped at the same router, forever, with no error ever surfacing to the sender, the classic "large transfers hang, small transfers succeed" symptom (small packets happen to fit under the constrained MTU and sail through fine, while anything larger vanishes without explanation).

Worked example
To reconstruct what happened from the capture: the fragmented IPv4 packets observed likely represent an EARLIER part of the same flow that happened to still get through (perhaps fragmented by an intermediate device before DF took full effect, or from a portion of traffic that didn't have DF set), while the ICMP message with next-hop MTU 1400 is the router's report on a LATER, DF-set packet it could not forward. To fix this for BOTH TCP and UDP traffic: for TCP, the most common resilient fix is MSS clamping on a network device at the edge (rewriting the TCP Maximum Segment Size option in SYN packets passing through, so TCP negotiates a small-enough segment size up front and the oversized-packet problem never occurs at all, PMTUD independent); for UDP, since there's no equivalent MSS negotiation, the application itself must either send appropriately small datagrams from the start or correctly handle PMTUD feedback (which requires NOT blocking the relevant ICMP messages on the path, the actual root-cause fix). In both cases, the truly correct long-term fix is ensuring ICMP "Fragmentation Needed"/"Packet Too Big" messages are explicitly PERMITTED through every firewall along the path, rather than working around their absence.

Trade-offs & pitfalls
MSS clamping is a pragmatic, widely-used workaround specifically because it doesn't depend on ICMP getting through at all, but it only helps TCP; it does nothing for UDP traffic hitting the exact same oversized-packet problem, which is why "block all ICMP" as a firewall policy is a genuinely bad default rather than a harmless-looking hardening step, it breaks a real, load-bearing part of how IP networking is supposed to self-correct.

Direct answer
Reno, NewReno, Cubic, and BBR represent an evolution in how TCP infers and reacts to congestion: the Reno family reacts to LOSS with a fixed halving of the window, Cubic grows more aggressively on high-bandwidth links using a cubic function of time since the last loss, and BBR abandons loss as the primary signal entirely, instead modeling the path's actual bandwidth and round-trip time directly.

Structured elaboration

• Reno: the classical algorithm. Slow start, congestion avoidance with linear (additive) growth, and on ANY loss, halves the window and re-enters a conservative recovery. Its big limitation on high-bandwidth-delay-product links is that halving the window after a single loss throws away a huge amount of earned capacity, and the subsequent linear regrowth takes a long time to recover it.
• NewReno: a refinement that fixes a specific weakness in Reno's fast recovery when MULTIPLE segments are lost within one window; Reno's original recovery logic could exit fast recovery prematurely and fall back to a slow, timeout-driven recovery for the second lost segment, while NewReno correctly stays in fast recovery until ALL the losses from that window are repaired.
• Cubic (the default on Linux for a long time): grows the window as a cubic function of the time elapsed since the last loss event, growing very slowly right after backing off, then accelerating, then leveling off as it approaches the window size where the last loss occurred, and probing gently past it. This makes Cubic much better at fully utilizing high-bandwidth, high-latency ("long fat") links than Reno's linear growth, since it isn't purely tied to round-trip-time-limited additive increase.
• BBR (Bottleneck Bandwidth and Round-trip propagation time): rather than reacting to loss at all, BBR periodically probes to directly estimate the bottleneck link's bandwidth and the path's minimum round-trip time, then paces its sending rate to match that estimate. This lets it largely ignore ordinary, non-congestive packet loss (which loss-based algorithms mistake for congestion), a real advantage on paths where a small amount of loss is normal and NOT actually a congestion signal (satellite links, some wireless links, or lossy long-haul fiber).

Worked example
For a large file transfer over a satellite link, characterized by very high round-trip time (often 500ms+) and some baseline non-congestive loss (a normal characteristic of the medium, not a sign of an overloaded path), BBR is generally the stronger choice: a loss-based algorithm like Cubic will repeatedly (and wrongly) interpret that baseline loss as congestion and needlessly shrink its window, capping throughput well below what the link can actually sustain, while BBR's bandwidth-and-RTT model isn't fooled by loss that isn't actually caused by queue buildup.

Trade-offs & pitfalls
BBR isn't a universal win: on a link SHARED with loss-based flows (Cubic, Reno), BBR's willingness to keep sending through non-congestive loss can let it grab a disproportionate share of a congested bottleneck's capacity from more conservative Reno/Cubic flows sharing that same link, an active area of real-world congestion-control fairness research, not a settled solved problem.