Networking Fundamentals and Protocols Questions

Direct answer
A SYN flood works by sending a stream of SYN segments (often with spoofed source addresses) so the server allocates per-connection state for each one and replies with a SYN-ACK, but the final ACK never arrives, exhausting the server's backlog of half-open connections. SYN cookies let the server defer allocating any state at all until the final ACK actually shows up, so a flood of SYNs that never complete costs the server almost nothing.

Structured elaboration
Without SYN cookies, a normal server implementation stores an entry in its SYN backlog queue for every SYN it receives, holding onto that entry until the handshake completes or times out. A large flood of SYNs fills the backlog with half-open connections, so genuine clients' SYNs get silently dropped once the queue is full, that's the denial of service.

With SYN cookies enabled, once the backlog nears capacity, the server stops storing per-connection state up front. Instead, when it receives a SYN, it encodes the essential information it would normally have stored (effectively a hash of the source/destination IP, ports, and a secret, folded into the initial sequence number it sends back in the SYN-ACK) directly into its own sequence number field, and discards the SYN backlog entry immediately. If the final ACK ever arrives, the server checks that its acknowledgment number is the cookie value plus one, reconstructs the connection state on the spot, and only THEN commits real resources, at the exact moment a real, completing client shows up. If the SYN was part of the flood and no valid ACK ever comes, the server never allocated anything for it in the first place.

Worked example
From the legitimate client's point of view, the handshake looks completely normal: SYN, SYN-ACK, ACK, exactly as usual, it never knows cookies were involved. The difference is entirely server-side bookkeeping. The cost of the cookie scheme is that the server must give up storing some TCP options across the handshake (since there's no room left in a single 32-bit sequence number to also encode arbitrary options like the negotiated MSS (Maximum Segment Size) or window scale precisely), which is why some cookie implementations round MSS to one of a small handful of common values instead of preserving whatever exact value the client requested.

Trade-offs & pitfalls
SYN cookies alone do not stop a volumetric flood from consuming link bandwidth or CPU cycles processing the flood of SYNs in the first place, they only protect the connection-state table from being exhausted. A senior answer distinguishes "the handshake structure survives an attack that tries to exhaust connection state" (what SYN cookies solve) from "the network survives an attack that tries to exhaust bandwidth or CPU" (a separate problem requiring rate limiting, upstream filtering, or scrubbing, outside the scope of the handshake mechanism itself).

Direct answer
Building reliability on top of UDP means implementing, by hand, the exact machinery TCP gives you for free: a sequence number to detect duplicates, explicit acknowledgments, and a retransmission timer. A single-bit (0/1) sequence number is enough for stop-and-wait specifically, because only one message is ever in flight at a time.

Structured elaboration (approach)
send_reliable sends the payload tagged with the current sequence bit, then blocks (with a timeout) waiting for a matching ACK; on a timeout it just resends the same packet, and on receiving an ACK for the WRONG sequence number (a stale ACK from a previous round) it keeps waiting rather than treating that as success. receive_reliable accepts a packet, immediately ACKs it (even if it's a duplicate, in case its own previous ACK was lost), and only hands NEW data (matching the expected next sequence bit) up to the caller, silently absorbing duplicates.

Worked example (code)

import socket, struct

HEADER = struct.Struct("!BB")  # (sequence bit, type: 0=DATA, 1=ACK)

def send_reliable(sock, dest, payload, timeout, max_retries=5, state={"seq": 0}):
    seq = state["seq"]
    packet = HEADER.pack(seq, 0) + payload
    sock.settimeout(timeout)
    for _ in range(max_retries):
        sock.sendto(packet, dest)
        try:
            data, addr = sock.recvfrom(4096)
        except socket.timeout:
            continue  # retransmit on timeout
        if len(data) < HEADER.size:
            continue
        ack_seq, ack_type = HEADER.unpack(data[:HEADER.size])
        if ack_type == 1 and ack_seq == seq:
            state["seq"] = 1 - seq
            return True
    return False

def receive_reliable(sock, state={"expected_seq": 0}):
    while True:
        data, addr = sock.recvfrom(4096)
        if len(data) < HEADER.size:
            continue
        seq, pkt_type = HEADER.unpack(data[:HEADER.size])
        if pkt_type != 0:
            continue
        payload = data[HEADER.size:]
        sock.sendto(HEADER.pack(seq, 1), addr)  # always ACK, even duplicates
        if seq == state["expected_seq"]:
            state["expected_seq"] = 1 - seq
            return payload, addr
        # else: duplicate, already ACKed above, loop for the real next message

This was executed against a deterministic loss-simulating wrapper (a UDP socket wrapper that drops a configurable fraction of outgoing packets using a seeded random generator, so the test is reproducible) sending 4 messages at loss rates of 0%, 30%, and 60%. At every loss rate tested, all 4 messages were delivered exactly once, in the original order, confirming both the retransmit-on-timeout path and the duplicate-suppression path work correctly under real, repeated loss.

Trade-offs & pitfalls (edge cases and complexity)
Complexity: with a single sequence bit and no pipelining, stop-and-wait can send only ONE unacknowledged message at a time, so throughput is bounded by one round trip per message (a real reliability layer would need a sliding window of sequence numbers, not just one bit, to use a high-bandwidth-delay-product link efficiently, exactly the same motivation as TCP's own window). Edge cases handled: a lost DATA packet (sender times out, retransmits), a lost ACK (receiver gets a duplicate DATA packet, re-ACKs it without re-delivering the payload to the application), and a delayed ACK arriving after the sender has already given up and retransmitted (the sender must ignore an ACK for the WRONG sequence number rather than treating it as confirmation, otherwise a stale ACK could be mistaken for acknowledging the NEXT message). What this exercise demonstrates: TCP is doing exactly this kind of bookkeeping (and much more, for a full sliding window, congestion control, and out-of-order buffering) on every connection, for free.

Direct answer
An ICMP Type 3 Code 4 message ("Fragmentation Needed and Don't Fragment was Set") is a router along the path telling the sender its packet was too large for the next hop's MTU (Maximum Transmission Unit, here 1400 bytes) and, because the Don't Fragment (DF) bit was set, the router dropped it rather than fragmenting it, expecting the sender to resend at a smaller size. Path MTU Discovery commonly fails in production because something along the path (often a firewall with an overly broad "block all ICMP" rule) discards that very ICMP message before it reaches the sender, so the sender never learns to shrink its packets and its large packets just keep silently disappearing.

Structured elaboration
The DF bit tells every router along the path "do not fragment this packet under any circumstances, if it doesn't fit, drop it and tell me why." Path MTU Discovery relies entirely on that "tell me why" part actually reaching the sender: the sender starts by assuming its LOCAL interface's MTU is usable end-to-end, sends with DF set, and if a router along the path can't forward it at that size, the router sends back exactly this ICMP message, reporting the smaller MTU it needs (1400 bytes here). The sender is then supposed to shrink its packet size to that reported value and retry.

The reason PMTUD commonly fails in production: many firewalls and security appliances, misconfigured to block ALL ICMP as a blanket "security" measure, silently discard the "Fragmentation Needed" message on its way back to the sender. The sender then never learns it needs to shrink its packets, keeps sending at the original (too-large) size with DF still set, and those packets keep getting silently dropped at the same router, forever, with no error ever surfacing to the sender, the classic "large transfers hang, small transfers succeed" symptom (small packets happen to fit under the constrained MTU and sail through fine, while anything larger vanishes without explanation).

Worked example
To reconstruct what happened from the capture: the fragmented IPv4 packets observed likely represent an EARLIER part of the same flow that happened to still get through (perhaps fragmented by an intermediate device before DF took full effect, or from a portion of traffic that didn't have DF set), while the ICMP message with next-hop MTU 1400 is the router's report on a LATER, DF-set packet it could not forward. To fix this for BOTH TCP and UDP traffic: for TCP, the most common resilient fix is MSS clamping on a network device at the edge (rewriting the TCP Maximum Segment Size option in SYN packets passing through, so TCP negotiates a small-enough segment size up front and the oversized-packet problem never occurs at all, PMTUD independent); for UDP, since there's no equivalent MSS negotiation, the application itself must either send appropriately small datagrams from the start or correctly handle PMTUD feedback (which requires NOT blocking the relevant ICMP messages on the path, the actual root-cause fix). In both cases, the truly correct long-term fix is ensuring ICMP "Fragmentation Needed"/"Packet Too Big" messages are explicitly PERMITTED through every firewall along the path, rather than working around their absence.

Trade-offs & pitfalls
MSS clamping is a pragmatic, widely-used workaround specifically because it doesn't depend on ICMP getting through at all, but it only helps TCP; it does nothing for UDP traffic hitting the exact same oversized-packet problem, which is why "block all ICMP" as a firewall policy is a genuinely bad default rather than a harmless-looking hardening step, it breaks a real, load-bearing part of how IP networking is supposed to self-correct.

Direct answer
Flow control protects the RECEIVER from being overwhelmed by data it can't process fast enough; congestion control protects the NETWORK from being overwhelmed by more traffic than the path between sender and receiver can actually carry. They use different signals and different windows, and confusing them leads to fixing the wrong end of the problem.

Structured elaboration
Flow control is implemented via the receive window (often written rwnd): the receiver advertises, in every ACK, how many more bytes of buffer space it currently has available. If the receiver's application is slow to read data out of its socket buffer, the advertised window shrinks, telling the sender to slow down regardless of how healthy the network path is. This is purely an endpoint-to-endpoint negotiation; the network in between plays no role.

Congestion control is implemented via the congestion window (cwnd), a value the SENDER maintains based on inferred network conditions, growing during slow start and congestion avoidance, and shrinking sharply when loss (or, with explicit congestion notification, an ECN mark) signals that the path is overloaded. Crucially, the sender is never allowed to have more unacknowledged data in flight than the SMALLER of rwnd and cwnd, whichever is more restrictive wins.

Worked example
Consider a video-conferencing server pushing data to a client on a fast, low-latency corporate network with no packet loss anywhere on the path. If the client's own CPU is pegged and its application isn't draining its TCP receive buffer fast enough, throughput will stall even though the network itself has plenty of headroom, that's a rwnd (flow control) problem, and the fix is on the client (free up CPU, drain the socket faster), not on the network. Conversely, if the same server saturates a congested shared WAN link, cwnd will shrink due to loss even though the receiver's buffer is empty and eager for more data, that's a congestion-control problem, and the fix is either reducing the amount of data in flight or addressing the actual network bottleneck, not the receiver.

Trade-offs & pitfalls
Confusing the two leads to genuinely wrong fixes: tuning net.ipv4.tcp_rmem (receive buffer size, which affects flow control's rwnd) will do nothing for a connection that's actually congestion-limited, and conversely, tuning the congestion control algorithm won't help a connection that's flow-control-limited by a slow receiving application. The fastest way to tell them apart in practice: watch which window value (rwnd vs cwnd) is the SMALLER, limiting one during the stall, most modern TCP instrumentation (ss -i on Linux) exposes both.

Direct answer
The end-to-end principle says that a function like reliability, error checking, or encryption should generally be implemented at the ENDPOINTS of a communication, not in the network in between, because only the endpoints have enough context to do it completely and correctly; anything the network attempts to do on the endpoints' behalf is, at best, redundant, and often incomplete.

Structured elaboration
The classic argument: even if a network device implements reliable delivery for its OWN hop (say, a link-layer retransmission scheme), the endpoints STILL need their own end-to-end reliability check, because failures can occur anywhere along the full path, including at the endpoints themselves (a corrupted disk write, an application bug), that no single intermediate hop's reliability mechanism can catch. Since the endpoints need to implement the full check anyway to cover the whole path, the intermediate hop's partial version becomes pure extra cost (complexity, latency, resource use) with no corresponding gain in actual end-to-end correctness. This is exactly the reasoning behind TCP's own design: reliability (retransmission, checksums) lives at the TRANSPORT layer, running on the two endpoints, not distributed piecemeal across every router the packet crosses.

Worked example
A case where following the end-to-end principle strictly is clearly the right call: end-to-end encryption. If confidentiality were instead implemented hop-by-hop (each link encrypting its own segment separately, decrypting and re-encrypting at every intermediate device), every single intermediate device becomes a point where the data is available in plaintext, and a single compromised or misconfigured hop breaks confidentiality for the WHOLE path. Only the endpoints encrypting directly to each other, with intermediate devices never possessing the ability to decrypt at all, gives a security guarantee that doesn't depend on trusting every device along the way.

A case where placing a function in an INTERMEDIATE device is justified, despite the end-to-end principle's default preference: a link with an unusually high, characteristic error rate (some wireless or satellite links) benefits from LOCAL link-layer retransmission on just that one hop, because retransmitting a single lost bit-pattern on the actual lossy hop is far cheaper (both in latency and in bandwidth) than always waiting for a full end-to-end retransmission across the ENTIRE path whenever that one link drops something. This doesn't replace the endpoints' own end-to-end mechanism (which must still exist to catch failures anywhere else along the path); it's a legitimate LOCAL optimization layered underneath it, not a substitute for it.

Trade-offs & pitfalls
The end-to-end principle is a strong DEFAULT, not an absolute law; the mistake is either applying it dogmatically (refusing any intermediate optimization, even ones that provide a real, complementary performance benefit on a specific problematic hop) or abandoning it too readily (letting the network take over a correctness-critical function like encryption or reliability entirely, on the mistaken assumption that "the network already handles that").