Networking Fundamentals and Protocols Questions

Direct answer
Bandwidth is the link's raw theoretical capacity; throughput is what's actually achieved end-to-end, which is always less than or equal to bandwidth; and goodput is throughput minus any overhead that isn't USEFUL application data (protocol headers, retransmissions of data that eventually succeeds anyway). An application commonly sees less than the rated bandwidth for several genuinely distinct reasons, and telling them apart is the fast path to the right fix.

Structured elaboration
Four common, distinct reasons throughput or goodput falls short of bandwidth:

1. Retransmissions: any lost segment that has to be resent consumes bandwidth without contributing new useful data, so a lossy path shows lower goodput even at unchanged raw throughput.
2. Protocol overhead: every layer's headers (Ethernet, IP, TCP) consume bytes that count toward raw throughput but never toward the application's actual USEFUL payload, goodput specifically excludes them.
3. Congestion-window or flow-control limits: if the connection's congestion window or receive window is smaller than the path's bandwidth-delay product, the sender is idle waiting for ACKs rather than continuously filling the pipe, capping achieved throughput well below the link's rated bandwidth regardless of loss.
4. Application-level inefficiency: an application that reads/writes in small chunks, adds its own serialization overhead, or simply isn't pipelining requests efficiently can bottleneck well below what the transport layer itself is capable of moving.

Worked example
The fastest way to localize which of these applies to a specific slow transfer: check retransmission counters first (ss -i's retrans field), if they're elevated, loss/retransmission is a real contributor. If retransmissions are low but the window (cwnd/rwnd from the same tool) is small relative to the path's bandwidth-delay product, the connection is window-limited rather than loss-limited, and no amount of "fixing loss" will help. If BOTH retransmissions are low and the window is comfortably larger than the bandwidth-delay product, yet throughput is still poor, suspect application-level inefficiency (measure CPU usage and how the application is actually issuing reads/writes) rather than anything at the transport layer at all.

Trade-offs & pitfalls
The most common diagnostic mistake is assuming a throughput shortfall is automatically a "network problem" and reaching straight for network-level tuning; a genuinely application-bound bottleneck (small, unbuffered writes; serialized rather than pipelined requests) is common and invisible to any amount of TCP-level tuning, wasting real effort until the actual bottleneck is correctly localized first.

Direct answer
Encapsulation is the process of wrapping data in a new header (and sometimes trailer) as it moves down the protocol stack, and stripping those headers back off as it moves up the stack on the receiving side. Each layer only understands its own header; it treats everything handed to it by the layer above as an opaque payload.

Structured elaboration
Follow a browser's HTTP request to a web server, header by header, going down the sender's stack:

1. Application layer: the browser produces an HTTP request (method, path, headers, body).
2. Transport layer: TCP wraps that request in a TCP segment, adding a header with source port, destination port, sequence number, acknowledgment number, and flags. The whole HTTP request becomes the segment's payload.
3. Network layer: IP wraps the TCP segment in an IP packet, adding a header with the source IP address and destination IP address. The TCP segment becomes the packet's payload.
4. Data link layer: Ethernet wraps the IP packet in a frame, adding a header with the source MAC address and destination MAC address (and a trailer with a frame check sequence for error detection). The IP packet becomes the frame's payload.
5. Physical layer: the frame is converted to bits and transmitted as electrical, optical, or radio signals.

On the receiving side, decapsulation runs in exactly the reverse order: the physical layer recovers bits into a frame, the data link layer strips the Ethernet header/trailer and hands the IP packet up, the network layer strips the IP header and hands the TCP segment up, the transport layer strips the TCP header and hands the HTTP request up, and the application layer finally sees the original request.

Worked example
Concretely, by the time the original HTTP request bytes reach the wire, they are surrounded by four layers of header (ignoring the physical layer, which isn't a header at all): [Ethernet header][IP header][TCP header][HTTP request bytes][Ethernet trailer]. Each intermediate device on the path (a switch, a router) only needs to look at the headers relevant to its own layer: a switch reads the Ethernet header to decide which port to forward the frame out of; a router strips the Ethernet framing entirely, reads the IP header to decide the next hop, and re-wraps the same IP packet in a NEW Ethernet frame addressed to the next hop's MAC address. The TCP header and the HTTP payload never change as the packet crosses the network; only the Layer 2 framing gets rewritten at each hop.

Trade-offs & pitfalls
The most common confusion is expecting the MAC addresses in the Ethernet header to stay constant end-to-end, they don't. Only the IP addresses (Network layer) stay constant from source to destination; the MAC addresses (Data Link layer) change at every hop, because Ethernet framing is only meaningful on a single link, not across the whole path.

Direct answer
Selective Acknowledgment (SACK) lets a receiver tell the sender exactly which non-contiguous blocks of data it has ALREADY received, so after a loss the sender only has to retransmit the specific missing segment(s), not everything that came after it.

Structured elaboration
Without SACK, TCP uses cumulative acknowledgment: an ACK only confirms "I have received everything up through this byte, contiguously." If segment 3 of a 10-segment flight is lost but segments 4 through 10 all arrive fine, the receiver can only ACK up through the end of segment 2, it has no way to tell the sender "I actually already have 4 through 10, I'm just missing 3." A sender using only cumulative ACKs, upon detecting the loss, may end up retransmitting segments 3 through 10 (everything the receiver hasn't cumulatively acknowledged), even though 4 through 10 were never actually lost.

With SACK enabled (negotiated via a permitted option in the handshake, then carried on subsequent ACKs), the receiver's ACK can include SACK blocks, explicit ranges of sequence numbers it holds that are NOT contiguous with the main acknowledged run, in the example above, a SACK block spanning segments 4 through 10. Now the sender knows precisely that only segment 3 needs retransmitting.

Worked example
Say a sender has segments with sequence ranges [1000-1500), [1500-2000), [2000-2500), ... up to [4500-5000), and segment [2000-2500) is lost in transit while everything else arrives. Without SACK: the receiver's ACKs stay pinned at ack=2000 (the last contiguous byte received) even as segments up through 5000 keep arriving; the sender, upon detecting the loss (via duplicate ACKs all saying ack=2000), knows only that SOMETHING after 2000 needs resending and, in older/naive implementations, could resend everything from 2000 onward. With SACK: the same duplicate ACKs at ack=2000 now also carry a SACK block like sack=2500-5000, telling the sender explicitly that only the single segment [2000-2500) is actually missing, so it retransmits exactly that one segment and nothing else.

Trade-offs & pitfalls
SACK is most valuable on connections with a large amount of data in flight (a large window relative to segment size) and where losses are isolated rather than in a solid burst, since that's exactly the scenario where "retransmit everything after the gap" wastes the most bandwidth compared to "retransmit only the gap." On a connection with a tiny window, or where an entire flight is lost at once (nothing left to selectively acknowledge), SACK provides little advantage.

Direct answer
A graceful close is a symmetric exchange where each direction of the connection is shut down independently with its own FIN/ACK pair; an abrupt close (RST) tears down both directions immediately and without acknowledgment, discarding any data still in flight.

Structured elaboration
Graceful close, the normal case: the side that's done sending data sends a FIN. The other side ACKs it and moves to CLOSE_WAIT (it can still send its own data; only ONE direction has closed so far). When that side is also done, it sends its own FIN, the original side ACKs it, and both directions are now closed. Because it takes a FIN and an ACK in EACH direction, a full graceful close is normally four messages (though the middle ACK and FIN are frequently combined into one segment in practice), and the side that sent the last ACK enters TIME_WAIT.

Abrupt close via RST: either side can send a RST at any point to say "this connection is invalid, stop immediately," with no ACK expected and no guarantee that data already sent (or in flight) will be delivered or acknowledged. A RST is what you see when a process crashes and the OS cleans up its sockets, when data arrives for a port nothing is listening on, or when an application deliberately aborts a connection instead of finishing a clean handshake-of-goodbyes.

Simultaneous close is the edge case where BOTH sides send a FIN before either has received the other's FIN. Both sides transition through a CLOSING state (rather than one going through FIN_WAIT and the other CLOSE_WAIT) before eventually reaching TIME_WAIT once both FINs are acknowledged. It's rare, but it's why the TCP state machine has a CLOSING state at all: it exists specifically for this race.

Worked example
A RST during the handshake itself, before a connection is even established, means something different from a RST on an established connection: it almost always means nothing is listening on that port (or a firewall/security-group is actively rejecting rather than silently dropping). This is one of the fastest network diagnostics available, a connection refused instantly (RST) tells you the path and host are reachable but the service isn't there, whereas a connection that hangs and eventually times out with no RST tells you the packet may be silently dropped somewhere in the path, a materially different failure to chase down.

Trade-offs & pitfalls
Asymmetric routing can produce a half-open connection that looks alive on one side even after the other side has actually closed: if the FIN from one side never arrives (dropped along an asymmetric or otherwise broken return path), that side can sit in FIN_WAIT or the other side in ESTABLISHED indefinitely from its own perspective, believing the connection is fine when it's not. Keepalive probes exist specifically to eventually detect and clear out this class of half-open zombie connection, since neither side's own state machine will notice on its own.

Direct answer
Building reliability on top of UDP means implementing, by hand, the exact machinery TCP gives you for free: a sequence number to detect duplicates, explicit acknowledgments, and a retransmission timer. A single-bit (0/1) sequence number is enough for stop-and-wait specifically, because only one message is ever in flight at a time.

Structured elaboration (approach)
send_reliable sends the payload tagged with the current sequence bit, then blocks (with a timeout) waiting for a matching ACK; on a timeout it just resends the same packet, and on receiving an ACK for the WRONG sequence number (a stale ACK from a previous round) it keeps waiting rather than treating that as success. receive_reliable accepts a packet, immediately ACKs it (even if it's a duplicate, in case its own previous ACK was lost), and only hands NEW data (matching the expected next sequence bit) up to the caller, silently absorbing duplicates.

Worked example (code)

import socket, struct

HEADER = struct.Struct("!BB")  # (sequence bit, type: 0=DATA, 1=ACK)

def send_reliable(sock, dest, payload, timeout, max_retries=5, state={"seq": 0}):
    seq = state["seq"]
    packet = HEADER.pack(seq, 0) + payload
    sock.settimeout(timeout)
    for _ in range(max_retries):
        sock.sendto(packet, dest)
        try:
            data, addr = sock.recvfrom(4096)
        except socket.timeout:
            continue  # retransmit on timeout
        if len(data) < HEADER.size:
            continue
        ack_seq, ack_type = HEADER.unpack(data[:HEADER.size])
        if ack_type == 1 and ack_seq == seq:
            state["seq"] = 1 - seq
            return True
    return False

def receive_reliable(sock, state={"expected_seq": 0}):
    while True:
        data, addr = sock.recvfrom(4096)
        if len(data) < HEADER.size:
            continue
        seq, pkt_type = HEADER.unpack(data[:HEADER.size])
        if pkt_type != 0:
            continue
        payload = data[HEADER.size:]
        sock.sendto(HEADER.pack(seq, 1), addr)  # always ACK, even duplicates
        if seq == state["expected_seq"]:
            state["expected_seq"] = 1 - seq
            return payload, addr
        # else: duplicate, already ACKed above, loop for the real next message

This was executed against a deterministic loss-simulating wrapper (a UDP socket wrapper that drops a configurable fraction of outgoing packets using a seeded random generator, so the test is reproducible) sending 4 messages at loss rates of 0%, 30%, and 60%. At every loss rate tested, all 4 messages were delivered exactly once, in the original order, confirming both the retransmit-on-timeout path and the duplicate-suppression path work correctly under real, repeated loss.

Trade-offs & pitfalls (edge cases and complexity)
Complexity: with a single sequence bit and no pipelining, stop-and-wait can send only ONE unacknowledged message at a time, so throughput is bounded by one round trip per message (a real reliability layer would need a sliding window of sequence numbers, not just one bit, to use a high-bandwidth-delay-product link efficiently, exactly the same motivation as TCP's own window). Edge cases handled: a lost DATA packet (sender times out, retransmits), a lost ACK (receiver gets a duplicate DATA packet, re-ACKs it without re-delivering the payload to the application), and a delayed ACK arriving after the sender has already given up and retransmitted (the sender must ignore an ACK for the WRONG sequence number rather than treating it as confirmation, otherwise a stale ACK could be mistaken for acknowledging the NEXT message). What this exercise demonstrates: TCP is doing exactly this kind of bookkeeping (and much more, for a full sliding window, congestion control, and out-of-order buffering) on every connection, for free.