Networking Fundamentals and Protocols Questions

Direct answer
The TCP/IP (Internet) model has four layers: Link, Internet, Transport, and Application. It maps onto the seven-layer OSI model by collapsing OSI's bottom two layers into "Link" and its top three into "Application."

Structured elaboration

Both models describe the same reality; they just carve it up at different granularity. OSI was designed as a general, vendor-neutral reference standard, drawn up before the protocols that would actually win were finalized, so it separates concerns the real internet protocol suite never bothered to split (nothing on the modern internet implements a distinct Session or Presentation layer as its own protocol). TCP/IP is the model that grew directly out of the protocols that were actually built and deployed, so it only has as many layers as the real stack needs to describe.

Worked example
Take a single HTTPS request. In TCP/IP terms: Application layer is HTTPS/TLS+HTTP, Transport is TCP, Internet is IP, Link is Ethernet or Wi-Fi. In OSI terms, the same request spans Layer 7 (HTTP semantics), arguably Layer 6 (TLS's encryption/format role, though in practice TLS libraries sit logically between transport and application), Layer 4 (TCP), Layer 3 (IP), and Layers 2/1 (the local network technology). Neither model is "more correct"; TCP/IP is more useful for describing what's actually running, OSI is more useful for a shared vocabulary when discussing where a NEW piece of functionality should live.

Trade-offs & pitfalls
A common mistake is treating the OSI-to-TCP/IP mapping as one-to-one instead of many-to-one. Session and Presentation don't disappear in TCP/IP, they're just not modeled as distinct layers because no widely deployed protocol maps cleanly onto only one of them. When someone asks "what OSI layer does TLS operate at," the honest answer is "it doesn't map cleanly, and that's expected," not a forced single number.

Direct answer
For large object-storage transfers, the levers worth investigating, roughly in order of impact, are: making sure window scaling and socket buffers are large enough for the path's bandwidth-delay product, enabling NIC-level offloads so the CPU isn't the bottleneck at high throughput, and deciding whether to use several parallel connections or one well-tuned connection based on whether the limiting factor is per-connection window size or something else entirely (like a single flow being unfairly rate-limited by a middlebox).

Structured elaboration
First, confirm the connection's window (after scaling) and OS socket buffers are large enough to cover the path's bandwidth-delay product, undersized buffers here silently cap throughput regardless of how good everything else is (this is the same BDP-sizing exercise as tuning any other high-bandwidth, high-latency transfer). Second, check NIC-level segmentation offloads: TSO/GSO let the OS hand large chunks of data to the NIC and have the NIC itself split them into wire-sized frames, and LRO does the reverse on receive, coalescing many small incoming frames before handing them to the OS; without these, the CPU has to do that segmentation/coalescing work itself, and at multi-gigabit throughput that CPU cost can become the actual bottleneck well before the network link itself is saturated. Third, decide on Nagle's algorithm (which delays sending small writes to coalesce them into fewer, larger segments): for large sequential transfers, Nagle is rarely the bottleneck since writes are already large, but for anything issuing many small writes interleaved with reads, disabling it (TCP_NODELAY) avoids needless latency.

Worked example
The parallel-versus-single-connection decision comes down to WHAT is actually being limited. If a single connection's throughput is capped by its own maximum achievable window (even after correct BDP-based tuning, some paths or middleboxes limit an individual flow's effective window more aggressively than the path's own capacity), opening several parallel connections lets the AGGREGATE throughput exceed what one connection alone could reach, effectively working around a per-flow limit by using multiple flows. But parallel connections add real complexity: more connection-management overhead, more complexity assembling the transferred object back together correctly if it's split across connections, and, on a link SHARED with other traffic, an unfair grab of a disproportionate share of available bandwidth compared to a well-behaved single flow. If the true bottleneck is the raw link capacity itself (not a per-flow cap), splitting into parallel connections doesn't help, and one well-tuned single connection is simpler and just as fast.

Trade-offs & pitfalls
It's tempting to reach for "just add more parallel connections" as a default fix for slow transfers, but that only helps when the limiting factor is genuinely per-connection (a window/RTT/middlebox constraint on a single flow), and on a genuinely bandwidth-saturated shared link, more parallel connections from one client mostly just take a larger, less fair share of that link away from other traffic rather than achieving any real net throughput gain.

Direct answer
TCP Fast Open (TFO) lets a client include application data directly in its SYN packet on a REPEAT connection to a server it has connected to before, skipping the usual "wait for the handshake to finish before sending anything" round trip; the risk is that some middleboxes, built assuming a SYN never carries a meaningful payload, may drop, strip, or misforward that data, or even the whole packet, since it doesn't look like a standard handshake to them.

Structured elaboration
TFO works by having the server issue a cryptographic cookie to the client the FIRST time they connect (during a normal handshake, no fast-open data yet). On any SUBSEQUENT connection attempt, the client includes that cookie plus its actual application data directly in the SYN packet. If the server validates the cookie, it can begin processing that data immediately, effectively saving a full round trip compared to the standard "handshake completes, THEN data starts flowing" sequence.

The middlebox risk comes from an assumption baked into a lot of older network equipment: that a SYN packet is control-plane-only and never carries a meaningful payload. Some middleboxes (older NAT devices, certain firewalls, some load balancers) built on that assumption may strip TCP options they don't recognize (breaking the cookie mechanism itself), drop SYN packets that carry payload as anomalous or suspicious, or in some documented cases, mishandle the connection entirely.

Worked example
The required fallback behavior: if a TFO attempt's SYN (carrying the cookie and data) gets no response, or gets a response that doesn't correctly acknowledge the fast-open data, a correct client implementation must FALL BACK to a standard handshake, retransmitting the SYN WITHOUT the fast-open data and cookie, and only sending the actual application data after the connection is fully, conventionally established. This fallback is what keeps TFO safe to enable broadly: in the worst case (a hostile middlebox on the path), the connection degrades to ordinary TCP behavior rather than failing outright, the client just loses the round-trip savings TFO was trying to provide, it never loses connectivity because of it.

Trade-offs & pitfalls
The whole design of TFO is built around the assumption that middlebox interference IS common enough to plan for, not a rare edge case, that's precisely why the fallback-to-standard-handshake behavior is a REQUIRED part of any compliant implementation, not an optional nicety. An implementation that assumes TFO will always succeed and doesn't implement the fallback path correctly risks connections silently failing specifically on paths that include one of these older devices, exactly the class of path where the round-trip savings would have mattered most (since those paths tend to be higher-latency, longer routes).

Direct answer
An ICMP Type 3 Code 4 message ("Fragmentation Needed and Don't Fragment was Set") is a router along the path telling the sender its packet was too large for the next hop's MTU (Maximum Transmission Unit, here 1400 bytes) and, because the Don't Fragment (DF) bit was set, the router dropped it rather than fragmenting it, expecting the sender to resend at a smaller size. Path MTU Discovery commonly fails in production because something along the path (often a firewall with an overly broad "block all ICMP" rule) discards that very ICMP message before it reaches the sender, so the sender never learns to shrink its packets and its large packets just keep silently disappearing.

Structured elaboration
The DF bit tells every router along the path "do not fragment this packet under any circumstances, if it doesn't fit, drop it and tell me why." Path MTU Discovery relies entirely on that "tell me why" part actually reaching the sender: the sender starts by assuming its LOCAL interface's MTU is usable end-to-end, sends with DF set, and if a router along the path can't forward it at that size, the router sends back exactly this ICMP message, reporting the smaller MTU it needs (1400 bytes here). The sender is then supposed to shrink its packet size to that reported value and retry.

The reason PMTUD commonly fails in production: many firewalls and security appliances, misconfigured to block ALL ICMP as a blanket "security" measure, silently discard the "Fragmentation Needed" message on its way back to the sender. The sender then never learns it needs to shrink its packets, keeps sending at the original (too-large) size with DF still set, and those packets keep getting silently dropped at the same router, forever, with no error ever surfacing to the sender, the classic "large transfers hang, small transfers succeed" symptom (small packets happen to fit under the constrained MTU and sail through fine, while anything larger vanishes without explanation).

Worked example
To reconstruct what happened from the capture: the fragmented IPv4 packets observed likely represent an EARLIER part of the same flow that happened to still get through (perhaps fragmented by an intermediate device before DF took full effect, or from a portion of traffic that didn't have DF set), while the ICMP message with next-hop MTU 1400 is the router's report on a LATER, DF-set packet it could not forward. To fix this for BOTH TCP and UDP traffic: for TCP, the most common resilient fix is MSS clamping on a network device at the edge (rewriting the TCP Maximum Segment Size option in SYN packets passing through, so TCP negotiates a small-enough segment size up front and the oversized-packet problem never occurs at all, PMTUD independent); for UDP, since there's no equivalent MSS negotiation, the application itself must either send appropriately small datagrams from the start or correctly handle PMTUD feedback (which requires NOT blocking the relevant ICMP messages on the path, the actual root-cause fix). In both cases, the truly correct long-term fix is ensuring ICMP "Fragmentation Needed"/"Packet Too Big" messages are explicitly PERMITTED through every firewall along the path, rather than working around their absence.

Trade-offs & pitfalls
MSS clamping is a pragmatic, widely-used workaround specifically because it doesn't depend on ICMP getting through at all, but it only helps TCP; it does nothing for UDP traffic hitting the exact same oversized-packet problem, which is why "block all ICMP" as a firewall policy is a genuinely bad default rather than a harmless-looking hardening step, it breaks a real, load-bearing part of how IP networking is supposed to self-correct.

Direct answer
TIME_WAIT is the state the side that sent the FINAL ACK of a connection close sits in for a fixed period (commonly twice the maximum expected segment lifetime, often around 60 seconds on Linux) before the connection's resources are fully released. It exists so a delayed, duplicate packet from an old connection can't be mistaken for part of a brand-new connection reusing the same address/port pair. A half-open connection is one where only one side still believes the connection is alive; the other side has already reset, crashed, or otherwise abandoned it without a clean FIN exchange.

Structured elaboration
TIME_WAIT exists to protect two things: (1) it guarantees the final ACK the closing side sent actually gets through, by giving time to retransmit it if the peer's FIN gets retransmitted (meaning the ACK was lost); (2) it prevents a stray, delayed packet from a previous incarnation of a connection (same 4-tuple: source IP, source port, destination IP, destination port) from being delivered into a brand new connection that happens to reuse the same 4-tuple before the old segments have had time to disappear from the network.

To detect a large number of sockets stuck in TIME_WAIT on Linux, ss -tan state time-wait | wc -l (or the older netstat -ant | grep TIME_WAIT | wc -l) gives a live count; watching this metric over time distinguishes a normal, self-draining backlog from a genuine problem.

Worked example
A server that closes millions of short-lived outbound connections per hour (for instance, a service making one HTTP call per request to an upstream) can exhaust its available ephemeral source ports if TIME_WAIT sockets accumulate faster than they expire, because each TIME_WAIT socket still holds its port reserved. The two standard mitigations are: raise the number of available client-side (ephemeral) ports and/or reuse connections via keepalive/connection pooling so fewer connections churn through TIME_WAIT in the first place; and, on the SERVER side specifically, enabling SO_REUSEADDR and, where safe, tcp_tw_reuse lets a new outgoing connection reuse a TIME_WAIT 4-tuple once TCP timestamps confirm it's safe to do so, rather than waiting out the full timer.

Trade-offs & pitfalls
Disabling or drastically shortening TIME_WAIT globally (rather than tuning port ranges or reuse settings) is the wrong fix: it reintroduces the exact correctness problem TIME_WAIT was designed to prevent, stray old packets landing in a new connection and corrupting it. The safe levers are reducing HOW MANY connections churn through the state (pooling, keepalive) and widening the ephemeral port range, not shrinking the safety window itself.