Networking Fundamentals and Protocols Questions

Direct answer
The general TCP-versus-UDP trade-offs (reliability, ordering, congestion control) still apply for a security-sensitive service, but the CONSEQUENCES of getting it wrong shift: for an authentication gateway, connection state and ordering guarantees typically matter enough that TCP (or a reliable protocol built carefully on UDP, like QUIC/DTLS) is usually the right call, while a telemetry pipeline or a DNS resolver often has different tolerance for loss that changes the calculus entirely.

Structured elaboration
For an authentication gateway: a security handshake (credential exchange, session-token issuance) usually MUST complete in full and in order, a partially-delivered or reordered authentication exchange is not just an inconvenience, it's a correctness and security hazard (imagine a session token being accepted before its associated authorization data has actually arrived). TCP's guarantees map naturally onto this requirement; a security-sensitive service choosing UDP for this kind of exchange would need to reimplement TCP-equivalent reliability AND get its security properties exactly right, real risk for very little benefit over just using TCP or a well-vetted reliable-over-UDP protocol like QUIC/DTLS.

For a telemetry pipeline: losing an occasional metric data point is usually an acceptable trade for lower overhead and not having a slow network path back up a whole telemetry stream behind one lost packet; UDP (or a lightweight, application-level "send and forget, tolerate gaps" scheme) is often the better fit, PROVIDED the pipeline's consumers are designed to tolerate gaps gracefully (which is a real design requirement, not automatic).

For a DNS resolver: DNS traditionally used UDP specifically because most queries and responses are small and a lost query is cheap to simply retry from the client, but DNS increasingly also uses TCP (mandatorily, for responses too large for a single UDP datagram, and increasingly for the added confidentiality of DNS-over-TLS/DNS-over-HTTPS), an example of a protocol that legitimately needs BOTH depending on the situation, rather than a single blanket choice.

Trade-offs & pitfalls
The security-flavored version of this question isn't really asking "which protocol is more secure", NEITHER TCP nor UDP provides confidentiality or authentication on its own, that's the job of a layer above (TLS, DTLS, application-level signing). It's asking whether the CORRECTNESS and availability properties each protocol provides (ordering, reliability, congestion behavior) match what a security-critical exchange actually needs, and getting that judgment wrong (for instance, building a custom "reliable enough" scheme on UDP for an auth exchange without the rigor of a vetted protocol) is a common, real source of security bugs.

Direct answer
The TCP/IP (Internet) model has four layers: Link, Internet, Transport, and Application. It maps onto the seven-layer OSI model by collapsing OSI's bottom two layers into "Link" and its top three into "Application."

Structured elaboration

Both models describe the same reality; they just carve it up at different granularity. OSI was designed as a general, vendor-neutral reference standard, drawn up before the protocols that would actually win were finalized, so it separates concerns the real internet protocol suite never bothered to split (nothing on the modern internet implements a distinct Session or Presentation layer as its own protocol). TCP/IP is the model that grew directly out of the protocols that were actually built and deployed, so it only has as many layers as the real stack needs to describe.

Worked example
Take a single HTTPS request. In TCP/IP terms: Application layer is HTTPS/TLS+HTTP, Transport is TCP, Internet is IP, Link is Ethernet or Wi-Fi. In OSI terms, the same request spans Layer 7 (HTTP semantics), arguably Layer 6 (TLS's encryption/format role, though in practice TLS libraries sit logically between transport and application), Layer 4 (TCP), Layer 3 (IP), and Layers 2/1 (the local network technology). Neither model is "more correct"; TCP/IP is more useful for describing what's actually running, OSI is more useful for a shared vocabulary when discussing where a NEW piece of functionality should live.

Trade-offs & pitfalls
A common mistake is treating the OSI-to-TCP/IP mapping as one-to-one instead of many-to-one. Session and Presentation don't disappear in TCP/IP, they're just not modeled as distinct layers because no widely deployed protocol maps cleanly onto only one of them. When someone asks "what OSI layer does TLS operate at," the honest answer is "it doesn't map cleanly, and that's expected," not a forced single number.

Direct answer
Over the life of a connection, TCP's congestion window grows exponentially in slow start, switches to growing linearly in congestion avoidance once it approaches a known safe ceiling, and reacts to loss with fast retransmit and fast recovery rather than always restarting from scratch.

Structured elaboration

1. Slow start: the connection begins with a small congestion window (historically 1 segment; modern stacks start higher, commonly around 10 segments per RFC 6928) and roughly DOUBLES the window every round trip, since each of the ACKs for the previous batch triggers sending two new segments. This continues until either loss occurs, or the window reaches a threshold called ssthresh (slow start threshold), at which point the sender switches strategies.
2. Congestion avoidance: once at or above ssthresh, growth switches from exponential to roughly linear (classically, additive increase of about one segment per round trip), a much more cautious probe for additional capacity.
3. Fast retransmit: if the sender sees three duplicate ACKs (the receiver repeatedly acknowledging the same byte, implying a specific segment is missing but LATER data did arrive), it retransmits the missing segment immediately, without waiting for the retransmission timer to expire, since three duplicate ACKs is strong, specific evidence of loss rather than simple reordering.
4. Fast recovery: after a fast retransmit, rather than collapsing all the way back to slow start, ssthresh is set to about half the current window, and the window itself is set near that halved value, so the sender doesn't have to re-earn all its previous progress from a window of one segment; it resumes near where it estimates the path can actually sustain.

Worked example
Picture a connection whose window has grown to 64 segments in flight when a single segment is lost and detected via three duplicate ACKs (not a full timeout). Fast retransmit resends the missing segment immediately. Fast recovery sets ssthresh to roughly 32 (half of 64) and the window to near that value, then resumes congestion avoidance's linear growth from there, rather than collapsing to slow start's small initial window and doubling all the way back up. Contrast this with a RETRANSMISSION TIMEOUT (no duplicate ACKs arrived at all, meaning the loss was severe enough that the whole flight of data went missing): that's a much stronger loss signal, and the sender resets ssthresh to half the current window but drops the actual window all the way back to slow start's minimum, since a timeout implies the path may be far more broken than a few duplicate ACKs would suggest.

Trade-offs & pitfalls
It's a common mistake to say TCP always halves its window on any loss and moves on; a full retransmission timeout is treated far more conservatively (full reset to slow start) than a fast-retransmit-detected loss (a much gentler recovery), because the ABSENCE of any duplicate ACKs at all is itself informative: it suggests either a much larger loss event or a badly congested/broken path, not just one unlucky dropped segment.

Direct answer
The TCP header carries, at minimum, a sequence number and acknowledgment number (for tracking and confirming data), the SYN/ACK/FIN/RST control flags (for connection setup and teardown), a window size (for flow control), and a set of options including MSS (Maximum Segment Size), window scale, SACK-permitted (Selective Acknowledgment), and timestamps (all negotiated at the handshake). If you could only check a few during a performance incident, window size and the options negotiated at the handshake (MSS, window scale, SACK) are the highest-value first checks, since they directly bound how efficiently the connection CAN perform, before even looking at anything dynamic.

Structured elaboration

• Sequence number: identifies the position, in bytes, of this segment's data within the overall byte stream; every byte sent gets a sequence number.
• Acknowledgment number: when the ACK flag is set, indicates the NEXT byte the receiver expects, effectively confirming everything before that point has arrived.
• Flags (SYN/ACK/FIN/RST): SYN initiates a connection, ACK confirms received data (present on nearly every segment after the handshake), FIN requests a graceful close, RST aborts the connection immediately.
• Window size: the receiver's advertised available buffer space (subject to the negotiated window SCALE factor from the handshake), the mechanism behind flow control.
• Options (MSS, window scale, SACK-permitted, timestamps): negotiated ONLY in the SYN/SYN-ACK exchange and fixed for the connection's lifetime; MSS caps the largest single segment, window scale extends the effective window size beyond the raw 16-bit field, SACK-permitted enables selective (rather than only cumulative) acknowledgment, and timestamps support accurate RTT measurement and protect against stale, wrapped sequence numbers.

Worked example
Triaging a performance incident with limited time, check the negotiated OPTIONS first: if window scale never negotiated successfully (visible by comparing the SYN and SYN-ACK), the connection is capped at a 64KB window for its ENTIRE lifetime regardless of anything else, a hard, structural ceiling worth ruling out before looking at anything dynamic. Then check the CURRENT window size value on live segments (has it collapsed to something small, suggesting a flow-control-limited receiver) alongside the flags (any unexpected RSTs indicating the connection is being torn down and re-established repeatedly, itself a red flag). Sequence and acknowledgment numbers matter most for confirming specific loss/retransmission behavior (comparing them across segments), a more detailed, second-pass check once the higher-level structural questions (options, window, flags) have been ruled out.

Trade-offs & pitfalls
It's easy to over-focus on sequence and acknowledgment numbers first because they feel like "the real data" of TCP's bookkeeping, but for a FIRST-PASS performance triage, the options negotiated once at the handshake (which structurally CAP what the connection can ever achieve) and the live window size (which shows whether that cap is even being approached) are higher-leverage checks, they answer "is there a hard ceiling here" before you spend time analyzing moment-to-moment sequence-level behavior.

Direct answer
For a WAN replication job stuck at 20% of theoretical throughput, walk the transport-layer stack in this order: congestion-control behavior first (is loss even happening, and if so is the algorithm reacting sensibly), then window sizing relative to the path's bandwidth-delay product, then socket buffers, then NIC-level settings, since each of these can independently cap throughput and the cheapest checks come first.

Structured elaboration

1. Congestion-control algorithm and loss: check whether the connection is experiencing any packet loss at all (via retransmit counters, ss -i's retrans field). If there's meaningful loss and the algorithm is a loss-based one like Cubic, and the path has ANY non-congestive baseline loss (common on long WAN paths), the algorithm may be needlessly throttling itself; switching to BBR is a plausible fix here specifically because it doesn't over-react to non-congestive loss the way Cubic does.
2. Window size versus bandwidth-delay product: compute the path's bandwidth-delay product (bandwidth times round-trip time) and compare it to the connection's actual window (ss -i's cwnd and the negotiated window scale). If the window can never grow large enough to cover the bandwidth-delay product, no amount of congestion-control tuning will help, the connection is fundamentally window-limited, not congestion-limited.
3. Socket buffer sizing: even with window scaling negotiated, if the OS's actual send/receive socket buffers (net.ipv4.tcp_wmem/tcp_rmem on Linux) are capped below what the window scale option would otherwise allow, the effective window is capped at the smaller of the two, check both.
4. NIC-level settings: segmentation/offload settings (TSO/GSO/LRO) and interface MTU (Maximum Transmission Unit) affect how efficiently the CPU can push bytes onto the wire; a misconfigured or disabled offload setting can bottleneck a fast link at the CPU rather than the network itself, worth ruling out especially if CPU utilization on the sending host is unexpectedly high relative to the achieved throughput.

Worked example
Suppose the link is rated at 1 Gbps with a 200ms round-trip time. The bandwidth-delay product is 1e9 bits/s times 0.2s = 2e8 bits = 25,000,000 bytes (25 MB). If the connection's actual window, even after scaling, tops out at 5 MB (one-fifth of the required 25 MB), the connection can never exceed roughly one-fifth of the link's rated throughput, which lines up suspiciously well with an observed 20%. That's the single most likely explanation to check FIRST, since it directly predicts the exact ratio being observed, before assuming something more exotic like a congestion-control mismatch.

Trade-offs & pitfalls
It's tempting to jump straight to "switch congestion-control algorithms" as the fix, but that's the wrong first move if the real limiter is window size: no algorithm change fixes a window that's structurally too small for the path's bandwidth-delay product. Always compute the bandwidth-delay product FIRST and check whether the achieved throughput lines up with a window-limited explanation before reaching for an algorithm change.