Networking Fundamentals and Protocols Questions

Direct answer
Bandwidth is the link's raw theoretical capacity; throughput is what's actually achieved end-to-end, which is always less than or equal to bandwidth; and goodput is throughput minus any overhead that isn't USEFUL application data (protocol headers, retransmissions of data that eventually succeeds anyway). An application commonly sees less than the rated bandwidth for several genuinely distinct reasons, and telling them apart is the fast path to the right fix.

Structured elaboration
Four common, distinct reasons throughput or goodput falls short of bandwidth:

1. Retransmissions: any lost segment that has to be resent consumes bandwidth without contributing new useful data, so a lossy path shows lower goodput even at unchanged raw throughput.
2. Protocol overhead: every layer's headers (Ethernet, IP, TCP) consume bytes that count toward raw throughput but never toward the application's actual USEFUL payload, goodput specifically excludes them.
3. Congestion-window or flow-control limits: if the connection's congestion window or receive window is smaller than the path's bandwidth-delay product, the sender is idle waiting for ACKs rather than continuously filling the pipe, capping achieved throughput well below the link's rated bandwidth regardless of loss.
4. Application-level inefficiency: an application that reads/writes in small chunks, adds its own serialization overhead, or simply isn't pipelining requests efficiently can bottleneck well below what the transport layer itself is capable of moving.

Worked example
The fastest way to localize which of these applies to a specific slow transfer: check retransmission counters first (ss -i's retrans field), if they're elevated, loss/retransmission is a real contributor. If retransmissions are low but the window (cwnd/rwnd from the same tool) is small relative to the path's bandwidth-delay product, the connection is window-limited rather than loss-limited, and no amount of "fixing loss" will help. If BOTH retransmissions are low and the window is comfortably larger than the bandwidth-delay product, yet throughput is still poor, suspect application-level inefficiency (measure CPU usage and how the application is actually issuing reads/writes) rather than anything at the transport layer at all.

Trade-offs & pitfalls
The most common diagnostic mistake is assuming a throughput shortfall is automatically a "network problem" and reaching straight for network-level tuning; a genuinely application-bound bottleneck (small, unbuffered writes; serialized rather than pipelined requests) is common and invisible to any amount of TCP-level tuning, wasting real effort until the actual bottleneck is correctly localized first.

Direct answer
The bandwidth-delay product (BDP) is the amount of data that can be "in flight" on a link at any instant, bandwidth multiplied by round-trip time, and it's the minimum window size TCP needs to keep the link fully utilized. On a 10 Gbps link with 150ms round-trip time, the BDP is large enough that the original 16-bit TCP window field (max 65,535 bytes) can represent only a tiny fraction of it, which is exactly why window scaling exists.

Structured elaboration
The BDP calculation is:
$\text{BDP (bits)} = \text{bandwidth (bits/s)} \times \text{RTT (s)}$
For a 10 Gbps link ($10 \times 10^9$ bits/s) with a 150ms ($0.150$ s) round-trip time:
$\text{BDP} = 10 \times 10^9 \times 0.150 = 1.5 \times 10^9 \text{ bits} = 187{,}500{,}000 \text{ bytes} \approx 187.5\text{ MB}$
The plain TCP window field is 16 bits, so the largest window it can express without scaling is $2^{16}-1 = 65{,}535$ bytes, about 0.035% of the 187.5 MB needed. Without more window than that, the sender would have to stop and wait for an ACK every 64KB, and at this RTT that caps throughput far below the link's actual 10 Gbps capacity, no matter how fast the link itself is.

The window scale option (negotiated ONLY during the handshake, in the SYN and SYN-ACK) adds a scale factor $S$ (0 to 14) that the receiver applies to its advertised window: the real window becomes $\text{advertised value} \times 2^S$. To cover a 187.5 MB requirement, we need the smallest $S$ such that $65{,}535 \times 2^S \geq 187{,}500{,}000$, which comes out to $S=12$ (giving a maximum representable window of $65{,}535 \times 4096 = 268{,}431{,}360$ bytes, comfortably above the 187.5 MB requirement).

Worked example
For a smaller, more common case, a 100 Mbps link with 100ms RTT, the BDP is:
$100 \times 10^6 \times 0.100 = 1 \times 10^7 \text{ bits} = 1{,}250{,}000 \text{ bytes} \approx 1.25\text{ MB}$
Here the minimum scale factor needed is $S=5$ (giving a max window of $65{,}535 \times 32 = 2{,}097{,}120$ bytes), a much smaller ask than the 10 Gbps case, illustrating that window scaling matters more the higher the bandwidth-delay product climbs, not RTT or bandwidth alone. Beyond window sizing, actually reaching close to line rate on a link like this also typically needs the OS socket buffers (net.ipv4.tcp_rmem/tcp_wmem on Linux) raised to match the negotiated window (a window the OS hasn't allocated buffer space for is wasted), and Selective Acknowledgment (SACK) enabled so a single lost segment somewhere in a large in-flight window doesn't force retransmission of everything after it.

Trade-offs & pitfalls
Window scaling is negotiated ONLY at connection setup; if either endpoint doesn't advertise the option in its SYN, the connection falls back to the un-scaled 64KB ceiling for its entire lifetime, a frequent, hard-to-spot cause of "high-bandwidth link, mysteriously capped throughput" that shows up when an old middlebox strips the option or a misconfigured host has scaling disabled.

Direct answer
Over the life of a connection, TCP's congestion window grows exponentially in slow start, switches to growing linearly in congestion avoidance once it approaches a known safe ceiling, and reacts to loss with fast retransmit and fast recovery rather than always restarting from scratch.

Structured elaboration

1. Slow start: the connection begins with a small congestion window (historically 1 segment; modern stacks start higher, commonly around 10 segments per RFC 6928) and roughly DOUBLES the window every round trip, since each of the ACKs for the previous batch triggers sending two new segments. This continues until either loss occurs, or the window reaches a threshold called ssthresh (slow start threshold), at which point the sender switches strategies.
2. Congestion avoidance: once at or above ssthresh, growth switches from exponential to roughly linear (classically, additive increase of about one segment per round trip), a much more cautious probe for additional capacity.
3. Fast retransmit: if the sender sees three duplicate ACKs (the receiver repeatedly acknowledging the same byte, implying a specific segment is missing but LATER data did arrive), it retransmits the missing segment immediately, without waiting for the retransmission timer to expire, since three duplicate ACKs is strong, specific evidence of loss rather than simple reordering.
4. Fast recovery: after a fast retransmit, rather than collapsing all the way back to slow start, ssthresh is set to about half the current window, and the window itself is set near that halved value, so the sender doesn't have to re-earn all its previous progress from a window of one segment; it resumes near where it estimates the path can actually sustain.

Worked example
Picture a connection whose window has grown to 64 segments in flight when a single segment is lost and detected via three duplicate ACKs (not a full timeout). Fast retransmit resends the missing segment immediately. Fast recovery sets ssthresh to roughly 32 (half of 64) and the window to near that value, then resumes congestion avoidance's linear growth from there, rather than collapsing to slow start's small initial window and doubling all the way back up. Contrast this with a RETRANSMISSION TIMEOUT (no duplicate ACKs arrived at all, meaning the loss was severe enough that the whole flight of data went missing): that's a much stronger loss signal, and the sender resets ssthresh to half the current window but drops the actual window all the way back to slow start's minimum, since a timeout implies the path may be far more broken than a few duplicate ACKs would suggest.

Trade-offs & pitfalls
It's a common mistake to say TCP always halves its window on any loss and moves on; a full retransmission timeout is treated far more conservatively (full reset to slow start) than a fast-retransmit-detected loss (a much gentler recovery), because the ABSENCE of any duplicate ACKs at all is itself informative: it suggests either a much larger loss event or a badly congested/broken path, not just one unlucky dropped segment.

Direct answer
Reno, NewReno, Cubic, and BBR represent an evolution in how TCP infers and reacts to congestion: the Reno family reacts to LOSS with a fixed halving of the window, Cubic grows more aggressively on high-bandwidth links using a cubic function of time since the last loss, and BBR abandons loss as the primary signal entirely, instead modeling the path's actual bandwidth and round-trip time directly.

Structured elaboration

• Reno: the classical algorithm. Slow start, congestion avoidance with linear (additive) growth, and on ANY loss, halves the window and re-enters a conservative recovery. Its big limitation on high-bandwidth-delay-product links is that halving the window after a single loss throws away a huge amount of earned capacity, and the subsequent linear regrowth takes a long time to recover it.
• NewReno: a refinement that fixes a specific weakness in Reno's fast recovery when MULTIPLE segments are lost within one window; Reno's original recovery logic could exit fast recovery prematurely and fall back to a slow, timeout-driven recovery for the second lost segment, while NewReno correctly stays in fast recovery until ALL the losses from that window are repaired.
• Cubic (the default on Linux for a long time): grows the window as a cubic function of the time elapsed since the last loss event, growing very slowly right after backing off, then accelerating, then leveling off as it approaches the window size where the last loss occurred, and probing gently past it. This makes Cubic much better at fully utilizing high-bandwidth, high-latency ("long fat") links than Reno's linear growth, since it isn't purely tied to round-trip-time-limited additive increase.
• BBR (Bottleneck Bandwidth and Round-trip propagation time): rather than reacting to loss at all, BBR periodically probes to directly estimate the bottleneck link's bandwidth and the path's minimum round-trip time, then paces its sending rate to match that estimate. This lets it largely ignore ordinary, non-congestive packet loss (which loss-based algorithms mistake for congestion), a real advantage on paths where a small amount of loss is normal and NOT actually a congestion signal (satellite links, some wireless links, or lossy long-haul fiber).

Worked example
For a large file transfer over a satellite link, characterized by very high round-trip time (often 500ms+) and some baseline non-congestive loss (a normal characteristic of the medium, not a sign of an overloaded path), BBR is generally the stronger choice: a loss-based algorithm like Cubic will repeatedly (and wrongly) interpret that baseline loss as congestion and needlessly shrink its window, capping throughput well below what the link can actually sustain, while BBR's bandwidth-and-RTT model isn't fooled by loss that isn't actually caused by queue buildup.

Trade-offs & pitfalls
BBR isn't a universal win: on a link SHARED with loss-based flows (Cubic, Reno), BBR's willingness to keep sending through non-congestive loss can let it grab a disproportionate share of a congested bottleneck's capacity from more conservative Reno/Cubic flows sharing that same link, an active area of real-world congestion-control fairness research, not a settled solved problem.

Direct answer
TCP's retransmission timeout (RTO) is computed from a smoothed estimate of round-trip time (SRTT) plus a term for how MUCH the round-trip time has been varying (RTTVAR), not from RTT alone, so the timer stays reasonably tight on a stable path and automatically widens on a noisy one.

Structured elaboration
The standard algorithm (RFC 6298) updates two running estimates on every new RTT sample:
$\text{RTTVAR} = (1-\beta)\cdot \text{RTTVAR} + \beta \cdot |\text{SRTT} - \text{RTT}_{\text{sample}}|$
$\text{SRTT} = (1-\alpha)\cdot \text{SRTT} + \alpha \cdot \text{RTT}_{\text{sample}}$
with the standard constants $\alpha = 1/8$ and $\beta = 1/4$. The retransmission timeout itself is then:
$\text{RTO} = \text{SRTT} + \max(\text{clock granularity}, 4 \cdot \text{RTTVAR})$
The intuition: if round-trip times are steady (RTTVAR is small), RTO sits close to the smoothed RTT, so genuine loss is detected quickly. If round-trip times are jittery (RTTVAR is large, common on congested or highly variable paths), RTO widens automatically, so ordinary jitter isn't mistaken for loss and doesn't trigger a storm of unnecessary retransmissions.

Retransmissions triggered purely by the RTO timer firing (as opposed to fast retransmit via duplicate ACKs) are treated as a MUCH stronger signal of serious congestion, since it means not even a single later segment's ACK arrived to trigger a duplicate-ACK-based fast retransmit; the response is to collapse all the way back to slow start, not the gentler fast-recovery halving.

Worked example
Suppose a connection's SRTT has settled around 50ms with RTTVAR around 10ms. RTO would be approximately $50 + 4\times10 = 90$ ms. If a burst of network jitter pushes several samples up to 120ms, RTTVAR grows to reflect that variability (say to 25ms), and RTO widens to roughly $70 + 4\times25 = 170$ ms (SRTT itself also shifts upward, more slowly, toward the new samples). This is why an environment full of many short-lived connections (each starting from scratch with no RTT history) behaves differently from one with few long-lived connections (which have had time to build a stable, well-calibrated SRTT/RTTVAR estimate): short-lived connections are stuck using a generic initial RTO (commonly 1 second per RFC 6298) until they've collected enough samples to calibrate, making them systematically slower to detect a REAL loss early in their life.

Trade-offs & pitfalls
A common mistake is assuming a single fixed RTO value would be simpler and just as effective; a fixed timeout either fires too eagerly on a naturally variable path (spurious retransmissions that waste bandwidth and can trigger unnecessary congestion-window collapses) or too slowly on a stable path (wasted time before a real loss is detected), the adaptive SRTT/RTTVAR scheme exists specifically to avoid both failure modes at once.