§8.4
Chapter 08 · Handling High-Latency Networks and Long RTT Effects

§8.4Designing Stream Concurrency for Long RTT and Bandwidth Variance

RFC 9000

Learning objective. Learn to size stream concurrency and flow-control windows so a long-RTT, variable-bandwidth path stays full: why concurrency comes from streams not connections, how the bandwidth-delay product sets the target, and where too few or too many streams both hurt.

Concurrency is free at connection start #

Two facts combine into the central design lever. First, HTTP/3 concurrency comes from streams, not connections: every request is one bidirectional stream, and many run at once over a single connection (§5.3). Second (and this is what makes it cheap on a long path), the stream limit is pre-granted in the transport parameters. initial_max_streams_bidi is delivered in the handshake, and "setting this parameter is equivalent to sending a MAX_STREAMS frame with the same value" ([RFC 9000 §18.2]). So a client may open streams up to that limit immediately, with no extra round trip to ask permission.

ServerClientlong RTT (200 ms) · pipe holds BDPbytes in flightSerial — one request at a time wastesthe pipeConcurrent — stream limits arepre-granted in transport params (noextra RTT)fill target: in-flight bytes ≈ BDP,bounded by initial_max_data +per-stream limitsGET /aresponse /a (1 RTT idle for the nextrequest)GET /bresponse /b (N objects → N roundtrips)GET /a · GET /b · GET /c · GET /d(open all at once)responses interleaved onindependent streams — pipe staysfull
Fig. 8.4-1Serial vs. concurrent requests on a 200 ms path. Issuing requests one at a time leaves the pipe idle for a round trip between each — N objects cost N round trips. Because the stream limit arrives pre-granted in the transport parameters, the client can open all the streams at once and let responses interleave on independent streams, keeping the pipe full.RFC 9000 §18.2, §4.6

The top half of Figure 8.4-1 is the cost of not using this: serial request/response leaves the pipe idle for a full RTT between each object, so fetching N objects costs N round trips. The bottom half opens all the streams at once, and the responses interleave: the pipe stays full and the whole batch costs about one round trip plus transfer time.

The target is the bandwidth-delay product #

The bandwidth-delay product is easiest to grasp as a pipe. Picture the network path from sender to receiver as a physical pipe: its length is the latency (how long a byte takes to traverse it) and its thickness is the bandwidth (how many bytes per second it can carry). The volume of that pipe (thickness × length) is the amount of data that fits inside it at once, and that volume is the bandwidth-delay product (BDP = bandwidth × RTT). To run the link at full speed you must keep the pipe full: there must always be about one BDP of data in flight, sent but not yet acknowledged.

window ≥ BDP — the pipe stays full enough data is in flight to cover the whole path → throughput = full bandwidth in-flight distance = one round trip (RTT) sender receiver ↕ bandwidth volume of the pipe = bandwidth × RTT = BDP — the bytes needed to keep it full window < BDP — the pipe runs half-empty the sender exhausts its window, then stalls waiting for ACKs → throughput = window ÷ RTT sender receiver idle — no data in flight here sender blocked, waiting for ACKs to free the window window (in flight) wasted capacity (BDP − window) no loss, no congestion — the link is simply not kept full. The fix is a bigger window, or more streams.
Fig. 8.4-2The pipe model of the bandwidth-delay product. The path is a pipe whose length is the RTT and whose thickness is the bandwidth; its volume is the BDP — the data that must be in flight to keep it full (top). If the sender's window is smaller than the BDP, it runs out of data to send and stalls waiting for ACKs, leaving the pipe half-empty and throughput capped at window ÷ RTT — with no loss and no congestion (bottom).RFC 9000 §4.3

This is the crux, and it is counter-intuitive: a path can be completely idle in the middle even though nothing is lost and the network is not congested. The sender fills its window, then must wait a full round trip for the first ACK to come back before it can send more, and while it waits, the far end of the pipe carries nothing. The only cure is to keep more data in flight, which is why QUIC states the flow-control form of the rule directly ([RFC 9000 §4.3]):

"If an endpoint cannot ensure that its peer always has available flow control credit that is greater than the peer's bandwidth-delay product on this connection, its receive throughput will be limited by flow control."

So the fill target is bounded by two things from §7.2: the connection window (initial_max_data) must exceed the BDP, and enough per-stream windows (initial_max_stream_data_*) must be open at once to reach that connection window. A single stream whose per-stream window is smaller than the BDP cannot fill the pipe by itself no matter how large the connection window is: you need either a bigger per-stream window or more concurrent streams. This is the quantitative link between concurrency and flow control: streams × per-stream window ≥ BDP ≤ connection window.

How the BDP shows up in real networks #

The reason this matters in practice is that the BDP varies enormously across real paths. Because bandwidth and RTT swing independently, their product spans nearly two orders of magnitude between a LAN and a satellite link:

bar length ∝ BDP (log scale) = bandwidth × RTT — RTT set by the path's dominant delay typical 1 MB default window paths to the right are flow-control limited by it LAN 1 Gbps × 1 ms same-site switching 125 KB Mobile 4G / 5G 50 Mbps × 50 ms radio-access scheduling 313 KB Metro fiber 1 Gbps × 10 ms ~100 km across a city 1.25 MB GEO satellite 50 Mbps × 600 ms up + down to 35,786 km orbit 3.75 MB Transcontinental 1 Gbps × 80 ms ~5,000 km of routed fiber 10 MB
Fig. 8.4-3The bandwidth-delay product across representative real networks (bar length is BDP on a log scale). Each RTT is set by the path's dominant delay — signal propagation over distance, radio-access scheduling, or local switching (see below). A LAN needs only ~125 KB in flight; a metro-fiber, satellite, or transcontinental path needs megabytes. A "typical" 1 MB default window fills the first two but starves the last three — the classic long-fat-network problem.RFC 9000 §4.3

Where those round-trip times come from. The RTTs are not arbitrary. Each is set by the dominant source of delay on that kind of path, and for most paths that source is the physical propagation delay: the time for the signal to cover the distance and come back. Light in optical fiber travels at roughly two-thirds of c, about 200 km per millisecond one way, so distance translates almost directly into latency:

  • LAN — ~1 ms. Endpoints in the same building; propagation is negligible, so the RTT is almost entirely switch and OS processing. (Often under 0.5 ms; 1 ms is a round figure.)
  • Metro fiber — ~10 ms. A path spanning a city, ~100 km, plus a few routing hops: a couple of milliseconds of propagation and the rest queueing and forwarding.
  • Mobile 4G/5G — ~50 ms. Here distance is not the driver; the RTT is dominated by the radio access network: air-interface scheduling and link-layer retransmission between handset and tower. (5G lowers this; 4G LTE is ~30–50 ms.)
  • Transcontinental — ~80 ms. Coast-to-coast is ~4,000 km, and real routed fiber runs longer (~5,000 km) and not in a straight line, so ~25 ms each way of propagation plus routing lands around 60–80 ms.
  • GEO satellite — ~600 ms. A geostationary satellite sits at 35,786 km altitude; the signal climbs to orbit and back down (~72,000 km) for a single hop, and a full client-to-server path often crosses that link at both ends: hundreds of milliseconds of pure propagation that no protocol can remove. (Low-Earth-orbit constellations like Starlink sit ~550 km up and behave much more like terrestrial links, ~40 ms.)

These are representative values, not measurements: the point is the shape, not the exact millisecond. Read the bars against the dashed line, a 1 MB flow-control window, a common library default:

  • LAN (1 Gbps × 1 ms ≈ 125 KB) and mobile (50 Mbps × 50 ms ≈ 313 KB) sit well under the line; a modest window fills them, and BDP is rarely the bottleneck.
  • Metro fiber (1 Gbps × 10 ms ≈ 1.25 MB), GEO satellite (50 Mbps × 600 ms ≈ 3.75 MB), and transcontinental (1 Gbps × 80 ms ≈ 10 MB) all exceed a 1 MB window. On these paths the connection tops out at window ÷ RTT regardless of how fast the link actually is.

That last group is what networking folklore calls a long fat network (LFN): high bandwidth and high delay, so a large BDP. It is exactly where an undersized window silently throttles throughput. The manifestation is unmistakable once you know the shape: a transfer that plateaus far below the link rate, on a clean path, with the congestion window wide open. Two numbers reveal it: a smoothed_rtt that is large and an in-flight-bytes ceiling that equals your advertised window. The fix is to raise the window (or autotune it, below) toward the peak BDP, not to hunt for packet loss that isn't there.

How many streams #

Both extremes hurt:

  • Too few streams underfills the pipe. If your workload is many small objects, one-at-a-time fetching wastes a round trip per object (Figure 8.4-1, top); if it is one large object, that single stream is capped at its per-stream window. Either way, in-flight data falls below the BDP.
  • Too many streams wastes resources and can add latency. Each open stream costs receive-buffer memory (§7.2), scheduler overhead, and (if you use QPACK's dynamic table) exposure to blocked streams under loss (§6.3). Past the point where the pipe is full, more streams add contention, not throughput.

The sweet spot is "enough concurrent streams to keep in-flight data at the BDP, and no more." For many small objects that means opening a batch at once rather than serially; for a few large objects it means giving each a per-stream window near the BDP rather than opening dozens of streams. The initial_max_streams_bidi a server advertises should comfortably exceed the concurrency a client actually needs, and the client should not open streams it cannot keep fed.

Bandwidth variance: size for the peak #

The BDP is not a constant. Bandwidth swings with congestion, radio conditions, and competing traffic, and RTT swings with queueing, so the BDP a connection needs to fill varies over its lifetime, sometimes by an order of magnitude. Two consequences:

  • Provision windows for the peak BDP, not the average. A window sized for the median bandwidth throttles the connection exactly when the link is fast and there is the most to gain. This is why autotuning (§7.2, [RFC 9000 §4.2]) matters more on variable paths than fixed windows: it grows the advertised credit from a live RTT estimate and the drain rate, tracking the BDP up as the path improves, and a receiver "MUST NOT wait" for a blocked signal before extending credit ([RFC 9000 §4.6]).
  • Don't over-commit memory to variance. Peak-BDP windows times high connection counts can be a large memory footprint (§7.5). Autotuning helps here too: it grows windows only for the connections that demonstrate they need them, rather than provisioning every connection for the peak up front.

A client fetches 20 small assets over a 250 KB-BDP path whose bandwidth halves and doubles with weather. Serial fetching would cost ~20 round trips — catastrophic at high RTT. Instead the client opens all 20 request streams at once (well within a server initial_max_streams_bidi of 100), using the limit pre-granted in the handshake, so the whole batch completes in about one round trip plus transfer. To keep the pipe full during the good-weather peaks, the server's initial_max_data sits above the peak BDP and autotuning lifts the per-stream and connection windows as the RTT estimate reveals headroom, so throughput tracks the link instead of a stale average.

Note

Independent streams remove transport head-of-line blocking (§4.4) but not application dependencies. If asset B's request cannot be formed until asset A's response is parsed (a discovered <script src>), no amount of concurrency helps: that is a dependency round trip, and it is exactly what 103 Early Hints (§8.3) exists to collapse. Concurrency fills the pipe with work you already know you need; it cannot manufacture work you haven't discovered yet.

In practice

editorial Match your concurrency to your object mix. Many small objects: issue requests in a batch, not a waterfall, and let a generous initial_max_streams_bidi and connection window carry them. The win is opening streams you already know you need in one flight. A few large objects: fewer streams, each with a per-stream window near the peak BDP, and lean on autotuning for the variance. The anti-pattern on long paths is a request waterfall (each request awaiting the previous response), which turns N objects into N round trips; if you see throughput far below the link rate with the congestion window open, look for serialization before you look at flow control.

Takeaways #

Stream concurrency is the lever that keeps a long-RTT pipe full, and it is cheap because the stream limit arrives pre-granted in the transport parameters: open the streams you need at once rather than serially. Size to the bandwidth-delay product: streams × per-stream window ≥ BDP ≤ connection window, provisioned for the peak BDP and adapted by autotuning as bandwidth varies. Too few streams underfill the pipe; too many waste memory and invite blocking. With the design principles in hand, §8.5 measures the components of end-to-end latency on a real emulated path.