Learning objective. Map the places a QUIC/HTTP-3 stack can be instrumented (which layer holds which state) and the three vantage points from which a connection can be observed (endpoint qlog, passive on-path, decrypted capture), so you can pick the right tool for the question you have.
Two axes: which layer, and which vantage point #
Observing QUIC means answering two questions. First, which layer holds the state you care about: the encrypted, multiplexed design means information is spread from the application down to the socket, and each layer exposes different things. Second, from where are you looking: your own endpoint, a point on the network path, or a capture you can decrypt. The two axes are independent, and choosing well is most of the skill.
The layers, and what each exposes #
From top to bottom, the state worth instrumenting:
- Application / HTTP-3. Requests and responses, stream lifecycle events, status codes, and (for a server) request handling time. This is where user-visible latency (time to first byte, time to last byte) is measured.
- QUIC transport. Frames, streams, flow-control limits, and connection-close signals. The frame and stream view tells you what was exchanged and whether ordering/limits were hit.
- Loss recovery and congestion control. This layer holds most of the performance signal: the RTT
estimator (
latest_rtt,min_rtt,smoothed_rtt,rttvar, [RFC 9002 §5]), the congestion controller (congestion_window,bytes_in_flight,ssthresh, [RFC 9002 §B.2]), loss events and the PTO count ([RFC 9002 §6]). When throughput or latency is wrong, the answer is almost always here. - TLS / crypto. The handshake, key updates, and the traffic secrets — the last of which is what a key log exports so an external tool can decrypt the connection (§11.3).
- UDP / IP. Datagram sizes, packet timing, ECN counts ([RFC 9000 §13.4]), and the peer addresses that reveal migration (§10.1).
The three vantage points #
The same connection looks different from three places, and each has a hard boundary:
Endpoint instrumentation (qlog). Your own stack can log everything above (RTT, congestion window, every frame) because it holds the state and the keys. The de facto format is qlog, a structured JSON event log the labs in this book already emit. Two caveats: it is a first-party view (you see your endpoint's state, not the peer's internals), and qlog is a tooling convention (an IETF work in progress), not part of RFC 9000/9001/9002. Implementations choose whether and what to emit. It is the richest view and the first place to look.
Passive on-path observation (no keys). A monitoring point on the network sees only what is not encrypted. That is a surprisingly useful set (datagram sizes, packet timing, connection IDs, ECN codepoints), and one purpose-built signal: the latency spin bit, which "enables passive latency monitoring from observation points on the network path" by toggling once per round trip so an observer can "measure the time between two spin bit toggle events to estimate the end-to-end RTT" ([RFC 9000 §17.4]). But the boundary is real: the payload is encrypted, and header protection hides the packet numbers: the packet-number field is "protected using a key that is derived separately" ([RFC 9001 §5.4]), so an observer cannot even count packets directly. (The spin bit is also optional and privacy-limited: endpoints may disable it, and MUST disable it on at least 1 in 16 paths ([RFC 9000 §17.4]), so treat on-path RTT as best-effort.)
Decrypted capture (Wireshark + key log). A packet capture plus the connection's TLS secrets lets a tool decrypt and dissect every frame in both directions — the byte-exact ground truth. The cost is that you must have the keys: an endpoint you control exports them to a key log file (an NSS/Wireshark convention, again not RFC-defined) that Wireshark reads to derive the packet-protection keys (§11.3). With the keys, nothing is hidden; without them, you are back to the passive view.
Choosing a vantage point #
Match the tool to the question:
| Question | Best vantage | Why |
|---|---|---|
| Why is throughput low? | endpoint qlog | cwnd, bytes_in_flight, loss live only at the endpoint |
| Is the path adding RTT/loss? | passive on-path | spin-bit RTT and timing need no keys, seen mid-path |
| What exact frames were exchanged? | decrypted capture | byte-exact, both directions, if you hold the keys |
| Did the connection migrate / reset? | qlog or capture | new addresses and CONNECTION_CLOSE reason codes |
The strongest debugging correlates all three: the same connection seen from inside (qlog), from the path (passive), and byte-exact (decrypted capture), lined up by time.
Worked example: a stalled download #
Suppose a large HTTP/3 download plateaus below the link rate. The vantage points partition the
investigation. The endpoint qlog shows whether bytes_in_flight is pinned at the congestion_window
(congestion-limited) or at the flow-control limit (flow-control-limited, §8.4), and whether
loss or PTO events are firing, usually enough to localize the cause. A passive on-path view of the
spin bit confirms whether the path RTT is stable or climbing (queueing) independent of what the endpoint
believes. And if the two disagree (the endpoint thinks it is sending but nothing advances), a decrypted
capture shows the exact frames, revealing, say, that MAX_DATA updates are not arriving. Each vantage
answers a different sub-question; together they converge.
The privacy properties that make QUIC hard to observe are deliberate, not accidental. Header protection hiding packet numbers ([RFC 9001 §5.4]), the spin bit's mandatory 1-in-16 opt-out ([RFC 9000 §17.4]), and connection-ID rotation on migration (§10.2) all exist to deny a passive observer the ability to track and fingerprint connections. As an operator of your own endpoint you sidestep all of it: you hold the keys and the internal state. That is exactly why endpoint instrumentation (qlog, key logs) is the backbone of QUIC observability, and passive on-path monitoring is a best-effort supplement.
editorial Instrument at the endpoint first, because it is the only vantage that sees congestion-control state and it is the one you fully control. Turn on qlog in your QUIC library in staging (and behind a flag in production, sampled: the logs are large), and make sure you can export a key log for the specific connections you need to decrypt. Reserve passive on-path monitoring for questions about the path itself (is a particular network segment adding latency or loss?), where the spin bit and timing are exactly right and no keys are needed. The anti-pattern is trying to debug a congestion problem from a passive capture you cannot decrypt — you will see sizes and timing but none of the state that actually explains the behavior.
Takeaways #
QUIC observability has two axes: which layer holds the state (application, transport, recovery/CC, crypto, UDP, with the performance signal concentrated in loss recovery and congestion control), and which vantage point you observe from (endpoint qlog, richest but one-sided; passive on-path, keyless but limited to spin bit/sizes/timing; decrypted capture, byte-exact but needs the keys). Pick the vantage by the question, and correlate all three for hard problems. The next section makes the recovery/CC layer concrete as a set of metrics worth tracking.