QPACK: Header Compression for HTTP over QUIC

An encoder MUST ensure that a header block which references a dynamic table entry is not received by the decoder after the referenced entry has been evicted. Hence the encoder needs to track information about each compressed header block that references the dynamic table until that header block is acknowledged by the decoder.¶

An encoder MUST NOT insert an entry into the dynamic table (or duplicate an existing entry) if doing so would evict an entry with unacknowledged references. For header blocks that might rely on the newly added entry, the encoder can use a literal representation and maybe insert the entry later.¶

To ensure that the encoder is not prevented from adding new entries, the encoder can avoid referencing entries that are close to eviction. Rather than reference such an entry, the encoder can emit a Duplicate instruction (see Section 4.3.3), and reference the duplicate instead.¶

Determining which entries are too close to eviction to reference is an encoder preference. One heuristic is to target a fixed amount of available space in the dynamic table: either unused space or space that can be reclaimed by evicting unreferenced entries. To achieve this, the encoder can maintain a draining index, which is the smallest absolute index in the dynamic table that it will emit a reference for. As new entries are inserted, the encoder increases the draining index to maintain the section of the table that it will not reference. If the encoder does not create new references to entries with an absolute index lower than the draining index, the number of unacknowledged references to those entries will eventually become zero, allowing them to be evicted.¶

Because QUIC does not guarantee order between data on different streams, a header block might reference an entry in the dynamic table that has not yet been received.¶

Each header block contains a Largest Reference (Section 4.5.1) which identifies the table state necessary for decoding. If the greatest absolute index in the dynamic table is less than the value of the Largest Reference, the stream is considered “blocked.” While blocked, header field data SHOULD remain in the blocked stream’s flow control window. When the Largest Reference is zero, the frame contains no references to the dynamic table and can always be processed immediately. A stream becomes unblocked when the greatest absolute index in the dynamic table becomes greater than or equal to the Largest Reference for all header blocks the decoder has started reading from the stream. If the decoder encounters a header block where the actual largest reference is not equal to the Largest Reference declared in the prefix, it MAY treat this as a stream error of type HTTP_QPACK_DECOMPRESSION_FAILED.¶

The SETTINGS_QPACK_BLOCKED_STREAMS setting (see Section 5) specifies an upper bound on the number of streams which can be blocked. An encoder MUST limit the number of streams which could become blocked to the value of SETTINGS_QPACK_BLOCKED_STREAMS at all times. Note that the decoder might not actually become blocked on every stream which risks becoming blocked. If the decoder encounters more blocked streams than it promised to support, it MUST treat this as a stream error of type HTTP_QPACK_DECOMPRESSION_FAILED.¶

An encoder can decide whether to risk having a stream become blocked. If permitted by the value of SETTINGS_QPACK_BLOCKED_STREAMS, compression efficiency can often be improved by referencing dynamic table entries that are still in transit, but if there is loss or reordering the stream can become blocked at the decoder. An encoder avoids the risk of blocking by only referencing dynamic table entries which have been acknowledged, but this could mean using literals. Since literals make the header block larger, this can result in the encoder becoming blocked on congestion or flow control limits.¶

In order to identify which dynamic table entries can be safely used without a stream becoming blocked, the encoder tracks the absolute index of the decoder’s Largest Known Received entry.¶

When blocking references are permitted, the encoder uses header block acknowledgement to identify the Largest Known Received index, as described in Section 4.4.2.¶

To acknowledge dynamic table entries which are not referenced by header blocks, for example because the encoder or the decoder have chosen not to risk blocked streams, the decoder sends a Table State Synchronize instruction (see Section 4.4.1).¶

• Complete processing of a header block
• Abandonment of a stream which might have remaining header blocks
• Receipt of new dynamic table entries

Knowledge that a header block with references to the dynamic table has been processed permits the encoder to evict entries to which no unacknowledged references remain, regardless of whether those references were potentially blocking (see Section 2.1.2). When a stream is reset or abandoned, the indication that these header blocks will never be processed serves a similar function; see Section 4.4.3.¶

The decoder chooses when to emit Table State Synchronize instructions (see Section 4.4.1). Emitting an instruction after adding each new dynamic table entry will provide the most timely feedback to the encoder, but could be redundant with other decoder feedback. By delaying a Table State Synchronize instruction, the decoder might be able to coalesce multiple Table State Synchronize instructions, or replace them entirely with Header Acknowledgements (see Section 4.4.2). However, delaying too long may lead to compression inefficiencies if the encoder waits for an entry to be acknowledged before using it.¶

To track blocked streams, the necessary Largest Reference value for each stream can be used. Whenever the decoder processes a table update, it can begin decoding any blocked streams that now have their dependencies satisfied.¶

The encoder decides how to update the dynamic table and as such can control how much memory is used by the dynamic table. To limit the memory requirements of the decoder, the dynamic table size is strictly bounded.¶

The decoder determines the maximum size that the encoder is permitted to use for the dynamic table. In HTTP/3, this value is determined by the SETTINGS_HEADER_TABLE_SIZE setting (see Section 5).¶

An encoder can choose to use less capacity than this maximum size (see Section 4.3.4), but the chosen size MUST stay lower than or equal to the maximum set by the decoder. Whenever the maximum size for the dynamic table is reduced, entries are evicted from the end of the dynamic table until the size of the dynamic table is less than or equal to the maximum size.¶

This mechanism can be used to completely clear entries from the dynamic table by setting a maximum size of 0, which can subsequently be restored.¶

The size of the dynamic table is the sum of the size of its entries.¶

The size of an entry is the sum of its name’s length in bytes (as defined in Section 4.1.2), its value’s length in bytes, and 32.¶

The size of an entry is calculated using the length of its name and value without any Huffman encoding applied.¶

MaxEntries is the maximum number of entries that the dynamic table can have. The smallest entry has empty name and value strings and has the size of 32. The MaxEntries is calculated as¶

MaxTableSize is the maximum size of the dynamic table as specified by the decoder (see Section 3.2.1).¶

Each entry possesses both an absolute index which is fixed for the lifetime of that entry and a relative index which changes based on the context of the reference. The first entry inserted has an absolute index of “1”; indices increase sequentially with each insertion.¶

The relative index begins at zero and increases in the opposite direction from the absolute index. Determining which entry has a relative index of “0” depends on the context of the reference.¶

On the encoder stream, a relative index of “0” always refers to the most recently inserted value in the dynamic table. Note that this means the entry referenced by a given relative index will change while interpreting instructions on the encoder stream.¶

Because frames from request streams can be delivered out of order with instructions on the encoder stream, relative indices are relative to the Base Index at the beginning of the header block (see Section 4.5.1). The Base Index is an absolute index. When interpreting the rest of the frame, the entry identified by Base Index has a relative index of zero. The relative indices of entries do not change while interpreting headers on a request or push stream.¶

A header block on the request stream can reference entries added after the entry identified by the Base Index. This allows an encoder to process a header block in a single pass and include references to entries added while processing this (or other) header blocks. Newly added entries are referenced using Post-Base instructions. Indices for Post-Base instructions increase in the same direction as absolute indices, but the zero value is one higher than the Base Index.¶

If the decoder encounters a reference on a request or push stream to a dynamic table entry which has already been evicted or which has an absolute index greater than the declared Largest Reference (see Section 4.5.1), it MUST treat this as a stream error of type HTTP_QPACK_DECOMPRESSION_FAILED.¶

If the decoder encounters a reference on the encoder stream to a dynamic table entry which has already been dropped, it MUST treat this as a connection error of type HTTP_QPACK_ENCODER_STREAM_ERROR.¶

The prefixed integer from Section 5.1 of [RFC7541] is used heavily throughout this document. The format from [RFC7541] is used unmodified. QPACK implementations MUST be able to decode integers up to 62 bits long.¶

The string literal defined by Section 5.2 of [RFC7541] is also used throughout. This string format includes optional Huffman encoding.¶

HPACK defines string literals to begin on a byte boundary. They begin with a single flag (indicating whether the string is Huffman-coded), followed by the Length encoded as a 7-bit prefix integer, and finally Length bytes of data. When Huffman encoding is enabled, the Huffman table from Appendix B of [RFC7541] is used without modification.¶

This document expands the definition of string literals and permits them to begin other than on a byte boundary. An “N-bit prefix string literal” begins with the same Huffman flag, followed by the length encoded as an (N-1)-bit prefix integer. The remainder of the string literal is unmodified.¶

A string literal without a prefix length noted is an 8-bit prefix string literal and follows the definitions in [RFC7541] without modification.¶

An addition to the header table where the header field name matches the header field name of an entry stored in the static table or the dynamic table starts with the ‘1’ one-bit pattern. The S bit indicates whether the reference is to the static (S=1) or dynamic (S=0) table. The 6-bit prefix integer (see Section 5.1 of [RFC7541]) that follows is used to locate the table entry for the header name. When S=1, the number represents the static table index; when S=0, the number is the relative index of the entry in the dynamic table.¶

The header name reference is followed by the header field value represented as a string literal (see Section 5.2 of [RFC7541]).¶

An addition to the header table where both the header field name and the header field value are represented as string literals (see Section 4.1) starts with the ‘01’ two-bit pattern.¶

The name is represented as a 6-bit prefix string literal, while the value is represented as an 8-bit prefix string literal.¶

Duplication of an existing entry in the dynamic table starts with the ‘000’ three-bit pattern. The relative index of the existing entry is represented as an integer with a 5-bit prefix.¶

The existing entry is re-inserted into the dynamic table without resending either the name or the value. This is useful to mitigate the eviction of older entries which are frequently referenced, both to avoid the need to resend the header and to avoid the entry in the table blocking the ability to insert new headers.¶

An encoder informs the decoder of a change to the size of the dynamic table using an instruction which begins with the ‘001’ three-bit pattern. The new maximum table size is represented as an integer with a 5-bit prefix (see Section 5.1 of [RFC7541]).¶

The new maximum size MUST be lower than or equal to the limit determined by the protocol using QPACK. A value that exceeds this limit MUST be treated as a connection error of type HTTP_QPACK_ENCODER_STREAM_ERROR. In HTTP/3, this limit is the value of the SETTINGS_HEADER_TABLE_SIZE parameter (see Section 5) received from the decoder.¶

Reducing the maximum size of the dynamic table can cause entries to be evicted (see Section 4.3 of [RFC7541]). This MUST NOT cause the eviction of entries with outstanding references (see Section 2.1.1). Changing the size of the dynamic table is not acknowledged as this instruction does not insert an entry.¶

The Table State Synchronize instruction begins with the ‘00’ two-bit pattern. The instruction specifies the total number of dynamic table inserts and duplications since the last Table State Synchronize or Header Acknowledgement that increased the Largest Known Received dynamic table entry (see Section 2.1.4). This is encoded as a 6-bit prefix integer. The encoder uses this value to determine which table entries might cause a stream to become blocked, as described in Section 2.2.1.¶

An encoder that receives an Insert Count equal to zero or one that increases Largest Known Received beyond what the encoder has sent MUST treat this as a connection error of type HTTP_QPACK_DECODER_STREAM_ERROR.¶

After processing a header block whose declared Largest Reference is not zero, the decoder emits a Header Acknowledgement instruction on the decoder stream. The instruction begins with the ‘1’ one-bit pattern and includes the request stream’s stream ID, encoded as a 7-bit prefix integer. It is used by the peer’s encoder to know when it is safe to evict an entry, and possibly update Largest Known Received.¶

The same Stream ID can be identified multiple times, as multiple header blocks can be sent on a single stream in the case of intermediate responses, trailers, and pushed requests. Since header frames on each stream are received and processed in order, this gives the encoder precise feedback on which header blocks within a stream have been fully processed.¶

If an encoder receives a Header Acknowledgement instruction referring to a stream on which every header block with a non-zero Largest Reference has already been acknowledged, that MUST be treated as a connection error of type HTTP_QPACK_DECODER_STREAM_ERROR.¶

When blocking references are permitted, the encoder uses acknowledgement of header blocks to update the Largest Known Received index. If a header block was potentially blocking, the acknowledgement implies that the decoder has received all dynamic table state necessary to process the header block. If the Largest Reference of an acknowledged header block was greater than the encoder’s current Largest Known Received index, the block’s Largest Reference becomes the new Largest Known Received.¶

The instruction begins with the ‘01’ two-bit pattern. The instruction includes the stream ID of the affected stream - a request or push stream - encoded as a 6-bit prefix integer.¶

A stream that is reset might have multiple outstanding header blocks with dynamic table references. When an endpoint receives a stream reset before the end of a stream, it generates a Stream Cancellation instruction on the decoder stream. Similarly, when an endpoint abandons reading of a stream it needs to signal this using the Stream Cancellation instruction. This signals to the encoder that all references to the dynamic table on that stream are no longer outstanding. A decoder with a maximum dynamic table size equal to zero MAY omit sending Stream Cancellations, because the encoder cannot have any dynamic table references.¶

An encoder cannot infer from this instruction that any updates to the dynamic table have been received.¶

Header data is prefixed with two integers, Largest Reference and Base Index.¶

An indexed header field representation identifies an entry in either the static table or the dynamic table and causes that header field to be added to the decoded header list, as described in Section 3.2 of [RFC7541].¶

If the entry is in the static table, or in the dynamic table with an absolute index less than or equal to Base Index, this representation starts with the ‘1’ 1-bit pattern, followed by the S bit indicating whether the reference is into the static (S=1) or dynamic (S=0) table. Finally, the relative index of the matching header field is represented as an integer with a 6-bit prefix (see Section 5.1 of [RFC7541]).¶

If the entry is in the dynamic table with an absolute index greater than Base Index, the representation starts with the ‘0001’ 4-bit pattern, followed by the post-base index (see Section 3.2.5) of the matching header field, represented as an integer with a 4-bit prefix (see Section 5.1 of [RFC7541]).¶

A literal header field with a name reference represents a header where the header field name matches the header field name of an entry stored in the static table or the dynamic table.¶

If the entry is in the static table, or in the dynamic table with an absolute index less than or equal to Base Index, this representation starts with the ‘01’ two-bit pattern. If the entry is in the dynamic table with an absolute index greater than Base Index, the representation starts with the ‘0000’ four-bit pattern.¶

The following bit, ‘N’, indicates whether an intermediary is permitted to add this header to the dynamic header table on subsequent hops. When the ‘N’ bit is set, the encoded header MUST always be encoded with a literal representation. In particular, when a peer sends a header field that it received represented as a literal header field with the ‘N’ bit set, it MUST use a literal representation to forward this header field. This bit is intended for protecting header field values that are not to be put at risk by compressing them (see Section 7.1 of [RFC7541] for more details).¶

For entries in the static table or in the dynamic table with an absolute index less than or equal to Base Index, the header field name is represented using the relative index of that entry, which is represented as an integer with a 4-bit prefix (see Section 5.1 of [RFC7541]). The S bit indicates whether the reference is to the static (S=1) or dynamic (S=0) table.¶

For entries in the dynamic table with an absolute index greater than Base Index, the header field name is represented using the post-base index of that entry (see Section 3.2.5) encoded as an integer with a 3-bit prefix.¶

An addition to the header table where both the header field name and the header field value are represented as string literals (see Section 4.1) starts with the ‘001’ three-bit pattern.¶

The fourth bit, ‘N’, indicates whether an intermediary is permitted to add this header to the dynamic header table on subsequent hops. When the ‘N’ bit is set, the encoded header MUST always be encoded with a literal representation. In particular, when a peer sends a header field that it received represented as a literal header field with the ‘N’ bit set, it MUST use a literal representation to forward this header field. This bit is intended for protecting header field values that are not to be put at risk by compressing them (see Section 7.1 of [RFC7541] for more details).¶

The name is represented as a 4-bit prefix string literal, while the value is represented as an 8-bit prefix string literal.¶