Google LLC

d@domenic.me

Google LLC

jbroman@chromium.org

Google LLC

nidhijaju@chromium.org

Web and Internet Transport HyperText Transfer Protocol http caching This specification defines an extension to HTTP Caching, changing how the URI query component impacts caching. It introduces the "No-Vary-Search" response header field, which allows origin servers to signal to caches that certain parts of the query component do not semantically affect the served response and can be ignored for cache matching purposes. About This Document The latest revision of this draft can be found at . Status information for this document may be found at . Discussion of this document takes place on the HTTP Working Group mailing list (), which is archived at . Working Group information can be found at . Source for this draft and an issue tracker can be found at .

Introduction HTTP caching is based on reusing resources which match across a number of cache keys, with the most important one being the presented target URI (). However, sometimes multiple URIs can represent the same resource. This leads to caches not always being as helpful as they could be: if the cache contains a response under one URI, but the response is then requested under another, the cached version will be ignored. The "No-Vary-Search" response header field defines a caching extension, as described in , that tackles a specific subset of this general problem, for when different URIs that differ only in their query component identify the same resource. It allows resources to declare that some or all parts of the query component do not semantically affect the served response, and thus can be ignored for cache matching purposes. This is achieved by interpreting the query component as a sequence of parameters encoded using the application/x-www-form-urlencoded format . For example, if the order of the parameters within the query component does not affect which resource is identified, this is indicated using No-Vary-Search: key-order If specific query parameters (e.g., ones indicating something for analytics) do not semantically affect the served resource, this is indicated using No-Vary-Search: params=("utm_source" "utm_medium" "utm_campaign") And if the resource instead wants to take an allowlist-based approach, where only certain known query parameters semantically affect the served response, they can use No-Vary-Search: except=("productId") Note that "cache busting", the practice of changing a part of the query component to create a distinct cache key and force retrieval of a newer response, can be made ineffective by the "No-Vary-Search" response header field. defines the new "No-Vary-Search" response header field, using the framework. and illustrate the data model for how the field value can be represented in specifications, and the process for parsing the raw output from the structured field parser into that data model. gives the key algorithm for comparing if two URLs are equivalent under the influence of the header field; notably, it leans on the decomposition of the query component into keys and values given by the application/x-www-form-urlencoded format specified in . (As such, this header field is not useful for URLs whose query component does not follow that format.) Finally, explains how to extend to take this new equivalence into account. From a deployment perspective, this extension is implemented by HTTP caches, including browser caches, content delivery networks, and forward proxies. Origin servers send the "No-Vary-Search" response header field to provide instructions to these caches. Caches that implement this extension use these instructions to determine when a previously stored response can be safely reused for a new request, even if the query components of the target URIs differ.

Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here. In this document, the terms "URI" and "URL" are used interchangeably, depending on context. "URI" is used in the context of , , and , whereas "URL" is used in the context of the algorithms specified in . The term "query parameters" in this document refers to the keys and values resulting from parsing a URL's query component using the application/x-www-form-urlencoded format . This document also adopts some conventions and notation typical in WHATWG and W3C usage, especially as it relates to algorithms. See , and in particular:

• its definition of lists, including the list literal notation « 1, 2, 3 ».
• its definition of strings, including their representation as code units.

(Other concepts used are called out using inline references.)

HTTP header field definition The "No-Vary-Search" response header field is a structured field whose value MUST be a dictionary (). It has the following constraints:

• If present, the key-order entry's value MUST be a boolean ().
• If present, the params entry's value MUST be an inner list of strings ().
• If present, the except entry's value MUST be an inner list of strings ().
• The except entry MUST NOT be present if the params entry is also present.

The dictionary MAY contain entries whose keys are not one of key-order, params, and except, but their meaning is not defined by this specification. Implementations of this specification will ignore such entries (but future documents might assign meaning to such entries). Future extensions to this dictionary MUST NOT restrict the set of URIs that are considered equivalent; they can only expand it. If a future extension requires restricting equivalence, it MUST be deployed as a new HTTP header field to ensure safety. The "No-Vary-Search" response header field is set by origin servers. Intermediaries MUST NOT insert, delete, or modify the field's value unless they are acting as the origin server for that response.

Data model A URL variation config consists of the following:

no-vary params

either the special value wildcard or a list of strings

vary params

either the special value wildcard or a list of strings

vary on key order

a boolean

The default URL variation config is a URL variation config whose no-vary params is an empty list, vary params is wildcard, and vary on key order is true. The obtain a URL variation config algorithm () ensures that all URL variation configs obey the following constraints:

• vary params is a list if and only if the no-vary params is wildcard; and
• no-vary params is a list if and only if the vary params is wildcard.

Parsing

Parse a URL variation config To parse a URL variation config given value:

1. If value is null, then return the default URL variation config.
2. Let result be a new URL variation config.
3. Set result's vary on key order to true.
4. If value["key-order"] exists:
   1. Let keyOrderValue be the item_or_inner_list component of the tuple value["key-order"] (ignoring any parameters).
   2. If keyOrderValue is not a boolean, then return the default URL variation config.
   3. Set result's vary on key order to the boolean negation of keyOrderValue.
5. If both value["params"] and value["except"] exist, then return the default URL variation config.
6. If neither value["params"] nor value["except"] exists:
   1. Set result's no-vary params to an empty list.
   2. Set result's vary params to wildcard.
7. If value["params"] exists:
   1. Let paramsValue be the item_or_inner_list component of the tuple value["params"] (ignoring any parameters).
   2. If paramsValue is not an inner list, then return the default URL variation config.
   3. Let paramsList be a list containing the bare_item component of each tuple in paramsValue (ignoring any parameters).
   4. If any item in paramsList is not a string, then return the default URL variation config.
   5. Set result's no-vary params to the result of applying parse a key () to each item in paramsList.
   6. Set result's vary params to wildcard.
8. Otherwise, if value["except"] exists:
   1. Let exceptValue be the item_or_inner_list component of the tuple value["except"] (ignoring any parameters).
   2. If exceptValue is not an inner list, then return the default URL variation config.
   3. Let exceptList be a list containing the bare_item component of each tuple in exceptValue (ignoring any parameters).
   4. If any item in exceptList is not a string, then return the default URL variation config.
   5. Set result's vary params to the result of applying parse a key () to each item in exceptList.
   6. Set result's no-vary params to wildcard.
9. Return result.

Obtain a URL variation config To obtain a URL variation config given an HTTP response () response:

1. Let fieldValue be the result of parsing the "No-Vary-Search" response header field from response as a Dictionary (). If parsing fails or the field is absent, let fieldValue be null.
2. Return the result of parsing a URL variation config () given fieldValue.

Examples The following illustrates how various inputs are parsed, in terms of their impact on the resulting no-vary params and vary params:

Input	Result
No-Vary-Search: key-order	no-vary params: (empty list) vary params: wildcard vary on key order: false
No-Vary-Search: params=("a")	no-vary params: « "a" » vary params: wildcard
No-Vary-Search: except=("x")	no-vary params: wildcard vary params: « "x" »
No-Vary-Search: params=()	no-vary params: (empty list) vary params: wildcard
No-Vary-Search: except=()	no-vary params: wildcard vary params: (empty list)

The following inputs are all invalid and will cause the default URL variation config to be returned:

Input	Explanation
No-Vary-Search: key-order="not a boolean"	key-order expects a boolean, not a string
No-Vary-Search: params="not an inner list"	params expects an inner list, not a string
No-Vary-Search: params=(not-a-string)	params items must be strings (tokens are invalid)
No-Vary-Search: params=?0	params expects an inner list, not a boolean
No-Vary-Search: params=?1	params expects an inner list, not a boolean
No-Vary-Search: params=?1, except=("x")	params and except cannot both be present
No-Vary-Search: params=("a"), except=("x")	params and except cannot both be present
No-Vary-Search: params=(), except=()	params and except cannot both be present
No-Vary-Search: except="not an inner list"	except expects an inner list, not a string
No-Vary-Search: except=(not-a-string)	except items must be strings (tokens are invalid)
No-Vary-Search: except=?1	except expects an inner list, not a boolean

The following inputs are valid, but somewhat unconventional. They are shown alongside their more conventional form.

Input	Conventional form
No-Vary-Search: key-order=?1	No-Vary-Search: key-order
No-Vary-Search: except=("x"), key-order	No-Vary-Search: key-order, except=("x")
No-Vary-Search: params=()	(omit the header field)
No-Vary-Search: key-order=?0	(omit the header field)

Parse a key To parse a key given an ASCII string keyString:

1. Let keyBytes be the isomorphic encoding of keyString.
2. Replace any 0x2B (+) in keyBytes with 0x20 (SP).
3. Let keyBytesDecoded be the percent-decoding of keyBytes.
4. Let keyStringDecoded be the UTF-8 decoding without BOM of keyBytesDecoded.
5. Return keyStringDecoded.

Examples The parse a key algorithm allows encoding non-ASCII key strings in the ASCII structured header field format, similar to how the application/x-www-form-urlencoded format allows encoding an entire entry list of keys and values in a URI (which is restricted to ASCII characters). For example: No-Vary-Search: params=("%C3%A9+%E6%B0%97") Notice that while the input string "%C3%A9+%E6%B0%97" consists entirely of ASCII characters (as required at the HTTP layer), the percent-decoding step used by the cache produces a non-ASCII result. This will result in a URL variation config whose no-vary params are « "é 気" ». Note that the "+" character in the encoded string is mapped to a space (SP). As explained in a later example, the canonicalization process during equivalence testing means this will treat as equivalent URIs such as:

• https://example.com/?é 気=1
• https://example.com/?é+気=2
• https://example.com/?%C3%A9%20気=3
• https://example.com/?%C3%A9+%E6%B0%97=4

and so on, since they all are parsed to having the same key "é 気".

Comparing Two URLs urlA and urlB are equivalent modulo variation config given a URL variation config variationConfig if the following algorithm returns true:

1. If the scheme, host, port, or path of urlA and urlB differ, then return false.
2. If variationConfig is equivalent to the default URL variation config, then:
   1. If urlA's query equals urlB's query, then return true.
   2. Return false.
   In this case, even URL pairs that might appear the same after running the application/x-www-form-urlencoded parser on their queries, such as https://example.com/a and https://example.com/a?, or https://example.com/foo?a=b&&&c and https://example.com/foo?a=b&c=, will be treated as inequivalent.
3. Let searchParamsA and searchParamsB be empty lists.
4. If urlA's query is not null, then set searchParamsA to the result of running the application/x-www-form-urlencoded parser given the isomorphic encoding of urlA's query.
5. If urlB's query is not null, then set searchParamsB to the result of running the application/x-www-form-urlencoded parser given the isomorphic encoding of urlB's query.
6. If variationConfig's no-vary params is a list, then:
   1. Set searchParamsA to a list containing those items pair in searchParamsA where variationConfig's no-vary params does not contain pair[0].
   2. Set searchParamsB to a list containing those items pair in searchParamsB where variationConfig's no-vary params does not contain pair[0].
7. Otherwise, if variationConfig's vary params is a list, then:
   1. Set searchParamsA to a list containing those items pair in searchParamsA where variationConfig's vary params contains pair[0].
   2. Set searchParamsB to a list containing those items pair in searchParamsB where variationConfig's vary params contains pair[0].
8. If variationConfig's vary on key order is false, then:
   1. Let keyLessThan be an algorithm taking as inputs two pairs (keyA, valueA) and (keyB, valueB), which returns whether keyA is code unit less than keyB.
   2. Set searchParamsA to the result of sorting searchParamsA in ascending order with keyLessThan.
   3. Set searchParamsB to the result of sorting searchParamsB in ascending order with keyLessThan.
9. If searchParamsA's size is not equal to searchParamsB's size, then return false.
10. Let i be 0.
11. While i < searchParamsA's size:
    1. If searchParamsA[i][0] does not equal searchParamsB[i][0], then return false.
    2. If searchParamsA[i][1] does not equal searchParamsB[i][1], then return false.
    3. Set i to i + 1.
12. Return true.

Examples Due to how the application/x-www-form-urlencoded parser canonicalizes query strings, there are some cases where query strings which do not appear obviously equivalent, will end up being treated as equivalent after parsing. So, for example, given any non-default value for the "No-Vary-Search" response header field, such as No-Vary-Search: key-order, we will have the following equivalences:

First Query	Second Query	Explanation
null	?	A null query is parsed the same as an empty string
?a=x	?%61=%78	Parsing performs percent-decoding
?a=é	?a=%C3%A9	Parsing performs percent-decoding
?a=%f6	?a=%ef%bf%bd	An invalid UTF-8 sequence and the literal U+FFFD character are both parsed as U+FFFD ( )
?a=x&&&&	?a=x	Parsing splits on & and discards empty strings
?a=	?a	Both parse as having an empty string value for a
?a=%20	?a= &	%20 is parsed as U+0020 SPACE
?a=+	?a= &	+ is parsed as U+0020 SPACE

Note that no Unicode normalization is performed during this comparison. For example, a query string of ?a=%C3%A9 (using the NFC encoding of é) and ?a=e%CC%81 (using the NFD encoding of é) will not be treated as equivalent.

Caching To reuse a stored response, requires that the presented target URI and that of the stored response match. If a cache implements the No-Vary-Search extension, this matching requirement is also satisfied if the URIs are equivalent modulo URL variation config () given the stored response's No-Vary-Search header. This document does not alter the requirements for cache invalidation (see ). A cache MAY invalidate stored responses for URIs that are equivalent modulo URL variation config, but is not required to do so. Therefore, state-changing requests might not invalidate all conceptually equivalent responses. Note that the "No-Vary-Search" response header field operates in addition to content negotiation and the Vary header field (see ). Cache implementations MAY fail to reuse a stored response whose target URI matches only modulo URL variation config, if the cache has a stored response with a more recent Date header field which:

• has a target URI which is equal to the presented target URI, excluding the query, and
• has a non-empty value for the "No-Vary-Search" response header field, and
• has a "No-Vary-Search" response header field value different from the stored response being considered for reuse.

When a cache has multiple stored responses with conflicting "No-Vary-Search" values, preferring the response with the most recent Date header field helps ensure caches converge on the origin's latest caching policy. To aid cache implementation efficiency, servers SHOULD NOT send different non-empty values for the "No-Vary-Search" response header field in response to requests for a given target URI path and authority over time, unless there is a need to update how they handle the query component. Doing so would cause cache implementations that use a strategy like the above to miss some stored responses that could otherwise have been reused.

Security Considerations The main risk to be aware of is a cache returning a response that was originally fetched from a URL different from the one requested. In a web browser, this could cause the user to see a response fetched from a URL different from the one displayed when they hovered a link, or the URL displayed in the URL bar. For shared caches, such as CDNs or forward proxies, returning a response for a different URL carries the risk of cross-user state leakage. If a server incorrectly declares that a query parameter does not affect the response, but that parameter actually dictates user-specific or sensitive content, the shared cache might serve one user's personalized response to another user. However, because the origin strictly controls the "No-Vary-Search" response header field, it is the origin's responsibility to ensure that ignored parameters are safe to disregard for all users. The "No-Vary-Search" response header field alters the algorithm that caches use for URI identifier comparison. As discussed in , altering identifier comparison logic can lead to security issues, primarily through "false positives" where two identifiers are incorrectly deemed equivalent. Because URL query parsing replaces invalid percent-encoded UTF-8 sequences with U+FFFD, lossy decoding can map distinct query strings onto the same cache key (for example, ?a=%f6 and ?a=%ef%bf%bd). Origins should not rely on invalid percent-encoded sequences being distinguishable, as this is a concrete example of the false positives warned about in . Incorrect configuration of this field can exacerbate cache poisoning or data leakage risks by causing such false positives. Origin servers MUST NOT declare a parameter as no-vary if doing so would bypass server processing required for safe response reuse. This includes parameters used for authorization, user identification, signature verification, user consent, routing, auditing, revocation, or any other security-sensitive operations. However, since the impact is limited to query parameters, this does not cross the relevant security boundary, which is the origin (). (See also the host from the perspective of web browser security UI ). Indeed, origins already have complete control over how they present URLs and response bodies, including on the client side via technology such as history.replaceState() or service workers.

Privacy Considerations This proposal is adjacent to the highly-privacy-relevant space of navigational tracking, which often uses query parameters to pass along user identifiers. If an origin were to encode user identifiers in its URI, this proposal can reduce user tracking by private caches, since preventing server processing of such user IDs bypasses the server in favor of the cache. It does not interfere with existing navigational tracking mitigations, or any known future ones being contemplated. However, this tracking reduction does not fully apply to shared caches (such as content delivery networks and forward proxies), which still receive the requests containing the identifiers. Furthermore, an errant configuration that incorrectly ignores parameters related to user identity or private state could expose cached content meant for one user to another. While this mistake can occur with standard caching, the "No-Vary-Search" response header field increases the surface area for such misconfigurations, making it critical that origins accurately classify their query parameters.

IANA Considerations

HTTP Field Names IANA is requested to enter the following into the Hypertext Transfer Protocol (HTTP) Field Name Registry (https://www.iana.org/assignments/http-fields/http-fields.xhtml):

Field Name:

No-Vary-Search

Status:

permanent

Structured Type:

Dictionary

Reference:

this document

Comments:

(none)

No-Vary-Search Dictionary Keys Registry IANA is requested to create a new registry, "No-Vary-Search Dictionary Keys", at https://www.iana.org/assignments/http-fields/. The registration policy is "IETF Review" (see ). A registration request MUST include the following fields:

• Key: the dictionary key for the "No-Vary-Search" response header field
• Description: a brief description of the key's purpose
• Reference: a pointer to the specification that defines the key

The initial contents of this registry are:

Key	Description	Reference
key-order	Indicates if query parameter order affects caching	this document
params	A list of query parameters that do not affect caching	this document
except	A list of query parameters that affect caching	this document

References Normative References Apple Inc. WHATWG Apple Inc. Google LLC WHATWG Apple Inc. WHATWG Informative References Apple Inc. WHATWG Brave Software, Inc. Google LLC n.d. W3C Privacy CG

Acknowledgments This document benefited from valuable reviews and suggestions by:

• Adam Rice
• Julian Reschke
• Kevin McNee
• Liviu Tinta
• Mark Nottingham
• Martin Thomson
• Valentin Gosu