DeNA Co, Ltd.

kazuhooku@gmail.com

mnot@mnot.net https://www.mnot.net/

Applications and Real-Time HTTP Internet-Draft This specification defines a HTTP/2 frame type to allow clients to inform the server of their cache’s contents. Servers can then use this to inform their choices of what to push to clients. Discussion of this draft takes place on the HTTP working group mailing list (ietf-http-wg@w3.org), which is archived at https://lists.w3.org/Archives/Public/ietf-http-wg/. Working Group information can be found at http://httpwg.github.io/; source code and issues list for this draft can be found at https://github.com/httpwg/http-extensions/labels/cache-digest.

HTTP/2 allows a server to “push” synthetic request/response pairs into a client’s cache optimistically. While there is strong interest in using this facility to improve perceived Web browsing performance, it is sometimes counterproductive because the client might already have cached the “pushed” response. When this is the case, the bandwidth used to “push” the response is effectively wasted, and represents opportunity cost, because it could be used by other, more relevant responses. HTTP/2 allows a stream to be cancelled by a client using a RST_STREAM frame in this situation, but there is still at least one round trip of potentially wasted capacity even then. This specification defines a HTTP/2 frame type to allow clients to inform the server of their cache’s contents using a Golomb-Rice Coded Set . Servers can then use this to inform their choices of what to push to clients.

The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in .

The CACHE_DIGEST frame type is 0xd (decimal 13).

The CACHE_DIGEST frame payload has the following fields: An unsigned, 16-bit integer indicating the length, in octets, of the Origin field. A sequence of characters containing the ASCII serialization of an origin (, Section 6.2) that the Digest-Value applies to. A sequence of octets containing the digest as computed in . The CACHE_DIGEST frame defines the following flags: RESET (0x1): When set, indicates that any and all cache digests for the applicable origin held by the recipient MUST be considered invalid. COMPLETE (0x2): When set, indicates that the currently valid set of cache digests held by the server constitutes a complete representation of the cache’s state regarding that origin, for the type of cached response indicated by the STALE flag. VALIDATORS (0x4): When set, indicates that the validators boolean in is true. STALE (0x8): When set, indicates that all cached responses represented in the digest-value are stale at the point in them that the digest was generated; otherwise, all are fresh.

A CACHE_DIGEST frame MUST be sent from a client to a server on stream 0, and conveys a digest of the contents of the client’s cache for the indicated origin. In typical use, a client will send one or more CACHE_DIGESTs immediately after the first request on a connection for a given origin, on the same stream, because there is usually a short period of inactivity then, and servers can benefit most when they understand the state of the cache before they begin pushing associated assets (e.g., CSS, JavaScript and images). Clients MAY send CACHE_DIGEST at other times. If the cache’s state is cleared, lost, or the client otherwise wishes the server to stop using previously sent CACHE_DIGESTs, it can send a CACHE_DIGEST with the RESET flag set. When generating CACHE_DIGEST, a client MUST NOT include cached responses whose URLs do not share origins with the indicated origin. Clients MUST NOT send CACHE_DIGEST frames on connections that are not authoritative (as defined in , 10.1) for the indicated origin. CACHE_DIGEST allows the client to indicate whether the set of URLs used to compute the digest represent fresh or stale stored responses, using the STALE flag. Clients MAY decide whether to only sent CACHE_DIGEST frames representing their fresh stored responses, their stale stored responses, or both. Clients can choose to only send a subset of the suitable stored responses of each type (fresh or stale). However, when the CACHE_DIGEST frames sent represent the complete set of stored responses of a given type, the last such frame SHOULD have a COMPLETE flag set, to indicate to the server that it has all relevant state of that type. Note that for the purposes of COMPLETE, responses cached since the beginning of the connection or the last RESET flag on a CACHE_DIGEST frame need not be included. CACHE_DIGEST can be computed to include cached responses’ ETags, as indicated by the VALIDATORS flag. This information can be used by servers to decide what kinds of responses to push to clients; for example, a stale response that hasn’t changed could be refreshed with a 304 (Not Modified) response; one that has changed can be replaced with a 200 (OK) response, whether the cached response was fresh or stale. CACHE_DIGEST has no defined meaning when sent from servers, and SHOULD be ignored by clients.

Given the following inputs: validators, a boolean indicating whether validators () are to be included in the digest; URLs', an array of (string URL, string ETag) tuples, each corresponding to the Effective Request URI (, Section 5.5) of a cached response and its entity-tag (if validators is true and if the ETag is available; otherwise, null); P, an integer that MUST be a power of 2 smaller than 2**32, that indicates the probability of a false positive that is acceptable, expressed as 1/P. digest-value can be computed using the following algorithm: Let N be the count of URLs’ members, rounded to the nearest power of 2 smaller than 2**32. Let hash-values be an empty array of integers. For each (URL, ETag) in URLs, compute a hash value () and append the result to hash-values. Sort hash-values in ascending order. Let digest-value be an empty array of bits. Write log base 2 of N to digest-value using 5 bits. Write log base 2 of P to digest-value using 5 bits. Let C be -1. For each V in hash-values: If V is equal to C, continue to the next V. Let D be the result of V - C - 1. Let Q be the integer result of D / P. Let R be the result of D modulo P. Write Q ‘0’ bits to digest-value. Write 1 ‘1’ bit to digest-value. Write R to digest-value as binary, using log2(P) bits. Let C be V If the length of digest-value is not a multiple of 8, pad it with 0s until it is.

Given: URL, an array of characters ETag, an array of characters validators, a boolean N, an integer P, an integer hash-value can be computed using the following algorithm: Let key be URL converted to an ASCII string by percent-encoding as appropriate . If validators is true and ETag is not null: Append ETag to key as an ASCII string, including both the weak indicator (if present) and double quotes, as per Section 2.3. Let hash-value be the SHA-256 message digest of key, expressed as an integer. Truncate hash-value to log2( N * P ) bits.

In typical use, a server will query (as per ) the CACHE_DIGESTs received on a given connection to inform what it pushes to that client; If a given URL has a match in a current CACHE_DIGEST with the STALE flag unset, it need not be pushed, because it is fresh in cache; If a given URL and ETag combination has a match in a current CACHE_DIGEST with the STALE flag set, the client has a stale copy in cache, and a validating response can be pushed; If a given URL has no match in any current CACHE_DIGEST, the client does not have a cached copy, and a complete response can be pushed. Servers MAY use all CACHE_DIGESTs received for a given origin as current, as long as they do not have the RESET flag set; a CACHE_DIGEST frame with the RESET flag set MUST clear any previously stored CACHE_DIGESTs for its origin. Servers MUST treat an empty Digest-Value with a RESET flag set as effectively clearing all stored digests for that origin. Clients are not likely to send updates to CACHE_DIGEST over the lifetime of a connection; it is expected that servers will separately track what cacheable responses have been sent previously on the same connection, using that knowledge in conjunction with that provided by CACHE_DIGEST. Servers MUST ignore CACHE_DIGEST frames sent on a stream other than 0.

Given: digest-value, an array of bits URL, an array of characters ETag, an array of characters validators, a boolean we can determine whether there is a match in the digest using the following algorithm: Read the first 5 bits of digest-value as an integer; let N be two raised to the power of that value. Read the next 5 bits of digest-value as an integer; let P be two raised to the power of that value. Let hash-value be the result of computing a hash value (). Let C be -1. Read ‘0’ bits from digest-value until a ‘1’ bit is found; let Q be the number of ‘0’ bits. Discard the ‘1’. Read log2(P) bits from digest-value after the ‘1’ as an integer; let R be its value. Let D be Q * P + R. Increment C by D + 1. If C is equal to hash-value, return ‘true’. Otherwise, return to step 5 and continue processing; if no match is found before digest-value is exhausted, return ‘false’.

A server can notify its support for CACHE_DIGEST frame by sending the ACCEPT_CACHE_DIGEST (0x7) SETTINGS parameter. If the server is tempted to making optimizations based on CACHE_DIGEST frames, it SHOULD send the SETTINGS parameter immediately after the connection is established. The value of the parameter is a bit-field of which the following bits are defined: FRESH (0x1): When set, it indicates that the server is willing to make use of a digest of freshly-cached responses. STALE (0x2): When set, it indicates that the server is willing to make use of a digest of stale-cached responses. Rest of the bits MUST be ignored and MUST be left unset when sending. The initial value of the parameter is zero (0x0) meaning that the server is not interested in seeing a CACHE_DIGEST frame. Some underlying transports allow the server’s first flight of application data to reach the client at around the same time when the client sends it’s first flight data. When such transport (e.g., TLS 1.3 in full-handshake mode) is used, a client can postpone sending the CACHE_DIGEST frame until it receives a ACCEPT_CACHE_DIGEST settings value. When the underlying transport does not have such property (e.g., TLS 1.3 in 0-RTT mode), a client can reuse the settings value found in previous connections to that origin to make assumptions.

This document registers the following entry in the Permanent Message Headers Registry, as per : Header field name: Cache-Digest Applicable protocol: http Status: experimental Author/Change controller: IESG Specification document(s): [this document] This document registers the following entry in the HTTP/2 Frame Type Registry, as per : Frame Type: CACHE_DIGEST Code: 0xd Specification: [this document] This document registers the following entry in the HTTP/2 Settings Registry, as per : Code: 0x7 Name: ACCEPT_CACHE_DIGEST Initial Value: 0x0 Reference: [this document]

The contents of a User Agent’s cache can be used to re-identify or “fingerprint” the user over time, even when other identifiers (e.g., Cookies ) are cleared. CACHE_DIGEST allows such cache-based fingerprinting to become passive, since it allows the server to discover the state of the client’s cache without any visible change in server behaviour. As a result, clients MUST mitigate for this threat when the user attempts to remove identifiers (e.g., “clearing cookies”). This could be achieved in a number of ways; for example: by clearing the cache, by changing one or both of N and P, or by adding new, synthetic entries to the digest to change its contents. TODO: discuss how effective the suggested mitigations actually would be. Additionally, User Agents SHOULD NOT send CACHE_DIGEST when in “privacy mode.”

In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. A Uniform Resource Identifier (URI) is a compact sequence of characters that identifies an abstract or physical resource. This specification defines the generic URI syntax and a process for resolving URI references that might be in relative form, along with guidelines and security considerations for the use of URIs on the Internet. The URI syntax defines a grammar that is a superset of all valid URIs, allowing an implementation to parse the common components of a URI reference without knowing the scheme-specific requirements of every possible identifier. This specification does not define a generative grammar for URIs; that task is performed by the individual specifications of each URI scheme. [STANDARDS-TRACK] Federal Information Processing Standard, FIPS This document defines the concept of an "origin", which is often used as the scope of authority or privilege by user agents. Typically, user agents isolate content retrieved from different origins to prevent malicious web site operators from interfering with the operation of benign web sites. In addition to outlining the principles that underlie the concept of origin, this document details how to determine the origin of a URI and how to serialize an origin into a string. It also defines an HTTP header field, named "Origin", that indicates which origins are associated with an HTTP request. [STANDARDS-TRACK] The Hypertext Transfer Protocol (HTTP) is a stateless application-level protocol for distributed, collaborative, hypertext information systems. This document provides an overview of HTTP architecture and its associated terminology, defines the "http" and "https" Uniform Resource Identifier (URI) schemes, defines the HTTP/1.1 message syntax and parsing requirements, and describes related security concerns for implementations. The Hypertext Transfer Protocol (HTTP) is a stateless application- level protocol for distributed, collaborative, hypertext information systems. This document defines HTTP/1.1 conditional requests, including metadata header fields for indicating state changes, request header fields for making preconditions on such state, and rules for constructing the responses to a conditional request when one or more preconditions evaluate to false. The Hypertext Transfer Protocol (HTTP) is a stateless \%application- level protocol for distributed, collaborative, hypertext information systems. This document defines HTTP caches and the associated header fields that control cache behavior or indicate cacheable response messages. This specification describes an optimized expression of the semantics of the Hypertext Transfer Protocol (HTTP), referred to as HTTP version 2 (HTTP/2). HTTP/2 enables a more efficient use of network resources and a reduced perception of latency by introducing header field compression and allowing multiple concurrent exchanges on the same connection. It also introduces unsolicited push of representations from servers to clients.This specification is an alternative to, but does not obsolete, the HTTP/1.1 message syntax. HTTP's existing semantics remain unchanged. This document describes the commonly used base 64, base 32, and base 16 encoding schemes. It also discusses the use of line-feeds in encoded data, use of padding in encoded data, use of non-alphabet characters in encoded data, use of different encoding alphabets, and canonical encodings. [STANDARDS-TRACK] Internet technical specifications often need to define a formal syntax. Over the years, a modified version of Backus-Naur Form (BNF), called Augmented BNF (ABNF), has been popular among many Internet specifications. The current specification documents ABNF. It balances compactness and simplicity with reasonable representational power. The differences between standard BNF and ABNF involve naming rules, repetition, alternatives, order-independence, and value ranges. This specification also supplies additional rule definitions and encoding for a core lexical analyzer of the type common to several Internet specifications. [STANDARDS-TRACK] This document defines the HTTP Cookie and Set-Cookie header fields. These header fields can be used by HTTP servers to store state (called cookies) at HTTP user agents, letting the servers maintain a stateful session over the mostly stateless HTTP protocol. Although cookies have many historical infelicities that degrade their security and privacy, the Cookie and Set-Cookie header fields are widely used on the Internet. This document obsoletes RFC 2965. [STANDARDS-TRACK] This document specifies version 1.3 of the Transport Layer Security (TLS) protocol. TLS allows client/server applications to communicate over the Internet in a way that is designed to prevent eavesdropping, tampering, and message forgery. This specification defines registration procedures for the message header fields used by Internet mail, HTTP, Netnews and other applications. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements.

On some web browsers that support Service Workers but not Cache Digests (yet), it is possible to achieve the benefit of using Cache Digests by emulating the frame using HTTP Headers. For the sake of interoperability with such clients, this appendix defines how a CACHE_DIGEST frame can be encoded as an HTTP header named Cache-Digest. The definition uses the Augmented Backus-Naur Form (ABNF) notation of with the list rule extension defined in , Appendix B.

digest-flag = token ]]>

A Cache-Digest request header is defined as a list construct of cache-digest-entities. Each cache-digest-entity corresponds to a CACHE_DIGEST frame. Digest-Value is encoded using base64url , Section 5. Flags that are set are encoded as digest-flags by their names that are compared case-insensitively. Origin is omitted in the header form. The value is implied from the value of the :authority pseudo header. Client MUST only send Cache-Digest headers containing digests that belong to the origin specified by the HTTP request. The example below contains one digest of fresh resource and has only the COMPLETE flag set.

Clients MUST associate Cache-Digest headers to every HTTP request, since Fetch - the HTTP API supported by Service Workers - does not define the order in which the issued requests will be sent to the server nor guarantees that all the requests will be transmitted using a single HTTP/2 connection. Also, due to the fact that any header that is supplied to Fetch is required to be end-to-end, there is an ambiguity in what a Cache-Digest header respresents when a request is transmitted through a proxy. The header may represent the cache state of a client or that of a proxy, depending on how the proxy handles the header.

Thanks to Adam Langley and Giovanni Bajo for their explorations of Golomb-coded sets. In particular, see http://giovanni.bajo.it/post/47119962313/golomb-coded-sets-smaller-than-bloom-filters, which refers to sample code. Thanks to Stefan Eissing for his suggestions.

Added definition of the Cache-Digest header. Introduce ACCEPT_CACHE_DIGEST SETTINGS parameter. Change intended status from Standard to Experimental.

Make the scope of a digest frame explicit and shift to stream 0.