Collection Synchronization for Web Distributed Authoring and Versioning (WebDAV)

One way to synchronize data between two entities is to use some form of synchronization token. The token defines the state of the data being synchronized at a particular point in time. It can then be used to determine what has changed between one point in time and another. This specification defines a new WebDAV report that is used to enable client-server collection synchronization based on such a token. In order to synchronize the contents of a collection between a server and client, the server provides the client with a synchronization token each time the synchronization report is executed. That token represents the state of the data being synchronized at that point in time. The client can then present that same token back to the server at some later time, and the server will return only those items that are new, have changed, or were deleted since that token was generated. The server also returns a new token representing the new state at the time the report was run. Typically, the first time a client connects to the server it will need to be informed of the entire state of the collection (i.e., a full list of all member URLs that are currently in the collection). That is done by the client sending an empty token value to the server. This indicates to the server that a full listing is required. As an alternative, the client might choose to do its first synchronization using some other mechanism on the collection (e.g., some other form of batch resource information retrieval such as PROPFIND, SEARCH, or specialized REPORTs such as those defined in CalDAV and CardDAV) and ask for the DAV:sync-token property to be returned. This property (defined in ) contains the same token that can be used later to issue a DAV:sync-collection report. In some cases, a server might only wish to maintain a limited amount of history about changes to a collection. In that situation, it will return an error to the client when the client presents a token that is "out of date". At that point, the client has to fall back to synchronizing the entire collection by re-running the report request using an empty token value. Typically, a client will use the synchronization report to retrieve the list of changes and will follow that with requests to retrieve the content of changed resources. It is possible that additional changes to the collection could occur between the time of the synchronization report and resource content retrieval, which could result in an inconsistent view of the collection. When clients use this method of synchronization, they need to be aware that such additional changes could occur and track them, e.g., by differences between the ETag values returned in the synchronization report and those returned when actually fetching resource content, by using conditional requests as described in , or by repeating the synchronization process until no changes are returned.

If the DAV:sync-collection report is implemented by a WebDAV server, then the server MUST list the report in the "DAV:supported-report-set" property on any collection that supports synchronization. To implement the behavior for this report, a server needs to keep track of changes to any member URLs and their mapped resources in a collection (as defined in Section 3 of ). This includes noting the addition of new member URLs, the changes to the mapped resources of existing member URLs, and the removal of member URLs. The server will track each change and provide a synchronization "token" to the client that describes the state of the server at a specific point in time. This "token" is returned as part of the response to the "sync-collection" report. Clients include the last token they got from the server in the next "sync-collection" report that they execute, and the server provides the changes from the previous state (represented by the token) to the current state (represented by the new token returned). The synchronization token itself MUST be treated as an "opaque" string by the client, i.e., the actual string data has no specific meaning or syntax. However, the token MUST be a valid URI to allow its use in an If precondition request header (see ). For example, a simple implementation of such a token could be a numeric counter that counts each change as it occurs and relates that change to the specific object that changed. The numeric value could be appended to a "base" URI to form the valid sync-token. Marshalling: The request-URI MUST identify a collection. The request body MUST be a DAV:sync-collection XML element (see ), which MUST contain one DAV:sync-token XML element, one DAV:sync-level element, and one DAV:prop XML element, and MAY contain a DAV:limit XML element. This report is only defined when the Depth header has value "0"; other values result in a 400 (Bad Request) error response. Note that , Section 3.6, states that if the Depth header is not present, it defaults to a value of "0". The response body for a successful request MUST be a DAV:&wj;multistatus XML element, which MUST contain one DAV:&wj;sync&nbhy;token element in addition to one DAV:response element for each member URL that was added, has had its mapped resource changed, or was deleted since the last synchronization operation as specified by the DAV:sync-token provided in the request. A given member URL MUST appear only once in the response. In the case where multiple member URLs of the request-URI are mapped to the same resource, if the resource is changed, each member URL MUST be returned in the response. The content of each DAV:response element differs depending on how the member was altered: For members that have changed (i.e., are new or have had their mapped resource modified), the DAV:response MUST contain at least one DAV:propstat element and MUST NOT contain any DAV:&wj;status element. For members that have been removed, the DAV:response MUST contain one DAV:status with a value set to '404 Not Found' and MUST NOT contain any DAV:propstat element. For members that are collections and are unable to support the DAV:sync-collection report, the DAV:response MUST contain one DAV:status with a value set to '403 Forbidden', a DAV:error containing DAV:supported-report or DAV:sync-traversal-supported (see for which is appropriate) and MUST NOT contain any DAV:propstat element. The conditions under which each type of change can occur are further described in . Preconditions: (DAV:valid-sync-token): The DAV:sync-token element value MUST be a valid token previously returned by the server for the collection targeted by the request-URI. Servers might need to invalidate tokens previously returned to clients. Doing so will cause the clients to fall back to doing full synchronization using the report, though that will not require clients to download resources that are already cached and have not changed. Even so, servers MUST limit themselves to invalidating tokens only when absolutely necessary. Specific reasons include: Servers might be unable to maintain all of the change data for a collection due to storage or performance reasons, e.g., servers might only be able to maintain up to 3 weeks worth of changes to a collection, or only up to 10,000 total changes, or not wish to maintain changes for a deleted collection. Change to server implementation: servers might be upgraded to a new implementation that tracks the history in a different manner, and thus previous synchronization history is no longer valid. Postconditions: (DAV:number-of-matches-within-limits): The number of changes reported in the response must fall within the client-specified limit. This condition might be triggered if a client requests a limit on the number of responses (as per ), but the server is unable to truncate the result set at or below that limit.

Servers MUST support only Depth:0 behavior with the DAV:&wj;sync&nbhy;collection report, i.e., the report targets only the collection being synchronized in a single request. However, clients do need to "scope" the synchronization to different levels within that collection -- specifically, immediate children (level "1") and all children at any depth (level "infinite"). To specify which level to use, clients MUST include a DAV:sync-level XML element in the request. When the client specifies the DAV:sync-level XML element with a value of "1", only appropriate internal member URLs (immediate children) of the collection specified as the request-URI are reported. When the client specifies the DAV:sync-level XML element with a value of "infinite", all appropriate member URLs of the collection specified as the request-URI are reported, provided child collections themselves also support the DAV:sync-collection report. DAV:sync-token values returned by the server are not specific to the value of the DAV:sync-level XML element used in the request. As such, clients MAY use a DAV:sync-token value from a request with one DAV:sync-level XML element value for a similar request with a different DAV:sync-level XML element value; however, the utility of this is limited. Note that when a server supports a DAV:sync-level XML element with a value of "infinite", it might not be possible to synchronize some child collections within the collection targeted by the report. When this occurs, the server MUST include a DAV:response element for the child collection with status 403 (Forbidden). The 403 response MUST be sent once, when the collection is first reported to the client. In addition, the server MUST include a DAV:error element in the DAV:&wj;response element, indicating one of two possible causes for this: The DAV:sync-collection report is not supported at all on the child collection. The DAV:error element MUST contain the DAV:&wj;supported-report element. The server is unwilling to report results for the child collection when a DAV:sync-collection report with the DAV:sync-level XML element set to "infinite" is executed on a parent resource. This might happen when, for example, the synchronization state of the collection resource is controlled by another subsystem. In such cases clients can perform the DAV:sync-collection report directly on the child collection instead. The DAV:error element MUST contain the DAV:sync-traversal-supported element.

When the DAV:sync-collection request contains an empty DAV:sync-token element, the server MUST return all member URLs of the collection (taking account of the DAV:sync-level XML element value as per , and optional truncation of the result set as per ) and it MUST NOT return any removed member URLs. All types of member (collection or non-collection) MUST be reported.

When the DAV:sync-collection request contains a valid value for the DAV:sync-token element, two types of member URL state changes can be returned (changed or removed). This section defines what triggers each of these to be returned. It also clarifies the case where a member URL might have undergone multiple changes between two synchronization report requests. In all cases, the DAV:sync-level XML element value (as per ) and optional truncation of the result set (as per ) are taken into account by the server.

A member URL MUST be reported as changed if it has been newly mapped as a member of the target collection since the request sync-token was generated (e.g., when a new resource has been created as a child of the collection). For example, this includes member URLs that have been newly mapped as the result of a COPY, MOVE, BIND, or REBIND request. All types of member URL (collection or non-collection) MUST be reported. In the case where a mapping between a member URL and the target collection was removed, then a new mapping with the same URI was created, the member URL MUST be reported as changed and MUST NOT be reported as removed. A member URL MUST be reported as changed if its mapped resource's entity tag value (defined in Section 3.11 of ) has changed since the request sync-token was generated. A member URL MAY be reported as changed if the user issuing the request was granted access to this member URL, due to access control changes. Collection member URLs MUST be returned as changed if they are mapped to an underlying resource (i.e., entity body) and if the entity tag associated with that resource changes. There is no guarantee that changes to members of a collection will result in a change in any entity tag of that collection, so clients cannot rely on a series of reports using the DAV:sync-level XML element value set to "1" at multiple levels to track all changes within a collection. Instead, a DAV:sync-level XML element with a value of "infinite" has to be used.

A member MUST be reported as removed if its mapping under the target collection has been removed since the request sync-token was generated, and it has not been remapped since it was removed. For example, this includes members that have been unmapped as the result of a MOVE, UNBIND, or REBIND operation. This also includes collection members that have been removed, including ones that themselves do not support the DAV:sync-collection report. If a member was added (and its mapped resource possibly modified), then removed between two synchronization report requests, it MUST be reported as removed. This ensures that a client that adds a member is informed of the removal of the member, if the removal occurs before the client has had a chance to execute a synchronization report. A member MAY be reported as removed if the user issuing the request no longer has access to this member, due to access control changes. For a report with the DAV:sync-level XML element value set to "infinite", where a collection is removed, the server MUST NOT report the removal of any members of the removed collection. Clients MUST assume that if a collection is reported as being removed, then all members of that collection have also been removed.

A server MAY limit the number of member URLs in a response, for example, to limit the amount of work expended in processing a request, or as the result of an explicit limit set by the client. If the result set is truncated, the response MUST use status code 207 (Multi-Status), return a DAV:multistatus response body, and indicate a status of 507 (Insufficient Storage) for the request-URI. That DAV:response element SHOULD include a DAV:error element with the DAV:&wj;number-of-matches-within-limits precondition, as defined in (Section 9.2). DAV:response elements for all the changes being reported are also included. When truncation occurs, the DAV:sync-token value returned in the response MUST represent the correct state for the partial set of changes returned. That allows the client to use the returned DAV:&wj;sync-token to fetch the next set of changes. In this way, the client can effectively "page" through the entire set of changes in a consistent manner. Clients MUST handle the 507 status on the request-URI in the response to the report. For example, consider a server that records changes using a strictly increasing integer to represent a "revision number" and uses that quantity as the DAV:sync-token value (appropriately encoded as a URI). Assume the last DAV:sync-token used by the client was "http://example.com/sync/10", and since then 15 additional changes to different resources have occurred. If the client executes a DAV:&wj;sync-collection request with a DAV:sync-token of "http://example.com/sync/10", without a limit, the server would return 15 DAV:response elements and a DAV:sync-token with value "http://example.com/sync/25". But if the server chooses to limit responses to at most 10 changes, then it would return only 10 DAV:&wj;response elements and a DAV:sync-token with value "http://example.com/sync/20", together with an additional DAV:&wj;response element for the request-URI with a status code of 507. Subsequently, the client can reissue the request with the DAV:sync-token value returned from the server and fetch the remaining 5 changes.

A client can limit the number of results returned by the server through use of the DAV:limit element (, Section 5.17) in the request body. This is useful when clients have limited space or bandwidth for the results. If a server is unable to truncate the result at or below the requested number, then it MUST fail the request with a DAV:number-of-matches-within-limits postcondition error. When the results can be correctly limited by the server, the server MUST follow the rules above for indicating a result set truncation to the client.

In this example, the client is making its first synchronization request to the server, so the DAV:sync-token element in the request is empty. It also asks for the DAV:getetag property and for a proprietary property. The server responds with the items currently in the targeted collection. The current synchronization token is also returned.

>> Request <<

]]>

>> Response << http://webdav.example.com/home/cyrusdaboo/test.doc "00001-abcd1" Box type A HTTP/1.1 200 OK http://webdav.example.com/home/cyrusdaboo/vcard.vcf "00002-abcd1" HTTP/1.1 200 OK HTTP/1.1 404 Not Found http://webdav.example.com/home/cyrusdaboo/calendar.ics "00003-abcd1" HTTP/1.1 200 OK HTTP/1.1 404 Not Found

http://example.com/ns/sync/1234

]]>

In this example, the client is making a synchronization request to the server and is using the DAV:sync-token element returned from the last report it ran on this collection. The server responds, listing the items that have been added, changed, or removed. The (new) current synchronization token is also returned.

>> Request <<

http://example.com/ns/sync/1234

]]>

>> Response << http://webdav.example.com/home/cyrusdaboo/file.xml "00004-abcd1" HTTP/1.1 200 OK HTTP/1.1 404 Not Found http://webdav.example.com/home/cyrusdaboo/vcard.vcf "00002-abcd2" HTTP/1.1 200 OK HTTP/1.1 404 Not Found http://webdav.example.com/home/cyrusdaboo/test.doc HTTP/1.1 404 Not Found

http://example.com/ns/sync/1238

]]>

In this example, the client is making its first synchronization request to the server, so the DAV:sync-token element in the request is empty. It also asks for the DAV:getetag property. The server responds with the items currently in the targeted collection but truncated at two items. The synchronization token for the truncated result set is returned.

>> Request <<

]]>

>> Response << http://webdav.example.com/home/cyrusdaboo/test.doc "00001-abcd1" HTTP/1.1 200 OK http://webdav.example.com/home/cyrusdaboo/vcard.vcf "00002-abcd1" HTTP/1.1 200 OK http://webdav.example.com/home/cyrusdaboo/ HTTP/1.1 507 Insufficient Storage

http://example.com/ns/sync/1233

]]>

In this example, the client is making its first synchronization request to the server, so the DAV:sync-token element in the request is empty. It requests a limit of 1 for the responses returned by the server. It also asks for the DAV:getetag property. The server responds with the items currently in the targeted collection, but truncated at one item. The synchronization token for the truncated result set is returned.

>> Request <<

]]>

>> Response << http://webdav.example.com/home/cyrusdaboo/test.doc "00001-abcd1" HTTP/1.1 200 OK http://webdav.example.com/home/cyrusdaboo/ HTTP/1.1 507 Insufficient Storage

http://example.com/ns/sync/1232

]]>

In this example, the client is making a synchronization request to the server with a valid DAV:sync-token element value. It requests a limit of 100 for the responses returned by the server. It also asks for the DAV:getetag property. The server is unable to limit the results to the maximum specified by the client, so it responds with a 507 status code and appropriate postcondition error code.

>> Request <<

http://example.com/ns/sync/1232

100 ]]>

>> Response <<

]]>

In this example, the client is making its first synchronization request to the server, so the DAV:sync-token element in the request is empty, and it is using DAV:sync-level set to "infinite". It also asks for the DAV:getetag property and for a proprietary property. The server responds with the items currently in the targeted collection. The current synchronization token is also returned. The collection /home/cyrusdaboo/collection1/ exists and has one child resource that is also reported. The collection /home/cyrusdaboo/collection2/ exists but has no child resources. The collection /home/cyrusdaboo/shared/ is returned with a 403 status indicating that a collection exists, but it is unable to report on changes within it in the scope of the current DAV:sync-level "infinite" report. Instead, the client can try a DAV:sync-collection report directly on the collection URI.

>> Request <<

infinite

]]>

>> Response << /home/cyrusdaboo/collection1/ "00001-abcd1" Box type A HTTP/1.1 200 OK /home/cyrusdaboo/collection1/test.doc "00001-abcd1" Box type A HTTP/1.1 200 OK /home/cyrusdaboo/collection2/ HTTP/1.1 404 Not Found HTTP/1.1 404 Not Found /home/cyrusdaboo/calendar.ics "00003-abcd1" HTTP/1.1 200 OK HTTP/1.1 404 Not Found /home/cyrusdaboo/shared/ HTTP/1.1 403 Forbidden

http://example.com/ns/sync/1234

]]>