Distributed Authoring and Versioning

Dave Sussman

It's a small world

A friend of mine uses the phrase 'geographical transparency'. It's a good phrase. It neatly sums up how the world is getting smaller. No longer is it necessary to physically be present to obtain information. The Internet has made accessing information from far and wide as easy as if it were on a local machine. In fact, that's the great beauty - you don't need to know where the information is physically stored. It's irrelevant.

This shrinking of the world has also changed the way we work. Because information is so easy to access and transfer, it's easy to work with others. Writing books is a good case in point. The Wrox Press book Professional Active Server Pages 2.0 had 8 authors, 3 editors and 15 technical reviewers, spread across the globe. I knew two of the people involved, and had met a few of the others once or twice, but many I'd not met. And still haven't. It didn't stop the book being successful.

The process of producing books like this is still very cumbersome. The authors mail in chapters and the editors send them to the reviewers in email. The reviewers add comments and mail the document back to the editors, who then have to merge the comments into a single document, which is then sent back to the author. The author then reworks the chapter and mails a finished copy back to the editors. It's a painful process.

This type of collaboration perhaps shows the extreme, but most published information requires some form of editing or reviewing. The problem we face is how to cope with that process in a more manageable way. There's a great quote from Jim Whitehead, current Chairman of the WebDAV working group, in late 1998 when they were working on the WebDAV specification:

The irony was intense. The frustrating, awkward nature of collaborative authoring over the Internet became increasingly evident as draft after draft of the WebDAV Distributed Authoring Protocol specification was edited, spelling out how collaboration could be much easier, much more fluid.

So this is what WebDAV is all about.

Dave Sussman David is a developer, trainer and author. He has authored and co-authored numerous titles for Wrox Press, including ADO 2.1 Programmer's Reference, Beginning Access 2000 VBA, Professional ASP 2.0, ASP 2.0 Programmer's Reference and Professional MTS MSMQ with VBA and ASP.

What does WebDAV do?

WebDAV doesn't do anything, because it is just a protocol for collaborative authoring. Its whole premise is that you should be able to edit resources in place, rather than having to copy them to a local location to edit them. The idea is to turn the Web into a writeable medium, instead of the read-only one we've got now. There are a few tools that allow remote editing and management, but they are all proprietary.

WebDAV is a standard that defines how you can accomplish certain tasks:

Ø       Edit documents in place.

Ø       Remotely access Web resources.

Ø       Control access to Web resources.

Ø       Ensure resource integrity.

What WebDAV aims to do is allow you to point at a Web resource, say a document stored on a Web server, and edit it in-place. It allows you to secure the document and control access to it, and to prevent others from editing the document at the same time.

One really important point to note about WebDAV is that it is an industry standard. It's been put together by a wide-ranging group of people and companies, and is controlled by the World Wide Web Consortium (W3C) and the Internet Engineering Taskforce (IETF).

History

So how did WebDAV come about? Well, in December 1995, a group of like-minded people met up at an IETF meeting to discuss remote authoring. So began a long series of meetings and email messages trying to work out how a standard remote authoring system could work, and more importantly, how it could actually be implemented.

In March 1997 the IETF approved the creation of working group to study this in more detail, and the series of meetings and mail messages became more prevalent and more important. It's amazing to think that many standards are created this way. Jim's quote is quite succinct - it shows that the product they were trying to create was exactly what they need to try and create it. That's recursion at it's best!

In November 1998 Version 10 of the WebDAV specification was finally approved as Proposed Standard, becoming RFC 2518.

So now we have a standard that people can implement. The standard lays down exactly how WebDAV should behave, so software vendors can write to this specification, and guarantee interoperability.

What makes up WebDAV?

WebDAV consists of three main areas, each heavily integrated with one another.

Namespaces

A namespace defines the scope of a URL, identifying the hierarchy to which WebDAV commands can be applied. This is pretty much the same as a standard Web namespace. Consider the following URL:

http://webdev.wrox.co.uk/public

This identifies the root directory, and thus the namespace. Underneath this directory you'll probably find multiple resources (files and directory), so the namespace contains collections of these resources. One of the points about a WebDAV server is that it manages the namespace and collection of resources.

Such management allows resources to be created, edited and deleted, as well as being copied and moved. In many ways this is the same as the operating system, which manages the file system on your local computer.

Properties

WebDAV properties simply define the characteristics of a resource - it's the metadata for the resource. Like most metadata, properties in WebDAV consist of name/value pairs. This is pretty much the same as a property in any other product. There are however, two things that are special about WebDAV properties.

The first is that the property name is a Uniform Resource Identifier (URI), such as a URL. For a URL, the property name would be unique within the domain, but not necessarily globally unique. These properties don't, therefore, need registering. A URI on the other hand, is a globally unique property name, and can be centrally registered.

The second point is that the values are well formed XML. This gives them some great advantages over normal properties:

Ø       They can hold multiple values. A property doesn't have to contain just a single value. Using XML allows multiple values to be easily represented.

Ø       They are extensible. Since XML is extensible, the properties are too.

Ø       They are language neutral. XML supports localization of languages, so properties can be represented in any language.

Ø       Availability of tools. There are plenty of XML parsing tools available, which means there is no need to build special tools to handle the properties.

Properties on a WebDAV server are typically divided into two types - live and static. Static properties are defined, and controlled by the server, and refer to the whole namespace of a server. Calling them static doesn't mean that their values don't change, but that their name and management is server controlled. A client can explicitly change the value of a static property. A live property, on the other hand, may only apply to part of the namespace. This is equivalent to properties that appear on every file in a file system (such as name, which would be a static property), and properties that appear only in certain directories (such as Internet Address in the Temporary Internet Files folder, which would be considered a live property).

One of the real benefits of tightly defining properties is that they allow property searching to be standardized. This is pretty important for future projects such as DASL (more later). Using schemas will allow various industries to define a standard set of properties suitable for their work. These schemas will be standardized and centrally registered.

Locking

Locking is a crucial point in any environment that allows collaboration. When more than one person has direct access to a resource, it's important that the lost-update problem (where one author overwrites another author's changes) is resolved. The only way to achieve this is with a sensible resource locking mechanism.

WebDAV supports two types of locking - exclusive write locks, and shared write locks. An exclusive write lock means that once locked, only the lock owner can update the resource. A shared write lock, on the other hand, allows multiple authors to simultaneously lock a resource. In WebDAV there is no such concept as a read-lock - you cannot stop people from reading a resource. WebDAV is about authoring, and preventing reads is contrary to the requirements. However, if WebDAV is integrated into a Web server, then the server can implement read-locks as part of its normal access rights.

Since any resource in a namespace can be a collection, the locking mechanism allows a depth to be specified to locks. This allows you to lock just an individual resource or all resources in a hierarchy.

Whilst WebDAV specifies the type of locking, it does not specify how things such as shared lock updates are to be handled. Although this is crucial to allow true multi-authoring and editing, you must remember that WebDAV is a protocol, not an implementation. The way shared updates are handled depends entirely upon the resource being edited, and cannot therefore be covered by the protocol.

To reduce administration of WebDAV servers, locks have a built in time-out. This means that when locking a resource you may have to refresh the lock to guarantee exclusivity on the resource.

Event Notification

Hand in hand with locking is the concept of notifications. If a resource is locked, how do you find out when that lock is released? The only way is to examine the resource properties to see if it is locked, which isn't an ideal situation. This also has implications on just reading resources. Because there's no concept of read-locks, a web resource can change at any time.

WebDAV doesn't define a notification mechanism, but this area is being examined as a general issue.

Implementation

Since WebDAV is just a specification, it doesn't in itself comprise any products. But a specification is useless without an implementation, and for this you need two parts. Since it's a protocol for allowing authoring in place, there's obviously a server side - you need some sort of server to support the protocol. And to be able to talk to this server, you also need some client software to talk to the server.

There are several ways to implement extensions to Web services.

Ø       Extending the URL. This is similar to the HTML GET method, where form details are added to the end of the URL. The advantage of this mechanism is that it works for existing browsers, with no change. The disadvantage is that is confuses the whole issue of URIs and HTTP, and makes extensibility harder to implement.

Ø       Extending POST. Extending the POST mechanism is quite a good solution, since it would allow the WebDAV commands to tunnel through firewalls and proxies. This is however, a disadvantage. Part of the role of a proxy or firewall is to protect Web servers, and pushing everything through as part of the POST information could lead to security breaches. Using POST means it's also more difficult to implement good access schemes.

Ø       Rewrite HTTP. Whilst it would be great to rewrite HTTP to work the way we now use the web, this is really an impractical solution. It will probably happen one day (as part of IP6), but no one in their right mind is going to insist on a rewrite until the whole world is ready (or forced) to take the plunge.

Ø       Extend HTTP. This seems the most logical solution, since HTTP 1.1 was intended to allow extensions. The only problem we have is that HTTP is not great at handling dual methods (such as moving files, which is implemented as a copy followed by a delete).

Extending HTTP 1.1

HTTP 1.1 was designed with extensibility in mind. It's quite a simple protocol, and contains a limited set of commands and headers, so WebDAV has extended this command set.

New Methods

The new methods that WebDAV has brought to HTTP are listed below:

Method	Description
PROPPATH	Allows properties to be retrieved.
PROPFIND	Allows properties to be updated.
COPY	Copies a resource from one location to another.
MOVE	Moves a resource from one location to another.
MKCOL	Creates a new collection
LOCK	Locks a resource.
UNLOCK	Unlocks a resource.

As well as adding new methods, the existing PUT, POST and GET methods are used.

New Headers

In conjunction with the new methods, there are also some new HTTP headers:

Header	Description
DAV	Identifies the version of DAV supported.
Depth	Identifies the depth in the hierarchy that should be applied to a command. For example, locking a resource could specify a depth of 0 to indicate just the resource is locked, a depth of 1 to indicate that the resource and its immediate children are locked, or a depth of infinity to indicate that locking applies to the resource and all its progeny.
Destination	The URI identifying the destination of a COPY or MOVE.
If	Similar functionality to the existing HTTP If header, allowing conditional headers.
Lock-Token	Identifies the unique lock token when LOCKing or UNLOCKing a resource.
Overwrite	Indicates whether an existing resource should be overwritten or not during a COPY or a MOVE.
Status-URI	Indicates the status codes on resources affected by the outstanding method.
TimeOut	Indicates the timeout details when LOCKing a resource.

New Response Code

WebDAV uses many existing status codes, but also adds the following:

Code	Description
102	Processing. The command has been accepted but has not yet completed.
207	Multi-status. The status provides status for multiple operations. The response body contains an XML element containing the status of the individual actions.
422	Unprocessable Entity. The command issued was valid, but the server is unable to process it.
423	Locked. The source of destination resource is locked.
424	Failed Dependency. The operation depended upon another action, and that action failed.
507	Insufficient Storage. The command could not be completed because the server doesn't have enough storage to complete the command.

Workings of HTTP

To understand how WebDAV works, it's necessary to understand a little of how HTTP works. This diagram should make things clear:

When you make a request to a Web page your browser sends the HTTP GET command, along with some header information. For example, the above request would generate the following HTTP command:

GET http://tigger/davids/Wrox/Temp/foo.htm

Along with the following headers:

User-Agent:

Host:

Connection:

What gets returned is the following:

HTTP/1.1 200 Ok

Server: Microsoft-IIS/5.0

Date: Fri, Aug 13 1999 11:03:00 GMT

Transfer-Encoding: chunked

Content-Type: text/html

<head> . . . </head>

The top line is a standard HTTP status code, indicating the command executed correctly. The next four lines are the HTTP headers and the rest of the lines contain the content.

Similarly, the POT command issues the following request:

POST http://tigger/davids/Wrox/Temp/login.asp

The return is:

HTTP/1.1 200 Ok

Server: Microsoft-IIS/5.0

Date: Fri, Aug 13 1999 11:03:00 GMT

Content-Length: 789

Content-Type: text/html

Set-Cookie: ASPSESSIONIDQQQQGGOOQ=HHH…

Cache-control: private

. . .

So underneath your URL you have three main portions.

Ø       The HTTP status line.

Ø       The HTTP headers.

Ø       The body of the response.

How WebDAV sits on HTTP

WebDAV extends HTTP with some new commands. Imagine a client program that edits documents remotely. It might do something like this:

When you open a document, the WebDAV client issues the new HTTP LOCK method. It then issues a PROPFIND to examine the properties of the resource, and then a GET to retrieve the resource. When saving the document it would issue a PUT to send the document back to the server, along with a PROPPATCH to update any properties. Finally the UNLOCK is issued.

Trying HTTP Commands

The simplest way to try out some of these HTTP commands is to use a tool like the HTTP Request Builder, written for an MSJ article on WebDAV. You can find part of the article, and a link for the tool, on http://www.microsoft.com/msj/0699/dav/dav.htm.

The tool looks like this:

You have the opportunity to pick a URL and select the HTTP method (verb), as well as adding HTTP headers. The command sent to the web server is shown in the Request section, and what's returned is shown in the Response window.

This tool isn't just for WebDAV - it's pretty useful for examining exactly what gets sent to and from Web servers.

ADO 2.5

You might wonder what a section on ADO is doing in a talk about WebDAV, but ADO 2.5 ships with an OLEDB Provider for Internet Publishing, which encapsulates the WebDAV functionality into a standard OLEDB Provider. This means that you can use the existing ADO methods to access a Web resource.

The features of this provider are based around three objects.

Ø       The new Record object, which maps to a node (or directory) in a namespace.

Ø       The existing Recordset object, which maps to a collection of nodes or files.

Ø       The new Stream object, which allows direct access to the contents (ie the body) or a resource.

What this gives you is a mapping between the ADO commands and the WebDAV HTTP methods:

ADO Command	HTTP Method
Record.Open	PROPFIND, GET
Record.Open . . . adLock . . .	LOCK, PROPFIND, GET
Record.Update	PROPPATCH, PUT
Record.Close	UNLOCK

Whilst the mapping isn't quite that simple, it gives you an idea of what's happing underneath ADO.

Work In Progress

It's important to understand that although WebDAV is a standard, the whole area of collaborative authoring is still a work in progress. In particular, there were some aspects of the initial specification that were split apart to become areas of their own. This happened for a variety of reasons, and allowed WebDAV to be ratified whilst work continued on other areas.

Advanced Collections

WebDAV has a mechanism for handling collections, but this doesn't handle some of the situations in an Internet scenario. Advanced collections are being designed to handle things such as referential members of collections, where a resource is a pointer to another resource. This has huge implications for locking, since when you lock a referential resource do you lock the resource or the target the resource points to? These sorts of issue are still being worked on.

Versioning

Given that DAV stands for Distributed Authoring and Versioning you might be surprised to see versioning as a work in progress, but unfortunately it is. This means that WebDAV is really WebDA.

Versioning is the ability to store previous versions of a resource and will give the ability to manage resource branching. This will allow a great deal of control over multiple authors performing parallel editing, and the merge process.

Access Control

Access control defines the ability to limit rights to a resource, by way of introducing Access Control Lists. For a true collaborative authoring scheme to work you must have some way of defining who has rights on resources, and what those rights are.

Software

A new standard isn't much use if there's no software to support it, but luckily this isn't the case with WebDAV. Even before it was a standard, software was being released, and there is a growing amount of support.

Server

At the server side, there are already quite a few products that support WebDAV:

Product	Available at
IIS5	www.microsoft.com/mind/0499/iis5/iis5.htm
CyberTeams Web Site Director	www.cyberteams.com/webdav
mod_dav, for Apache	www.webdav.org/mod_dav
PyDAV, Python based	sandbox.xerox.com/webdav

This is just a short list of what's available, and you should check the WebDAV web site (shown in the More Information section) for a full list. What's particularly exciting is that support for WebDAV is being integrated into free products, which means you're not locked into a proprietary product.

Client

The list is just as promising for WebDAV clients:

Product	Available from
WebDAV Explorer	www.ics.uci.edu/~webdav
Sitecopy	www.lyra.org/sitecopy
Microsoft IE5 & Office 2000 Web Folders	www.microsoft.com/office
OLEDB 2.5 Internet Publishing Provider	www.microsoft.com/data

Again some of these products are free, so there's no reason why you can't play with WebDAV now.

Related Topics

WebDAV doesn't stand-alone in the arena of Internet technologies being developer. Three others that are related are:

Ø       DAV Searching and Locating (DASL). This is a work in progress, which will define the standard for searching DAV repositories.

Ø       Event Notifications. Currently, the ability to be notified about events in DAV (such as locks being released) isn't in place, but initiatives like the Workshop on Internet Scale Event Notifications and the IETF Event Notification Service are looking at this.

Ø       Resource Description Framework (RDF), which is the W3C framework for metadata.

Uber Protocol

One employee at Adobe, who has worked on the WebDAV specification, described WebDAV as an Uber protocol. It's not really such a far fetched idea, because once all a of the various extensions (versioning, access control, etc.) are finished you have a protocol that can:

Ø       Allow resources to be created, deleted, moved and copied.

Ø       Allow remote editing of a resource.

Ø       Control access to that resource.

Ø       Maintain versioning information of that resource.

Does this sound familiar? In theory, WebDAV could replace all of the following:

Ø       File transfer systems, such as FTP.

Ø       News systems, such as NNTP.

Ø       Version control systems, such as CVS or SourceSafe.

Ø       Mail systems, such as IMAP or POP.

There is huge potential here. This extended form of HTTP could become the de-facto standard for any file-based operation. A single, truly global standard. Wow.

More Information

For more information on WebDAV, and the technologies that surround it, you should check out the following resources:

Ø       The WebDAV Working Group, at http://www.ics.uci.edu/pub/ietf/webdav.

Ø       The DASL Working Group at www.ics.uci.edu/pub/ietf/dasl

Ø       The WebDAV Resource, at www.webdav.org

There are plenty of articles to read, and plenty of software to try out.

The Versioning Working Group is still working on the specification. You can join the working group by simply joining the mailing list (details are on the sites above). So, if you want to have a say in a future protocol, join up.

Summary

So what does WebDAV really mean to the ASP programmer? After all, that's what you really want to know isn't it? Initially you might think 'not much', and initially you might be right. As it stands WebDAV is quite powerful, but without the extensions it lacks the greatness to make it truly magnificent.

WebDAV gives you the power to edit and manage documents from remote locations. So for Web developers, it's even easier to stay at home in bed, err, I mean telework. For many home workers this could be a significant point.

It's still got a long way to go, but this could be a tool that will change the way you work. Forever. Not many products have achieved that.