StoRM GridHTTPs Server’s WebDAV interface

version: 3.0.0

http(s)://[gridhttps.hostname]:[port]/webdav/[storage-area]/

http(s)://[gridhttps.hostname]:[port]/[storage-area]/

Table of contents

• Introduction
• What is WebDAV?
• SRM operations via WebDAV
• Service installation and configuration
• Using WebDAV
  • Access data via browser
  • cURLs
  • Firefox RESTClient plugin
  • Cyberduck

Introduction  

Each Storage Area that supports HTTP or HTTPS transfer protocols can be accessed through the WebDAV interface provided by the StoRM GridHTTPs server component. This WebDAV interface conceals the details of the SRM protocol and allows users to mount remote Grid storage areas as a volume, directly on their own desktop.

To access the Storage Area’s data users have to provide the right credentials. For example, if the Storage Area A is owned by the VO X, user has to provide a valid VOMS proxy. If the Storage Area B is owned by the VO Y but permits a read-only access to anonymous (see examples section), user has to provide a valid VOMS proxy only if he wants to write data. And so on. See the examples section to other storage area configuration examples.

What is WebDAV?  

Web Distributed Authoring and Versioning (WebDAV) protocol consists of a set of methods, headers, and content-types ancillary to HTTP/1.1 for the management of resource properties, creation and management of resource collections, URL namespace manipulation, and resource locking. The purpose of this protocol is to present a Web content as a writable medium in addition to be a readable one. WebDAV on Wikipedia and the WebDAV website provide information on this protocol.

In a few words, the WebDAV protocol mainly abstracts concepts such as resource properties, collections of resources, locks in general, and write locks specifically. These abstractions are manipulated by the WebDAV-specific HTTP methods and the extra HTTP headers used with WebDAV methods. The WebDAV added methods include:

• PROPFIND - used to retrieve properties, stored as XML, from a web resource. It is also overloaded to allow one to retrieve the collection structure (a.k.a. directory hierarchy) of a remote system.
• PROPPATCH - used to change and delete multiple properties on a resource in a single atomic act.
• MKCOL - used to create collections (a.k.a. a directory).
• COPY - used to copy a resource from one URI to another.
• MOVE - used to move a resource from one URI to another.
• LOCK - used to put a lock on a resource. WebDAV supports both shared and exclusive locks.
• UNLOCK - used to remove a lock from a resource.

While the status codes provided by HTTP/1.1 are sufficient to describe most error conditions encountered by WebDAV methods, there are some errors that do not fall neatly into the existing categories, so the WebDAV specification defines some extra status codes. Since some WebDAV methods may operate over many resources, the Multi-Status response has been introduced to return status information for multiple resources. WebDAV uses XML for property names and some values, and also uses XML to marshal complicated requests and responses.

SRM operations via WebDAV  

Starting from EMI3 version, StoRM GridHTTPs server exposes a WebDAV interface to allow users to access Storage-Areas data via browser or by mounting it from a remote host.

GridHTTPs’ WebDAV implementation is based on Milton open source java library that acts as an API and HTTP protocol handler for adding the WebDAV support to web applications. Milton is not a full server in itself. It is able to expose any existing data source (e.g. CMS, hibernate pojos, etc) through a WebDAV interface.

As seen in the chapter before, through a WebDAV interface we are allowed to manipulate resources and collections of them. So it is simple to understand that a WebDAV resource for StoRM GridHTTPs WebDAV implementation will be a file, while WebDAV collections will be directories of a file-system. Every WebDAV method needs to be mapped to one or more SRM operations that have to be transparent to the final users. StoRM GridHTTPs maps the HTTP/WebDAV methods with the SRM operations as shown by the following table:

For each method, a 401 Unauthorized can be obtained if user doesn’t provide the necessary credentials.

Service installation and configuration  

The WebDAV interface is provided by StoRM GridHTTPs component. Therefore, if you want to install a WebDAV access point to your data you have to install StoRM GridHTTPs metapackage RPM (do not forget to satisfy all the pre-requisites shown in the sys-admin guide before):

  [~]# yum install emi-storm-gridhttps-mp

To configure storm-gridhttps-server you need to fill the requested YAIM variables as described in the basic and advanced StoRM GridHTTPs sys-admin configuration guides. A good explanation of the required YAIM variable is available in:

/opt/glite/yaim/examples/siteinfo/services/se_storm_gridhttps

The service uses (by default) ports 8443 and 8085, so open them on your firewall.

The service needs to be installed on a machine on which storm file system is mounted. If you need, you can install the StoRM GridHTTPs on differents hosts (that share the same data, e.g. hosts are GPFS clients) and use them as a pool (see StoRM BackEnd configuration on sys-admin guide). To start the service:

  [~]# service storm-gridhttps-server start

Using WebDAV  

The StoRM GridHTTPs WebDAV server listens on two ports, one for the unencrypted HTTP connections and another for the SSL encrypted HTTP requests. Their default values are:

• HTTP: 8085
• HTTP over SSL: 8443

To access storage areas’ data, users can use:

• a browser (if the storage area can be accessed by anonymous or via a valid personal certificate)
• cURLs (mandatory if you need to provide a valid x509 proxy credential)
• a third-party WebDAV client (Cyberduck, Firefox RestClient plugin, …)

You can also develop a client on your own, for example by using the Apache Jackrabbit API.

Access data via browser  

Users can use browsers to easily read data of storage areas that are:

• HTTP readable (see the storage area configuration described in the StoRM BackEnd configuration on sys-admin guide)
• not HTTP readable, not associated to a particular VO, eventually access-filtered through users’ DN.

Using a browser, users can navigate through the storage areas’ directories and download/open files.

cURLs  

The best way to use the WebDAV service is using cURL command. cURL is a command line tool for transferring data with URL syntax (see cURL website). With cURLs we can do anonymous requests or provide our x509 credentials: personal certificate, plain Grid proxy, VOMS proxy. The following examples suppose that user has his/her personal certificate (usercert.pem) and key (userkey.pem) in $HOME/.globus directory, and his/her proxy in $X509_USER_PROXY.

Anonymous cURLs

Assuming that:

• ghttps.hostname is the hostname where your WebDAV service is available
• free is the name of a R/W from anonymous Storage Area

and knowing that unencrypted connections have 8085 as default port, the following table show various cURLs, one for each HTTP/WebDAV method.

Using x509 credentials

If you need to present an x509 certificate to be authorized, you can add to your cURL command the following options:

--cert path/to/usercert.pem --key path/to/userkey.pem --capath /path/to/the/trustdir

For example, assuming that:

• A is a storage area readable and writable only with an x509 certificate that has O=”INFN”
• $HOME/.globus/usercert.pem and $HOME/.globus/userkey.pem are user’s certificate and private key
• $HOME/.globus/usercert.pem has O=”INFN”
• the trust directory with CA informations is in /etc/grid-security/certificates

and knowing that encrypted connections has 8443 as default port, we can perform a cURL like this:

curl --verbose -X GET https://gridhttps.hostname:8443/webdav/A/ --cert $HOME/.globus/usercert.pem --key $HOME/.globus/userkey.pem --capath /etc/grid-security/certificates

and retrieve the list of file/directories in the root directory of the storage area A.

Using proxy credentials

If you need to present an x509 proxy - plain or with VOMS extensions - to be authorized, you can add to your cURL command the following options:

--cert path/to/yourproxy --capath /path/to/the/trustdir

For example, assuming that:

• B is a storage area readable and writable only with an x509 VOMS proxy for dteam VO
• $X509_USER_PROXY contains the path to the user’s VOMS proxy
• the trust directory with CA informations is in /etc/grid-security/certificates

and knowing that encrypted connections has 8443 as default port, we can perform a cURL like this:

curl --verbose -X GET https://gridhttps.hostname:8443/webdav/B/ --cert $X509_USER_PROXY --capath /etc/grid-security/certificates

and retrieve the list of file/directories in the root directory of the storage area B.

Firefox RESTClient plugin  

There’s a useful Firefox plugin, named RESTClient, that can be used as a debugger for RESTful web services. RESTClient supports all HTTP methods RFC2616 (HTTP/1.1) and RFC2518 (WebDAV). You can construct custom HTTP request (custom method with resources URI and HTTP request Body) to directly test requests against a server.

Cyberduck  

To connect to HTTP readable storage area you can use several clients. One of this is Cyberduck. Cyberduck is an open source FTP and SFTP, WebDAV, Cloud Files, Google Docs, and Amazon S3 client for Mac OS X and Windows (as of version 4.0) licensed under the GPL. To configure it, add a new connection and insert:

• server: the FQDN of the gridhttps host
• port: the unencrypted HTTP port, default 8085
• select anonymous login
• specify /storage-area-name as remote path

This configuration is the same for lots of WebDAV clients, alternatives to Cyberduck.