Hadoop WebDAV server is based the earlier work and is released under an Apache License, Version 2.0
Notice
The WebDAV server is built on the top of Hadoop distribution, so make sure you have sources from hadoop-0.16.* or hadoop-0.17.* unpacked in some location. WebDAV server may be deployed on any node of Hadoop subnet, inside or outside of cluster. It uses some libraries from Hadoop's lib/ dir, stores logs into logs/ dir. See Installation section for more details.
We provide both sources and binary files: see attachment at the bottom.
History of the hadoop WebDAV serverWe used sources at https://issues.apache.org/jira/browse/HADOOP-496 and after fixing URL-decoding bug in it corrected server code:
- to enable authorization dialog with the client and
- to support authentication and HDFS-permissions
We also provided start/stop scripts for this server.
Compilation
Download the hadoop WebDAV server source from attachment below.
WebDAV uses some libs from Hadoop distribution, so you have to update build.xml file and specify value of "hadoop.dir" property to point to your unpacked Hadoop distribution.
Download and put into ./lib dir following jars that Hadoop-WebDAV depends on:
- jackrabbit-jcr-commons-1.4-SNAPSHOT.jar
- jackrabbit-jcr-server-1.4-SNAPSHOT.jar
- jackrabbit-webdav-1.4-SNAPSHOT.jar
- commons-collections-3.2.jar
- slf4j-api-1.3.0.jar
- slf4j-log4j12-1.3.0.jar
- xercesImpl-2.8.1.jar
- jcr-1.0.jar
You can use following links:
http://mirror.olnevhost.net/pub/apache/jackrabbit/binaries/ http://repo1.maven.org/maven2/commons-collections/commons-collections/3.2/commons-collections-3.2.jar http://repo1.maven.org/maven2/org/slf4j/slf4j-api/1.3.0/slf4j-api-1.3.0.jar http://repo1.maven.org/maven2/org/slf4j/slf4j-log4j12/1.3.0/slf4j-log4j12-1.3.0.jar http://repo1.maven.org/maven2/xerces/xercesImpl/2.8.1/xercesImpl-2.8.1.jar http://repo1.maven.org/maven2/javax/jcr/jcr/1.0/jcr-1.0.jar
Then use 'ant' to compile the sources, find hdfs_webdav.jar in build/ dir.
Installation
1. Unpack webdav.tar.bz2 distribution to any location (for example, /home/user/webdav). 2. Modify config file hadoop-webdav.sh to satisfy your configuration:
- HADOOP_WEBDAV_HOST, HADOOP_WEBDAV_PORT - address and port WebDAV server will listen to.
- HADOOP_WEBDAV_HDFS - The name of the HDFS, e.g. namenode:port in case if you run WebDAV server on nodes that are different from the master. If this parameter is not specified, WebDAV will try determine name of the FS from 'fs.default.name' parameter, specified in hadoop-site.xml of your Hadoop installation.
- HADOOP_WEBDAV_CLASSPATH parameter should point to lib directory from where you unpacked WebDAV distribution.
3. Copy neccessary files to Hadoop distribution.
- Put bin/start-webdav.sh and bin/stop-webdav.sh to Hadoop's bin directory (for example, /home/user/hadoop/bin).
- Put files from conf/ dir (hadoop-webdav.sh, accounts.properties,jetty.xml, web.xml) to Hadoop's config dir (/home/user/hadoop/conf/).
Permission configurationCorrect account_properties file in ./conf directory for correct account information. See more details here.
Running
In order to start WebDAV server run from Hadoop's home dir (for example, /home/user/hadoop):
# ./bin/start-webdav.sh
To stop the WebDAV server you have to run
# ./bin/stop-webdav.sh
Known client-specific issues for Windows "native" clients- On Windows you can mount WebDAV resources only when WebDAV server runs on the port 80. So be sure to configure server properly.
- Windows clients need service "WebClient" to be running. On Windows 2003 server it's not running by default. To change this
- go to 'Control Panel' > 'Administrative Tools'
- run 'Computer Management' applet, select 'Services and Applications' > 'Services',
- find WebClient, open its properties, change Startup type to Automatic.
- double check that it actually runs
- Another issue with Windows XP's and later version of Windows, is that the client cannot download files longer than 50Mb from a WebDav server. This is supposed to be a security feature -- see http://support.microsoft.com/kb/900900/en-us for discussion and justifications. You can fix this by editing registry manually, as suggested in the article above
|