webdav frontend #451

New Issue

warner · 2008-06-03T05:24:57Z

warner commented

2008-06-03 05:24:57 +00:00

I suspect that the most convenient front-end in the long run is going to be a WEBDAV server. Most operating systems know how to mount DAV shares, we wouldn't need any special per-platform code, and our webapi server is pretty close to being a DAV server anyways.

OTOH there is some funky locking that must be done, and there might be some impedance mismatches that need to be resolved.

I suspect that the most convenient front-end in the long run is going to be a WEBDAV server. Most operating systems know how to mount DAV shares, we wouldn't need any special per-platform code, and our webapi server is pretty close to being a DAV server anyways. OTOH there is some funky locking that must be done, and there might be some impedance mismatches that need to be resolved.

warner added the

labels 2008-06-03 05:24:57 +00:00

warner added this to the undecided milestone 2008-06-03 05:24:57 +00:00

warner commented

2008-09-09 20:12:45 +00:00

I looked a little bit at this, and I think it's feasible. Our existing RESTful interface is pretty close to what a webdav client appears to use. The main difference is that directory listing uses a special PROPsomething method (instead of a special GET), and servers can implement locking. The PROPFIND/PROPSET methods use a bizarre XML syntax (as befits a microsoft-inspired protocol :-).

The best way to pursue this will start by building the apple calendar server (http://trac.calendarserver.org/), which includes a webdav server written in twisted (using web2). If we could get this server running inside a tahoe node, I think the rest would be pretty easy. Getting twisted.web2 into a tahoe process when it isn't in the regular twisted distribution is a hassle, but should be possible. (web2 was recently removed from twisted SVN, so it could spend some time maturing. The challenge is to manage the namespaces properly). web2 doesn't play well with Nevow, but I think we might want a separate port/server for the webdav stuff anyways, so that wouldn't be a problem.

Once that's running, the question is how people should use it. One possibility that might make regular users a bit more comfortable would be to give the tahoe node a table of username/password/rootcap, and have the webdav server implement HTTP Digest authentication, and teach the twisted.vfs layer that when a request shows up for /foo/bar.txt via username Bob, that means it should turn that into a GET for /uri/BOB's-rootcap/foo/bar.txt . Another possibility is that users could tell their clients to mount <http://localhost:8123/dav/ROOTCAP>, which would be more capabilitiesish.

After we build this, we might want to look into caching dirnodes for some limited period of time (5 seconds?), since at least Mac OS-X likes to look for a lot of hidden OS-X -specific files in each directory (to decide how to arrange the icons, where to open the window, etc).

The unix tool 'cadaver' is a DAV client that acts a lot like the old FTP client, and is useful for running tests. The easiest way to get a server running is to enable mod_dav in apache.

I looked a little bit at this, and I think it's feasible. Our existing RESTful interface is pretty close to what a webdav client appears to use. The main difference is that directory listing uses a special PROPsomething method (instead of a special GET), and servers can implement locking. The PROPFIND/PROPSET methods use a bizarre XML syntax (as befits a microsoft-inspired protocol :-). The best way to pursue this will start by building the apple calendar server (<http://trac.calendarserver.org/>), which includes a webdav server written in twisted (using web2). If we could get this server running inside a tahoe node, I think the rest would be pretty easy. Getting twisted.web2 into a tahoe process when it isn't in the regular twisted distribution is a hassle, but should be possible. (web2 was recently removed from twisted SVN, so it could spend some time maturing. The challenge is to manage the namespaces properly). web2 doesn't play well with Nevow, but I think we might want a separate port/server for the webdav stuff anyways, so that wouldn't be a problem. Once that's running, the question is how people should use it. One possibility that might make regular users a bit more comfortable would be to give the tahoe node a table of username/password/rootcap, and have the webdav server implement HTTP Digest authentication, and teach the twisted.vfs layer that when a request shows up for /foo/bar.txt via username Bob, that means it should turn that into a GET for /uri/BOB's-rootcap/foo/bar.txt . Another possibility is that users could tell their clients to mount `<http://localhost:8123/dav/ROOTCAP>`, which would be more capabilitiesish. After we build this, we might want to look into caching dirnodes for some limited period of time (5 seconds?), since at least Mac OS-X likes to look for a lot of hidden OS-X -specific files in each directory (to decide how to arrange the icons, where to open the window, etc). The unix tool 'cadaver' is a DAV client that acts a lot like the old FTP client, and is useful for running tests. The easiest way to get a server running is to enable mod_dav in apache.

tahoe-lafs modified the milestone from undecided to eventually

2010-02-11 03:36:36 +00:00

davidsarah commented

2010-04-04 22:35:33 +00:00

See wiki/GSoCIdeas#WebDAVSupport

See [wiki/GSoCIdeas#WebDAVSupport](wiki/GSoCIdeas#WebDAVSupport)

zooko commented

2010-10-19 03:09:32 +00:00

There exists a very old project to add webdav to twisted: http://akadav.sourceforge.net/

And a branch within twisted: (@@http://www.mail-archive.com/twisted-web@twistedmatrix.com/msg02169.html@@)

There exists a very old project to add webdav to twisted: <http://akadav.sourceforge.net/> And a branch within twisted: (@@http://www.mail-archive.com/twisted-web@twistedmatrix.com/msg02169.html@@)

davidsarah commented

2010-10-20 02:18:30 +00:00

Replying to zooko:

There exists a very old project to add webdav to twisted: http://akadav.sourceforge.net/

This has a dependency on PyXML, which is unmaintained.

And a branch within twisted: (@@http://www.mail-archive.com/twisted-web@twistedmatrix.com/msg02169.html@@)

This code depended on Twisted web2, which is dead. Progress on reanimating this branch seems slow.

I think that wsgidav is probably a better choice, because it seems to be actively maintained.

Replying to [zooko](/tahoe-lafs/trac-2024-07-25/issues/451#issuecomment-66875): > There exists a very old project to add webdav to twisted: <http://akadav.sourceforge.net/> This has a dependency on [PyXML](http://sourceforge.net/projects/pyxml/files/), which is unmaintained. > And a branch within twisted: (@@http://www.mail-archive.com/twisted-web@twistedmatrix.com/msg02169.html@@) This code depended on Twisted web2, which is dead. [Progress on reanimating this branch](@@http://twistedmatrix.com/trac/ticket/3081#[comment:-1](/tahoe-lafs/trac-2024-07-25/issues/451#issuecomment--1)@@) seems slow. I think that [wsgidav](http://code.google.com/p/wsgidav/) is probably a better choice, because it [seems to be actively maintained](http://code.google.com/p/wsgidav/source/list).

warner commented

2010-10-20 03:39:29 +00:00

Yay! A new option for WebDAV!

But, doesn't WSGI usually means synchronous+blocking? twisted.web can call out to WSGI applications, but usually it's hard for a WSGI server to call out to Deferred-returning APIs. But, maybe there's a way to hack it in. It'd be awesome to get DAV working.

Yay! A new option for WebDAV! But, doesn't WSGI usually means synchronous+blocking? twisted.web can call out to WSGI applications, but usually it's hard for a WSGI server to call out to Deferred-returning APIs. But, maybe there's a way to hack it in. It'd be awesome to get DAV working.

davidsarah commented

2010-10-20 04:27:40 +00:00

Replying to warner:

Yay! A new option for WebDAV!

But, doesn't WSGI usually means synchronous+blocking?

Yes. However this appears to say that someone got wsgidav working with Twisted's wsgi-container (see code in comment 10).

As you say, there might be problems calling back into Tahoe's APIs, since wsgi-container works by using deferToThread. But maybe you can have a WSGI thread block until the main Tahoe thread has responded to a message from it.

Replying to [warner](/tahoe-lafs/trac-2024-07-25/issues/451#issuecomment-66877): > Yay! A new option for WebDAV! > > But, doesn't WSGI usually means synchronous+blocking? Yes. However [this](http://code.google.com/p/wsgidav/issues/detail?id=41) appears to say that someone got wsgidav working with Twisted's wsgi-container (see code in comment 10). As you say, there might be problems calling back into Tahoe's APIs, since wsgi-container works by using `deferToThread`. But maybe you can have a WSGI thread block until the main Tahoe thread has responded to a message from it.

davidsarah commented

2011-01-03 20:34:00 +00:00

Zope2 apparently has a WebDAV server, as does Zope3 (source of Zope3's implementation).

It appears to be fairly tightly dependent on the rest of Zope (example for COPY/MOVE), but is still worth looking at for inspiration, and for how it has solved client-compatibility issues.

[Zope2 apparently has a WebDAV server](http://wiki.zope.org/zope2/ZopeAndWebDAV), [as does Zope3](https://launchpad.net/z3c.dav) ([source of Zope3's implementation](http://svn.zope.org/z3c.dav/)). It appears to be fairly tightly dependent on the rest of Zope ([example for COPY/MOVE](http://svn.zope.org/z3c.dav/trunk/src/z3c/dav/copymove.py?view=markup)), but is still worth looking at for inspiration, and for how it has solved client-compatibility issues.

zooko commented

2011-03-10 23:45:04 +00:00

I just discovered pyWebDAV. I heard about it because it had a security vulnerability announced on http://lwn.net .

I just discovered [pyWebDAV](http://code.google.com/p/pywebdav/). I heard about it because it had a security vulnerability announced on <http://lwn.net> .

tahoe-lafs added

normal

and removed

major

labels 2012-06-30 22:24:19 +00:00

tahoe-lafs modified the milestone from eventually to undecided

2012-06-30 22:24:19 +00:00

amontero commented

2013-12-28 17:18:43 +00:00

Call to Twisted/WSGI/webserver wizards: Could not #2144 help achieve WebDAV, just using an out-of-the-box web server?

Sign in to join this conversation.