Tahoe-LAFS as website hosting storage backend #2144

New Issue

tahoe-lafs · 2013-12-28T16:48:51Z

amontero commented

2013-12-28 16:48:51 +00:00

While playing with Tahoe, I've done a bit of research out of curiosity about one possible usage scenario, that instead of keeping in my notes I think it is a use case of interest.

I was thinking in a Tahoe-LAFS grid being the storage backend for a web app served by common httpd servers, such as nginx.
Depending on your app requirements, you grid can range from a read-only cap webroot (that could be tamper-proof for free!) to more complex writtable grids, depending on your app.
Your web server recieves a website GET and, instead of reading from a local filesystem directory at /var/www, it reads from a dircap as the webroot. It could even execute scripts, making for a webroot dir not hackable even if the webserver is compromised. As far as the script cares, it's being run from a readonly filesystem.

Tahoe-LAFS would be the perfect match for distributedly hosting static sites inside anonymity networks, like Tor. Lots of other usages might fit for web app farms, where you push code by changing the webroot caps of the currently served code to your newly and securely deployed code DIR-IMM cap. You can make them write files in a completely separate bucket-storage grid in a S3 fashion. Whatever your crazy app needs.

To accomplish this, I've done a bit of research and nginx looks like a good fit and well documented.
The approaches I've found are:

lafs-rpg:
Can not do this as of now. Nejucomo provided helpful comments at https://bitbucket.org/nejucomo/lafs-rpg/issue/12/run-tahoe-hosted-scripts
However, it could be a good starting point, perhaps. Just adding some more nginx config wizardry could be the way, not sure.
FUSE under nginx
Available now.
Performance?
Documented at http://uwsgi-docs.readthedocs.org/en/latest/tutorials/ReliableFuse.html
uWSGI
Docs for nginx: http://uwsgi-docs.readthedocs.org/en/latest/Nginx.html
The newest, sexiest and cloudfull kid on the block.
Also available under Apache: http://uwsgi-docs.readthedocs.org/en/latest/Apache.html
There is nothing done for nginx, but there are even plugins available for:
GlusterFS: http://uwsgi-docs.readthedocs.org/en/latest/GlusterFS.html
GridFS: http://uwsgi-docs.readthedocs.org/en/latest/GridFS.html
Twisted:
To my knowledge, Tahoe is built on this lib. No Python knowledge here. Perhaps it has nothing to do.
If possible, this would be the closest to the grid?
Just found mentioned at:
http://uwsgi-docs.readthedocs.org/en/latest/DynamicApps.html
http://uwsgi-docs.readthedocs.org/en/latest/WebServers.html#twisted

Alongside: could not this help achieve WebDAV, just using nginx's out-of-the-box? #451

Now it's just an idea, maybe it's doable and just needs a howto. Since I haven't found none, at least linking this issue in the UseCases page could bring interested parties on the same page. Opening Tahoe-LAFS usages to webops community could be a huge boost.

While playing with Tahoe, I've done a bit of research out of curiosity about one possible usage scenario, that instead of keeping in my notes I think it is a use case of interest. I was thinking in a Tahoe-LAFS grid being the storage backend for a web app served by common httpd servers, such as nginx. Depending on your app requirements, you grid can range from a read-only cap webroot (that could be tamper-proof for free!) to more complex writtable grids, depending on your app. Your web server recieves a website GET and, instead of reading from a local filesystem directory at /var/www, it reads from a dircap as the webroot. It could even execute scripts, making for a webroot dir not hackable even if the webserver is compromised. As far as the script cares, it's being run from a readonly filesystem. Tahoe-LAFS would be the perfect match for distributedly hosting static sites inside anonymity networks, like Tor. Lots of other usages might fit for web app farms, where you push code by changing the webroot caps of the currently served code to your newly and securely deployed code DIR-IMM cap. You can make them write files in a completely separate bucket-storage grid in a S3 fashion. Whatever your crazy app needs. To accomplish this, I've done a bit of research and nginx looks like a good fit and well documented. The approaches I've found are: * lafs-rpg: * Can not do this *as of now*. Nejucomo provided helpful comments at <https://bitbucket.org/nejucomo/lafs-rpg/issue/12/run-tahoe-hosted-scripts> * However, it could be a good starting point, perhaps. Just adding some more nginx config wizardry could be the way, not sure. * FUSE under nginx * Available now. * Performance? * Documented at <http://uwsgi-docs.readthedocs.org/en/latest/tutorials/ReliableFuse.html> * uWSGI * Docs for nginx: <http://uwsgi-docs.readthedocs.org/en/latest/Nginx.html> * The newest, sexiest and cloudfull kid on the block. * Also available under Apache: <http://uwsgi-docs.readthedocs.org/en/latest/Apache.html> * There is nothing done for nginx, but there are even plugins available for: * GlusterFS: <http://uwsgi-docs.readthedocs.org/en/latest/GlusterFS.html> * GridFS: <http://uwsgi-docs.readthedocs.org/en/latest/GridFS.html> * Twisted: * To my knowledge, Tahoe is built on this lib. No Python knowledge here. Perhaps it has nothing to do. * If possible, this would be the closest to the grid? * Just found mentioned at: * <http://uwsgi-docs.readthedocs.org/en/latest/DynamicApps.html> * <http://uwsgi-docs.readthedocs.org/en/latest/WebServers.html#twisted> Alongside: could not this help achieve WebDAV, just using nginx's out-of-the-box? #451 Now it's just an idea, maybe it's doable and just needs a howto. Since I haven't found none, at least linking this issue in the [UseCases](wiki/UseCases) page could bring interested parties on the same page. Opening Tahoe-LAFS usages to webops community could be a huge boost.