split uploader/downloader into "txlafs" library #2786

New Issue

warner · 2016-05-10T18:21:16Z

warner commented

2016-05-10 18:21:16 +00:00

At today's devchat, I asked folks for opinions about the shape of the tahoe application, based on questions in https://tahoe-lafs.org/pipermail/tahoe-dev/2016-May/009745.html . Afterwards, Meejah and I had an idea.

slowly rearrange tahoe's internals to make the uploader and downloader a bit more isolated: things like Timing and History should be passed into the Filenode instances, rather than being ambiently available to persistent Uploader and Downloader objects
then draw a dotted line around the client-only functionality: NodeMaker, the src/allmydata/immutable/ and /mutable/ directories, and the IntroducerClient
call that part "txlafs" and split it into a separate library, making Tahoe-LAFS (the application) depend upon txlafs (the library)

Meejah took some notes on what the interface to txlafs would look like. Basically you create a "grid" or "lafs" object around an Introducer FURL (or eventually a URL, if we can move to an HTTP-based introducer). Then you call upload and download methods on that object, or maybe a get_node(filecap) like the nodemaker does now.

The goal would be to allow new applications to be written that use Tahoe-LAFS storage technology, but which don't need to coordinate with a pre-configured pre-running Tahoe client node, and which don't need the additional dependencies that the Tahoe application currently uses to support the HTTP-based webapi, SFTP, magic-folders, the storage server, etc.

These application developers might prefer a one-off filecap=upload(data) API, to something that involves more objects (like Filenodes and mutable Versions).

Connections are only made in response to upload/download requests. The library might be allowed to retain those connections in-between requests (for performance), but the caller shouldn't need to think too much about that. This is the opposite of tahoe's current Client object, which is the central hub through which everything gets started, and which must be fully running before you can do anything with it.

At today's devchat, I asked folks for opinions about the shape of the tahoe application, based on questions in <https://tahoe-lafs.org/pipermail/tahoe-dev/2016-May/009745.html> . Afterwards, Meejah and I had an idea. * slowly rearrange tahoe's internals to make the uploader and downloader a bit more isolated: things like Timing and History should be passed *into* the Filenode instances, rather than being ambiently available to persistent Uploader and Downloader objects * then draw a dotted line around the client-only functionality: `NodeMaker`, the `src/allmydata/immutable/` and `/mutable/` directories, and the IntroducerClient * call that part "txlafs" and split it into a separate library, making Tahoe-LAFS (the application) depend upon `txlafs` (the library) Meejah took some notes on what the interface to txlafs would look like. Basically you create a "grid" or "lafs" object around an Introducer FURL (or eventually a URL, if we can move to an HTTP-based introducer). Then you call upload and download methods on that object, or maybe a `get_node(filecap)` like the nodemaker does now. The goal would be to allow new applications to be written that use Tahoe-LAFS storage technology, but which don't need to coordinate with a pre-configured pre-running Tahoe client node, and which don't need the additional dependencies that the Tahoe application currently uses to support the HTTP-based webapi, SFTP, magic-folders, the storage server, etc. These application developers might prefer a one-off `filecap=upload(data)` API, to something that involves more objects (like Filenodes and mutable Versions). Connections are only made in response to upload/download requests. The library might be allowed to retain those connections in-between requests (for performance), but the caller shouldn't need to think too much about that. This is the opposite of tahoe's current Client object, which is the central hub through which everything gets started, and which must be fully running before you can do anything with it.