implement CHK download helper #268

New Issue

warner · 2008-01-09T23:25:31Z

warner commented

2008-01-09 23:25:31 +00:00

The upload-specific helper described in #116 is insufficient to achieve all
the properties we're hoping for. The helper as originally envisioned in that
ticket was specifically for uploading CHK files: it did not encompass
downloading files (since there's no bandwidth benefit for those), nor did it
include mutable files (since by our SDMF rules, these files are small and the
bandwidth of them isn't too much of a burden).

To prevent the clients from making connections to all storage servers, we
need a helper that will also mediate CHK downloads and mutable uploads and
downloads. I'm making a separate ticket for this since it's more work than
I'd originally realized. I think I can still get it done in the original
schedule.

Zooko and I have talked about the need or lack of need for this
connection-minimization. It may be the case that minimizing connections is
not all that important, at least with a reasonable number of clients. So it's
possible that we'll want the #116 upload-only helper to get us resumability
and 1x uploads, but not the all-purpose helper described in this ticket to
get us minimal connections and isolation between clients and storage servers.

The upload-specific helper described in #116 is insufficient to achieve all the properties we're hoping for. The helper as originally envisioned in that ticket was specifically for uploading CHK files: it did not encompass downloading files (since there's no bandwidth benefit for those), nor did it include mutable files (since by our SDMF rules, these files are small and the bandwidth of them isn't too much of a burden). To prevent the clients from making connections to all storage servers, we need a helper that will also mediate CHK downloads and mutable uploads and downloads. I'm making a separate ticket for this since it's more work than I'd originally realized. I think I can still get it done in the original schedule. Zooko and I have talked about the need or lack of need for this connection-minimization. It may be the case that minimizing connections is not all that important, at least with a reasonable number of clients. So it's possible that we'll want the #116 upload-only helper to get us resumability and 1x uploads, but not the all-purpose helper described in this ticket to get us minimal connections and isolation between clients and storage servers.

warner added the

labels 2008-01-09 23:25:31 +00:00

warner added this to the 0.8.0 (Allmydata 3.0 Beta) milestone 2008-01-09 23:25:31 +00:00

warner self-assigned this 2008-01-09 23:25:31 +00:00

warner commented

2008-01-24 01:48:13 +00:00

I'm narrowing the focus of this ticket to just the CHK download helper. The
mutable-file helper has been moved off to #283, and the CHK-upload helper
remains in #116.

The benefit of implementing just the CHK-download helper is that downloads
will use a single TCP connection, improving bandwidth fairness. Specifically
this means that if both Tahoe and some other program are both downloading a
file over the same DSL line, they will get roughly the same bandwidth.
Without this feature, at our default 3-of-10 encoding, the Tahoe download
will get about three times as much bandwidth as the other program.

If all four helpers are implemented (#116, #268, and #283), then we can
change the introducer configuration to isolate the clients from the storage
servers.

I'm narrowing the focus of this ticket to just the CHK download helper. The mutable-file helper has been moved off to #283, and the CHK-upload helper remains in #116. The benefit of implementing just the CHK-download helper is that downloads will use a single TCP connection, improving bandwidth fairness. Specifically this means that if both Tahoe and some other program are both downloading a file over the same DSL line, they will get roughly the same bandwidth. Without this feature, at our default 3-of-10 encoding, the Tahoe download will get about three times as much bandwidth as the other program. If all four helpers are implemented (#116, #268, and #283), then we can change the introducer configuration to isolate the clients from the storage servers.