allow users to disable use of helper: direct uploads might be faster #398

Closed
opened 2008-04-25 00:57:29 +00:00 by warner · 5 comments

It occurred to me and Zandr that clients with fast uplinks may achieve faster
throughput by not using the helper at all. The protocol improvements in
are nice and all, but really it's the helper that's getting overloaded. The
purpose of the helper was to allow folks with slow uplinks to avoid paying
the 3x expansion penalty on the wrong end of the wire, but folks with
multi-megabit uplinks don't really need this help. At those speeds, sending
3x shares directly to the storage servers will get faster uploads than
sending 1x ciphertext to an overloaded helper.

We might manage to farm out the helpers enough to improve performance to make
this not true, but fundamentally the goal of a distributed system is to
reduce centralization and push the work out to the edge. The helper is a
concession to improving user experience with slow uplinks, but on fast
uplinks it should probably not be used.

So the task is to give advanced end users of the commercial service a way to
not use the helper. Simply deleting or renaming BASEDIR/helper.furl and
restarting the node would suffice, but it might be nice to have a better UI
for this. The goal is to let users with fast uplinks try it both ways and see
which gives them better service.

Eventually, we might consider having the node automatically decide whether to
use the helper or not, based upon the upload speeds measured by the first few
direct uploads.

It occurred to me and Zandr that clients with fast uplinks may achieve faster throughput by not using the helper at all. The protocol improvements in #397 are nice and all, but really it's the helper that's getting overloaded. The purpose of the helper was to allow folks with slow uplinks to avoid paying the 3x expansion penalty on the wrong end of the wire, but folks with multi-megabit uplinks don't really need this help. At those speeds, sending 3x shares directly to the storage servers will get faster uploads than sending 1x ciphertext to an overloaded helper. We might manage to farm out the helpers enough to improve performance to make this not true, but fundamentally the goal of a distributed system is to reduce centralization and push the work out to the edge. The helper is a concession to improving user experience with slow uplinks, but on fast uplinks it should probably not be used. So the task is to give advanced end users of the commercial service a way to not use the helper. Simply deleting or renaming BASEDIR/helper.furl and restarting the node would suffice, but it might be nice to have a better UI for this. The goal is to let users with fast uplinks try it both ways and see which gives them better service. Eventually, we might consider having the node automatically decide whether to use the helper or not, based upon the upload speeds measured by the first few direct uploads.
warner added the
code-encoding
major
task
1.0.0
labels 2008-04-25 00:57:29 +00:00
warner added this to the eventually milestone 2008-04-25 00:57:29 +00:00
warner modified the milestone from eventually to undecided 2008-06-01 20:48:08 +00:00

If you love this ticket, you might also like (Measure how segment size affects upload/download speed.) and (add streaming (on-line) upload to HTTP interface).

If you love this ticket, you might also like #809 (Measure how segment size affects upload/download speed.) and #320 (add streaming (on-line) upload to HTTP interface).
davidsarah commented 2009-12-30 00:12:36 +00:00
Owner

If you like this ticket, you might also like : 'sometimes a helper doesn't help (if you need to upload less than K shares)'.

If you like this ticket, you might also like #613: 'sometimes a helper doesn't help (if you need to upload less than K shares)'.

was a duplicate of this one.

#213 was a duplicate of this one.
davidsarah commented 2010-12-29 20:17:46 +00:00
Owner

Replying to zooko:

was a duplicate of this one.

Well, not really a duplicate. Rather, the case where a client can either be connected to a small LAN-based grid or to a larger grid, is another case that might be helped by automatic choice of whether to use the helper.

Replying to [zooko](/tahoe-lafs/trac-2024-07-25/issues/398#issuecomment-66195): > #213 was a duplicate of this one. Well, not really a duplicate. Rather, the case where a client can either be connected to a small LAN-based grid or to a larger grid, is another case that might be helped by automatic choice of whether to use the helper.
daira commented 2013-07-17 14:15:33 +00:00
Owner

This related to Allmydata's service, which is no longer relevant. If anyone thinks it is really important to dynamically choose whether to use a helper (I'm skeptical), then please file another ticket.

This related to Allmydata's service, which is no longer relevant. If anyone thinks it is really important to dynamically choose whether to use a helper (I'm skeptical), then please file another ticket.
tahoe-lafs added the
invalid
label 2013-07-17 14:15:33 +00:00
daira closed this issue 2013-07-17 14:15:33 +00:00
Sign in to join this conversation.
No Milestone
No Assignees
3 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Reference: tahoe-lafs/trac-2024-07-25#398
No description provided.