Don't use helper for local nodes, only remote ones #2376

New Issue

zooko · 2015-02-06T17:42:08Z

zooko commented

2015-02-06 17:42:08 +00:00

I (Zooko) rescued this from the spam filter. I didn't write it — wilruvvo did:

When sending data to storage nodes on the same network (or even on the same computer), it is clearly more efficient to transfer the data directly rather than uploading it to a helper which then sends it back. The helper should only be used for nodes at remote locations.

Possible heuristics for deciding if a node is 'local'

localhost
same subnet
reachable on a RFC1918 address
user-specified address range

Here are some related tickets:

#610 upload should take better advantage of existing shares
#613 sometimes a helper hurts instead of helping (if you need to upload less than K shares)
#868 use helper to bypass NAT/firewall
#1207 Detect RFC1918/invalid addresses somehow

Having a heuristic for detecting which nodes are local/remote would also help the following use cases:

Ensure at least K shares are 'local' so data remain available if the internet connection is down.
Ensure at least K shares are 'remote' so data remain available if all nodes on one site are lost.

helper upload

I (Zooko) rescued this from the spam filter. I didn't write it — wilruvvo did: When sending data to storage nodes on the same network (or even on the same computer), it is clearly more efficient to transfer the data directly rather than uploading it to a helper which then sends it back. The helper should only be used for nodes at remote locations. Possible heuristics for deciding if a node is 'local' * localhost * same subnet * reachable on a RFC1918 address * user-specified address range Here are some related tickets: #610 upload should take better advantage of existing shares #613 sometimes a helper hurts instead of helping (if you need to upload less than K shares) #868 use helper to bypass NAT/firewall #1207 Detect RFC1918/invalid addresses somehow Having a heuristic for detecting which nodes are local/remote would also help the following use cases: * Ensure at least K shares are 'local' so data remain available if the internet connection is down. * Ensure at least K shares are 'remote' so data remain available if all nodes on one site are lost. helper upload