tahoe storage over ssh tunnel #2843

New Issue

tahoe-lafs · 2016-11-08T16:47:31Z

gregbk commented

2016-11-08 16:47:31 +00:00

Hello,

Tahoe LAFS on docker node work great8
I am now trying to secure connection between storage nodes(on internet) and the introducer+client(private network).

I want to use ssh tunneling to allow client to push on storage node.

From each node
I opened a reverse tunnel: ssh -LR 1111:localhost:1111 CLIENT_IP

Tahoe.cfg:

[node]
nickname = tdengine
reveal-IP-address = true
web.port = tcp:3456:interface=127.0.0.1
web.static = public_html
tub.port = tcp:1111
tub.location = tcp:127.0.0.1:1111
#log_gatherer.furl =
#timeout.keepalive =
#timeout.disconnect =
#ssh.port = 8022
#ssh.authorized_keys_file = ~/.ssh/authorized_keys
[client]
introducer.furl = pb://zyadrwufzm34fwquu6oz6ktqu2e4phlg@tcp:INTRODUCER_IP:41464/uqrzlcn5etmnrb5x7rzbhkgq6ctoakrb
helper.furl =
#stats_gatherer.furl =
... rest is default

From introducer web page, I can't see my node in green
From Client

netstat -plunt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1112/sshd       
tcp        0      0 127.0.0.1:1111          0.0.0.0:*               LISTEN      4694/sshd: root 
tcp6       0      0 :::3456                 :::*                    LISTEN      4051/docker-proxy
tcp6       0      0 :::22                   :::*                    LISTEN      1112/sshd       
tcp6       0      0 ::1:1111                :::*                    LISTEN      4694/sshd: root

flogtool tail /root/.tahoe/private/logport.furl:

14:24:31.208 L20 []#1395 received 1 announcements (v2)
14:24:31.214 L20 []#1396 announcement for nickname 'tdengine', service=storage: {u'nonce': u'zyzg462q2enwfkjdjizccdgwplrahg3gb3seum6rgi5wqs73lt2a', u'app-versions': {u'Nevow': u'0.14.2', u'foolscap': u'0.12.4', u'cffi': u'1.8.3', u'Twisted': u'16.4.1', u'twisted': u'16.4.1', u'attrs': u'16.2.0', u'simplejson': u'3.8.2', u'pyasn1-modules': u'0.0.8', u'six': u'1.10.0', u'OpenSSL': u'1.0.1t', u'platform': u'Linux-debian_8.6-x86_64-64bit_ELF', u'zope.interface': u'unknown', u'PyYAML': u'3.12', u'cryptography': u'1.5.2', u'python': u'2.7.12', u'pycparser': u'2.14', u'idna': u'2.1', u'zfec': u'1.4.24', u'pycryptopp': u'0.7.1.869544967005693312591928092448767568728501330214', u'ipaddress': u'1.0.17', u'tahoe-lafs': u'unknown', u'pycrypto': u'2.6.1', u'pyOpenSSL': u'16.1.0', u'characteristic': u'14.3.0', u'service-identity': u'16.0.0', u'enum34': u'1.1.6', u'shutilwhich': u'1.1.0', u'setuptools': u'27.3.0', u'pyasn1': u'0.1.9'}, u'seqnum': 6, u'nickname': u'tdengine', u'anonymous-storage-FURL': u'pb://6ayuenhqyxfcz6iuafpdy3fni26xnzz3@tcp:127.0.0.1:1111/brkvatvmt42wk25vgmn4xnazx6l4mgnv', u'service-name': u'storage', u'version': 0, u'my-version': u'tahoe-lafs/unknown', u'permutation-seed-base32': u'rnlexqv5bm7em5ycu6pi3c55ompqab234f63r3b22xwcpfzhwaua', u'oldest-supported': u'1.0.0'}
14:24:31.214 L10 []#1397 replacing old announcement: {u'nonce': u'zyzg462q2enwfkjdjizccdgwplrahg3gb3seum6rgi5wqs73lt2a', u'app-versions': {u'Nevow': u'0.14.2', u'foolscap': u'0.12.4', u'cffi': u'1.8.3', u'Twisted': u'16.4.1', u'twisted': u'16.4.1', u'attrs': u'16.2.0', u'cryptography': u'1.5.2', u'ipaddress': u'1.0.17', u'six': u'1.10.0', u'OpenSSL': u'1.0.1t', u'platform': u'Linux-debian_8.6-x86_64-64bit_ELF', u'zope.interface': u'unknown', u'PyYAML': u'3.12', u'pyasn1': u'0.1.9', u'simplejson': u'3.8.2', u'python': u'2.7.12', u'pycparser': u'2.14', u'zfec': u'1.4.24', u'pycryptopp': u'0.7.1.869544967005693312591928092448767568728501330214', u'pyasn1-modules': u'0.0.8', u'pycrypto': u'2.6.1', u'tahoe-lafs': u'unknown', u'enum34': u'1.1.6', u'characteristic': u'14.3.0', u'service-identity': u'16.0.0', u'pyOpenSSL': u'16.1.0', u'shutilwhich': u'1.1.0', u'setuptools': u'27.3.0', u'idna': u'2.1'}, u'seqnum': 6, u'oldest-supported': u'1.0.0', u'anonymous-storage-FURL': u'pb://6ayuenhqyxfcz6iuafpdy3fni26xnzz3@tcp:127.0.0.1:1111/brkvatvmt42wk25vgmn4xnazx6l4mgnv', u'service-name': u'storage', u'version': 0, u'my-version': u'tahoe-lafs/unknown', u'permutation-seed-base32': u'rnlexqv5bm7em5ycu6pi3c55ompqab234f63r3b22xwcpfzhwaua', u'nickname': u'tdengine'}
14:24:31.345 L20 []#1398 TubConnector created from xjcagnk5v2ghjykrbmoz7pivqovowc65 to 6ayuenhqyxfcz6iuafpdy3fni26xnzz3
14:24:31.345 L20 []#1399 considering hint: tcp:127.0.0.1:1111
14:24:31.345 L20 []#1400 connecting to hint: tcp:127.0.0.1:1111
14:24:31.648 L20 []#1401 connection refused for tcp:127.0.0.1:1111
14:24:31.650 L20 []#1402 connectorFinished (<foolscap.connection.TubConnector object at 0x7f1f1fa5ab10 from xjcagnk5v2ghjykrbmoz7pivqovowc65 to 6ayuenhqyxfcz6iuafpdy3fni26xnzz3>)
14:24:34.443 L20 []#1403 TubConnector created from xjcagnk5v2ghjykrbmoz7pivqovowc65 to 6ayuenhqyxfcz6iuafpdy3fni26xnzz3
14:24:34.445 L20 []#1404 considering hint: tcp:127.0.0.1:1111
14:24:34.445 L20 []#1405 connecting to hint: tcp:127.0.0.1:1111
14:24:34.753 L20 []#1406 connection refused for tcp:127.0.0.1:1111
14:24:34.753 L20 []#1407 connectorFinished (<foolscap.connection.TubConnector object at 0x7f1f1fa5a610 from xjcagnk5v2ghjykrbmoz7pivqovowc65 to 6ayuenhqyxfcz6iuafpdy3fni26xnzz3>)

...

14:15:29.746 L20 []#1357 negotiationFailed
 FAILURE:
 [CopiedFailure instance: Traceback from remote host -- Traceback (most recent call last):
   File "/usr/local/lib/python2.7/site-packages/twisted/internet/posixbase.py", line 597, in _doReadOrWrite
     why = selectable.doRead()
   File "/usr/local/lib/python2.7/site-packages/twisted/internet/tcp.py", line 208, in doRead
     return self._dataReceived(data)
   File "/usr/local/lib/python2.7/site-packages/twisted/internet/tcp.py", line 214, in _dataReceived
     rval = self.protocol.dataReceived(data)
   File "/usr/local/lib/python2.7/site-packages/twisted/internet/endpoints.py", line 116, in dataReceived
     return self._wrappedProtocol.dataReceived(data)
 --- <exception caught here> ---
   File "/usr/local/lib/python2.7/site-packages/foolscap/negotiate.py", line 384, in dataReceived
     self.handlePLAINTEXTClient(header)
   File "/usr/local/lib/python2.7/site-packages/foolscap/negotiate.py", line 523, in handlePLAINTEXTClient
     % lines[0])
 foolscap.tokens.BananaError: BananaError: ("not right, got 'HTTP/1.1 500 Internal Server Error: unknown TubID gzssqpbugmn6uzxgyjyf6twxgtdxfd55', expected 101 Switching Protocols",)
 ]
14:15:29.753 L20 []#1358 connectorFinished (<foolscap.connection.TubConnector object at 0x7f1f204110d0 from nvq6jlmugj4vyvh76bvwwzplsefnouj6 to gzssqpbugmn6uzxgyjyf6twxgtdxfd55>)
14:15:44.819 L20 []#1359 TubConnector created from jot7orbnlfaye5vqrskkv7n6tb7wln3u to 6ayuenhqyxfcz6iuafpdy3fni26xnzz3
14:15:44.820 L20 []#1360 considering hint: tcp:127.0.0.1:1111
14:15:44.820 L20 []#1361 connecting to hint: tcp:127.0.0.1:1111
14:15:45.127 L20 []#1362 connection refused for tcp:127.0.0.1:1111
14:15:45.128 L20 []#1363 connectorFinished (<foolscap.connection.TubConnector object at 0x7f1f1fac7510 from jot7orbnlfaye5vqrskkv7n6tb7wln3u to 6ayuenhqyxfcz6iuafpdy3fni26xnzz3>)
14:15:53.694 L20 []#1364 TubConnector created from s2khye6nlphlv4lex6htv44izn2lz7ng to vgvcxuhqvykujzfajjfv5pyxxepd5rsq
14:15:53.694 L20 []#1365 considering hint: tcp:185.19.30.51:3457
14:15:53.695 L20 []#1366 connecting to hint: tcp:185.19.30.51:3457
14:16:23.697 L28 []#1367
 FAILURE:
 [CopiedFailure instance: Traceback from remote host -- Traceback (most recent call last):
 Failure: twisted.internet.error.TimeoutError: User timeout caused connection failure.
 ]
14:16:23.697 L20 []#1368 connectorFinished (<foolscap.connection.TubConnector object at 0x7f1f1fa5aa90 from s2khye6nlphlv4lex6htv44izn2lz7ng to vgvcxuhqvykujzfajjfv5pyxxepd5rsq>)

Can you help me understand what's the best way to do that?
I want something easy, and not tor.

Thank you and good day!
Greg.

Hello, Tahoe LAFS on docker node work great8 I am now trying to secure connection between storage nodes(on internet) and the introducer+client(private network). I want to use ssh tunneling to allow client to push on storage node. * From each node I opened a reverse tunnel: `ssh -LR 1111:localhost:1111 CLIENT_IP` Tahoe.cfg: ``` [node] nickname = tdengine reveal-IP-address = true web.port = tcp:3456:interface=127.0.0.1 web.static = public_html tub.port = tcp:1111 tub.location = tcp:127.0.0.1:1111 #log_gatherer.furl = #timeout.keepalive = #timeout.disconnect = #ssh.port = 8022 #ssh.authorized_keys_file = ~/.ssh/authorized_keys [client] introducer.furl = pb://zyadrwufzm34fwquu6oz6ktqu2e4phlg@tcp:INTRODUCER_IP:41464/uqrzlcn5etmnrb5x7rzbhkgq6ctoakrb helper.furl = #stats_gatherer.furl = ... rest is default ``` * From introducer web page, I can't see my node in green * From Client ``` netstat -plunt Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1112/sshd tcp 0 0 127.0.0.1:1111 0.0.0.0:* LISTEN 4694/sshd: root tcp6 0 0 :::3456 :::* LISTEN 4051/docker-proxy tcp6 0 0 :::22 :::* LISTEN 1112/sshd tcp6 0 0 ::1:1111 :::* LISTEN 4694/sshd: root ``` `flogtool tail /root/.tahoe/private/logport.furl`: ``` 14:24:31.208 L20 []#1395 received 1 announcements (v2) 14:24:31.214 L20 []#1396 announcement for nickname 'tdengine', service=storage: {u'nonce': u'zyzg462q2enwfkjdjizccdgwplrahg3gb3seum6rgi5wqs73lt2a', u'app-versions': {u'Nevow': u'0.14.2', u'foolscap': u'0.12.4', u'cffi': u'1.8.3', u'Twisted': u'16.4.1', u'twisted': u'16.4.1', u'attrs': u'16.2.0', u'simplejson': u'3.8.2', u'pyasn1-modules': u'0.0.8', u'six': u'1.10.0', u'OpenSSL': u'1.0.1t', u'platform': u'Linux-debian_8.6-x86_64-64bit_ELF', u'zope.interface': u'unknown', u'PyYAML': u'3.12', u'cryptography': u'1.5.2', u'python': u'2.7.12', u'pycparser': u'2.14', u'idna': u'2.1', u'zfec': u'1.4.24', u'pycryptopp': u'0.7.1.869544967005693312591928092448767568728501330214', u'ipaddress': u'1.0.17', u'tahoe-lafs': u'unknown', u'pycrypto': u'2.6.1', u'pyOpenSSL': u'16.1.0', u'characteristic': u'14.3.0', u'service-identity': u'16.0.0', u'enum34': u'1.1.6', u'shutilwhich': u'1.1.0', u'setuptools': u'27.3.0', u'pyasn1': u'0.1.9'}, u'seqnum': 6, u'nickname': u'tdengine', u'anonymous-storage-FURL': u'pb://6ayuenhqyxfcz6iuafpdy3fni26xnzz3@tcp:127.0.0.1:1111/brkvatvmt42wk25vgmn4xnazx6l4mgnv', u'service-name': u'storage', u'version': 0, u'my-version': u'tahoe-lafs/unknown', u'permutation-seed-base32': u'rnlexqv5bm7em5ycu6pi3c55ompqab234f63r3b22xwcpfzhwaua', u'oldest-supported': u'1.0.0'} 14:24:31.214 L10 []#1397 replacing old announcement: {u'nonce': u'zyzg462q2enwfkjdjizccdgwplrahg3gb3seum6rgi5wqs73lt2a', u'app-versions': {u'Nevow': u'0.14.2', u'foolscap': u'0.12.4', u'cffi': u'1.8.3', u'Twisted': u'16.4.1', u'twisted': u'16.4.1', u'attrs': u'16.2.0', u'cryptography': u'1.5.2', u'ipaddress': u'1.0.17', u'six': u'1.10.0', u'OpenSSL': u'1.0.1t', u'platform': u'Linux-debian_8.6-x86_64-64bit_ELF', u'zope.interface': u'unknown', u'PyYAML': u'3.12', u'pyasn1': u'0.1.9', u'simplejson': u'3.8.2', u'python': u'2.7.12', u'pycparser': u'2.14', u'zfec': u'1.4.24', u'pycryptopp': u'0.7.1.869544967005693312591928092448767568728501330214', u'pyasn1-modules': u'0.0.8', u'pycrypto': u'2.6.1', u'tahoe-lafs': u'unknown', u'enum34': u'1.1.6', u'characteristic': u'14.3.0', u'service-identity': u'16.0.0', u'pyOpenSSL': u'16.1.0', u'shutilwhich': u'1.1.0', u'setuptools': u'27.3.0', u'idna': u'2.1'}, u'seqnum': 6, u'oldest-supported': u'1.0.0', u'anonymous-storage-FURL': u'pb://6ayuenhqyxfcz6iuafpdy3fni26xnzz3@tcp:127.0.0.1:1111/brkvatvmt42wk25vgmn4xnazx6l4mgnv', u'service-name': u'storage', u'version': 0, u'my-version': u'tahoe-lafs/unknown', u'permutation-seed-base32': u'rnlexqv5bm7em5ycu6pi3c55ompqab234f63r3b22xwcpfzhwaua', u'nickname': u'tdengine'} 14:24:31.345 L20 []#1398 TubConnector created from xjcagnk5v2ghjykrbmoz7pivqovowc65 to 6ayuenhqyxfcz6iuafpdy3fni26xnzz3 14:24:31.345 L20 []#1399 considering hint: tcp:127.0.0.1:1111 14:24:31.345 L20 []#1400 connecting to hint: tcp:127.0.0.1:1111 14:24:31.648 L20 []#1401 connection refused for tcp:127.0.0.1:1111 14:24:31.650 L20 []#1402 connectorFinished (<foolscap.connection.TubConnector object at 0x7f1f1fa5ab10 from xjcagnk5v2ghjykrbmoz7pivqovowc65 to 6ayuenhqyxfcz6iuafpdy3fni26xnzz3>) 14:24:34.443 L20 []#1403 TubConnector created from xjcagnk5v2ghjykrbmoz7pivqovowc65 to 6ayuenhqyxfcz6iuafpdy3fni26xnzz3 14:24:34.445 L20 []#1404 considering hint: tcp:127.0.0.1:1111 14:24:34.445 L20 []#1405 connecting to hint: tcp:127.0.0.1:1111 14:24:34.753 L20 []#1406 connection refused for tcp:127.0.0.1:1111 14:24:34.753 L20 []#1407 connectorFinished (<foolscap.connection.TubConnector object at 0x7f1f1fa5a610 from xjcagnk5v2ghjykrbmoz7pivqovowc65 to 6ayuenhqyxfcz6iuafpdy3fni26xnzz3>) ... 14:15:29.746 L20 []#1357 negotiationFailed FAILURE: [CopiedFailure instance: Traceback from remote host -- Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/twisted/internet/posixbase.py", line 597, in _doReadOrWrite why = selectable.doRead() File "/usr/local/lib/python2.7/site-packages/twisted/internet/tcp.py", line 208, in doRead return self._dataReceived(data) File "/usr/local/lib/python2.7/site-packages/twisted/internet/tcp.py", line 214, in _dataReceived rval = self.protocol.dataReceived(data) File "/usr/local/lib/python2.7/site-packages/twisted/internet/endpoints.py", line 116, in dataReceived return self._wrappedProtocol.dataReceived(data) --- <exception caught here> --- File "/usr/local/lib/python2.7/site-packages/foolscap/negotiate.py", line 384, in dataReceived self.handlePLAINTEXTClient(header) File "/usr/local/lib/python2.7/site-packages/foolscap/negotiate.py", line 523, in handlePLAINTEXTClient % lines[0]) foolscap.tokens.BananaError: BananaError: ("not right, got 'HTTP/1.1 500 Internal Server Error: unknown TubID gzssqpbugmn6uzxgyjyf6twxgtdxfd55', expected 101 Switching Protocols",) ] 14:15:29.753 L20 []#1358 connectorFinished (<foolscap.connection.TubConnector object at 0x7f1f204110d0 from nvq6jlmugj4vyvh76bvwwzplsefnouj6 to gzssqpbugmn6uzxgyjyf6twxgtdxfd55>) 14:15:44.819 L20 []#1359 TubConnector created from jot7orbnlfaye5vqrskkv7n6tb7wln3u to 6ayuenhqyxfcz6iuafpdy3fni26xnzz3 14:15:44.820 L20 []#1360 considering hint: tcp:127.0.0.1:1111 14:15:44.820 L20 []#1361 connecting to hint: tcp:127.0.0.1:1111 14:15:45.127 L20 []#1362 connection refused for tcp:127.0.0.1:1111 14:15:45.128 L20 []#1363 connectorFinished (<foolscap.connection.TubConnector object at 0x7f1f1fac7510 from jot7orbnlfaye5vqrskkv7n6tb7wln3u to 6ayuenhqyxfcz6iuafpdy3fni26xnzz3>) 14:15:53.694 L20 []#1364 TubConnector created from s2khye6nlphlv4lex6htv44izn2lz7ng to vgvcxuhqvykujzfajjfv5pyxxepd5rsq 14:15:53.694 L20 []#1365 considering hint: tcp:185.19.30.51:3457 14:15:53.695 L20 []#1366 connecting to hint: tcp:185.19.30.51:3457 14:16:23.697 L28 []#1367 FAILURE: [CopiedFailure instance: Traceback from remote host -- Traceback (most recent call last): Failure: twisted.internet.error.TimeoutError: User timeout caused connection failure. ] 14:16:23.697 L20 []#1368 connectorFinished (<foolscap.connection.TubConnector object at 0x7f1f1fa5aa90 from s2khye6nlphlv4lex6htv44izn2lz7ng to vgvcxuhqvykujzfajjfv5pyxxepd5rsq>) ``` Can you help me understand what's the best way to do that? I want something easy, and not tor. Thank you and good day! Greg.

tahoe-lafs added the

labels 2016-11-08 16:47:31 +00:00

tahoe-lafs added this to the undecided milestone 2016-11-08 16:47:31 +00:00

exarkun commented

2020-12-09 14:50:46 +00:00

The error in the ticket description seems to indicate that the wrong Tub has been reached. Perhaps the tunnel was set up to point at the wrong place. Or, Docker was mentioned, so perhaps the original Tub's state was lost when an ephemeral Docker instance was destroyed. A new one replaced it but can't prove it is the expected Tub (since it isn't).

It's hard to say more without all of the details of the setup. Also this ticket is quite stale so I doubt all of those details still exist, and maybe no one even cares anymore.

I'd encourage folks to use the IRC channel and the mailing list for support-style requests like this. The issue tracker does not get prompt attention.

The error in the ticket description seems to indicate that the wrong Tub has been reached. Perhaps the tunnel was set up to point at the wrong place. Or, Docker was mentioned, so perhaps the original Tub's state was lost when an ephemeral Docker instance was destroyed. A new one replaced it but can't prove it is the expected Tub (since it isn't). It's hard to say more without all of the details of the setup. Also this ticket is quite stale so I doubt all of those details still exist, and maybe no one even cares anymore. I'd encourage folks to use the IRC channel and the mailing list for support-style requests like this. The issue tracker does not get prompt attention.

exarkun added the

cannot reproduce

label 2020-12-09 14:50:46 +00:00

exarkun closed this issue

2020-12-09 14:50:46 +00:00

Sign in to join this conversation.