tcp hole-punching! #169

New Issue

zooko · 2007-10-09T03:49:16Z

zooko commented

2007-10-09 03:49:16 +00:00

Did you know that it is actually possible to do tcp hole-punching (for some NAT situations) and that there exists source code which is an almost-drop-in-replacement for connectTCP() and listenTCP(), and that this source code is going to be open sourced soon?

zooko added the

labels 2007-10-09 03:49:16 +00:00

zooko added this to the eventually milestone 2007-10-09 03:49:16 +00:00

ghazel commented

2007-10-09 03:52:42 +00:00

As a matter of fact I did! Now if I could only make a Trac ticket email me every weekend to remind me to do it...

zooko commented

2007-10-19 23:11:23 +00:00

I just reminded ghazel.

This might make a fun project for The Hack Fest next Friday...

I just reminded ghazel. This might make a fun project for The Hack Fest next Friday...

ghazel commented

2007-10-31 04:27:10 +00:00

I didn't get to it at the hack fest, but I did get Tahoe running, woo!
I've been reviewing the timeout cases in Stunted, to make sure it won't retry forever. Perhaps I'll put up a Google Code page about it, so you can check it out.

I didn't get to it at the hack fest, but I did get Tahoe running, woo! I've been reviewing the timeout cases in Stunted, to make sure it won't retry forever. Perhaps I'll put up a Google Code page about it, so you can check it out.

zooko commented

2007-11-01 16:59:37 +00:00

What if I want it to keep retrying forever? (Or until I tell it to stop.)

warner modified the milestone from eventually to undecided

2008-06-01 21:08:28 +00:00

jsgf commented

2008-08-18 18:03:31 +00:00

Any progress on this? I'd like to look at this if it still needs doing.

ghazel: Do you have a library available?

Any progress on this? I'd like to look at this if it still needs doing. ghazel: Do you have a library available?

zooko commented

2008-08-19 18:11:35 +00:00

Please post to this ticket and/or tahoe-dev if you do anything on this.

zooko commented

2009-09-19 21:47:23 +00:00

Assigning to ghazel to answer jsgf's question. :-)

zooko commented

2009-09-19 22:13:21 +00:00

If you like this ticket, you may also like #49 (UPnP), #50 (STUNT/ICE), #445 (implement relay: allow storage servers behind NAT), and #754 (merge manually specified tub location with autodetected tub location).

davidsarah commented

2010-03-12 17:45:58 +00:00

The Google Summer of Code project would be "make Tahoe work through firewalls", not necessarily this specific ticket. That might require use of more than one technique.

exarkun commented

2020-01-17 23:43:11 +00:00

I wonder what code zooko was referring to. It sounds like Vertex but I don't recall there ever being a time when Vertex wasn't open source, so maybe not. In any case, Vertex is open source and ... might still work? I don't know. Even if it does I suppose in 2020 it might not be the best option anyway.

I wonder what code zooko was referring to. It sounds like Vertex but I don't recall there ever being a time when Vertex *wasn't* open source, so maybe not. In any case, Vertex *is* open source and ... might still work? I don't know. Even if it does I suppose in 2020 it might not be the best option anyway.

maylee commented

2021-04-01 03:25:29 +00:00

Choosing this ticket at random. Is this something that fits into any of our current milestones, or should we make a milestone this can be bucketed into?

lpirl commented

2021-04-01 07:39:53 +00:00

Maybe "Grid Management", if any? As in: "It makes a grid easier to manage when servers need no extra firewall/NAT configuration". Although this might be more an operation issue but oh well. $ .02

exarkun commented

2021-04-01 12:46:53 +00:00

To choose a milestone, let's work backwards from an end-user-facing goal. What is that in this case? Real servers rarely have to deal with NAT. They just get public IP addresses. Perhaps the goal is related to operation by "home" users? That is, to allow me to run a listening Tahoe node (so, not just a storage client - a storage server and/or an introducer) on my consumer-grade network connection where I have no publicly routeable address to make my servers listen on.

Oops, wait. Is that really relevant? Can I TCP hole-punch through carrier-grade NAT (CGN)? Wikipedia vaguely nods towards "yes" (no citations). All of my hole-punching experience is with consumer-grade routers so I can't say first-hand. You have to be able to hole-punch through CGN to meaningfully improve the home user experience now, I think.

Well ... maybe there are mild ease-of-use improvements from just defeating consumer-grade router NAT. Tahoe could deal with UPnP instead of making me click buttons on my router.

Suppose we can do some or all of those things. The end result is that Tahoe storage nodes and introducers can run on my low-end network connection. This helps out ... friendnet operation, perhaps? A group of "friends" are more likely to have a NAT'd server among them (it's not until you have two that you actually need help here. still.).

If you wanted a remote control for your client node then the same requirement would apply, I suppose. You want the remote control to be able to connect back to your client node regardless of intervening network topology. There's a lot of other work required before we know if/how we can have such remote controls though.

I think "Grid Management" is essentially orthogonal. It's actually about efforts related to ticket:2916 which is a very specific kind of grid management.

If we had some kind of "friendnet" milestone it might make sense there.
If we had some kind of "remote control" milestone it might make sense there.
It might even make sense if we sliced the problem differently and had a "maximum network compatibility" milestone that included all of the hole punching, UPnP, STUN, etc tickets.

None of those milestones exist yet, though, and I wouldn't suggest creating them just to have a place to put this ticket. I think we need to plan a roadmap first and then we'll see what milestones are on it and we can sort tickets into them. There will be lots of tickets that don't fall into a milestone in the near-term future based on such a roadmap and that's fine (or at least, that's life).

To choose a milestone, let's work backwards from an end-user-facing goal. What is that in this case? Real servers rarely have to deal with NAT. They just get public IP addresses. Perhaps the goal is related to operation by "home" users? That is, to allow me to run a listening Tahoe node (so, not just a storage client - a storage server and/or an introducer) on my consumer-grade network connection where I have no publicly routeable address to make my servers listen on. Oops, wait. Is that really relevant? Can I TCP hole-punch through carrier-grade NAT (CGN)? Wikipedia vaguely nods towards "yes" (no citations). All of my hole-punching experience is with consumer-grade routers so I can't say first-hand. You have to be able to hole-punch through CGN to meaningfully improve the home user experience now, I think. Well ... maybe there are mild ease-of-use improvements from just defeating consumer-grade router NAT. Tahoe could deal with UPnP instead of making me click buttons on my router. Suppose we can do some or all of those things. The end result is that Tahoe storage nodes and introducers can run on my low-end network connection. This helps out ... friendnet operation, perhaps? A group of "friends" are more likely to have a NAT'd server among them (it's not until you have two that you actually need help here. still.). If you wanted a remote control for your client node then the same requirement would apply, I suppose. You want the remote control to be able to connect back to your client node regardless of intervening network topology. There's a lot of other work required before we know if/how we can have such remote controls though. I think "Grid Management" is essentially orthogonal. It's actually about efforts related to ticket:2916 which is a very specific kind of grid management. If we had some kind of "friendnet" milestone it might make sense there. If we had some kind of "remote control" milestone it might make sense there. It might even make sense if we sliced the problem differently and had a "maximum network compatibility" milestone that included all of the hole punching, UPnP, STUN, etc tickets. None of those milestones exist yet, though, and I wouldn't suggest creating them just to have a place to put this ticket. I think we need to plan a roadmap first and then we'll see what milestones are on it and we can sort tickets into them. There will be *lots* of tickets that don't fall into a milestone in the near-term future based on such a roadmap and that's fine (or at least, that's life).

Sign in to join this conversation.