tahoe create-node
should require --location
and/or --hostname
, and not autodetect
#2773
Labels
No Label
0.2.0
0.3.0
0.4.0
0.5.0
0.5.1
0.6.0
0.6.1
0.7.0
0.8.0
0.9.0
1.0.0
1.1.0
1.10.0
1.10.1
1.10.2
1.10a2
1.11.0
1.12.0
1.12.1
1.13.0
1.14.0
1.15.0
1.15.1
1.2.0
1.3.0
1.4.1
1.5.0
1.6.0
1.6.1
1.7.0
1.7.1
1.7β
1.8.0
1.8.1
1.8.2
1.8.3
1.8β
1.9.0
1.9.0-s3branch
1.9.0a1
1.9.0a2
1.9.0b1
1.9.1
1.9.2
1.9.2a1
LeastAuthority.com automation
blocker
cannot reproduce
cloud-branch
code
code-dirnodes
code-encoding
code-frontend
code-frontend-cli
code-frontend-ftp-sftp
code-frontend-magic-folder
code-frontend-web
code-mutable
code-network
code-nodeadmin
code-peerselection
code-storage
contrib
critical
defect
dev-infrastructure
documentation
duplicate
enhancement
fixed
invalid
major
minor
n/a
normal
operational
packaging
somebody else's problem
supercritical
task
trivial
unknown
was already fixed
website
wontfix
worksforme
No Milestone
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Reference: tahoe-lafs/trac-2024-07-25#2773
Loading…
Reference in New Issue
Block a user
No description provided.
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
This splits out the most user-visible aspect of #2491. The
create-node
commands that create listening nodes should always be told what hostname or IP address(es) to advertise, rather than guessing by using IP-address autodetection functions.That means the following commands should require a
--location=
argument:create-key-generator(deleted in #2783)These commands should also accept (but not require) a
--port=
argument, which says what port the server should listen on. These will be endpoint descriptors, so things liketcp:12345
. Both location and port get written into tahoe.cfg. Unlike--location=
,--port=
is not mandatory, and the code should pick something sensible if it isn't provided.I'm undecided on what exactly counts as "sensible". One option is just to allocate a free one inside
create-node
(different for each server). Another is for use to choose a port number for Tahoe servers to listen on (now, in this ticket, maybe register it with IANA or something), and let the option default to that. A third is a hybrid: attempt to listen on the default port duringcreate-node
and record that (in tahoe.cfg) if that succeeds, but if it's already claimed by some other process, allocate and record a random one.Note: if the server is using a Tor "onion service", then the user should not be obligated to figure this out: the
--tor
option should provision and register the XYZ.onion location automatically. So--location=
and--port=
should not be required (and in fact should probably be rejected) if--tor
is active.But for plain TCP locations, the server should stop trying to figure it out on its own, and have the node-constructing command get this information from the user.
Hm, I realized I'm not sure whether
--location=
is supposed to be just a hostname (example.com
), or a full Foolscap connection hint (tcp:example.com:12345
).If it's just a hostname, then it makes sense for
--port
to be optional, because the command can build a connection hint from the given hostname and a self-allocated port.If it's a full connection hint, then you basically must provide a
--port
that matches the hint:--location=tcp:example.com:12345 --port=12345
. If you have some funky non-1-to-1 port-mapping firewall going on, you might provide different port numbers (where--location
points at the external one, and--port
provides the internal one). You'd do the same to use an externally-configured Tor onion service,--location=tor:xyz.onion:80 --port=tcp:12345
.I suppose we could say that if you pass
--location
, but not--port
, and the location is in the formtcp:HOSTNAME:PORT
, then we glean--port
from the last component of the location, but this feels a bit magic to me.We could also say that if you provide
--location=tcp:HOSTNAME:PORT1
and a different--port=PORT2
, then we could pretend that you really said--location=tcp:HOSTNAME:PORT2
, but that would be even more magical, and would prevent setting up the portmapping example above.I guess I'm most inclined to be fully explicit.
--location=
would be a fulltcp:HOSTNAME:PORT
connection hint (or a comma-separated list of them).--port=
would be a full endpoint specification (sotcp:PORT
ortcp:PORT:interface=127.0.0.1
), with maybe the "tcp:" being optional.Oh, what if we have a separate argument name
--hostname=
? Then we'd have the following possibilities:--hostname=example.com
: self-allocate a port, use location=tcp:example.com:PORT
, and port=tcp:PORT
--hostname=example.com --port=12345
: usetcp:example.com:12345
andtcp:12345
--location=tcp:example.com:12345 --port=tcp:12345
: use those as location and port--hostname
and--location
, or to provide--location
without--port
.And then using an automatically-configured Tor onion service would still be something different, like
--tor
or--listen-tor
, which builds the service, then sets--location
and--port
internally.`tahoe create-node` should require `--location`, and not autodetectto `tahoe create-node` should require `--location` and/or `--hostname`, and not autodetect#1478 would be easier with something like this in place.
At today's devchat, meejah and I worked something out.
The node creation commands are responsible for writing
tub.location
andtub.port
intotahoe.cfg
, with the same semantics as we have now:tub.location
is a foolscap hint list (e.g. "tcp:HOSTNAME:PORT,tcp:HOST2:PORT2"), andtub.port
is a twisted server endpoint specifiction ("tcp:12345" or "tcp:12345:interface=127.0.0.1"). These commands will also write information about Tor setup, when necessary. All allocation is finished by the timecreate-node
exits, so nothing dynamic needs to happend at runtime (tahoe start
).Then the server-like node-creation commands (
create-server
,create-introducer
, andcreate-node
if it means client+server) have a couple of different allowable casestahoe create-server --hostname=HOST
: This allocates a TCP port (maybe an IANA-standardized Tahoe port, maybe just any free one), and sets location totcp:HOSTNAME:PORT
and port totcp:PORT
.--listen-tor
: This allocates an onion address (insidecreate-server
, which will need to start up a reactor, maybe start a tor instance, and talk to Tor's control port to create the private key), then sets location totor:XYZ.onion:PORT1
and port totcp:PORT2:interface=127.0.01
. If we're able to use a unix socket for the inbound connections from tor, the port will be something likeunix:PATH
instead.--listen-i2p
: same, but for I2Pany combination of the above three
--location= --port=
: This explicitly provides both the FURL connection hints and the listening port.--location
and--port
must be provided as a pair (both, or neither, but never just one). If they are provided, none of--hostname
/--listen-tor
/--listen-i2p
are accepted.In the outbound direction, clients will automatically use Tor hints if possible. At node startup, they'll attempt to import txtorcon, and if that works, they'll register a foolscap connection handler for "tor" hints. The handler will provide Tor endpoints on request (synchronously). Then, when
.connect()
is called on one of those hints, the handler will attempt to spin up a Tor instance. If this fails (e.g. because Tor is not actually installed), the connection will fail, no big deal. Same story for i2p: try to importtxi2p
, if that works install the connection handler (which might not work), if not, don't.tahoe.cfg
will have an option to not use Tor, for the benefit of folks operating in an environment where they have txtorcon and Tor installed, but for whatever reason don't want want Tahoe to use them. I don't think this is common enough to need a CLI argument: folks can just edit tahoe.cfg after node creation.The client-like node creation commands (
create-client
, maybecreate-node
) will then have an argument like--only-tor
which indicates that all outbound TCP connections ought to route through Tor instead. It will also have an--anonymous
flag that enforces restrictions on the other settings: it requires --only-tor, it looks at the server settings and complains if it sees non-onion hints in--location
or non-localhost listening ports in--port
.The client-like commands will also take something like
--socks-port
, which registers a foolscap connection handler (for both tcp and tor) that sends everything to the given SOCKS server. This might be a Tor instance (in which case both tcp:HOST:PORT and tor:XYZ.onion:PORT will work), or it might be a plain SOCKS daemon (so tor: won't work).All commands will accept
--tor-controlport=endpoint
, which means txtorcon ought to talk to the pre-existing Tor instance at that control port, rather than launching its own. This gets written into tahoe.cfg somehow.It might be a good idea to have the node creation commands, when asked to use tor, check to see if it will actually work. That means spinning up a Tor executable (or connecting to the
--tor-controlport=
provided) and seeing if Tor can get the full descriptor list. Maybe even making a connection to a tahoe-lafs.org -hosted onion service to make sure it works.And looking towards the future, meejah had a great idea about a sort of "setup wizard".
tahoe create-node --web
could run a server and pop open your web browser to a control panel. It could then run ifconfig and see if your box seemed to have a public IP address, or try to get a UPnP port, or provide instructions for setting up a firewall port-forwarding, etc. Then it could ask the user for permission to confirm connectivity, by having a service on tahoe-lafs.org connect back down to the purported address.In the end, we really want running a server to be as easy as running a client, but the networking world in which we live doesn't make that trivial. I think
--hostname=
is about the simplest possible tool (but only works if you have a public-IP VPS server of some sort), and--location= --port=
is the second-simplest tool.In f57d1e9/trunk:
BTW, that 2773-stats patch fixed the last of our "pending deprecations": warnings we'd get from running the test suite against the latest trunk version of Foolscap and Twisted.
moving most tickets from 1.12 to 1.13 so we can release 1.12 with magic-folders
At some point on IRC, I think we (maybe str4d?) suggested we use
--listen=tor
/--listen=i2p
, instead of--listen-tor
/--listen-i2p
. Talking with dawuud today, we thought of:none
)although I don't think we should feel obligated to support automatically-configured multiple listeners on the first step.
The first step is probably to change the create-node functions to be async, which means changing
runner.py
to accept a Deferred back from the dispatch function, and spin up a reactor, and wait for the Deferred to finish before shutting it down and returning the exit code. Twisted has a utility for thistwisted.internet.task.react()
, which I'm using in magic-wormhole, although it callssys.exit(rc)
itself, so we might only want to use it in the run-by-human=true path.Second step is to add --location/--port. Third is to add --listen=tcp. Fourth is to implement --listen=tor or i2p.
Although we could put off the async create-node until just before implementing --listen=tor, because we've got a synchronous
allmydata.util.iputil.allocate_tcp_port()
.It's looking like we're pulling this into 1.12, since it's turning into the "private-IP AND magic-folders" release. If it looks like trouble, we can push it back out again.
Note: #2490 is about
--listen=tor/i2p
and automatic server setup. This ticket (#2773) is just about --port/--location/--hostname, the non-automatic setup. #2773 may include--listen=tcp
and/or--listen=none
, but should not block on the async-ification of node creation, or on anything involving onion server creation.1.12 will block on #2773, since I think we want those setup functions to make manually-configured tahoe servers easier to build. But I'm going to leave #2490 in the next (1.13) milestone, unless it happens to get finished early (in which case, yay! 1.12 is even cooler).
So I'm going to suggest that the order of operations is:
this PR comes pretty close to the above steps 1 & 2:
https://github.com/tahoe-lafs/tahoe-lafs/pull/336
the tests don't pass. not sure why.
The
create-node
command requires--hostname=localhost
to be added to get the previous behaviour. I know that often Explicit Is Better Than Implicit, but I'm not clear on what the advantage is of requiring this.Hm.. the previous behavior was that
tahoe create-node
(with no arguments) would 1: listen on all interfaces, 2: auto-detect the host's IP addresses, 3: put all detected IP addresses in the Tub location hints. So you'd wind up with the equivalent of e.g.tub.port = tcp:12345 ; tub.location = tcp:127.0.0.1:12345,tcp:8.8.8.8:12345
.I believe we decided that automatic IP address detection was bad/confusing/unhelpful, and automatic detection of externally-unreachable IPs like 127.0.0.1 and RFC1918 addresses was nigh-useless. (I originally pushed back against this, because I run test networks on my local box all the time, but I finally agreed that the majority use case is more important).
So I think we agreed that running a server on a box with a public IP can/should be one argument more verbose: you have to know enough about networking to know the hostname (or public IP address) of that box, and you provide it when setting up the node.
So as far as use cases go:
tahoe create-client
and none of this matters, no extra arguments are neededtahoe create-node --hostname=MYHOSTNAME
and you'll wind up with an allocated TCP port and a matchingtub.port
/tub.location
tahoe create-node --port=tcp:LOCALPORT --location=tcp:EXTERNALIP:EXTERNALPORT
--port= --location=
to make it work--port= --location=
and then when #2490 lands, we add:
tahoe create-node --listen=tor
(or i2p) to automatically set up the .onion/.i2p addressIn a8899c8/trunk:
In 5a195e2/trunk: