separate "gateway state directory" from "client state directory" #1310

New Issue

zooko · 2011-01-15T05:14:19Z

zooko commented

2011-01-15 05:14:19 +00:00

updating the Description for clarity:

This ticket is about the proposal to have separate directories for holding the state/configuration of the LAFS gateway from the state/configuration of the LAFS client. In the current (Tahoe-LAFS v1.10) code, both of those things are maintained in one shared directory, called the "node directory" or "base directory". The state therein is actually non-overlapping:

things a client (i.e. the "tahoe" command-line tool) uses out of the base directory:
- the node.url file to find out how to connect to the gateway
- the private/backupdb.sqlite file to find out what files have already been uploaded
- the private/aliases file holding aliases to caps
things a gateway uses out of the base directory:
- everything else that is stored in there except for the node.url, private/aliases and private/backupdb.sqlite files

So the client never uses any of the files that are kept in that directory for the gateway's purposes, and the gateway never uses any of the files that are kept in that directory for the client's purposes.

---- original Description follows

I use multiple grids (pub grid, volunteergrid, and a private family grid), and I just now had a confusing error where I ran tahoe backup and it completed quickly but produced a backup directory full of links to files with 0 shares each.

What happened, of course, was that I had previously run tahoe backup --node-url=<http://127.0.0.1:3458/> to backup these files to my family grid, and now I was running tahoe backup --node-url=<http://127.0.0.1:3457/> to backup these files to the volunteergrid, but I was unwittingly using the same backupdb.sqlite.

I wonder if, when the --node-url option is present, then the CLI shouldn't look into ~/.tahoe at all. Most of the configuration and state in ~/.tahoe is specific to the gateway that the --node-url points to, and the CLI will ignore it anyway and instead whatever configuration is in the tahoe-base-dir that is used by the gateway will take effect.

The only exception that I can think of right away is the private/backupdb.sqlite. Is that the only thing that affects the CLI when --node-url is present? Maybe it should be kept in a different directory.

I think I'm a bit confused about this. I'm not sure what all it means that there exists a ~/.tahoe when I'm actually using a gateway which runs as a separate user process, is specified by the --node-url option, and it has its own ~/.tahoe in its own user account. As a work-around and a way to gain clarity, I'll probably start specifying --node-directory in addition to --node-url, but this really feels wrong as it isn't a node directory at all! It is a CLI directory. :-)

updating the Description for clarity: This ticket is about the proposal to have separate directories for holding the state/configuration of the LAFS gateway from the state/configuration of the LAFS client. In the current (Tahoe-LAFS v1.10) code, both of those things are maintained in one shared directory, called the "node directory" or "base directory". The state therein is actually non-overlapping: * things a client (i.e. the "tahoe" command-line tool) uses out of the base directory: - the `node.url` file to find out how to connect to the gateway - the `private/backupdb.sqlite` file to find out what files have already been uploaded - the `private/aliases` file holding aliases to caps * things a gateway uses out of the base directory: - everything else that is stored in there *except* for the `node.url`, `private/aliases` and `private/backupdb.sqlite` files So the client never uses any of the files that are kept in that directory for the gateway's purposes, and the gateway never uses any of the files that are kept in that directory for the client's purposes. ---- original Description follows I use multiple grids (pub grid, volunteergrid, and a private family grid), and I just now had a confusing error where I ran `tahoe backup` and it completed quickly but produced a backup directory full of links to files with 0 shares each. What happened, of course, was that I had previously run `tahoe backup --node-url=<http://127.0.0.1:3458/>` to backup these files to my family grid, and now I was running `tahoe backup --node-url=<http://127.0.0.1:3457/>` to backup these files to the volunteergrid, but I was unwittingly using the same `backupdb.sqlite`. I wonder if, when the `--node-url` option is present, then the CLI shouldn't look into `~/.tahoe` at all. Most of the configuration and state in `~/.tahoe` is specific to the gateway that the `--node-url` points to, and the CLI will ignore it anyway and instead whatever configuration is in the tahoe-base-dir that is used by the gateway will take effect. The only exception that I can think of right away is the `private/backupdb.sqlite`. Is that the only thing that affects the CLI when `--node-url` is present? Maybe it should be kept in a different directory. I think I'm a bit confused about this. I'm not sure what all it means that there exists a `~/.tahoe` when I'm actually using a gateway which runs as a separate user process, is specified by the `--node-url` option, and it has its own `~/.tahoe` in its own user account. As a work-around and a way to gain clarity, I'll probably start specifying `--node-directory` in addition to `--node-url`, but this really feels wrong as it isn't a node directory at all! It is a CLI directory. :-)

zooko added the

labels 2011-01-15 05:14:19 +00:00

zooko added this to the undecided milestone 2011-01-15 05:14:19 +00:00

warner was assigned by zooko

2011-01-15 05:14:19 +00:00

davidsarah commented

2011-01-15 10:18:24 +00:00

Duplicate of #977.

I think I'm a bit confused about this. I'm not sure what all it means that there exists a ~/.tahoe when I'm actually using a gateway which runs as a separate user process, is specified by the --node-url option, and it has its own ~/.tahoe in its own user account.

I also found that confusing.

Duplicate of #977. > I think I'm a bit confused about this. I'm not sure what all it means that there exists a ~/.tahoe when I'm actually using a gateway which runs as a separate user process, is specified by the --node-url option, and it has its own ~/.tahoe in its own user account. I also found that confusing.

tahoe-lafs added the

duplicate

label 2011-01-15 10:18:24 +00:00

tahoe-lafs closed this issue

2011-01-15 10:18:24 +00:00

zooko commented

2011-01-15 17:46:24 +00:00

I'm not sure that I agree that this is just a duplicate of #977 (backupdb should store which grid it is scoped to). What about the parts of ~/.tahoe which don't have to do with backup at all? It is still confusing that the command

tahoe put --node-url=http://127.0.0.1:3456 --node-directory=~/.tahoe-volunteergrid MYFILE

will not use any of your configuration from your ~/.tahoe-volunteergrid directory and indeed may not touch the grid you call "volunteergrid" at all.

I'm not sure that I agree that this is just a duplicate of #977 (backupdb should store which grid it is scoped to). What about the parts of `~/.tahoe` which don't have to do with backup at all? It is still confusing that the command `tahoe put --node-url=http://127.0.0.1:3456 --node-directory=~/.tahoe-volunteergrid MYFILE` will not use any of your configuration from your `~/.tahoe-volunteergrid` directory and indeed may not touch the grid you call "volunteergrid" at all.

zooko removed the

duplicate

label 2011-01-15 17:46:24 +00:00

zooko reopened this issue

2011-01-15 17:46:24 +00:00

zooko changed title from ~~backupdb.sqlite (and all other state in ~/.tahoe?) should be scoped to the --node-url option if it is present~~ to If --node-url is present then --node-directory is mostly but not entirely ignored.

2011-01-15 17:46:24 +00:00

zooko commented

2011-01-15 19:37:14 +00:00

Hm, I'm not sure about this, but one potential solution to this issue would be for there to be a --cli-directory= option which points to a directory that contains nothing but an optional backupdb and a node URL. Hm, in fact that's all that the cli ever uses the so-called --node-directory for, right? It is really not true that you ever tell the CLI what the node directory is. You really tell it where to find the node URL and then it contacts a node with that URL and that node uses whatever node directory it is configured to use. So the option to the CLI named --node-directory is a misnomer.

Hm, I'm not sure about this, but one potential solution to this issue would be for there to be a `--cli-directory=` option which points to a directory that contains nothing but an optional backupdb and a node URL. Hm, in fact that's all that the cli ever uses the so-called `--node-directory` for, right? It is really not true that you ever tell the CLI what the node directory is. You really tell it where to find the node URL and then it contacts a node with that URL and that node uses whatever node directory it is configured to use. So the option to the CLI named `--node-directory` is a misnomer.

mlakewood commented

2011-01-21 07:22:41 +00:00

In addition to this i've experienced the following problem. If I create a client node in the default directory ie .tahoe through tahoe create-client command, and then modify the web.port = tcp:3456:interface=127.0.0.1 to be say web.port = tcp:3456:interface=10.0.10.89 when you try to run a command it will thow and exception about trying to connect to the client. This is because in node.url its still pointing at http://127.0.0.1:3456 . If you change this to point at 10.0.10.89:3456 then all is good. however if you stop and start the tahoe client then the node.url is reset to http://127.0.0.1:3456 . which is a bit of a usability problem.. node.url seems like a configuration file, but gets overridden with a potentially invalid default. It seems to me node.url shouldn't be overriden on tahoe start.

[to make all the examples use the same port number, which I believe is what mlakewood intended --Zooko]edited

In addition to this i've experienced the following problem. If I create a client node in the default directory ie .tahoe through tahoe create-client command, and then modify the **web.port = tcp:3456:interface=127.0.0.1** to be say **web.port = tcp:3456:interface=10.0.10.89** when you try to run a command it will thow and exception about trying to connect to the client. This is because in node.url its still pointing at <http://127.0.0.1:3456> . If you change this to point at 10.0.10.89:3456 then all is good. however if you stop and start the tahoe client then the node.url is reset to <http://127.0.0.1:3456> . which is a bit of a usability problem.. node.url seems like a configuration file, but gets overridden with a potentially invalid default. It seems to me node.url shouldn't be overriden on tahoe start. [to make all the examples use the same port number, which I believe is what mlakewood intended --Zooko]edited

zooko commented

2011-01-21 07:56:20 +00:00

Straw man proposal (I'm sleepy!):

Remove --node-directory from the CLI and document the fact that users can write tahoe --node-url=cat ~/.tahoe/node.url ls $DIRCAP if they wish. This syntax will make it clear to the user who inspects it that the tahoe command is not reading the contents of ~/.tahoe directory aside from the contents of that one file.

Straw man proposal (I'm sleepy!): Remove `--node-directory` from the CLI and document the fact that users can write `tahoe --node-url=`cat ~/.tahoe/node.url` ls $DIRCAP` if they wish. This syntax will make it clear to the user who inspects it that the `tahoe` command is not reading the contents of `~/.tahoe` directory aside from the contents of that one file.

warner commented

2011-01-21 17:41:41 +00:00

CLI tools look in --node-directory for things that are generated
by:

the tahoe client node, to help the CLI tools find the node
the CLI tools themselves, that they want to retrieve later

At present, the only from-node-to-CLI things are node.url, to find
the webapi port. In the future, I'm considering adding a few more files,
starting with a $NODEDIR/private/control.key which would help a
tahoe webopen --control-panel derive a secret URL that points to
node-control functions, outside the scope of any particular filecap.
Another possibility is an accounting.secret, which would let the
CLI tools create webapi URLs that include authority to use space on a
grid (rather than the ambient authority that currently grants storage
rights to anyone who can talk to the webapi port). Also,
$NODEDIR/private/control.furl could be used by CLI commands that
wanted foolscap-based access (which would be easier for some purposes
than HTTP-based access).

The set of from-CLI-to-CLI things in there currently includes:

private/aliases: allows "tahoe ls home:" instead of "tahoe ls
URI:DIR2:blahblahblahblahblah"
private/backupdb.sqlite: used by 'tahoe backup', potentially
'tahoe cp'

I named it --node-directory because you can set it equal to a
tahoe client node's base directory (which is what I usually mean by
"$NODEDIR") and then the CLI tool will correctly find everything it
needs to work without any extra effort on your part (and the secret
things that it writes will go into a well-named directory that's already
chmod go-rwx). And, because --node-directory is consistent
across all tahoe commands, including tahoe create-node and
tahoe start. So you could make a one-line shell script which did
tahoe --node-directory=~/.other $* named othertahoe and use
that for everything and it would Just Work (well, if we fixed the option
parser to look for --node-directory before the command instead of
after).

I've gotta run, I'll come back to this ticket later to chime in about
the proposal. At first glance, --cli-directory does sound more
accurate, but I think the parallelism of --node-directory
everywhere is valuable.

CLI tools look in `--node-directory` for things that are generated by: * the tahoe client node, to help the CLI tools find the node * the CLI tools themselves, that they want to retrieve later At present, the only from-node-to-CLI things are `node.url`, to find the webapi port. In the future, I'm considering adding a few more files, starting with a `$NODEDIR/private/control.key` which would help a `tahoe webopen --control-panel` derive a secret URL that points to node-control functions, outside the scope of any particular filecap. Another possibility is an `accounting.secret`, which would let the CLI tools create webapi URLs that include authority to use space on a grid (rather than the ambient authority that currently grants storage rights to anyone who can talk to the webapi port). Also, `$NODEDIR/private/control.furl` could be used by CLI commands that wanted foolscap-based access (which would be easier for some purposes than HTTP-based access). The set of from-CLI-to-CLI things in there currently includes: * `private/aliases`: allows "tahoe ls home:" instead of "tahoe ls URI:DIR2:blahblahblahblahblah" * `private/backupdb.sqlite`: used by 'tahoe backup', potentially 'tahoe cp' I named it `--node-directory` because you can set it equal to a tahoe client node's base directory (which is what I usually mean by "$NODEDIR") and then the CLI tool will correctly find everything it needs to work without any extra effort on your part (and the secret things that it writes will go into a well-named directory that's already `chmod go-rwx`). And, because `--node-directory` is consistent across all tahoe commands, including `tahoe create-node` and `tahoe start`. So you could make a one-line shell script which did `tahoe --node-directory=~/.other $*` named `othertahoe` and use that for everything and it would Just Work (well, if we fixed the option parser to look for `--node-directory` before the command instead of after). I've gotta run, I'll come back to this ticket later to chime in about the proposal. At first glance, `--cli-directory` does sound more accurate, but I think the parallelism of `--node-directory` everywhere is valuable.

zooko commented

2011-03-11 21:25:17 +00:00

I still have the feeling that tahoe-lafs gateways and tahoe-lafs clients are separate objects (for example, I prefer to run my gateways and my clients under separate user accounts on the same machine sometimes), and they should not share state in this way. I find it confusing.

Replying to warner:

I've gotta run, I'll come back to this ticket later to chime in about
the proposal. At first glance, --cli-directory does sound more
accurate, but I think the parallelism of --node-directory
everywhere is valuable.

I have the feeling that this is really wrong -- that you shouldn't say --node-directory to a gateway to mean the directory where it stores its persistent state such as the node.pem and also say --node-directory to a client to mean the directory where it stores its persistent state such as the backupdb.sqlite. It is the "bad" kind of parallelism, when the underlying thing is different, so the interface to it ought to be different too. :-)

(Also, I think it would help us all think and communicate more precisely if we stopped using the ambiguous word "node"... Except in those cases where we actually mean a Tahoe-LAFS process which runs more than one of (server, gateway, introducer) or which runs an unspecified service.)

I still have the feeling that tahoe-lafs gateways and tahoe-lafs clients are separate objects (for example, I prefer to run my gateways and my clients under separate user accounts on the same machine sometimes), and they should not share state in this way. I find it confusing. Replying to [warner](/tahoe-lafs/trac-2024-07-25/issues/1310#issuecomment-82055): > I've gotta run, I'll come back to this ticket later to chime in about > the proposal. At first glance, `--cli-directory` does sound more > accurate, but I think the parallelism of `--node-directory` > everywhere is valuable. I have the feeling that this is really wrong -- that you shouldn't say `--node-directory` to a gateway to mean the directory where it stores its persistent state such as the `node.pem` and also say `--node-directory` to a client to mean the directory where it stores its persistent state such as the `backupdb.sqlite`. It is the "bad" kind of parallelism, when the underlying thing is different, so the interface to it *ought* to be different too. :-) (Also, I think it would help us all think and communicate more precisely if we stopped using the ambiguous word "node"... Except in those cases where we actually mean a Tahoe-LAFS process which runs more than one of (server, gateway, introducer) or which runs an unspecified service.)

gdt commented

2011-03-11 21:39:38 +00:00

I broadly concur with zooko in comment 7. Sort of related, as I've been setting up storage nodes in a private grid, I find myself wishing for the grid parameters to be in a separate file from the local parameters, so that I could copy the grid-params.conf file around and just drop it in, and not have that overwrite the node id.

Beyond this, I think that probably one should have a directory to represent client access, and that would then have the aliases file, the backupdb, and a pointer to the WAPI. The gateway node directory would then be separate. Once this is done, the CLI probably should lose the ability to specify the WAPI URL directly, and only point to client directories.

I broadly concur with zooko in comment 7. Sort of related, as I've been setting up storage nodes in a private grid, I find myself wishing for the grid parameters to be in a separate file from the local parameters, so that I could copy the grid-params.conf file around and just drop it in, and not have that overwrite the node id. Beyond this, I think that probably one should have a directory to represent client access, and that would then have the aliases file, the backupdb, and a pointer to the WAPI. The gateway node directory would then be separate. Once this is done, the CLI probably should lose the ability to specify the WAPI URL directly, and only point to client directories.

zooko commented

2011-09-26 20:22:47 +00:00

I'm thinking about configuration right now, so I'm interested in this ticket. Commenting here to remind myself about it and to let everyone know that I might get around to working on it sometime.

I'm thinking about configuration right now, so I'm interested in this ticket. Commenting here to remind myself about it and to let everyone know that I *might* get around to working on it sometime.

zooko changed title from ~~If --node-url is present then --node-directory is mostly but not entirely ignored.~~ to separate "gateway state directory" from "client state directory"

2013-11-27 20:07:40 +00:00

daira commented

2013-11-28 22:08:57 +00:00

I wouldn't be opposed to adding --cli-directory, but I would be opposed to not having it default to the same thing as --node-directory. Not doing that would break people's existing setups and would be a serious usability regression.

I wouldn't be opposed to adding `--cli-directory`, but I *would* be opposed to not having it default to the same thing as `--node-directory`. Not doing that would break people's existing setups and would be a serious usability regression.

amontero commented

2013-12-03 16:09:26 +00:00

Added private/aliases to client-side files in issue description. Also opened #2127, since docs provided no clear answer.

Added `private/aliases` to client-side files in issue description. Also opened #2127, since docs provided no clear answer.

warner commented

2014-09-22 21:40:20 +00:00

(circling back to this ticket thanks to zooko's link from #2045, which is about larger-scale changes to the node's and code's directory layout)

Rereading zooko's initial issue, I found myself tempted to yell out "don't do that!". I guess I've always optimized tahoe's frontend- and setup- management tools for the common case of a single "gateway" per (user*computer) tuple. I really want the instructions to be as simple as "tahoe create; tahoe start; tahoe webopen". I don't want to complicate that for the sake of the less-common use case of multiple nodes/gateways/clients/whatevers.

Partly that indicates a lack of universality in our design (which we've always known about, and always regretted, but also know better than to try and fix, because it's very hard, certainly distracting, probably confusing, and slightly impossible). There's no "one true grid" (#2009). Grids can't be too big (#235, #444), increasing the demand for using multiple ones, in particular if you want to use tahoe to share files with other people. There are no "grid identifiers" in filecaps (#403), so if you want to use multiple distinct grids, you need multiple distinct client nodes ("gateways" in zooko's lexicon) and must be careful to give the right filecap to the right node (WAPI port / gateway process).

A side-note on terminology mismatches: I think (and talk) about tahoe in terms of three pieces:

1: frontends (CLI scripts, web browsers, FTP/SFTP clients), all

talking over a network connection (WAPI or others) to the client node

2: client nodes, which respond to WAPI requests, perform the

upload/download/encode/decode algorithms and make connections to server nodes

3: server nodes, which are (so far) agnostic about file-encoding

formats and just respond to PUT/GET-share requests from client nodes

I think Zooko thinks/talks in the same three pieces but with different names:

1: clients (CLI scripts, web browsers, FTP/SFTP clients)
2: gateways
3: servers

A related angle is the imperfect distinction between the functions performed by pieces 1 and 2. tahoe backup is a good example: this is currently a CLI command, but I feel that it should really be moved into the client node (#1018). Backup is more of an ongoing process than a one-off action (#643). A one-shot CLI command needs to be run from cron to make it into a process, and then it doesn't have enough information to coordinate with other (overlapping) runs (#2062, #2053). I'd like to have backups be managed through some sort of control panel (#1588, #1587), where you can express your priorities and preferences about what you want to be backed up and how much network/CPU it's allowed to consume, and then the backup agent handles the rest. This control panel should also be a place to check in on the process, especially for progress reports during the long initial upload.

There are two big blockers for this sort of long-running agent. The first is how/whether to split this from the long-running code that knows how to upload/download files. We had a good discussion about this at the 2011 summit. Zooko's mental model, which uses the word "gateway", helps make this a bit more clear: my desired backup-manager process would live in an "Agent", and the upload/download stuff would live in the "Gateway", and maybe the Agent would use te Gateway but not vice-versa. Depending upon the value of caching server connections and generally having a long-term relationship with servers (tracking uptime/speed/reliability), it might even make sense for the Gateway functionality to not live in a long-term process, and instead be a short-lifetime library that gets loaded on demand (imagine if "tahoe put" were standalone, maybe learning about cached server information from a sqlite database, but establishing its own server connections as necessary).

The other is how to safely talk to this agent, honoring our objcap no-ambient-authority style: some sort of restricted-access web-based control panel (#674, wiki/Summit2Day2#ControlPanel) which I've been prototyping externally in my "toolbed" and "petmail" projects (but it's very JS-heavy).

Anyways, that was a long diversion away from the main point: the use of a single NODEDIR to manage the states and configurations of all these pieces (client-ish stuff, agent-ish stuff, gateway-ish stuff, heck even server-ish stuff) is ideal for one-grid cases, and confusing for multiple-grid cases.

I'm warming slightly to the --cli-directory= idea. Maybe by splitting these different bits of functionality into separate subdirs, putting all of them in the single NODEDIR by default, but making it clear that e.g. CLI commands only touch stuff in NODEDIR/cli/* . Then make it possible to either override the top-level --nodedir=, or a CLI-functionality-specific --cli-directory=.

It's worth remembering the overlap between these components, though. The gateway writes out a node.url file, the frontend commands read it. Control panels will involve access keys being read or written to agent-accessible databases. We might be able to statically construct enough of this that we don't need to think about ongoing config-directory-based communication between components after initial tahoe create, but maybe not.

But I still think it may be easier to tell people "don't do that" and try to make the one-grid-per-user case work better.

(circling back to this ticket thanks to zooko's link from #2045, which is about larger-scale changes to the node's and code's directory layout) Rereading zooko's initial issue, I found myself tempted to yell out "don't do that!". I guess I've always optimized tahoe's frontend- and setup- management tools for the common case of a single "gateway" per (user*computer) tuple. I really want the instructions to be as simple as "tahoe create; tahoe start; tahoe webopen". I don't want to complicate that for the sake of the less-common use case of multiple nodes/gateways/clients/whatevers. Partly that indicates a lack of universality in our design (which we've always known about, and always regretted, but also know better than to try and fix, because it's very hard, certainly distracting, probably confusing, and slightly impossible). There's no "one true grid" (#2009). Grids can't be too big (#235, #444), increasing the demand for using multiple ones, in particular if you want to use tahoe to share files with other people. There are no "grid identifiers" in filecaps (#403), so if you want to use multiple distinct grids, you need multiple distinct client nodes ("gateways" in zooko's lexicon) and must be careful to give the right filecap to the right node (WAPI port / gateway process). A side-note on terminology mismatches: I think (and talk) about tahoe in terms of three pieces: * 1: frontends (CLI scripts, web browsers, FTP/SFTP clients), all > talking over a network connection (WAPI or others) to the client node * 2: client nodes, which respond to WAPI requests, perform the > upload/download/encode/decode algorithms and make connections to server nodes * 3: server nodes, which are (so far) agnostic about file-encoding > formats and just respond to PUT/GET-share requests from client nodes I think Zooko thinks/talks in the same three pieces but with different names: * 1: clients (CLI scripts, web browsers, FTP/SFTP clients) * 2: gateways * 3: servers A related angle is the imperfect distinction between the functions performed by pieces 1 and 2. `tahoe backup` is a good example: this is currently a CLI command, but I feel that it should really be moved into the client node (#1018). Backup is more of an ongoing process than a one-off action (#643). A one-shot CLI command needs to be run from cron to make it into a process, and then it doesn't have enough information to coordinate with other (overlapping) runs (#2062, #2053). I'd like to have backups be managed through some sort of control panel (#1588, #1587), where you can express your priorities and preferences about what you want to be backed up and how much network/CPU it's allowed to consume, and then the backup agent handles the rest. This control panel should also be a place to check in on the process, especially for progress reports during the long initial upload. There are two big blockers for this sort of long-running agent. The first is how/whether to split this from the long-running code that knows how to upload/download files. We had a good [discussion](wiki/Summit2Day1#AgentGatewaysplit) about this at the [2011 summit](wiki/Summit2011). Zooko's mental model, which uses the word "gateway", helps make this a bit more clear: my desired backup-manager process would live in an "Agent", and the upload/download stuff would live in the "Gateway", and maybe the Agent would use te Gateway but not vice-versa. Depending upon the value of caching server connections and generally having a long-term relationship with servers (tracking uptime/speed/reliability), it might even make sense for the Gateway functionality to *not* live in a long-term process, and instead be a short-lifetime library that gets loaded on demand (imagine if "tahoe put" were standalone, maybe learning about cached server information from a sqlite database, but establishing its own server connections as necessary). The other is how to safely talk to this agent, honoring our objcap no-ambient-authority style: some sort of restricted-access web-based control panel (#674, [wiki/Summit2Day2#ControlPanel](wiki/Summit2Day2#ControlPanel)) which I've been prototyping externally in my "toolbed" and "petmail" projects (but it's very JS-heavy). Anyways, that was a long diversion away from the main point: the use of a single NODEDIR to manage the states and configurations of all these pieces (client-ish stuff, agent-ish stuff, gateway-ish stuff, heck even server-ish stuff) is ideal for one-grid cases, and confusing for multiple-grid cases. I'm warming slightly to the `--cli-directory=` idea. Maybe by splitting these different bits of functionality into separate subdirs, putting all of them in the single NODEDIR by default, but making it clear that e.g. CLI commands only touch stuff in NODEDIR/cli/* . Then make it possible to either override the top-level `--nodedir=`, or a CLI-functionality-specific `--cli-directory=`. It's worth remembering the overlap between these components, though. The gateway writes out a `node.url` file, the frontend commands read it. Control panels will involve access keys being read or written to agent-accessible databases. We might be able to statically construct enough of this that we don't need to think about ongoing config-directory-based communication between components after initial `tahoe create`, but maybe not. But I still think it may be easier to tell people "don't do that" and try to make the one-grid-per-user case work better.

zooko commented

2014-09-22 21:49:11 +00:00

Replying to warner:

Rereading zooko's initial issue, I found myself tempted to yell out "don't do that!". I guess I've always optimized tahoe's frontend- and setup- management tools for the common case of a single "gateway" per (user*computer) tuple. I really want the instructions to be as simple as "tahoe create; tahoe start; tahoe webopen". I don't want to complicate that for the sake of the less-common use case of multiple nodes/gateways/clients/whatevers.
…
Anyways, that was a long diversion away from the main point: the use of a single NODEDIR to manage the states and configurations of all these pieces (client-ish stuff, agent-ish stuff, gateway-ish stuff, heck even server-ish stuff) is ideal for one-grid cases, and confusing for multiple-grid cases.

I'm not sure if the "separate dirs for separate state" approach is any more complicated, at least for the end user.

Specifically, for the proposal Brian suggests here:

I'm warming slightly to the --cli-directory= idea. Maybe by splitting these different bits of functionality into separate subdirs, putting all of them in the single NODEDIR by default, but making it clear that e.g. CLI commands only touch stuff in NODEDIR/cli/* . Then make it possible to either override the top-level --nodedir=, or a CLI-functionality-specific --cli-directory=.

What impact would this have on the end-user-visible CLI flow, such as the tahoe create; tahoe start; tahoe webopen that Brian mentioned?

Replying to [warner](/tahoe-lafs/trac-2024-07-25/issues/1310#issuecomment-82062): > > Rereading zooko's initial issue, I found myself tempted to yell out "don't do that!". I guess I've always optimized tahoe's frontend- and setup- management tools for the common case of a single "gateway" per (user*computer) tuple. I really want the instructions to be as simple as "tahoe create; tahoe start; tahoe webopen". I don't want to complicate that for the sake of the less-common use case of multiple nodes/gateways/clients/whatevers. … > Anyways, that was a long diversion away from the main point: the use of a single NODEDIR to manage the states and configurations of all these pieces (client-ish stuff, agent-ish stuff, gateway-ish stuff, heck even server-ish stuff) is ideal for one-grid cases, and confusing for multiple-grid cases. I'm not sure if the "separate dirs for separate state" approach *is* any more complicated, at least for the end user. Specifically, for the proposal Brian suggests here: > I'm warming slightly to the `--cli-directory=` idea. Maybe by splitting these different bits of functionality into separate subdirs, putting all of them in the single NODEDIR by default, but making it clear that e.g. CLI commands only touch stuff in NODEDIR/cli/* . Then make it possible to either override the top-level `--nodedir=`, or a CLI-functionality-specific `--cli-directory=`. What impact would this have on the end-user-visible CLI flow, such as the `tahoe create; tahoe start; tahoe webopen` that Brian mentioned?

warner commented

2014-09-22 22:25:25 +00:00

Hm. Ok, imagine this (I'm not sure I like it yet, but let's think about it):

emphasize well-defined boundaries of responsibility in non-server-side docs: "Gateway Process", "Agent Process", "CLI Tools" (oneshot)
users must create/configure the three things separately
first: tahoe create-gateway GATEWAYDIR. This directory holds human-edited config files and machine-edited state files, but only for gateway-related things: cached introducer data, server connection data, historical server reliability, etc. Maybe cached ciphertext (download caching, or upload caching to satisfy Shawn Willden's request for faster backup-and-shutdown handoff of responsibility from Agent to Gateway)
second: tahoe create-agent AGENTDIR GATEWAYDIR, where the AGENTDIR holds agent-related state, and the GATEWAYDIR is only used to figure out how to contact the gateway. Maybe tahoe create-agent AGENTDIR GATEWAYURL instead (except does it need to access anything else?).
third: tahoe configure-clients CLIDIR GATEWAYURL.

Except:

how do we want to split responsibility between CLI (client) scripts and the gateway? Like, where should the encoding parameters be? When we add accounting, which side holds the key?
what about CLI scripts that are supposed to talk to the Agent instead of the Gateway? tahoe configure-backup?

I kind of expect that tripling the number of commands that people have to run would roughly triple the perceived level of complexity, even if it exposes distinct components as separate conceptual entities in a way that's less confusing than conflating them.

Hm. Ok, imagine this (I'm not sure I like it yet, but let's think about it): * emphasize well-defined boundaries of responsibility in non-server-side docs: "Gateway Process", "Agent Process", "CLI Tools" (oneshot) * users must create/configure the three things separately * first: `tahoe create-gateway GATEWAYDIR`. This directory holds human-edited config files and machine-edited state files, but only for gateway-related things: cached introducer data, server connection data, historical server reliability, etc. Maybe cached ciphertext (download caching, or upload caching to satisfy Shawn Willden's request for faster backup-and-shutdown handoff of responsibility from Agent to Gateway) * second: `tahoe create-agent AGENTDIR GATEWAYDIR`, where the AGENTDIR holds agent-related state, and the GATEWAYDIR is only used to figure out how to contact the gateway. Maybe `tahoe create-agent AGENTDIR GATEWAYURL` instead (except does it need to access anything else?). * third: `tahoe configure-clients CLIDIR GATEWAYURL`. Except: * how do we want to split responsibility between CLI (client) scripts and the gateway? Like, where should the encoding parameters be? When we add accounting, which side holds the key? * what about CLI scripts that are supposed to talk to the Agent instead of the Gateway? `tahoe configure-backup`? I kind of expect that tripling the number of commands that people have to run would roughly triple the perceived level of complexity, even if it exposes distinct components as separate conceptual entities in a way that's less confusing than conflating them.

Sign in to join this conversation.