graceful handling of capabilities in a format from the future that you can't understand #708

New Issue

zooko · 2009-05-14T21:12:34Z

zooko commented

2009-05-14 21:12:34 +00:00

In the future, we will rev the format of LAFS files and directories. (Don't worry, this will happen very gracefully -- you can confidently continue to use the current format for as long as you wish without having to deal compatibility issues. So relax -- we know how to do this right.)

One detail that will help with the transition (when/if you choose to make the transition) is for older Tahoe clients such as v1.5 (which isn't out yet, but will be old by then) to fail in some clear and graceful manner when confronted with a capability of a new form that it doesn't know how to use. In the WUI and the CLI this is very simple: return a nice error message. If v1.5 is able to recognize that this is a capability from the future (as opposed to a corrupted or truncated capability or something else such as a normal old Web URL) and can tell the user so, so much the better.

But what about capabilities from the future which it finds in a directory? Looking at [NewDirectoryNode._unpack_contents()]source:src/allmydata/dirnode.py@20090411225205-92b7f-7adfb89cb4db4ac7d28427934dea3d2c108f6476#L194, it looks like it calls self._create_node() on each capability that it finds in a directory. _create_node() will raise an exception if its argument is not a known kind of capability, so that means using the current release of Tahoe (v1.4.1) you can't view or use a directory if any of the entries in it have capabilities that you don't know how to use.

It would be better if those entries which have an unusable capability were greyed-out (marked as unusable) but the rest of the directory remained usable. Again, if Tahoe v1.5 has a good way to discriminate between capabilities that look like they're from the future and capabilities that look like they're just messed up, the WUI could display some sort of "help" link explaining to the user why those entries are greyed out.

In the future, we will rev the format of LAFS files and directories. (Don't worry, this will happen very gracefully -- you can confidently continue to use the current format for as long as you wish without having to deal compatibility issues. So relax -- we know how to do this right.) One detail that will help with the transition (*when/if* you choose to make the transition) is for older Tahoe clients such as v1.5 (which isn't out yet, but will be old by then) to fail in some clear and graceful manner when confronted with a capability of a new form that it doesn't know how to use. In the WUI and the CLI this is very simple: return a nice error message. If v1.5 is able to recognize that this is a capability from the future (as opposed to a corrupted or truncated capability or something else such as a normal old Web URL) and can tell the user so, so much the better. But what about capabilities from the future which it finds in a directory? Looking at [NewDirectoryNode._unpack_contents()]source:src/allmydata/dirnode.py@20090411225205-92b7f-7adfb89cb4db4ac7d28427934dea3d2c108f6476#L194, it looks like it calls `self._create_node()` on each capability that it finds in a directory. `_create_node()` will raise an exception if its argument is not a known kind of capability, so that means using the current release of Tahoe (v1.4.1) you can't view or use a directory if *any* of the entries in it have capabilities that you don't know how to use. It would be better if those entries which have an unusable capability were greyed-out (marked as unusable) but the rest of the directory remained usable. Again, if Tahoe v1.5 has a good way to discriminate between capabilities that look like they're from the future and capabilities that look like they're just messed up, the WUI could display some sort of "help" link explaining to the user why those entries are greyed out.

zooko added the

labels 2009-05-14 21:12:34 +00:00

zooko added this to the eventually milestone 2009-05-14 21:12:34 +00:00

zooko commented

2009-05-14 22:22:05 +00:00

By the way, this could be implemented as a special case of #683 (handle arbitrary URIs in directories).

zooko commented

2009-06-10 18:07:22 +00:00

This is one of those issues which can cause more trouble for future releases if we don't do something about it in the current release, so I hope to address it for Tahoe-v1.5 (ASAP). (I guess I intend the keyword "backwards-compatibility" to flag such issues.)

zooko modified the milestone from eventually to 1.5.0

2009-06-10 18:07:22 +00:00

zooko commented

2009-06-11 19:52:11 +00:00

I meant "forward-compatibility": http://allmydata.org/pipermail/tahoe-dev/2009-June/001968.html

I meant "forward-compatibility": <http://allmydata.org/pipermail/tahoe-dev/2009-June/001968.html>

zooko commented

2009-06-11 19:52:41 +00:00

No, actually I meant "forward-compatibility".

zooko commented

2009-07-02 14:18:43 +00:00

Brian is working on this.

warner was assigned by zooko

2009-07-02 14:18:43 +00:00

zooko commented

2009-07-03 04:26:03 +00:00

Over on ticket #683, comment 5, Brian wrote:

changeset:ef1b6ae8e312af21 changes the way dirnodes are processed to tolerate unrecognized URIs. This should make tahoe-1.5 able to survive new formats that come from the future (i.e. if a 1.5 client tries to read or modify a directory which has new-format entries which were placed there by some 1.6-or-beyond version). It's at least a start.

As I understand it, the fact that we can't add unknown caps into a directory means that we can't copy directories which contain caps from the future. (If we do copy such a directory then the entries in it which had new-style caps will be omitted from the newly created copy of the directory). In theory it should be possible to do that safely just by copying the write-cap field from the entry in the source dir into the write-cap field of the newly created entry in the target dir, and likewise copying the read-cap.

Brian: would there be any danger in that approach?

I don't know how important it would be for clients from the past to be able to copy your new-style caps.

Over on [ticket #683, comment 5](/tahoe-lafs/trac-2024-07-25/issues/683#issuecomment-70727), Brian wrote: > changeset:ef1b6ae8e312af21 changes the way dirnodes are processed to tolerate unrecognized URIs. This should make tahoe-1.5 able to survive new formats that come from the future (i.e. if a 1.5 client tries to read or modify a directory which has new-format entries which were placed there by some 1.6-or-beyond version). It's at least a start. As I understand it, the fact that we can't add unknown caps into a directory means that we can't copy directories which contain caps from the future. (If we do copy such a directory then the entries in it which had new-style caps will be omitted from the newly created copy of the directory). In theory it should be possible to do that safely just by copying the write-cap field from the entry in the source dir into the write-cap field of the newly created entry in the target dir, and likewise copying the read-cap. Brian: would there be any danger in that approach? I don't know how important it would be for clients from the past to be able to copy your new-style caps.

zooko added the

fixed

label 2009-07-03 04:26:03 +00:00

zooko closed this issue

2009-07-03 04:26:03 +00:00

warner commented

2009-07-03 16:21:37 +00:00

The internal 'move' method does just that, and the JSON representation of a directory includes all the information we have about the unknown object (i.e. both the writecap and the readcap). What I don't know is how the CLI-side "tahoe cp" works, specifically if the put-lots-of-caps-at-once dirnode webapi operation will accept the same "unknown cap" structure that the JSON representation hands down. Also, I wanted to discourage people from adding new unknown caps to a directory (because they might just be adding complete junk, or a typo, or a blank string, and it'd be nice to detect that early), so the current code is liberal in what it accepts from the backend, but strict in what it accepts from the frontend, and this might prevent the frontend-based tools from doing this sort of copy.

So yes, I think that approach would be safe, and it might already work. (of course we have no way to tell if the unknown-cap is a file or a directory, or something even more exotic, so we might be creating a hardlink to a mutable directoryish-thing when the rest of the copy operation was trying to make a deepcopy of individual files).

The test would need to go in test_cli.py where it tests the "tahoe cp" operation. grep around the test suite for UnknownNode, you have to be a bit sneaky to get the cap-from-the-future into a directory to start with.

The internal 'move' method does just that, and the JSON representation of a directory includes all the information we have about the unknown object (i.e. both the writecap and the readcap). What I don't know is how the CLI-side "tahoe cp" works, specifically if the put-lots-of-caps-at-once dirnode webapi operation will accept the same "unknown cap" structure that the JSON representation hands down. Also, I wanted to discourage people from adding new unknown caps to a directory (because they might just be adding complete junk, or a typo, or a blank string, and it'd be nice to detect that early), so the current code is liberal in what it accepts from the backend, but strict in what it accepts from the frontend, and this might prevent the frontend-based tools from doing this sort of copy. So yes, I think that approach would be safe, and it might already work. (of course we have no way to tell if the unknown-cap is a file or a directory, or something even more exotic, so we might be creating a hardlink to a mutable directoryish-thing when the rest of the copy operation was trying to make a deepcopy of individual files). The test would need to go in test_cli.py where it tests the "tahoe cp" operation. grep around the test suite for UnknownNode, you have to be a bit sneaky to get the cap-from-the-future into a directory to start with.

warner added

code-dirnodes

and removed

code-encoding

labels 2009-07-03 16:21:37 +00:00

davidsarah commented

2009-11-23 03:26:42 +00:00

Copying caps from the future is now #839.

Sign in to join this conversation.