write a tool to copy all mutable shares onto a single or fewer disks #1277

New Issue

tahoe-lafs · 2010-12-07T05:12:55Z

davidsarah commented

2010-12-07 05:12:55 +00:00

This could potentially help to retrieve the data of allmydata.com customers without having to put all of the disks online at once, by allowing us to build the directory tree. (All directories at that time will have been mutable.)

The tahoe debug dump-share command has code to determine whether a share is mutable.

This could potentially help to retrieve the data of allmydata.com customers without having to put all of the disks online at once, by allowing us to build the directory tree. (All directories at that time will have been mutable.) The `tahoe debug dump-share` command has [code to determine whether a share is mutable](http://tahoe-lafs.org/trac/tahoe-lafs/browser/trunk/src/allmydata/scripts/debug.py#L44).

tahoe-lafs added the

labels 2010-12-07 05:12:55 +00:00

tahoe-lafs added this to the soon (release n/a) milestone 2010-12-07 05:12:55 +00:00

davidsarah commented

2010-12-08 02:16:08 +00:00

This has been rather cursorily tested and seems to work. It stops on any exception, which might not be what we want.

kevan commented

2010-12-08 17:58:42 +00:00

I can review this.

kevan commented

2010-12-11 21:18:24 +00:00

When I write a utility script in Python (especially when its functionality is abstracted in a way that allows reuse), I prefer to stick the bits that rely on the script running as a standalone program in an

if __name__=="__main__":

block. This won't, AFAIK, change the way the script runs if you run it as a standalone program, but it will also allow you to import bits and pieces of the script from within other programs or the REPL; the latter, at least, can be very useful for debugging. Without something like that, this doesn't work:

viking-2:tahoe-1277 kacarstensen$ python
Python 2.6.1 (r261:67515, Dec  6 2008, 16:42:21) 
[GCC 4.0.1 (Apple Computer, Inc. build 5370)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from copy_mutable_shares import make_dirs
Usage: python copy_mutable_shares.py SRC_BASEDIR DEST_BASEDIR
viking-2:tahoe-1277 kacarstensen$

In your script, it's only the last four statements that need that. Just a thought.

The script executes without issue on my machine.

shortsi should be short_si to be consistent with your other variable names. I also think it is easier to read as short_si.

I don't have a strong opinion on how the script should handle exceptions, since I think there are good arguments for both what it does now and for other behaviors. Maybe we should get secorp to look at this ticket, and tell us what he (or his helpers) would want the script to do.

I've satisfied myself that your mutable file check is exhaustive, since the state of [http://tahoe-lafs.org/trac/tahoe-lafs/browser/trunk/docs/mutable.txt?rev=8ba10d0155cddeb3 mutable.txt before the time that SDMF was finished] suggests that there was only ever one mutable container version, which is the one you're checking for.

(The mutable.txt document says that the magic string should start with a lowercase t, which does not agree with the implementation. I guess the specification should be fixed at some point)

When I write a utility script in Python (especially when its functionality is abstracted in a way that allows reuse), I prefer to stick the bits that rely on the script running as a standalone program in an ``` if __name__=="__main__": ``` block. This won't, AFAIK, change the way the script runs if you run it as a standalone program, but it will also allow you to import bits and pieces of the script from within other programs or the REPL; the latter, at least, can be very useful for debugging. Without something like that, this doesn't work: ``` viking-2:tahoe-1277 kacarstensen$ python Python 2.6.1 (r261:67515, Dec 6 2008, 16:42:21) [GCC 4.0.1 (Apple Computer, Inc. build 5370)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from copy_mutable_shares import make_dirs Usage: python copy_mutable_shares.py SRC_BASEDIR DEST_BASEDIR viking-2:tahoe-1277 kacarstensen$ ``` In your script, it's only the last four statements that need that. Just a thought. The script executes without issue on my machine. `shortsi` should be `short_si` to be consistent with your other variable names. I also think it is easier to read as `short_si`. I don't have a strong opinion on how the script should handle exceptions, since I think there are good arguments for both what it does now and for other behaviors. Maybe we should get secorp to look at this ticket, and tell us what he (or his helpers) would want the script to do. I've satisfied myself that your mutable file check is exhaustive, since the state of [http://tahoe-lafs.org/trac/tahoe-lafs/browser/trunk/docs/mutable.txt?rev=8ba10d0155cddeb3 mutable.txt before the time that SDMF was finished] suggests that there was only ever one mutable container version, which is the one you're checking for. (The mutable.txt document says that the magic string should start with a lowercase t, which does not agree with the implementation. I guess the specification should be fixed at some point)

davidsarah commented

2010-12-12 01:40:01 +00:00

Attachment copy_mutable_shares.py (2406 bytes) added

Script to copy mutable shares, with changes from kevan's review.

**Attachment** copy_mutable_shares.py (2406 bytes) added Script to copy mutable shares, with changes from kevan's review.

copy_mutable_shares.py

2.3 KiB

davidsarah commented

2010-12-12 01:53:15 +00:00

Replying to kevan:

(The mutable.txt document says that the magic string should start with a lowercase t, which does not agree with the implementation. I guess the specification should be fixed at some point)

Fixed in changeset:7da1885531508c25.

Replying to [kevan](/tahoe-lafs/trac-2024-07-25/issues/1277#issuecomment-81488): > (The mutable.txt document says that the magic string should start with a lowercase t, which does not agree with the implementation. I guess the specification should be fixed at some point) Fixed in changeset:7da1885531508c25.

meejah commented

2021-03-30 18:41:12 +00:00

Ticket retargeted after milestone closed

meejah modified the milestone from soon (release n/a) to soon

2021-03-30 18:41:12 +00:00

Sign in to join this conversation.