"tahoe backup --prefix": override pathname root for snapshots #632

New Issue

warner · 2009-02-18T17:42:53Z

warner commented

2009-02-18 17:42:53 +00:00

I've been perusing the LVM documentation, looking at the "snapshot" feature, which of course could be used for easy coherent backups (to avoid the concern that someone will change a file while you're backing it up, thus getting neither the old version nor the new finished version). But the current implementation of tahoe's backupdb uses os.path.abspath as the lookup key. So if we were to snapshot, say, /home into /mnt/home-snapshot-TIMESTAMP before running 'tahoe backup' against /mnt/home-snapshot-TIMESTAMP, the backupdb filenames would be completely different than the previous day's /mnt/home-snapshot-OLDTIMESTAMP.

So the useful feature would be to give 'tahoe backup' a --root or a --strip-prefix argument, which would tell it to remove some portion of the absolute pathname (and perhaps replace it with some other string) before doing the backupdb lookup. This would allow /mnt/home-snapshot-TIMESTAMP/foo to be compared against the previously-uploaded /mnt/home-snapshot-OLDTIMESTAMP/foo .

I've been perusing the LVM documentation, looking at the "snapshot" feature, which of course could be used for easy coherent backups (to avoid the concern that someone will change a file while you're backing it up, thus getting neither the old version nor the new finished version). But the current implementation of tahoe's backupdb uses `os.path.abspath` as the lookup key. So if we were to snapshot, say, /home into /mnt/home-snapshot-TIMESTAMP before running 'tahoe backup' against /mnt/home-snapshot-TIMESTAMP, the backupdb filenames would be completely different than the previous day's /mnt/home-snapshot-OLDTIMESTAMP. So the useful feature would be to give 'tahoe backup' a `--root` or a `--strip-prefix` argument, which would tell it to remove some portion of the absolute pathname (and perhaps replace it with some other string) before doing the backupdb lookup. This would allow /mnt/home-snapshot-TIMESTAMP/foo to be compared against the previously-uploaded /mnt/home-snapshot-OLDTIMESTAMP/foo .

warner added the

labels 2009-02-18 17:42:53 +00:00

warner added this to the undecided milestone 2009-02-18 17:42:53 +00:00

zooko commented

2009-02-18 18:07:34 +00:00

By the way, I had a good idea for a technique to avoid missed or inconsistent backups of a file at the cost of having delayed (even indefinitely delayed) backup of that file. It is written in this message:

http://allmydata.org/pipermail/tahoe-dev/2008-September/000809.html

I do hope that the makers of backup tools (Brian, Shawn, Francois?, ?) think about this, because the failure modes of the naive approach are rare and ugly, and my technique to avoid it doesn't seem too complicated.

Of course, what you really want is something much more sophisticated, such as a file-system-wide cheap snapshot. NTFS has something along those lines -- Volume Shadow Something Or Other, and of course ZFS is really good at that, and hopefully someday BTRFS and others will be. But for now, please consider whether my technique described in that mail will work.

By the way, I had a good idea for a technique to avoid missed or inconsistent backups of a file at the cost of having delayed (even indefinitely delayed) backup of that file. It is written in this message: <http://allmydata.org/pipermail/tahoe-dev/2008-September/000809.html> I do hope that the makers of backup tools (Brian, Shawn, Francois?, ?) think about this, because the failure modes of the naive approach are rare and ugly, and my technique to avoid it doesn't seem too complicated. Of course, what you *really* want is something much more sophisticated, such as a file-system-wide cheap snapshot. NTFS has something along those lines -- Volume Shadow Something Or Other, and of course ZFS is really good at that, and hopefully someday BTRFS and others will be. But for now, please consider whether my technique described in that mail will work.

warner commented

2010-01-27 22:39:29 +00:00

#848 was a duplicate of this one, merging them.

warner changed title from ~~backupdb: override pathname root for snapshots~~ to "tahoe backup --prefix": override pathname root for snapshots

2010-01-27 22:39:29 +00:00

Sign in to join this conversation.