the timers used by tahoe backup to trigger repair for unhealthy files should be configurable #915

New Issue

tahoe-lafs · 2010-01-18T03:54:56Z

davidsarah commented

2010-01-18 03:54:56 +00:00

Reported by Kyle Markley:

I am nervous about synchronization between backupdb.sqlite and the grid. As happened recently, if a large chunk of the grid goes down, some files may go below their minimum number of shares. I could repair those files by re-uploading them, but the backupdb assumes those files are fine. So those files won't be reuploaded and they'll stay broken on the grid -- possibly forever. I'll think they're backed up, but they aren't! I'd like to have some way for a deep-check to inject a dose of reality into my backupdb.

For extra credit, I'd like behavior that's a hybrid between backup and deep-check. Run a backup operation, but check every file as we go along, and repair or re-upload the file if the grid doesn't have enough shares.

(Actually it should be if there are insufficient servers of happiness for the file, as per #614.)

Reported by Kyle Markley: > I am nervous about synchronization between backupdb.sqlite and the grid. As happened recently, if a large chunk of the grid goes down, some files may go below their minimum number of shares. I could repair those files by re-uploading them, but the backupdb assumes those files are fine. So those files won't be reuploaded and they'll stay broken on the grid -- possibly forever. I'll think they're backed up, but they aren't! I'd like to have some way for a deep-check to inject a dose of reality into my backupdb. > For extra credit, I'd like behavior that's a hybrid between backup and deep-check. Run a backup operation, but check every file as we go along, and repair or re-upload the file if the grid doesn't have enough shares. (Actually it should be if there are insufficient servers of happiness for the file, as per #614.)

tahoe-lafs added the

labels 2010-01-18 03:54:56 +00:00

tahoe-lafs added this to the undecided milestone 2010-01-18 03:54:56 +00:00

warner commented

2010-01-18 22:43:09 +00:00

Take a look at source:docs/backupdb.txt . Clients of the backupdb (like "tahoe backup") are required to eventually do a filecheck on the files that they skip uploading. It uses an age-since-last-check -based probability. The timer values are set such that all files will be checked at least every two months. If the check indicates unhealthy, the file is re-uploaded.

It would probably be useful to make these timers configurable. Setting them to "zero" would give you the check-each-time behavior you'd like. source:src/allmydata/scripts/backupdb.py#L163 (NO_CHECK_BEFORE and ALWAYS_CHECK_AFTER in BackupDB_v2) are the values that would need configuring.

Take a look at source:docs/backupdb.txt . Clients of the backupdb (like "tahoe backup") are required to eventually do a filecheck on the files that they skip uploading. It uses an age-since-last-check -based probability. The timer values are set such that all files will be checked at least every two months. If the check indicates unhealthy, the file is re-uploaded. It would probably be useful to make these timers configurable. Setting them to "zero" would give you the check-each-time behavior you'd like. source:src/allmydata/scripts/backupdb.py#L163 (`NO_CHECK_BEFORE` and `ALWAYS_CHECK_AFTER` in `BackupDB_v2`) are the values that would need configuring.

tahoe-lafs changed title from ~~tahoe backup should trigger repair for unhealthy files~~ to the timers used by tahoe backup to trigger repair for unhealthy files should be configurable

2010-01-20 07:06:06 +00:00

tahoe-lafs modified the milestone from undecided to 1.7.0

2010-02-01 19:46:49 +00:00

zooko modified the milestone from 1.7.0 to eventually

2010-06-19 01:01:43 +00:00

Sign in to join this conversation.