Automatically schedule repair process (and backups?) #643

New Issue

tahoe-lafs · 2009-02-25T11:45:23Z

mmore commented

2009-02-25 11:45:23 +00:00

1.have Tahoe an automatic repair, i.e. , the introducer monitor the replicas numbers, for example if we have configuration of 10 /3 , so introducer trigger repair process when the replicas became only 4 to keep the file available?
2. what are the threshold that triggers the repair process?

1.have Tahoe an automatic repair, i.e. , the introducer monitor the replicas numbers, for example if we have configuration of 10 /3 , so introducer trigger repair process when the replicas became only 4 to keep the file available? 2. what are the threshold that triggers the repair process?

tahoe-lafs added the

labels 2009-02-25 11:45:23 +00:00

tahoe-lafs added this to the undecided milestone 2009-02-25 11:45:23 +00:00

terrell commented

2009-02-25 15:21:27 +00:00

As I understand it, the repairer will try to keep 10 copies (in your configuration) on the grid at all times. If the checker determines that the current number of good shares is less than 10, the repairer will create/repair enough to get back up to 10.

Is this an enhancement request for a separate configurable repair threshold (some number < N) ?

As I understand it, the repairer will try to keep 10 copies (in your configuration) on the grid at all times. If the checker determines that the current number of good shares is less than 10, the repairer will create/repair enough to get back up to 10. Is this an enhancement request for a separate configurable repair threshold (some number < N) ?

mmore commented

2009-02-26 15:15:46 +00:00

Replying to terrell:

As I understand it, the repairer will try to keep 10 copies (in your configuration) on the grid at all times. If the checker determines that the current number of good shares is less than 10, the repairer will create/repair enough to get back up to 10.

Is this an enhancement request for a separate configurable repair threshold (some number < N) ?

indeed , my explanation was: currently has Tahoe automatic repair ?, i.e the lost share can be rebuild in order to keep the file available.

Replying to [terrell](/tahoe-lafs/trac-2024-07-25/issues/643#issuecomment-69984): > As I understand it, the repairer will try to keep 10 copies (in your configuration) on the grid at all times. If the checker determines that the current number of good shares is less than 10, the repairer will create/repair enough to get back up to 10. > > Is this an enhancement request for a separate configurable repair threshold (some number < N) ? indeed , my explanation was: currently has Tahoe automatic repair ?, i.e the lost share can be rebuild in order to keep the file available.

zooko commented

2009-03-05 04:14:26 +00:00

It is currently not "automatic" in the sense that there is an established process which you can tell it: "Please repair the following set of files on the following repeating schedule.". Such a process could be built, for example with a bash script or a short Python program, using the wapi or the cli to trigger the repair processes.

Brian Warner has been writing code like that, to trigger repair processes at certain times. I think it isn't quite documented yet but already runs. Also, you could write your own, which might actually be better as the exact set of file to repair and the exact schedule of when to repair them might fit your use better.

It is currently not "automatic" in the sense that there is an established process which you can tell it: "Please repair the following set of files on the following repeating schedule.". Such a process could be built, for example with a bash script or a short Python program, using the wapi or the cli to trigger the repair processes. Brian Warner has been writing code like that, to trigger repair processes at certain times. I think it isn't quite documented yet but already runs. Also, you could write your own, which might actually be better as the exact set of file to repair and the exact schedule of when to repair them might fit your use better.

zooko commented

2009-08-04 15:24:58 +00:00

This introductory article in arstechnica.com reminds me that I've had many conversations with people introducing them to Tahoe-LAFS in which I have to explain that the repair process is not automatic. Anytime I have to repeatedly explain something to newcomers, this makes me think it ought to be changed so that the newcomers's assumptions are true. In this case, that means making a scheduler which automatically kicks off a repair process and runs it in the background.

(The comment by thraxil after the arstechnica.com article incorrectly states that this is currently done automatically. In fact, the user has to learn a special command-line tool and run it themselves or configure it to run as a cron job or whatever.)

This introductory article in arstechnica.com reminds me that I've had many conversations with people introducing them to Tahoe-LAFS in which I have to explain that the repair process is not automatic. Anytime I have to repeatedly explain something to newcomers, this makes me think it ought to be changed so that the newcomers's assumptions are true. In this case, that means making a scheduler which automatically kicks off a repair process and runs it in the background. (The comment by thraxil after the arstechnica.com article incorrectly states that this is currently done automatically. In fact, the user has to learn a special command-line tool and run it themselves or configure it to run as a cron job or whatever.)

zooko commented

2009-08-04 15:25:05 +00:00

(http://allmydata.org/pipermail/tahoe-dev/2009-August/002509.html)

zooko commented

2009-10-27 04:35:23 +00:00

Kevin Reid posted a cron job to do this http://allmydata.org/pipermail/tahoe-dev/2009-October/003012.html and I linked to it from the wiki/RelatedProjects page.

Kevin Reid posted a cron job to do this <http://allmydata.org/pipermail/tahoe-dev/2009-October/003012.html> and I linked to it from the [wiki/RelatedProjects](wiki/RelatedProjects) page.

zooko commented

2009-10-27 04:36:30 +00:00

So, even though Kevin has demonstrated how to do it with a cron job, this ticket still remains open to make it an integrated and easy-to-configure feature of Tahoe-LAFS itself.

davidsarah commented

2009-10-27 19:43:59 +00:00

To schedule tasks from Python under both Unix and Windows:

http://pypi.python.org/pypi/python-crontab/0.7
http://tarekziade.wordpress.com/2007/11/01/scheduling-tasks-in-windows-with-pywin32/

To schedule tasks from Python under both Unix and Windows: <http://pypi.python.org/pypi/python-crontab/0.7> <http://tarekziade.wordpress.com/2007/11/01/scheduling-tasks-in-windows-with-pywin32/>

davidsarah commented

2009-10-27 19:45:06 +00:00

Better link: http://pypi.python.org/pypi/python-crontab/

Better link: <http://pypi.python.org/pypi/python-crontab/>

tahoe-lafs changed title from ~~Tahoe Repair process~~ to Automatically schedule repair process

2009-12-29 19:43:58 +00:00

davidsarah commented

2009-12-29 19:46:38 +00:00

An earlier ticket discussing a repair service (now partly overtaken by other changes) was #483, now marked as a duplicate of #543.

davidsarah commented

2010-12-12 23:45:04 +00:00

Replying to davidsarah:

To schedule tasks from Python under [...] Windows:

http://tarekziade.wordpress.com/2007/11/01/scheduling-tasks-in-windows-with-pywin32/

However, see #1274 (eliminate pywin32 dependency). The code on that page could be translated to use ctypes, although it's slightly more complicated than usual because it uses COM.

Replying to [davidsarah](/tahoe-lafs/trac-2024-07-25/issues/643#issuecomment-69991): > To schedule tasks from Python under [...] Windows: > > <http://tarekziade.wordpress.com/2007/11/01/scheduling-tasks-in-windows-with-pywin32/> However, see #1274 (eliminate pywin32 dependency). The code on that page could be translated to use ctypes, although it's slightly more complicated than usual because it uses COM.

davidsarah commented

2011-12-12 04:43:46 +00:00

amiller had some difficulty configuring tahoe backup to run from cron. If we were to add automatic scheduling of repair, we could make it schedule tahoe backup as well, with little extra work.

amiller had some difficulty configuring `tahoe backup` to run from cron. If we were to add automatic scheduling of repair, we could make it schedule `tahoe backup` as well, with little extra work.

tahoe-lafs modified the milestone from undecided to eventually

2011-12-12 04:43:46 +00:00

tahoe-lafs changed title from ~~Automatically schedule repair process~~ to Automatically schedule repair process (and backups?)

2011-12-12 04:43:46 +00:00

zhan0903 commented

2013-07-03 16:49:52 +00:00

Is there some implementation about automatical repair? I found it is not convenient to repair only one file or directory every time you run the repair tool

sometimes, you even donnot know which file should be repaired, so I think it will be good to make this tool run as a service and check files automatically based on some principles or algorithm

actually, I would like to do something about that

Is there some implementation about automatical repair? I found it is not convenient to repair only one file or directory every time you run the repair tool sometimes, you even donnot know which file should be repaired, so I think it will be good to make this tool run as a service and check files automatically based on some principles or algorithm actually, I would like to do something about that

Sign in to join this conversation.