Alter mutable files to use servers of happiness #1057

New Issue

tahoe-lafs · 2010-05-25T17:21:27Z

kevan commented

2010-05-25 17:21:27 +00:00

Servers of happiness is more robust (or can be made more robust) than the upload health metric currently used by mutable files. Tahoe-LAFS would also be more consistent and easier to understand if it used only one upload health metric. We should look into altering mutable files to use servers of happiness as their upload health metric.

tahoe-lafs added the

code-mutable

major

enhancement

labels 2010-05-25 17:21:27 +00:00

tahoe-lafs added this to the 1.8.0 milestone 2010-05-25 17:21:27 +00:00

zooko commented

2010-05-27 22:14:05 +00:00

If you like this ticket (and I hope you do, because we need somebody to step up and totally revamp mutable files to be faster, more robust, easier to understand, less buggy, etc.!) then you might like these tickets: #540 (inappropriate "uncoordinated write error" after handling a server failure), #546 (mutable-file surprise shares raise inappropriate UCWE), #893 (UCWE when mapupdate gives up too early, then server errors require replacement servers), #232 (Peer selection doesn't rebalance shares on overwrite of mutable file.), #270 (test for interrupted writes of mutable files), #394 (mutable publish: add timing charts to measure RTT), #474 (uncaught exception in mutable-retrieve: UCW between mapupdate and retrieve), #547 (mapupdate(MODE_WRITE) triggers on a false boundary), #548 (mutable publish sends queries to servers that have already been asked), #549 (MODE_WRITE mapupdate: maybe increase epsilon to handle large batches of new servers better), #667 (KeyError in mutable download), #846 (allmydata.test.test_system.SystemTest.test_mutable sometimes hangs on a slow machine), #1004 (how to fix 'multiple versions are recoverable'?), #1042 (KeyError exception seen in log when doing a mutable publish on the pubgrid).

If you like this ticket (and I hope you do, because we need *somebody* to step up and totally revamp mutable files to be faster, more robust, easier to understand, less buggy, etc.!) then you might like these tickets: #540 (inappropriate "uncoordinated write error" after handling a server failure), #546 (mutable-file surprise shares raise inappropriate UCWE), #893 (UCWE when mapupdate gives up too early, then server errors require replacement servers), #232 (Peer selection doesn't rebalance shares on overwrite of mutable file.), #270 (test for interrupted writes of mutable files), #394 (mutable publish: add timing charts to measure RTT), #474 (uncaught exception in mutable-retrieve: UCW between mapupdate and retrieve), #547 (mapupdate(MODE_WRITE) triggers on a false boundary), #548 (mutable publish sends queries to servers that have already been asked), #549 (MODE_WRITE mapupdate: maybe increase epsilon to handle large batches of new servers better), #667 (KeyError in mutable download), #846 (allmydata.test.test_system.SystemTest.test_mutable sometimes hangs on a slow machine), #1004 (how to fix 'multiple versions are recoverable'?), #1042 (KeyError exception seen in log when doing a mutable publish on the pubgrid).

zooko commented

2010-08-14 06:44:37 +00:00

Putting this in Milestone 1.9.0. Kevan: do you think you can work on this for 1.9.0?

zooko modified the milestone from 1.8.0 to 1.9.0

2010-08-14 06:44:37 +00:00

kevan commented

2010-08-14 17:19:00 +00:00

Sure. I should be able to have it done by then (assuming that its due date is still October 31st).

davidsarah commented

2011-07-12 22:52:57 +00:00

Kevan: please set the Milestone to 'soon' if you don't have a patch for 1.9.0.

tahoe-lafs modified the milestone from 1.9.0 to soon

2011-07-13 01:39:13 +00:00

markberger modified the milestone from soon to 1.11.0

2013-07-22 18:54:06 +00:00

markberger commented

2013-07-22 19:56:12 +00:00

This is the link to my branch: https://github.com/markberger/tahoe-lafs/tree/1057-mutable-servers-of-happiness

This is the link to my branch: <https://github.com/markberger/tahoe-lafs/tree/1057-mutable-servers-of-happiness>

markberger commented

2013-07-24 15:55:19 +00:00

Mutable files now must pass the servers-of-happiness test before they are uploaded. Basic tests have been added to test this new behavior.

markberger commented

2013-08-01 14:24:05 +00:00

Here is the pull request: https://github.com/tahoe-lafs/tahoe-lafs/pull/55

Here is the pull request: <https://github.com/tahoe-lafs/tahoe-lafs/pull/55>

markberger modified the milestone from soon to 1.11.0

2013-08-28 15:33:36 +00:00

zooko commented

2013-11-13 22:10:02 +00:00

Daira says that this branch has the same problem that Servers-of-Happiness had for static (immutable) files — #1382. That problem is that the upload strategy is not designed to achieve optimal happiness (it isn't the Upload Strategy Of Happiness), but then after the upload strategy has done its work, then the Servers-of-Happiness criterion is applied to it, which may cause that upload to be marked a failure.

Having looked at the branch, it looks plausible to me that it does have that problem.

Daira and I think this ticket should be moved out of Milestone 1.11 and into Milestone 1.12.

Daira says that this branch has the same problem that Servers-of-Happiness had for static (immutable) files — #1382. That problem is that the upload strategy is not designed to achieve optimal happiness (it isn't the Upload Strategy Of Happiness), but then after the upload strategy has done its work, then the Servers-of-Happiness criterion is applied to it, which may cause that upload to be marked a failure. Having looked at [the branch](https://github.com/markberger/tahoe-lafs/tree/1057-mutable-servers-of-happiness), it looks plausible to me that it does have that problem. Daira and I think this ticket should be moved out of Milestone 1.11 and into Milestone 1.12.

zooko modified the milestone from 1.11.0 to 1.12.0

2013-11-13 22:10:02 +00:00

zooko commented

2013-11-14 16:47:13 +00:00

To prevent that issue we would need to fix #2060.