KeyError in mutable download #667

New Issue

warner · 2009-03-24T23:55:00Z

warner commented

2009-03-24 23:55:00 +00:00

Zooko observed a KeyError exception raised during mutable-file retrieval, in #651 (specifically at source:git/src/allmydata/mutable/retrieve.py?rev=4061258c#L157). This is a bug in the retrieval code, probably triggered by running out of valid shares. Either the self.remaining_sharemap mapping should not be modified while download() is iterating over it, or more likely download() must be prepared for remaining_sharemap to change during the iteration.

Zooko observed a KeyError exception raised during mutable-file retrieval, in #651 (specifically at source:git/src/allmydata/mutable/retrieve.py?rev=4061258c#L157). This is a bug in the retrieval code, probably triggered by running out of valid shares. Either the `self.remaining_sharemap` mapping should not be modified while `download()` is iterating over it, or more likely `download()` must be prepared for `remaining_sharemap` to change during the iteration.

warner added the

labels 2009-03-24 23:55:00 +00:00

warner added this to the 1.6.0 milestone 2009-03-24 23:55:00 +00:00

tahoe-lafs changed title from ~~KeyError in mutable publish~~ to KeyError in mutable download

2009-12-30 01:46:00 +00:00

zooko modified the milestone from 1.6.0 to eventually

2010-01-26 16:56:11 +00:00

zooko commented

2010-05-27 22:10:26 +00:00

It's really bothering me that mutable file upload and download behavior is so finicky, buggy, inefficient, hard to understand, different from immutable file upload and download behavior, etc. So I'm putting a bunch of tickets into the "1.8" Milestone. I am not, however, at this time, volunteering to work on these tickets, so it might be a mistake to put them into the 1.8 Milestone, but I really hope that someone else will volunteer or that I will decide to do it myself. :-)

zooko modified the milestone from eventually to 1.8.0

2010-05-27 22:10:26 +00:00

tahoe-lafs modified the milestone from 1.8.0 to 1.9.0

2010-08-10 04:14:17 +00:00

zooko commented

2010-08-10 04:40:29 +00:00

If you like this ticket, you're probably the kind of weirdo who would like #474 (uncaught exception in mutable-retrieve: UCW between mapupdate and retrieve), #548 (mutable publish sends queries to servers that have already been asked), #540 (inappropriate "uncoordinated write error" after handling a server failure), #546 (mutable-file surprise shares raise inappropriate UCWE), #547 (mapupdate(MODE_WRITE) triggers on a false boundary), #548 (mutable publish sends queries to servers that have already been asked), #549 (MODE_WRITE mapupdate: maybe increase epsilon to handle large batches of new servers better). And we need more weirdos like you!

davidsarah commented

2011-07-23 20:31:03 +00:00

This seems unlikely to be fixed for 1.9.

tahoe-lafs modified the milestone from 1.9.0 to soon

2011-07-23 20:31:03 +00:00

davidsarah commented

2012-12-06 22:15:31 +00:00

The description of this ticket probably refers to the code as it was before the start of Kevan's MDMF patches, i.e. up to and including [darcs revision 4772]source:trunk/src/allmydata/mutable/retrieve.py@4772#L157 / [git revision 4061258c]source:git/src/allmydata/mutable/retrieve.py?rev=4061258c#L157. The code changes a lot for MDMF and this bug might have been incidentally fixed -- in particular, there is no longer any indexing into self.remaining_sharemap, and all of the loops over its keys are done synchronously with the keys having been copied beforehand.

If that's correct, this ticket can be closed as "cannot reproduce".

(There was a vaguely similar bug in source:git/src/allmydata/mutable/publish.py that is easy to confuse with this one, but that was fixed in #1749.)

The description of this ticket probably refers to the code as it was before the start of Kevan's MDMF patches, i.e. up to and including [darcs revision 4772]source:trunk/src/allmydata/mutable/retrieve.py@4772#L157 / [git revision 4061258c]source:git/src/allmydata/mutable/retrieve.py?rev=4061258c#L157. The code changes a lot for MDMF and this bug might have been incidentally fixed -- in particular, there is no longer any indexing into `self.remaining_sharemap`, and all of the loops over its keys are done synchronously with the keys having been copied beforehand. If that's correct, this ticket can be closed as "cannot reproduce". (There was a vaguely similar bug in source:git/src/allmydata/mutable/publish.py that is easy to confuse with this one, but that was fixed in #1749.)

davidsarah commented

2012-12-06 22:19:23 +00:00

Editing description to refer to the right revision.

tahoe-lafs added the

cannot reproduce

label 2013-05-03 01:01:44 +00:00

daira closed this issue

2013-05-03 01:01:44 +00:00

Sign in to join this conversation.