modifying SDMF file fails when tahoe.cfg's k/N differs from file #1510

New Issue

warner · 2011-08-27T20:44:07Z

warner commented

2011-08-27 20:44:07 +00:00

I found a regression in the SDMF publish code when I happened to try to update an old directory that I'd created with k=3/N=20. My current tahoe.cfg uses k=3/N=10. The exception is a NotEnoughServersError that wraps an AssertionError in mutable.layout.SDMFSlotWriteProxy.finish_publishing, where it notices that it doesn't have all the share pieces it needs (it's missing sharedata, blockhashes, and sharehashes, but it has verification_key, encprivkey, and signature).

The publish process has several dictionaries that are keyed by share number. It looks like some of them are being filled with shnums that depend upon what the current tahoe.cfg has for shares.total, whereas others are filled according to which existing shares were actually found. If tahoe.cfg has changed in the meantime, finish_publishing() will be called on shnums that were never handled in the earlier methods.

I've attached a unit test which exercises this case in a pretty narrow way.

This is a blocker for 1.9

I found a regression in the SDMF publish code when I happened to try to update an old directory that I'd created with k=3/N=20. My current tahoe.cfg uses k=3/N=10. The exception is a NotEnoughServersError that wraps an AssertionError in `mutable.layout.SDMFSlotWriteProxy.finish_publishing`, where it notices that it doesn't have all the share pieces it needs (it's missing sharedata, blockhashes, and sharehashes, but it has verification_key, encprivkey, and signature). The publish process has several dictionaries that are keyed by share number. It looks like some of them are being filled with shnums that depend upon what the current tahoe.cfg has for `shares.total`, whereas others are filled according to which existing shares were actually found. If tahoe.cfg has changed in the meantime, `finish_publishing()` will be called on shnums that were never handled in the earlier methods. I've attached a unit test which exercises this case in a pretty narrow way. This is a blocker for 1.9

warner added the

labels 2011-08-27 20:44:07 +00:00

warner added this to the 1.9.0 milestone 2011-08-27 20:44:07 +00:00

warner commented

2011-08-27 20:44:38 +00:00

Attachment 1510-test.diff (1355 bytes) added

unit test to exercise the failure

**Attachment** 1510-test.diff (1355 bytes) added unit test to exercise the failure

1510-test.diff

1.3 KiB

warner commented

2011-08-27 21:15:03 +00:00

Attachment update-parms.diff (780 bytes) added

here's the fix: update k/N after retrieval, so we use the same values on subsequent publish

**Attachment** update-parms.diff (780 bytes) added here's the fix: update k/N after retrieval, so we use the same values on subsequent publish

update-parms.diff

780 B

Brian Warner <warner@lothar.com> commented

2011-08-27 22:51:05 +00:00

In changeset:370e6f271e40945b:

SDMF: update filenode with correct k/N after Retrieve. Fixes #1510.

Without this, we get a regression when modifying a mutable file that was
created with more shares (larger N) than our current tahoe.cfg . The
modification attempt creates new versions of the (0,1,..,newN-1) shares, but
leaves the old versions of the (newN,..,oldN-1) shares alone (and throws a
assertion error in SDMFSlotWriteProxy.finish_publishing in the process).

The mixed versions that result (some shares with e.g. N=10, some with N=20,
such that both versions are recoverable) cause problems for the Publish code,
even before MDMF landed. Might be related to refs #1390 and refs #1042.

In changeset:370e6f271e40945b: ``` SDMF: update filenode with correct k/N after Retrieve. Fixes #1510. Without this, we get a regression when modifying a mutable file that was created with more shares (larger N) than our current tahoe.cfg . The modification attempt creates new versions of the (0,1,..,newN-1) shares, but leaves the old versions of the (newN,..,oldN-1) shares alone (and throws a assertion error in SDMFSlotWriteProxy.finish_publishing in the process). The mixed versions that result (some shares with e.g. N=10, some with N=20, such that both versions are recoverable) cause problems for the Publish code, even before MDMF landed. Might be related to refs #1390 and refs #1042. ```

tahoe-lafs added the

fixed

label 2011-08-27 22:51:05 +00:00

tahoe-lafs closed this issue

2011-08-27 22:51:05 +00:00

Sign in to join this conversation.