Prevent uncoordinated writes locally. #265

New Issue

nejucomo · 2008-01-08T17:16:07Z

nejucomo commented

2008-01-08 17:16:07 +00:00

Tahoe does not attempt to solve uncoordinated writes, generally. However, it could make it hard for a single user, using a single node, to trigger uncoordinated writes, by detecting this behavior locally.

nejucomo added the

labels 2008-01-08 17:16:07 +00:00

nejucomo added this to the undecided milestone 2008-01-08 17:16:07 +00:00

zooko commented

2008-03-27 05:21:13 +00:00

We just had a conversation with MikeB and Peter about how the allmydata.com 3.0 native Windows integration has some uncoordinated writes due to links of child directories and unlinks of any kind being uncoordinated.

It would be good to serialize those to prevent errors, but it would require a bit of restructuring of MikeB's Windows integration code.

On the way out to the taxi, Nathan reminded me of this idea -- that the Tahoe node could serialize those for you. We could probably do it in Tahoe more easily than Mike could do it in the Window integration.

The first step, of course, would be to make unit tests which issue two successive updates through the WAPI to one mutable file, which makes sure that the first update doesn't complete before the second one is issued (by using a Fake object for the mutable file), and which makes sure that the second update doesn't get issued to the Fake mutable file object by the wapi layer until after the first update finishes.

We just had a conversation with MikeB and Peter about how the allmydata.com 3.0 native Windows integration has some uncoordinated writes due to links of child directories and unlinks of any kind being uncoordinated. It would be good to serialize those to prevent errors, but it would require a bit of restructuring of MikeB's Windows integration code. On the way out to the taxi, Nathan reminded me of this idea -- that the Tahoe node could serialize those for you. We could probably do it in Tahoe more easily than Mike could do it in the Window integration. The first step, of course, would be to make unit tests which issue two successive updates through the WAPI to one mutable file, which makes sure that the first update doesn't complete before the second one is issued (by using a Fake object for the mutable file), and which makes sure that the second update doesn't get issued to the Fake mutable file object by the wapi layer until after the first update finishes.

zooko self-assigned this 2008-03-27 05:21:13 +00:00

zooko commented

2008-03-27 16:31:49 +00:00

Oh wait, even better is to do this serialization in the [MutableFileNode object]source:src/allmydata/mutable.py@2339#L1607, and for the wapi layer to simply guarantee that whenever you make a call to a mutable file or directory, that if a MutableFileNode object or Dirnode object for that file or directory is already in memory that it uses it rather than creating a new object pointing to the same mutable file or directory.

Oh wait, even better is to do this serialization in the [MutableFileNode object]source:src/allmydata/mutable.py@2339#L1607, and for the wapi layer to simply guarantee that whenever you make a call to a mutable file or directory, that if a [MutableFileNode](wiki/MutableFileNode) object or Dirnode object for that file or directory is already in memory that it uses it rather than creating a new object pointing to the same mutable file or directory.

warner commented

2008-03-27 17:59:30 +00:00

Yeah, I've wondered if the Client should keep a weak-value-dict that maps read-cap to filenode, so it could make a promise that there will never be more than a single filenode for any given mutable file. Then the MutableFileNode.update method would need internal locking: only one update may be active at a time.

Yeah, I've wondered if the Client should keep a weak-value-dict that maps read-cap to filenode, so it could make a promise that there will never be more than a single filenode for any given mutable file. Then the `MutableFileNode.update` method would need internal locking: only one update may be active at a time.

warner commented

2008-04-24 23:25:39 +00:00

The MutableFileNode object now does internal serialization: if a single instance is asked to perform an operation while it's in the middle of another operation, the second will wait until the first has finished.

To make this useful for external webapi callers, we still need to implement the weak-value-dict scheme. Basically client.create_node_from_uri() should look up the URI in a weak-value-dict, and return the value if found. If not, it should create a new node, then add it to the weak-value-dict before returning it.

It will still be possible to get, say, a write-node and a read-node to the same file in the same process, but these won't collide with each other in bad ways (only annoying ones: a read could fail because the write-node modified the shares as the read was trying to fetch them).

The `MutableFileNode` object now does internal serialization: if a single instance is asked to perform an operation while it's in the middle of another operation, the second will wait until the first has finished. To make this useful for external webapi callers, we still need to implement the weak-value-dict scheme. Basically client.create_node_from_uri() should look up the URI in a weak-value-dict, and return the value if found. If not, it should create a new node, then add it to the weak-value-dict before returning it. It will still be possible to get, say, a write-node and a read-node to the same file in the same process, but these won't collide with each other in bad ways (only annoying ones: a read could fail because the write-node modified the shares as the read was trying to fetch them).

warner added

code-mutable

and removed

code-frontend

labels 2008-04-24 23:44:31 +00:00

warner commented

2008-04-24 23:46:07 +00:00