don't do lease-renewal automatically #1893

New Issue

zooko · 2012-12-11T18:45:05Z

zooko commented

2012-12-11 18:45:05 +00:00

Currently lease-renewal happens when you explicitly tell your client to create a file or to renew leases, and also on some other occasions. I think that the "other occasions" are only when you update a mutable file, but I'm not sure. I don't think this is appropriate because in the future you might want to update a file without thereby agreeing to pay for the file's preservation.

To close this ticket:

Read the source code and write down on this ticket all the ways that leases can get added or renewed.
Write a patch that removes the lease renewal from all cases except for when the client is either creating a file or sending an "Add or Renew Lease" request.
Of course, unit tests of step 2 are required!

Brian: please let us know your opinion of this.

Currently lease-renewal happens when you explicitly tell your client to create a file or to renew leases, and also on some other occasions. I *think* that the "other occasions" are only when you update a mutable file, but I'm not sure. I don't think this is appropriate because in the future you might want to update a file without thereby agreeing to pay for the file's preservation. To close this ticket: 1. Read the source code and write down on this ticket all the ways that leases can get added or renewed. 2. Write a patch that removes the lease renewal from all cases except for when the client is either creating a file or sending an "Add or Renew Lease" request. 3. Of course, unit tests of step 2 are required! Brian: please let us know your opinion of this.

zooko added the

labels 2012-12-11 18:45:05 +00:00

zooko added this to the undecided milestone 2012-12-11 18:45:05 +00:00

daira commented

2013-05-24 22:19:51 +00:00

Isn't the fact that you're modifying a file a pretty strong heuristic indication that you want to keep it?

zooko commented

2013-05-24 22:39:09 +00:00

Replying to daira:

Isn't the fact that you're modifying a file a pretty strong heuristic indication that you want to keep it?

I think we should separate those two. My original motivation for this ticket had to do with performance and engineering -- I didn't want to have to deal with lease-renewal in every place and at every time that I have to deal with mutation. But, once I started thinking about it, I saw no reason why updating a file should obligate you to pay for its survival. Maybe it is your file, and you asked me to write into it, and I am willing to give you the data you request, but I am not willing to pay for its maintenance.

Replying to [daira](/tahoe-lafs/trac-2024-07-25/issues/1893#issuecomment-90677): > Isn't the fact that you're modifying a file a pretty strong heuristic indication that you want to keep it? I think we should separate those two. My original motivation for this ticket had to do with performance and engineering -- I didn't want to have to deal with lease-renewal in every place and at every time that I have to deal with mutation. But, once I started thinking about it, I saw no reason why updating a file should obligate you to pay for its survival. Maybe it is your file, and you asked me to write into it, and I am willing to give you the data you request, but I am not willing to pay for its maintenance.

zooko commented

2013-05-24 22:41:33 +00:00

Replying to daira:

Isn't the fact that you're modifying a file a pretty strong heuristic indication that you want to keep it?

Also, I'm not really keen on using heuristics for this. I would rather have simpler rules that are easier for everyone (developers, users) to remember and make them more accurate in their predictions of which files will be retained. In fact, as per #1832, I would actually prefer a very simple rule: I agree to store this file for you until you tell me otherwise! (As long as I still have a relationship with you -- e.g. you are a customer in good standing, or you are still my friend, or whatever.)

Replying to [daira](/tahoe-lafs/trac-2024-07-25/issues/1893#issuecomment-90677): > Isn't the fact that you're modifying a file a pretty strong heuristic indication that you want to keep it? Also, I'm not really keen on using heuristics for this. I would rather have simpler rules that are easier for everyone (developers, users) to remember and make them more accurate in their predictions of which files will be retained. In fact, as per #1832, I would actually prefer a *very* simple rule: I agree to store this file for you until you tell me otherwise! (As long as I still have a relationship with you -- e.g. you are a customer in good standing, or you are still my friend, or whatever.)

exarkun commented

2019-08-09 12:42:38 +00:00

Separating these actions also makes accounting easier. "Accept a change to a mutable share" and "store some data for a period of time" are potentially separate accounting operations. Combining these in the implementation makes it impossible to separate them from each other, either in practice or for accounting purposes.

I think a first step (apart from those outlined by zooko) in this direction will be to refactor the implementation so that there is clear separation between data service operation (eg writing to a mutable share) and the lease operation. In this first step, there will be no change in behavior. The entrypoint used by clients will behave the same, it will just be implemented differently.

The refactoring will pave the way for a protocol change (later) where these operations can be invoked separately by clients. Clients which wish to participate in a system that uses some form of accounting can then invoke them separately and servers which wish to put such accounting into place can reject the combined operation and require clients to perform the two parts separately (if, indeed, both parts are desired by the client at any particular point in time).

Separating these actions also makes accounting easier. "Accept a change to a mutable share" and "store some data for a period of time" are potentially separate accounting operations. Combining these in the implementation makes it impossible to separate them from each other, either in practice or for accounting purposes. I think a first step (apart from those outlined by zooko) in this direction will be to refactor the implementation so that there is clear separation between data service operation (eg writing to a mutable share) and the lease operation. In this first step, there will be no change in behavior. The entrypoint used by clients will behave the same, it will just be implemented differently. The refactoring will pave the way for a protocol change (later) where these operations *can* be invoked separately by clients. Clients which wish to participate in a system that uses some form of accounting can then invoke them separately and servers which wish to put such accounting into place can reject the combined operation and require clients to perform the two parts separately (if, indeed, both parts are desired by the client at any particular point in time).

warner commented

2019-08-09 16:12:03 +00:00

Sounds reasonable to me. The only non-orthogonality I see is that the initial upload of a mutable file really also ought to arrange for its preservation. If "create mutable share" and "establish lease on mutable share" are completely independent API calls, then a correctly function server would be allowed to delete the share inbetween the two calls, which won't serve anybody well.

exarkun commented

2019-08-09 18:12:54 +00:00

The only non-orthogonality I see is that the initial upload of a mutable file really also ought to arrange for its preservation.

This makes sense. It's awkward to express with the current protocol since there is no distinction on the wire between initial upload and subsequent writes. I don't know what the best resolution to this is. The protocol could change to have separate "create" and "write" steps, as is the case for immutable shares. This introduces extra round trips. I wonder if eliminating those round trips was the motivation for making the mutable share protocol what it is now.

A different option might be to add arguments to the mutable share write API. Then create and initial write can still happen in one round trip but if this is the desired behavior the client will have to express it. On the other hand, it sounds like this creates a requirement for more distributed consensus (client and servers agreeing whether a share is already stored or not) in the system and that probably brings with it a load of problems.

> The only non-orthogonality I see is that the initial upload of a mutable file really also ought to arrange for its preservation. This makes sense. It's awkward to express with the current protocol since there is no distinction on the wire between initial upload and subsequent writes. I don't know what the best resolution to this is. The protocol could change to have separate "create" and "write" steps, as is the case for immutable shares. This introduces extra round trips. I wonder if eliminating those round trips was the motivation for making the mutable share protocol what it is now. A different option might be to add arguments to the mutable share write API. Then create and initial write can still happen in one round trip but if this is the desired behavior the client will have to express it. On the other hand, it sounds like this creates a requirement for more distributed consensus (client and servers agreeing whether a share is already stored or not) in the system and that probably brings with it a load of problems.

exarkun commented

2019-08-15 12:16:34 +00:00

Refactoring is done over at https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3241

Refactoring is done over at <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3241>

Sign in to join this conversation.