increase maximum mutable share size #1778

New Issue

tahoe-lafs · 2012-06-22T15:43:34Z

davidsarah commented

2012-06-22 15:43:34 +00:00

No description provided.

tahoe-lafs added the

labels 2012-06-22 15:43:34 +00:00

tahoe-lafs added this to the 1.9.2 milestone 2012-06-22 15:43:34 +00:00

david-sarah@jacaranda.org commented

2012-06-22 15:46:51 +00:00

In changeset:33e2d2962e2bc6cc:

Change the maximum mutable share size to 69105 TB, and add a maximum-mutable-share-size field to the version announcement. Includes a test. refs #1778

In changeset:33e2d2962e2bc6cc: ``` Change the maximum mutable share size to 69105 TB, and add a maximum-mutable-share-size field to the version announcement. Includes a test. refs #1778 ```

david-sarah@jacaranda.org commented

2012-06-22 15:47:28 +00:00

In changeset:5526/1.9.2:

Change the maximum mutable share size to 69105 TB, and add a maximum-mutable-share-size field to the version announcement. Includes a test. refs #1778

In changeset:5526/1.9.2: ``` Change the maximum mutable share size to 69105 TB, and add a maximum-mutable-share-size field to the version announcement. Includes a test. refs #1778 ```

david-sarah <david-sarah@jacaranda.org> commented

2012-06-22 16:46:45 +00:00

In changeset:33e2d2962e2bc6cc:

Change the maximum mutable share size to 69105 TB, and add a maximum-mutable-share-size field to the version announcement. Includes a test. refs #1778

In changeset:33e2d2962e2bc6cc: ``` Change the maximum mutable share size to 69105 TB, and add a maximum-mutable-share-size field to the version announcement. Includes a test. refs #1778 ```

zooko was assigned by tahoe-lafs

2012-06-23 23:29:03 +00:00

zooko commented

2012-07-03 02:47:37 +00:00

Hm...

        version = { "http://allmydata.org/tahoe/protocols/storage/v1" :
                    { "maximum-immutable-share-size": remaining_space,
                      "maximum-mutable-share-size": MAX_MUTABLE_SHARE_SIZE,
                      "tolerates-immutable-read-overrun": True,
                      "delete-mutable-shares-with-zero-length-writev": True,
                      "fills-holes-with-zero-bytes": True,
                      "prevents-read-past-end-of-share-data": True,
                      },
                    "application-version": str(allmydata.__full_version__),
                    }

Wouldn't it be more consistent for maximum-mutable-share-size to be equal to remaining_space? Also more meaningful to potential future clients that are curious about approximately big of a mutable share they could upload. With this version (above), they would have to look at maximum-immutable-share-size in order to answer that question, and they would ignore maximum-mutable-share-size.

Hm... ``` version = { "http://allmydata.org/tahoe/protocols/storage/v1" : { "maximum-immutable-share-size": remaining_space, "maximum-mutable-share-size": MAX_MUTABLE_SHARE_SIZE, "tolerates-immutable-read-overrun": True, "delete-mutable-shares-with-zero-length-writev": True, "fills-holes-with-zero-bytes": True, "prevents-read-past-end-of-share-data": True, }, "application-version": str(allmydata.__full_version__), } ``` Wouldn't it be more consistent for `maximum-mutable-share-size` to be equal to `remaining_space`? Also more meaningful to potential future clients that are curious about approximately big of a mutable share they could upload. With this version (above), they would have to look at `maximum-immutable-share-size` in order to answer that question, and they would ignore `maximum-mutable-share-size`.

zooko commented

2012-07-03 02:48:46 +00:00

But otherwise I like the patch -- it makes minimal changes to the code (which is appropriate for the imminent 1.9.2 release) and it includes a unit test.

davidsarah commented

2012-07-03 04:35:52 +00:00

The current storage code doesn't enforce remaining_space for mutables (#390). The filesystem will fill up at some point, but that would impose a limit of

min(MAX_MUTABLE_SHARE_SIZE, total_remaining_disk_space)

rather than remaining_space. We can always compute the limit more precisely if we fix #390; I don't think there's a significant forward-compatibility problem there.

The current storage code doesn't enforce `remaining_space` for mutables (#390). The filesystem will fill up at some point, but that would impose a limit of ``` min(MAX_MUTABLE_SHARE_SIZE, total_remaining_disk_space) ``` rather than `remaining_space`. We can always compute the limit more precisely if we fix #390; I don't think there's a significant forward-compatibility problem there.

zooko commented

2012-07-03 16:26:48 +00:00

Okay, good enough for 1.9.2 IMO.

tahoe-lafs added the

fixed

label 2012-07-03 16:27:49 +00:00

tahoe-lafs closed this issue

2012-07-03 16:27:49 +00:00

warner commented

2012-07-03 16:34:07 +00:00

I'm cool with it. At this point, I somewhat prefer fixed-max-size advertisements over min(MAXSIZE, remaining_disk_space), mostly because the latter will prevent uploads if the disk is full when the node is started (and the announcement produced), but then later frees up some space. Really, it's a tradeoff between delivering information ahead of time (in the announcement) versus discovering available space at the time of upload.

I'm a little bit weirded out by the funny magic 69105TB limit (subtle in-jokes like that are cute, but at least one person is going to be flummoxed in the future trying to figure out why the limit is that particular number and not e.g. 69106TB). But I don't have any 70 petabyte mutable files that need uploading, nor do I anticipate anyone else having that need this decade, so I don't really care very much.

I'm cool with it. At this point, I somewhat prefer fixed-max-size advertisements over `min(MAXSIZE, remaining_disk_space)`, mostly because the latter will prevent uploads if the disk is full when the node is started (and the announcement produced), but then later frees up some space. Really, it's a tradeoff between delivering information ahead of time (in the announcement) versus discovering available space at the time of upload. I'm a little bit weirded out by the funny magic 69105TB limit (subtle in-jokes like that are cute, but at least one person is going to be flummoxed in the future trying to figure out why the limit is that particular number and not e.g. 69106TB). But I don't have any 70 petabyte mutable files that need uploading, nor do I anticipate anyone else having that need this decade, so I don't really care very much.

david-sarah@jacaranda.org commented

2012-07-16 16:33:55 +00:00

In changeset:5885/cloud-backend:

[rebased for cloud-backend] Change the maximum mutable share size to 69105 TB, and add a maximum-mutable-share-size field to the version announcement. Includes a test. refs #1778

In changeset:5885/cloud-backend: ``` [rebased for cloud-backend] Change the maximum mutable share size to 69105 TB, and add a maximum-mutable-share-size field to the version announcement. Includes a test. refs #1778 ```

Sign in to join this conversation.