HTTP storage protocol should not support requests beyond the end of the share #3907

New Issue

itamarst · 2022-07-05T20:12:40Z

itamarst commented

2022-07-05 20:12:40 +00:00

Currently requests beyond the end of the share result in truncated result. This is a weird thing to do from an HTTP perspective, and I suspect is only done because Foolscap maps directly to function calls; more structured information in a HTTP response ought to make it unnecessary. So:

The HTTP protocol will not support this; reading beyond the end will result in HTTP error.
The HTTP protocol will grow HEAD endpoints for mutable and immutables shares to get their length.
Given the length of a share, the IStorageServer emulation layer can emulate this behavior by truncating requests to the actual length of the share. If a mutable is written half-way this in theory can result in error, but oh well.

Currently requests beyond the end of the share result in truncated result. This is a weird thing to do from an HTTP perspective, and I suspect is only done because Foolscap maps directly to function calls; more structured information in a HTTP response ought to make it unnecessary. So: 1. The HTTP protocol will not support this; reading beyond the end will result in HTTP error. 2. The HTTP protocol will grow HEAD endpoints for mutable and immutables shares to get their length. 3. Given the length of a share, the `IStorageServer` emulation layer can emulate this behavior by truncating requests to the actual length of the share. If a mutable is written half-way this in theory can result in error, but oh well.

itamarst added the

labels 2022-07-05 20:12:40 +00:00

itamarst added this to the HTTP Storage Protocol milestone 2022-07-05 20:12:40 +00:00

itamarst commented

2022-07-18 16:42:42 +00:00

After further thought, returning truncated responses is probably fine. However, we still need to ensure the response headers are correct, which is not the case now.

After further thought, returning truncated responses is probably fine. However, we still need to ensure the response headers are correct, which is _not_ the case now.

itamarst commented

2022-07-18 17:09:19 +00:00

Address all TODOs mentioning 3907, using new proposed design where truncated responses are acceptable so long as they are internally consistent.
The client should validate the length of response matches what is claimed in the Content-Range header. If it doesn't, this is either a bug, or in the case of mutables suggests a write happened while reading. Either way the data should be thrown away.

1. Address all TODOs mentioning 3907, using new proposed design where truncated responses are acceptable so long as they are internally consistent. 2. The client should validate the length of response matches what is claimed in the Content-Range header. If it doesn't, this is either a bug, or in the case of mutables suggests a write happened while reading. Either way the data should be thrown away.

GitHub <noreply@github.com> commented

2022-07-22 16:37:21 +00:00

In 26f15f4/trunk:

Merge pull request #1204 from tahoe-lafs/3907-http-response-lengths

Improved HTTP response length handling

Fixes ticket:3907

In [26f15f4/trunk](/tahoe-lafs/trac-2024-07-25/commit/26f15f461c035a1e0b9223c16692b9c066971c44): ``` Merge pull request #1204 from tahoe-lafs/3907-http-response-lengths Improved HTTP response length handling Fixes ticket:3907 ```

tahoe-lafs added the

fixed

label 2022-07-22 16:37:21 +00:00

GitHub <noreply@github.com> closed this issue

2022-07-22 16:37:21 +00:00

Sign in to join this conversation.