let the get_buckets() response include the first block #1109

New Issue

zooko · 2010-07-06T21:27:02Z

zooko commented

2010-07-06 21:27:02 +00:00

We could optimize out a round trip in download if, when asking whether a storage server has blocks, by sending it a get_buckets(), you could optionally specify that if it does have one of those then could it please send you one of them. Then the reply would be a set of buckets (remote references to an object on the server) plus the full data of one block.

Brian: what do you think?

We could optimize out a round trip in download if, when asking whether a storage server has blocks, by sending it a `get_buckets()`, you could optionally specify that if it *does* have one of those then could it please send you one of them. Then the reply would be a set of buckets (remote references to an object on the server) plus the full data of one block. Brian: what do you think?

zooko added the

labels 2010-07-06 21:27:02 +00:00

zooko added this to the 1.8.0 milestone 2010-07-06 21:27:02 +00:00

warner was assigned by zooko

2010-07-06 21:27:02 +00:00

warner commented

2010-07-18 01:30:12 +00:00

Yeah, in general, I think a more stateless immutable share-read interface would be better. The mutable share interface (which was written about 6 months later) is stateless, and that makes life a bit easier. That interface takes a read vector and a set of share numbers, with an empty set meaning "all shares that you are holding", and I think the same interface would work for immutable files.

Such an interface assumes that the server can efficiently open+seek+read the same file several times in quick succession (closing the filehandle between each call), whereas the current stateful interface keeps the filehandle open for the entire download. I suspect that most modern OS/filesystems cache recently-opened files and make this fairly quick, but it's worth doing some benchmarks to be certain. Also, we'd want to consider the interaction with GC and/or external tools which delete shares: keeping the filehandle open means a download will survive the share being deleted in the middle, whereas a stateless interface would not.

A stateless interface would also make us slightly more resistant to a DoS attack in which the attacker opens lots of shares at once and tries to fill the file-descriptor table.

I'd want to leave the signature of get_buckets() alone, and add a new method instead. The server-version-information dictionary could be used to advertise the availability of such a method.

And overall, yeah, I'd like to optimize out that extra round trip, because my new downloader (#798) can currently retrieve a small file in just two roundtrips, and with this fix we could get that down to just a single roundtrip, which would be great.

I was probably a bit over-enthusiastic about using Foolscap remote references when I wrote the immutable interface... incidentally, one conceivable benefit of the stateful interface could come up in server-driven share migration. That code (living in server A) could talk to server B and send it a "please copy my share" message, passing it the remote-reference to the share's BucketReader, or a client could use Foolscap's third-party-reference ("Gifts") feature to let A and B move the data directly between themselves without requiring client-side bandwidth for the copy. Of course, since all shares are publically readable, there's no authority-reducing benefit to doing it with bucket references over simply telling someone the storage-index and having them do the reads themselves. But when Accounting shows up, it might wind up to be handy to have a bucket-plus-read-authority object available to pass around.

Yeah, in general, I think a more stateless immutable share-read interface would be better. The mutable share interface (which was written about 6 months later) is stateless, and that makes life a bit easier. That interface takes a read vector and a set of share numbers, with an empty set meaning "all shares that you are holding", and I think the same interface would work for immutable files. Such an interface assumes that the server can efficiently open+seek+read the same file several times in quick succession (closing the filehandle between each call), whereas the current stateful interface keeps the filehandle open for the entire download. I suspect that most modern OS/filesystems cache recently-opened files and make this fairly quick, but it's worth doing some benchmarks to be certain. Also, we'd want to consider the interaction with GC and/or external tools which delete shares: keeping the filehandle open means a download will survive the share being deleted in the middle, whereas a stateless interface would not. A stateless interface would also make us slightly more resistant to a DoS attack in which the attacker opens lots of shares at once and tries to fill the file-descriptor table. I'd want to leave the signature of get_buckets() alone, and add a new method instead. The server-version-information dictionary could be used to advertise the availability of such a method. And overall, yeah, I'd like to optimize out that extra round trip, because my new downloader (#798) can currently retrieve a small file in just two roundtrips, and with this fix we could get that down to just a single roundtrip, which would be great. I was probably a bit over-enthusiastic about using Foolscap remote references when I wrote the immutable interface... incidentally, one conceivable benefit of the stateful interface could come up in server-driven share migration. That code (living in server A) could talk to server B and send it a "please copy my share" message, passing it the remote-reference to the share's `BucketReader`, or a client could use Foolscap's third-party-reference ("Gifts") feature to let A and B move the data directly between themselves without requiring client-side bandwidth for the copy. Of course, since all shares are publically readable, there's no authority-reducing benefit to doing it with bucket references over simply telling someone the storage-index and having them do the reads themselves. But when Accounting shows up, it might wind up to be handy to have a bucket-plus-read-authority object available to pass around.

zooko commented

2010-07-18 02:07:48 +00:00

So perhaps for v1.8.0 we would add a new method called something like get_buckets_data() which implements the stateless interface, like [slot_readv()]source:src/allmydata/interfaces.py@4410#L165. Then new downloaders could use that, perhaps invoking that on the first K servers that they know of which support it, and invoking get_buckets() on the remaining N-K servers which they will need to use only if some of the first K servers fail.

So perhaps for v1.8.0 we would add a new method called something like `get_buckets_data()` which implements the stateless interface, like [slot_readv()]source:src/allmydata/interfaces.py@4410#L165. Then new downloaders could use that, perhaps invoking that on the first `K` servers that they know of which support it, and invoking `get_buckets()` on the remaining `N-K` servers which they will need to use only if some of the first `K` servers fail.

zooko modified the milestone from 1.8.0 to soon

2010-08-14 06:43:47 +00:00

Sign in to join this conversation.