Abuse of HTTP status 410 Gone #1993

New Issue

kpreid · 2013-05-30T01:00:03Z

kpreid commented

2013-05-30 01:00:03 +00:00

NotEnoughSharesError, NoSharesError, and UnrecoverableFileError, at least, are being reported using HTTP status code 410 Gone, which is a severe misuse of the code, as 410 means that the resource is known to be forevermore unavailable. Per RFC 2616 section 10.4.11:

The requested resource is no longer available at the server and no forwarding address is known. This condition is expected to be considered permanent. Clients with link editing capabilities SHOULD delete references to the Request-URI after user approval. If the server does not know, or has no facility to determine, whether or not the condition is permanent, the status code 404 (Not Found) SHOULD be used instead.

All of these errors indicate that the gateway is currently unable to fulfill the request (as any of them could result from temporary partition in the grid), not permanent deletion. 410 would be appropriate if, for example, a mutable file were put into a revoked, “no content and cannot be written to further”, state, but not for anything less drastic. (Tahoe is unusual in having even the architectural possibility of having enough confidence to correctly answer 410!)

The most appropriate response codes would be, I think, 404 for NoSharesError (because the grid has no knowledge of the file) and 503 for NotEnoughSharesError (because the grid knows the file exists but cannot be served). UnrecoverableFileError appears to be a conflation of the two in the case of mutable files, and so I see no good answer there but to introduce a distinction between the two cases.

Regardless, 410 should not be used in any of these cases.

I noticed this via https://tahoe-lafs.org/pipermail/tahoe-dev/2013-May/008313.html .

NotEnoughSharesError, NoSharesError, and UnrecoverableFileError, at least, are being reported using HTTP status code 410 Gone, which is a severe misuse of the code, as 410 means that the resource is **known to be** forevermore unavailable. Per RFC 2616 section 10.4.11: The requested resource is no longer available at the server and no forwarding address is known. This condition is expected to be considered permanent. Clients with link editing capabilities SHOULD delete references to the Request-URI after user approval. If the server does not know, or has no facility to determine, whether or not the condition is permanent, the status code 404 (Not Found) SHOULD be used instead. All of these errors indicate that the gateway is *currently* unable to fulfill the request (as any of them could result from temporary partition in the grid), not permanent deletion. 410 would be appropriate if, for example, a mutable file were put into a revoked, “no content and cannot be written to further”, state, but not for anything less drastic. (Tahoe is unusual in having even the architectural possibility of having enough confidence to correctly answer 410!) The most appropriate response codes would be, I think, 404 for NoSharesError (because the grid has no knowledge of the file) and 503 for NotEnoughSharesError (because the grid knows the file exists but cannot be served). UnrecoverableFileError appears to be a conflation of the two in the case of mutable files, and so I see no good answer there but to introduce a distinction between the two cases. Regardless, 410 should not be used in any of these cases. I noticed this via <https://tahoe-lafs.org/pipermail/tahoe-dev/2013-May/008313.html> .

kpreid added the

labels 2013-05-30 01:00:03 +00:00

kpreid added this to the undecided milestone 2013-05-30 01:00:03 +00:00

warner commented

2013-05-30 18:21:45 +00:00

Sounds good to me. Note that NoSharesError could be interpreted as an even-less-healthy version of NotEnoughSharesError, where there are so few shares that we couldn't find even a single one. So there might be an argument for reporting 503 in both cases.

If 410 means "it will never exist", does 404 mean "it might come back someday"? Also, does 410 imply anything about whether or not it used to exist? Are there any normal-web-server situations that would correctly produce a 410?

Sounds good to me. Note that `NoSharesError` could be interpreted as an even-less-healthy version of `NotEnoughSharesError`, where there are so few shares that we couldn't find even a single one. So there might be an argument for reporting 503 in both cases. If 410 means "it will never exist", does 404 mean "it might come back someday"? Also, does 410 imply anything about whether or not it used to exist? Are there any normal-web-server situations that would correctly produce a 410?

kpreid commented

2013-05-30 19:08:04 +00:00

Note that NoSharesError could be interpreted as an even-less-healthy version of NotEnoughSharesError, where there are so few shares that we couldn't find even a single one. So there might be an argument for reporting 503 in both cases.

I agree that 503 is not-wrong, but it is commonly understood that 404 can result from servers being temporarily broken; I think it is more valuable that to have the property that any bogus URL yields a 404.

If your grid is so flaky that you can lose all shares of a file, that's another problem entirely. (Actually: what if the gateway is not connected to enough storage servers that enough (properly spread) shares could not possibly be found? That would be an appropriate time for a 503 if no shares are found, since it is likely that the answer will be different when the grid is in better condition.)

If 410 means "it will never exist", does 404 mean "it might come back someday"?

404 is most fitting for "I haven't heard of that URL", but also

  This status code is commonly used when the server does not wish to reveal exactly why the request has been refused, or when no other   response is applicable.

If the server knows that you can't have it right now, 403. If the server is unable to supply it but expects to in the future, 503.

Also, does 410 imply anything about whether or not it used to exist?

It would be peculiar for the resource to not have used to exist, but 410 at least indicates that the server knows about the hypothetical currently-non-existing resource.

But really, we should be thinking about the operational implications, from which perspective 410 means "The link you followed is known to be obsolete and so you should remove it."

Are there any normal-web-server situations that would correctly produce a 410?

If I deliberately remove something from my web site, it would be appropriate for me to arrange for a 410 response. (Of course, then "I decided to put it back" is problematic, which is a reason 410 doesn't see more use.)

> Note that NoSharesError could be interpreted as an even-less-healthy version of NotEnoughSharesError, where there are so few shares that we couldn't find even a single one. So there might be an argument for reporting 503 in both cases. I agree that 503 is not-wrong, but it is commonly understood that 404 can result from servers being temporarily broken; I think it is more valuable that to have the property that any bogus URL yields a 404. If your grid is so flaky that you can lose all shares of a file, that's another problem entirely. (Actually: what if the gateway is not *connected to* enough storage servers that enough (properly spread) shares could not possibly be found? That would be an appropriate time for a 503 if no shares are found, since it is likely that the answer will be different when the grid is in better condition.) > If 410 means "it will never exist", does 404 mean "it might come back someday"? 404 is most fitting for "I haven't heard of that URL", but also This status code is commonly used when the server does not wish to reveal exactly why the request has been refused, or when no other response is applicable. If the server knows that you can't have it right now, 403. If the server is unable to supply it but expects to in the future, 503. > Also, does 410 imply anything about whether or not it used to exist? It would be peculiar for the resource to not have used to exist, but 410 at least indicates that the server knows *about* the hypothetical currently-non-existing resource. But really, we should be thinking about the operational implications, from which perspective 410 means "The link you followed is known to be obsolete and so you should remove it." > Are there any normal-web-server situations that would correctly produce a 410? If I deliberately remove something from my web site, it would be appropriate for me to arrange for a 410 response. (Of course, then "I decided to put it back" is problematic, which is a reason 410 doesn't see more use.)

daira commented

2013-05-30 19:14:47 +00:00

Duplicate of #1764. I'll copy the discussion here to that ticket.

tahoe-lafs added the

duplicate

label 2013-05-30 19:14:47 +00:00

daira closed this issue

2013-05-30 19:14:47 +00:00

Sign in to join this conversation.