Extend external interfaces for operation monitoring. #83
Labels
No Label
0.2.0
0.3.0
0.4.0
0.5.0
0.5.1
0.6.0
0.6.1
0.7.0
0.8.0
0.9.0
1.0.0
1.1.0
1.10.0
1.10.1
1.10.2
1.10a2
1.11.0
1.12.0
1.12.1
1.13.0
1.14.0
1.15.0
1.15.1
1.2.0
1.3.0
1.4.1
1.5.0
1.6.0
1.6.1
1.7.0
1.7.1
1.7β
1.8.0
1.8.1
1.8.2
1.8.3
1.8β
1.9.0
1.9.0-s3branch
1.9.0a1
1.9.0a2
1.9.0b1
1.9.1
1.9.2
1.9.2a1
LeastAuthority.com automation
blocker
cannot reproduce
cloud-branch
code
code-dirnodes
code-encoding
code-frontend
code-frontend-cli
code-frontend-ftp-sftp
code-frontend-magic-folder
code-frontend-web
code-mutable
code-network
code-nodeadmin
code-peerselection
code-storage
contrib
critical
defect
dev-infrastructure
documentation
duplicate
enhancement
fixed
invalid
major
minor
n/a
normal
operational
packaging
somebody else's problem
supercritical
task
trivial
unknown
was already fixed
website
wontfix
worksforme
No Milestone
No Assignees
3 Participants
Notifications
Due Date
No due date set.
Reference: tahoe-lafs/trac-2024-07-25#83
Loading…
Reference in New Issue
Block a user
No description provided.
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
I'd like to see the external interfaces (rest, xml-rpc, foolscap) support operation monitoring, so that external clients can display operation progress and perhaps control the operations (cancel, for instance).
The rest api could include an operation id in response headers for new operations. Separate urls could be used to query progress or list current operations.
Currently you can see progress of e.g. an upload (
tahoe put
) by observing how much of the file you have been able to upload. The server (which ideally is running on your localhost anyway) will not accept the file faster than it can encrypt, encode, and distribute the shares.Likewise, the progress of download is apparent by how much of the leading cleartext segments of the files have been delivered to you. :-)
What do you think?
Oh, cancellation is implemented by closing the HTTP connection before you've finished up/down load.
I'm pretty pleased with this design so far...
Replying to zooko:
+1. The request in this ticket seems like unnecessary complexity. I suggest wontfix.
(Note that #92 is about showing upload progress/completion to the user in the WUI; that would still be useful.)
So I definitely would have preferred the simplicity of using in-band progress indicators and cancellation as described in comment:60816, but Brian persuaded me that this just wasn't good enough. The part of his argument that I remember being unable to counter was that we have some operations that take longer than an HTTP connection can reliably last. For example if you want to do a deep-verify-and-repair which is going to walk a large directory structure and download every bit of every share of every file and, if necessary, upload replacement shares. This could take days or weeks or months, and if your control of the process is a single HTTP connection then you're quite likely to suffer a network glitch which closes your TCP connection or encounter some kind of stupid timeout in an HTTP proxy or something.
(The way I like to think of this is that the comms abstraction of TCP is insufficiently robust -- there isn't a widely understood and implemented way to force your HTTP transaction to outlive temporary disconnections of the underlying TCP connection. That means that HTTP, while a wonderful lingua franca for some protocols, can't be used for long-running operations or operations which cannot be cannot be safely retried when the first try might or might not have failed to get through.)
So, Brian went ahead and invented "operation handles", documented here: source:docs/frontends/webapi.txt@4112#L203.
Hm, reading those docs again, I see this new text:
Oh dear, so it appears that neither the operation-handles nor the single HTTP connection is really good enough in all dimensions. Hm.
So what shall we do with this ticket? I guess we'll close it as "fixed", and then maybe open a new ticket saying "Make operation-handle-querying use only a little memory" and maybe open a new ticket saying "Invent robust HTTP so that streaming operations handles can be used on operations that last longer than a TCP connection lasts".
I'm not actually going to open either of those two tickets right now. I just took painkillers for my knee (recuperating from surgery).
If Brian, Nathan, or David-Sarah (or anyone) have any ideas on how to follow-up on this by all means post to the list or comment on this or some other ticket.
Replying to zooko:
This is #857.
Not a Tahoe bug :-)
Get well soon!