I want an indicator of progress on a backup #1587

New Issue

zooko · 2011-11-19T01:20:50Z

zooko commented

2011-11-19 01:20:50 +00:00

When running "tahoe backup" on a directory, I'd like an indicator about approximately how much of the total job is has done and approximately how much longer it will probably take.

zooko added the

labels 2011-11-19 01:20:50 +00:00

zooko added this to the undecided milestone 2011-11-19 01:20:50 +00:00

dsoprea commented

2017-12-24 02:06:15 +00:00

What's actually blocking this ticket?

I was a little surprised that there wasn't already a mechanism to do this from the beginning, but I'm even more surprised that this major ticket has been open for six years and nary a comment. Thoughts?

Thanks (and Merry Christmas).

What's actually blocking this ticket? I was a little surprised that there wasn't already a mechanism to do this from the beginning, but I'm even more surprised that this major ticket has been open for six years and nary a comment. Thoughts? Thanks (and Merry Christmas).

exarkun commented

2018-03-13 12:26:20 +00:00

One minor improvement that could be made relatively easily here is just to report progress by file count. This isn't the most accurate kind of progress report. However, it would be an incremental step towards something based on actual size of data transfer required. The current implementation doesn't know how many files it is working on, it discovers them as it goes. Reporting on file-count-based progress would require looking at all files in advance - which will also be required for a data-size-based progress report.

I suggest that, to replace the current behavior of no-output, we have something like:

$ tahoe backup src dst
 Found 123 files to back up.
 Backing up 1/123...  0:00:0 elapsed...
 Backing up 2/123...  0:00:7 elapsed...
 .
 .
 .
 123 files uploaded (0 reused), 0 files skipped, 2 directories created (0 reused), 0 
directories skipped
 backup done, elapsed time: 0:00:41

This serves three purposes. First, it gives the user some feedback some very rough feedback on the size of the job they've just initiated. Second, it gives them some very rough indication of what kind of progress has been made through the job (and that some progress is still being made). Third, it gives them a running report of how much time has been spent on this job up to the current point.

A variation on this could be that the Backing up ... line is updated rather than being re-written for each line. I'm not sure what assumptions the Tahoe-LAFS CLI is happy making about its output destination though. Are basic control bytes are acceptable? If so, an updating status line with \b is easy enough.

I imagine in the future this could be improved to something like:

$ tahoe backup src dst
 Found 456 bytes in 123 files to back up.
 Backing up file 1/123, 20/456 bytes...  0:00:0 elapsed... estimated completion 1:23:45
 .
 .
 .
 123 files uploaded (0 reused), 0 files skipped, 2 directories created (0 reused), 0 
directories skipped
 backup done, elapsed time: 0:00:41

Perhaps also including a b/s transfer rate. And the "Backing up ..." line could again either be updating or repeating (if updating, some basic tabular formatting might be nice).

Thoughts?

One minor improvement that could be made relatively easily here is just to report progress by file count. This isn't the most accurate kind of progress report. However, it would be an incremental step towards something based on actual size of data transfer required. The current implementation doesn't know how many files it is working on, it discovers them as it goes. Reporting on file-count-based progress would require looking at all files in advance - which will also be required for a data-size-based progress report. I suggest that, to replace the current behavior of no-output, we have something like: ``` $ tahoe backup src dst Found 123 files to back up. Backing up 1/123... 0:00:0 elapsed... Backing up 2/123... 0:00:7 elapsed... . . . 123 files uploaded (0 reused), 0 files skipped, 2 directories created (0 reused), 0 directories skipped backup done, elapsed time: 0:00:41 ``` This serves three purposes. First, it gives the user some feedback some _very_ rough feedback on the size of the job they've just initiated. Second, it gives them some _very_ rough indication of what kind of progress has been made through the job (and that _some_ progress is still being made). Third, it gives them a running report of how much time has been spent on this job up to the current point. A variation on this could be that the `Backing up ...` line is updated rather than being re-written for each line. I'm not sure what assumptions the Tahoe-LAFS CLI is happy making about its output destination though. Are basic control bytes are acceptable? If so, an updating status line with \b is easy enough. I imagine in the future this could be improved to something like: ``` $ tahoe backup src dst Found 456 bytes in 123 files to back up. Backing up file 1/123, 20/456 bytes... 0:00:0 elapsed... estimated completion 1:23:45 . . . 123 files uploaded (0 reused), 0 files skipped, 2 directories created (0 reused), 0 directories skipped backup done, elapsed time: 0:00:41 ``` Perhaps also including a b/s transfer rate. And the "Backing up ..." line could again either be updating or repeating (if updating, some basic tabular formatting might be nice). Thoughts?

GitHub <noreply@github.com> commented

2018-03-27 11:47:06 +00:00

In 6ec5005/trunk:

Merge pull request #474 from exarkun/1587.basic-progress-report

Basic progress reporting for `tahoe backup`

Fixes ticket:1587

In [6ec5005/trunk](/tahoe-lafs/trac-2024-07-25/commit/6ec5005ccd4c24b68c6308a0f7b1870c452af17b): ``` Merge pull request #474 from exarkun/1587.basic-progress-report Basic progress reporting for `tahoe backup` Fixes ticket:1587 ```

tahoe-lafs added the

fixed

label 2018-03-27 11:47:06 +00:00

GitHub <noreply@github.com> closed this issue

2018-03-27 11:47:06 +00:00

Sign in to join this conversation.