babcf632da
util/time_format: new routine to parse dates like 2009-03-18, switch expirer to use it. I'd prefer to use 18-Mar-2009, but it is surprisingly non-trivial to build a parser that will take UTC dates instead of local dates
2009-03-18 17:58:14 -07:00
8eaee28550
expirer: change setup, config options, in preparation for adding tahoe.cfg controls
2009-03-18 17:21:38 -07:00
fffab0d724
expirer: track mutable-vs-immutable sharecounts and sizes, report them on the web status page for comparison
2009-03-18 13:25:04 -07:00
406fdba61f
add utility function to parse durations, for lease-expiration config
2009-03-17 00:01:17 -07:00
24ab5ec26f
expirer: add mode to expire only-mutable or only-immutable shares
2009-03-16 23:51:18 -07:00
475e7b8804
tahoe_ls.py: add comment about error cases to improve
2009-03-16 22:12:06 -07:00
c7254c5f1d
GC: add date-cutoff -based expiration, add proposed docs
2009-03-16 22:10:41 -07:00
2b525a42d3
test_deepcheck: remove the 10s timeout: our dapper buildslave requires 30s, and the reduced timeout was only there because this tests fails by timeout rather than explicitly
2009-03-15 17:51:34 -07:00
f0b0ad1c8f
tahoe cp -r: add --caps-only flag, to write filecaps into local files instead of actual file contents. Used only for debugging and as a quick tree-comparison tool.
2009-03-15 16:19:58 -07:00
6e57576f2e
dirnode deep_traverse: insert a turn break (fireEventually) at least once every 100 files, otherwise a CHK followed by more than 158 LITs can overflow the stack, sort of like #237 .
2009-03-13 16:31:35 -07:00
e59164e0bb
consolidate: remove pointless 'else' after for loop
2009-03-13 01:27:51 -07:00
ae64ddf8e6
consolidate: add eta, flush stdout
2009-03-13 01:24:51 -07:00
06bf443e91
consolidate: tolerate unicode dirnames
2009-03-12 23:54:02 -07:00
1a741fdb03
dirnode.py: when doing deep-traverse, walk each directory in alphabetical order, to make things like 'manifest' more predictable
2009-03-12 23:50:46 -07:00
a18f8d4cc7
consolidator: add progress to scan-old-directory passes
2009-03-12 22:47:28 -07:00
ddb2f1a6e3
consolidator: fix cycle detection to not trigger on merely shared directories, add snapshot counter to progress
2009-03-12 21:22:29 -07:00
ddc9a7ae74
consolidator: re-use more directories, add total directories seen-vs-used counts
2009-03-12 20:48:01 -07:00
809ec25ffa
tahoe_backup.py: tolerate more time formats
2009-03-12 18:16:00 -07:00
1c24707f19
consolidator: add more verbose traversal of directories
2009-03-12 16:29:00 -07:00
760688a224
consolidate: create multiple numbered backups of the original Archives directory, not just the first time
2009-03-12 16:04:27 -07:00
c4c6a62954
add 'tahoe debug consolidate' command, to merge directories created by repeated 'tahoe cp -r' or the allmydata win32 backup tool, into the form that would have been created by 'tahoe backup'.
2009-03-12 13:56:06 -07:00
969b758bf5
tahoe_cp.py: return 0 for success, instead of None
2009-03-12 13:53:45 -07:00
55de7c3fed
setup: specify in the debian/control files that tahoe is compatible with Python 2.6
2009-03-11 15:59:02 -07:00
cc9adba8a3
setup: update the debian/copyright text to reflect the current licences
2009-03-11 08:29:52 -07:00
1ccd426a34
expirer: fix prediction math, thanks to Zandr for the catch
2009-03-09 13:42:17 -07:00
8708045a98
storage.expirer: oops, fix upgrade-handler code
2009-03-08 20:55:16 -07:00
df3f7f93e0
storage.expirer: handle upgrades better
2009-03-08 20:42:20 -07:00
a68ad06254
storage.expirer: exercise the last missing line of webstatus code
2009-03-08 20:38:28 -07:00
6d6049430c
expirer: tolerate corrupt shares, add them to the state and history for future examination
2009-03-08 20:08:40 -07:00
1a98521c3d
storage/immutable: raise a specific error upon seeing a bad version number, instead of using assert. Also wrap to 80cols.
2009-03-08 20:07:32 -07:00
02b40ec499
storage: add test coverage for BucketReader.__repr__
2009-03-08 20:05:27 -07:00
6d7319c588
storage/mutable: raise a specific error upon seeing bad magic, instead of using assert
2009-03-08 19:02:01 -07:00
7e7167f2e5
nodeadmin: node stops itself if a hotline file hasn't been touched in 120 seconds now, instead of in 60 seconds
...
A test failed on draco (MacPPC) because it took 67.1 seconds to get around to running the test, and the node had already stopped itself when the hotline file was 60 seconds old.
2009-03-07 20:50:39 -07:00
5675b4e7e0
expirer: make web display a bit more consistent
2009-03-07 16:14:42 -07:00
df045650e0
web/storage.py: tolerate unknown-future displays, I'm not sure why LeaseCrawler.test_unpredictable_future didn't catch this
2009-03-07 16:02:43 -07:00
192c161be7
tahoe_cp.py: improve error reporting slightly: don't json-interpret HTTP errors, pass through tahoe webapi error messages
2009-03-07 05:40:51 -07:00
2c2c3c2295
test_cli.py: assert that 'ls' on an unrecoverable file now gives a better error message
2009-03-07 05:08:15 -07:00
67d7600781
test_web: test behavior of broken-dirnode GET, both html and json
2009-03-07 04:57:07 -07:00
950200fece
web: when a dirnode can't be read, emit a regular HTML page but with the child-table and upload-forms replaced with an apologetic message. Make sure to include the 'get info' links so the user can do a filecheck
2009-03-07 04:56:01 -07:00
badd79671c
web/common: split out exception-to-explanation+code mapping to a separate humanize_failure() function, so it can be used by other code. Add explanation for mutable UnrecoverableFileError.
2009-03-07 04:54:08 -07:00
9c7fafd5af
test_cli: use explicit (and stable) testdirs, instead of using self.mktemp
2009-03-07 03:04:28 -07:00
091cfd04ca
test_storage: solaris doesn't appear to give the same block count as other platforms, so don't assert as much about 'diskbytes' recovered
2009-03-07 02:45:18 -07:00
24ebdb3954
tahoe_cp.py: don't be verbose unless --verbose is passed: default is one line of success
2009-03-07 02:44:21 -07:00
38a9ab4c33
setup.cfg: use allmydata.org as a mirror for some of our necessary dependencies
2009-03-06 22:57:53 -07:00
0dee2a6036
storage: add a lease-checker-and-expirer crawler, plus web status page.
...
This walks slowly through all shares, examining their leases, deciding which
are still valid and which have expired. Once enabled, it will then remove the
expired leases, and delete shares which no longer have any valid leases. Note
that there is not yet a tahoe.cfg option to enable lease-deletion: the
current code is read-only. A subsequent patch will add a tahoe.cfg knob to
control this, as well as docs. Some other minor items included in this patch:
tahoe debug dump-share has a new --leases-only flag
storage sharefile/leaseinfo code is cleaned up
storage web status page (/storage) has more info, more tests coverage
space-left measurement on OS-X should be more accurate (it was off by 2048x)
(use stat .f_frsize instead of f_bsize)
2009-03-06 22:45:17 -07:00
908a460665
setup: comment-out the use of the directory full of dependency packages on the test grid -- the test grid is too unreliable! :-(
2009-03-06 22:43:32 -07:00
32c541cffc
setup: create a "make deb-lenny-head" target
...
I made this patch by copying [20090305220021-92b7f-89d987c7d05306b5cb03a64f2956a652c10a7296] and changing the name from "intrepid" to "lenny". I haven't tested it.
2009-03-06 13:10:57 -07:00
e9199a89ab
trivial: remove unused import detected by pyflakes, and remove trailing whitespace
2009-03-05 16:32:04 -07:00
341c7aad83
setup: enable build of .debs for Ubuntu Intrepid, thanks to DarKNesS_WolF
2009-03-05 16:00:21 -07:00
5e90d82a02
trivial: use more specific function for ascii-encoding storage index
2009-02-22 11:57:51 -07:00
Brian Warner