tahoe-lafs/tahoe-lafs
tahoe-lafs/tahoe-lafs
2
0
Fork 0
Code Issues Pull Requests 23 Packages Projects Releases 1 Wiki Activity
5,912 Commits 200 Branches 89 Tags 162 MiB
b3bd6a1279
Commit Graph

5912 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
David-Sarah Hopwood
0e7e3bc51e OpenStack service: add AuthenticationClient. 
Configure using properties relevant to OpenStack.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 01:33:54 +01:00
David-Sarah Hopwood
2dc48bc8c5 Add stub OpenStack container impl. as a copy of S3 container impl. 
Generalize the container instantiation to work for either.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 01:33:54 +01:00
David-Sarah Hopwood
3c54924ecd Fix interface violations introduced in cloud merge. 
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 01:33:54 +01:00
Daira Hopwood
9160181d83 Make backupdb use dbutil. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2014-04-09 01:33:54 +01:00
Daira Hopwood
d29cfe15a5 Comment changes for ticket ref #1784 
Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
 2014-04-09 01:33:54 +01:00
David-Sarah Hopwood
598cd91f70 Makefile: have 'make tmpfstest' unmount and remove stale temp directories. 
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 01:33:54 +01:00
David-Sarah Hopwood
5438f4b35b Makefile: the timing for 'make tmpfstest' should exclude filesystem 
mounting/unmounting and entering the password if needed.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 01:33:54 +01:00
Daira Hopwood
d6d759f590 Makefile: allow tmpfs size to be more easily overridden, and use 500 MiB by default (rebased). 
(The kernel will only allocate space that is used; the limit is just in case
tests write more than expected.)

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2014-04-09 01:33:54 +01:00
David-Sarah Hopwood
cd67298d66 test_runner.py: add test for 'tahoe debug trial'. 
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 01:33:54 +01:00
Daira Hopwood
9b18949c91 Fixes to tests. Some tests are applied to multiple backends. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2014-04-09 01:33:54 +01:00
David-Sarah Hopwood
c896cfc2c1 Fixes to test infrastructure. 
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 01:33:54 +01:00
David-Sarah Hopwood
03e02eeece Miscellaneous corrections and additions. 
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 01:33:54 +01:00
David-Sarah Hopwood
b2b91a6aaf Changes to crawler classes (ShareCrawler and AccountingCrawler). 
Pass in a Clock to allow (in theory) deterministic testing, although this isn't used yet by tests.
Simplify the generic ShareCrawler code by not attempting to track state during processing
of a single prefix.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 01:33:54 +01:00
Daira Hopwood
a79d3d69fb Changes to fileutil. 
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 01:33:54 +01:00
David-Sarah Hopwood
97268cc95f Fix bugs in Accountant. 
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 01:33:54 +01:00
David-Sarah Hopwood
d0d17ff152 Simplify Account. 
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 01:33:54 +01:00
Daira Hopwood
230e57906d Changes to debug.py. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2014-04-09 01:33:53 +01:00
Daira Hopwood
cdbc1bcf36 Changes to node classes (Node, Client and StorageServer). 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2014-04-09 01:33:53 +01:00
Daira Hopwood
26aa98b9f4 Changes to Bucket{Reader,Writer} and disk backend (rebased). 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2014-04-09 01:33:53 +01:00
David-Sarah Hopwood
5a5622ce1d Changes and additions to interface documentation. 
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 01:33:53 +01:00
Daira Hopwood
7202791c3f Add new files for cloud merge (rebased). 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2014-04-09 01:33:33 +01:00
David-Sarah Hopwood
8faca7bc72 Move BucketWriter and BucketReader to storage/bucket.py. 
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 00:47:02 +01:00
David-Sarah Hopwood
434f781432 Move code around and add new directories for cloud backend merge. 
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 00:47:02 +01:00
David-Sarah Hopwood
8c92b50a33 Add dependency on our fork of txAWS (0.2.1.post5). 
Add 'six' to ignorable package list because it is a dependency of txAWS via python-dateutil.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 00:47:02 +01:00
Daira Hopwood
61727bf2ec .gitignore: changes to facilitate cloud backend merge (rebased). 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2014-04-09 00:46:57 +01:00
Daira Hopwood
c24a0b8270 Add documentation for each storage backend (rebased). 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2014-04-09 00:43:30 +01:00
David-Sarah Hopwood
8834a34a7e Add test_leasedb.py. 
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 00:33:34 +01:00
David-Sarah Hopwood
d99a093168 leasedb: use a semantic primary key (storage_index, shnum, account_id), rather than an integer, for the leases table. 
Take advantage of this to simplify add_or_renew_leases.
Fix a bug in add_starter_lease (which is not used yet) when a starter lease already exists.
Clean up leftover accesses to self._dirty.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 00:33:34 +01:00
David-Sarah Hopwood
1f61319128 Rename 'buckets' to 'sharesets' on storage status page. 
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 00:33:34 +01:00
David-Sarah Hopwood
68cb1b4e74 Remove unused files storage/lease.py and storage/expirer.py. 
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 00:33:34 +01:00
Daira Hopwood
0162c8bf69 Main leasedb changes (rebased). 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2014-04-09 00:33:21 +01:00
David-Sarah Hopwood
dbd6321f37 Remove the 'original-*' and 'configured-*' lease crawler output that won't be supported by the accounting crawler. 
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 00:26:05 +01:00
David-Sarah Hopwood
74d6f4a19b Cosmetics. 
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 00:26:05 +01:00
David-Sarah Hopwood
503a9dfa82 test_storage.py: ss -> server for cases that will remain a server after the server/account split. 
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 00:26:05 +01:00
David-Sarah Hopwood
a17fe86d69 Asyncify crawlers. Note that this breaks tests for the LeaseCrawler 
(which is going away, to be replaced by the AccountingCrawler).

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 00:26:05 +01:00
David-Sarah Hopwood
a67b54662e scripts/debug.py: remove display of lease information and secrets. 
This version replaces the expiration field with '-' instead of '0', per Zooko's comments.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 00:26:04 +01:00
David-Sarah Hopwood
79e4766b22 Remove the [storage]expire.{mutable,immutable} options. 
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 00:26:04 +01:00
David-Sarah Hopwood
91c66e8867 Remove support for [storage]debug_discard option. 
(BucketWriter.throw_out_all_data is kept because it might still be useful.)

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 00:26:04 +01:00
David-Sarah Hopwood
f05887e558 Add new files for leasedb. 
Authors: Brian Warner <warner@lothar.com> and David-Sarah Hopwood
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 00:26:04 +01:00
David-Sarah Hopwood
f3f8d3fd7b Changes to specification of add_lease and renew_lease in RIStorageServer. 
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 00:26:04 +01:00
David-Sarah Hopwood
5da509a7c2 docs/garbage-collection.rst: update text for leasedb changes. 
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 00:26:04 +01:00
David-Sarah Hopwood
d4184cffc2 Use "PRAGMA synchronous = OFF" for dbutil. 
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 00:26:04 +01:00
David-Sarah Hopwood
dfeb188d32 Add util/dbutil.py: open/create/update sqlite databases given some schema. 
Author: Brian Warner <warner@lothar.com>
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 00:26:04 +01:00
David-Sarah Hopwood
814620206a util/fileutil.py: add get_used_space. This version does not use FilePath. 
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 00:26:04 +01:00
David-Sarah Hopwood
7d120c3c58 Some useful Deferred utilities. Addresses Zooko's review comments. 
HookMixin and WaitForDelayedCallsMixin have been added for leasedb,
the rest are from the cloud backend branch, with minor improvements.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
 2014-04-09 00:26:04 +01:00
Daira Hopwood
3719978215 misc/simulators/hashbasedsig.py: revert last patch; the original code was correct. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2014-02-25 16:57:57 +00:00
Daira Hopwood
40c12fc736 misc/simulators/hashbasedsig.py: fix minor miscalculation bug. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2014-02-25 16:55:19 +00:00
Daira Hopwood
b262f656f4 misc/simulators/hashbasedsig.py: the cost of labels was being double-counted. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2014-02-25 01:04:03 +00:00
Daira Hopwood
946a9f9ddd misc/simulators/hashbasedsig.py: fix an incorrect comment. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2014-02-25 00:58:54 +00:00
Daira Hopwood
6ca5f5fc39 Use bigger random one-time keys in timing_safe_compare. fixes #2165 
Follows this advice from Marsh Ray and Solar Designer: https://twitter.com/zooko/status/431105294777597952

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2014-02-24 20:43:23 +00:00