Commit Graph

6523 Commits

Author SHA1 Message Date
Brian Warner b6c0297dee Include unminified versions of d3+jquery. Addresses #2208.
This should hopefully satisfy the Debian requirement to include original
sources. The old minified files for d3 and jquery were 63k and 91k
respectively, while the new unminified files are 133k and 293k.
2014-04-25 16:16:05 -07:00
Brian Warner a724c09ab8 release docs: recommend Twine for package upload. Closes #2086. 2014-04-25 15:26:23 -07:00
Daira Hopwood 4889129f37 Minor comment fix. refs #1847
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2014-04-21 22:41:23 +01:00
Daira Hopwood 77767e9e12 Remove ugly shadowing of Client.DEFAULT_ENCODING_PARAMETERS. fixes #1847
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2014-04-21 22:41:23 +01:00
Zooko Wilcox-O'Hearn 17736575f4 remove lzip, add cffi, cryptography.io 2014-04-15 16:24:36 +00:00
Zooko Wilcox-O'Hearn 77b536c7a4 remove darcs tools from show-tool-versions 2014-04-11 14:34:01 +00:00
Daira Hopwood 1cddcb6bc3 Leasedb and cloud backend merge is scheduled for v1.12.0.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2014-04-09 05:34:08 +01:00
Daira Hopwood 05fb4039ab tests: rename *WithMockCloudBackend to *WithCloudBackendAndMockContainer.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2014-04-09 01:58:07 +01:00
Daira Hopwood 3df9bb7934 Add the 'tahoe admin ls-container' command and tests. fixes #1759
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2014-04-09 01:57:52 +01:00
Daira Hopwood 64d259c142 Add backend support for listing container contents. refs #1759
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2014-04-09 01:57:52 +01:00
Daira Hopwood 6c4f51056c Refactoring to move ContainerItem and ContainerListing.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2014-04-09 01:57:51 +01:00
Daira Hopwood 32547bbd10 Move a utility function into allmydata.util.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2014-04-09 01:57:51 +01:00
Daira Hopwood e0ffe8bfcd S3Container: fix an oversight in the parent constructor call.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2014-04-09 01:57:51 +01:00
Daira Hopwood de8b40188b Fix #2206. Includes refactoring of ContainerRetryMixin into CommonContainerMixin, and rearrangement of the code to initialize HTTPClientMixin.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2014-04-09 01:57:51 +01:00
Daira Hopwood ea3111d24a S3 container: treat an empty GET bucket response as an error.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2014-04-09 01:57:51 +01:00
Daira Hopwood a09ffad7d2 Fix wrong arguments to CorruptStoredShareError constructor in immutable share classes.
(Bug pointed out by Mark_B.)

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2014-04-09 01:57:51 +01:00
Mark Berger 78d8ed2e05 Fixes fd leak in leasedb (ticket #2015)
Brian correctly diagnozed this issue and suggested the fix in Weekly Dev
Chat of 2013-07-09. This patch has been manually tested by Zooko using his
fdleakfinder and a unit test has been added to ensure that database connections
are properly closed. Without this patch, the 1819-cloud-merge branch uses
more than 1020 fds and will start failing on operating systems with a low fds
limit.

The portions affecting leasedb.py were written by Zooko and Daira.
2014-04-09 01:57:51 +01:00
Daira Hopwood 5b31c24535 Accounting crawler: re-enable share deletion. refs #1987
Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:57:51 +01:00
Daira Hopwood a389ee7196 Cloud backend: log URLs for HTTP responses, as well as requests.
Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:57:51 +01:00
Daira Hopwood 47e487ce8a Cloud backend: suppress unhandled TaskStopped exceptions from FileBodyProducer.
Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:57:51 +01:00
Daira Hopwood f1662b8162 Cloud backend: change the ChunkCache's replacement policy to LRU. fixes #1885
Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:57:51 +01:00
Daira Hopwood 95182fe4ec test_storage.py: clean up arguments to FakeAccount methods.
Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:57:51 +01:00
Daira Hopwood c23e01c2c0 Accounting crawler: make share deletion conditional (defaulting to False for now). refs #1987, #1921
Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:57:51 +01:00
Daira Hopwood efd86e32a1 Azure: change 'container_name' config entry to 'container' for consistency with OpenStack.
Also fix a hidden bug in a test.

Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:57:51 +01:00
Daira Hopwood b71637b2ce docs/backends/cloud.rst: cosmetics and caveat about support for 'create-container'. refs #1971
Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:57:51 +01:00
Daira Hopwood d7fa405462 Add tests for 'tahoe admin create-container'. refs #1971
Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:57:51 +01:00
Daira Hopwood 1e8bedb444 Refactor mock_cloud methods to simplify and make them more like CommonContainerMixin.
Also make sure that stripping of data arguments for logging is correct for all containers.

Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:57:51 +01:00
Daira Hopwood dd809272a0 Move failure handling for create_container to make it more testable. refs #1971
Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:57:51 +01:00
Daira Hopwood 722b26c711 Fix incorrect calls to create and delete S3 buckets in S3Container. refs #1971
Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:57:51 +01:00
Daira Hopwood 0f840c4514 Add and document "tahoe admin create-container" command (rebased). refs #1971
Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:57:51 +01:00
Daira Hopwood 5800e5e5b2 A missing basedir should cause an error if we try to read the config. refs #1971
Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:44:25 +01:00
Daira Hopwood 11087d6dfa Refactoring to make node config accessible without actually creating a Node. refs #1971
Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:44:25 +01:00
Daira Hopwood e786fcf1f7 Refactoring to make backend configuration accessible from outside Client. refs #1971
Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:44:25 +01:00
Daira Hopwood 194935ba86 Help for admin commands: cosmetics and new tests.
Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:44:25 +01:00
Itamar Turner-Trauring 92599d01b6 Actually run the creation code against real Azure service, and corresponding bug fix. 2014-04-09 01:44:25 +01:00
Itamar Turner-Trauring fb9065a99f Implement method to create containers in Microsoft Azure storage backend. 2014-04-09 01:44:25 +01:00
Itamar Turner-Trauring 0c9fab28d6 Add documentation for Microsoft Azure Blob Service cloud storage backend config. 2014-04-09 01:44:25 +01:00
Daira Hopwood 51cc8907ef Implement dumping of chunked shares and fix system tests. fixes #1959
Also, change 'assert' to '_assert' in debug.py.

Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:44:25 +01:00
Daira Hopwood b897936045 Cleanup: improve error reporting for DataTooLargeError
(and fix an off-by-one error in MutableDiskShare._write_share_data).

Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:44:25 +01:00
Daira Hopwood 181c91e747 Make the cloud backend report corrupted shares correctly. fixes #1566
Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:44:25 +01:00
Daira Hopwood 634cce0773 Add CorruptStoredShareError as a superclass for all corrupt share errors raised by a storage server (rebased). refs #1566
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2014-04-09 01:44:25 +01:00
Daira Hopwood 64bf33bdb6 Improve skip messages for Azure tests.
Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:44:25 +01:00
Daira Hopwood bfa799347e Use 'fail*' rather than 'assert*' methods for consistency with other tests.
Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:44:25 +01:00
Daira Hopwood ea6d988b52 Test that share methods are called with the shareset locked. refs #1869
Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:44:25 +01:00
Daira Hopwood 48f58ac8d2 Fix a pyflakes warning.
Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:44:25 +01:00
Daira Hopwood 8ae9169e5b Lock remote operations on sharesets. fixes #1869
Signed-off-by: Daira Hopwood <david-sarah@jacaranda.org>
2014-04-09 01:44:25 +01:00
Daira Hopwood 220de5d355 Cosmetic: fix trailing whitespace.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2014-04-09 01:44:11 +01:00
Itamar Turner-Trauring 6d46d15e38 Add Google and Azure backends to setup.py. 2014-04-09 01:33:56 +01:00
Itamar Turner-Trauring db68ab19d7 Some improvements and bug fixes.
1. Discard body even if response code indicates a problem, when doing cloud backend HTTP requests. I believe this was triggering a bug in Twistd.
2. Google backend retries on 403 and other 4xx codes, not just 401.
3. More logging.
2014-04-09 01:33:56 +01:00
Itamar Turner-Trauring 5d774342af Fix bug when using oauth2client 1.1 instead of 1.0 (returned HTTP header was unicode rather than the expected bytes). 2014-04-09 01:33:56 +01:00