storage server deletes garbage shares itself instead of waiting for crawler to notice them #1833

Open
opened 2012-10-30 22:39:10 +00:00 by zooko · 3 comments

Currently, the "lease crawler" or "accounting crawler" is responsible for deleting shares that have lost all their leases (by cancellation or expiry).

I propose that this be done instead by the storage server maintaining a persistent set of shares to be deleted. When lease-updating step (which, in #666, is synchronous and fast) has identified a share that has no more leases, the share's id gets added to the persistent set of shares to delete. A long-running, persistent, duty-cycle-limited processes deletes those shares from the backend and removes their ids from the set of shares-to-delete. This is cleaner and more efficient than using a crawler, which has to visit all shares and which never stops twitching, since this has to visit only shares that have been marked as to-delete, and it quiesces when there is nothing to delete.

This is part of an "overarching ticket" to eliminate most uses of crawler — ticket #1834.

Currently, the "lease crawler" or "accounting crawler" is responsible for deleting shares that have lost all their leases (by cancellation or expiry). I propose that this be done instead by the storage server maintaining a persistent set of shares to be deleted. When lease-updating step (which, in #666, is synchronous and fast) has identified a share that has no more leases, the share's id gets added to the persistent set of shares to delete. A long-running, persistent, duty-cycle-limited processes deletes those shares from the backend and removes their ids from the set of shares-to-delete. This is cleaner and more efficient than using a crawler, which has to visit *all* shares and which never stops twitching, since this has to visit only shares that have been marked as to-delete, and it quiesces when there is nothing to delete. This is part of an "overarching ticket" to eliminate most uses of crawler — ticket #1834.
zooko added the
code-storage
normal
enhancement
1.9.2
labels 2012-10-30 22:39:10 +00:00
zooko added this to the undecided milestone 2012-10-30 22:39:10 +00:00
davidsarah commented 2012-11-19 19:17:08 +00:00
Owner

Relevant to #1862 (removing shares more quickly than one day).

Relevant to #1862 (removing shares more quickly than one day).
Author

This would fix #1987.

This would fix #1987.
Author

This might fix #1921.

This might fix #1921.
Sign in to join this conversation.
No Milestone
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Reference: tahoe-lafs/trac-2024-07-25#1833
No description provided.