design+build the Usage/Aggregator service #468

New Issue

warner · 2008-06-17T01:58:37Z

warner commented

2008-06-17 01:58:37 +00:00

As part of the Accounting project, we'll be building a "Usage" service, which
will help keep track of how much space is used per account. Here are my notes
from last week:

creating a Tahoe node with a storage server component creates a Usage
component by default. You can also create a node with a Usage component
but nothing else: we call this an Aggregator. The Usage component
subscribes to any co-resident Storage component, and can also subscribe to
other Storage services in other nodes. The Usage component provides a web
interface (localhost:8124/usage) that provides different views of a table
that shows usage (in bytes) per (server,account) pair. It also provides a
sum across all servers. If the Usage component is only subscribing to a
single storage server, the table will only have one column, and the sum
will be the same as the table. It can also return information about just a
subset of accounts (to return data in batches, since there might be a
million accounts). The Usage component will consult an optional "petname
database", which maps account to name. It will have an HTML table display,
and a machine-readable JSON interface.

Allmydata will run an Aggregator that subscribes to all prodnet storage
servers. Peter's SQL database stuff will then make JSON queries of the
aggregator to keep track of how much each customer is using.

In a friendnet deployment, the storage server admin can use their local
Usage service to find out how much each of their friends is using on that
one server. If the friendnet members exchange Usage-subscription-FURLs
symmetrically, they will be able to see how much each member is using
across the whole friendnet.

When we implement Invitations, the code that negotiates the invitation
will populate the petname database.

As part of the Accounting project, we'll be building a "Usage" service, which will help keep track of how much space is used per account. Here are my notes from last week: creating a Tahoe node with a storage server component creates a Usage component by default. You can also create a node with a Usage component but nothing else: we call this an Aggregator. The Usage component subscribes to any co-resident Storage component, and can also subscribe to other Storage services in other nodes. The Usage component provides a web interface (localhost:8124/usage) that provides different views of a table that shows usage (in bytes) per (server,account) pair. It also provides a sum across all servers. If the Usage component is only subscribing to a single storage server, the table will only have one column, and the sum will be the same as the table. It can also return information about just a subset of accounts (to return data in batches, since there might be a million accounts). The Usage component will consult an optional "petname database", which maps account to name. It will have an HTML table display, and a machine-readable JSON interface. Allmydata will run an Aggregator that subscribes to all prodnet storage servers. Peter's SQL database stuff will then make JSON queries of the aggregator to keep track of how much each customer is using. In a friendnet deployment, the storage server admin can use their local Usage service to find out how much each of their friends is using on that one server. If the friendnet members exchange Usage-subscription-FURLs symmetrically, they will be able to see how much each member is using across the whole friendnet. When we implement Invitations, the code that negotiates the invitation will populate the petname database.