clarify policy about what versions of dependencies Tahoe-LAFS requires #1452

New Issue

zooko · 2011-07-28T04:07:58Z

zooko commented

2011-07-28 04:07:58 +00:00

There are conflicting ideas about how tight to make the requirements Tahoe-LAFS should place on dependencies (in source:src/allmydata/_auto_deps.py). One way to do it would to be require versions of dependencies which we know work with Tahoe-LAFS (i.e. they don't have any critical bugs that are triggered by the way Tahoe-LAFS uses them nor fatal incompatibilities with Tahoe-LAFS or other code that Tahoe-LAFS uses). This would be the tightest policy. The other extreme—the loosest policy—is to allow any version unless we know for sure that particular version has a critical bug or incompatibility.

I personally find it a bit dubious if we specify that Tahoe-LAFS requires a version of Package X greater than or equal to Version N, if we don't have any buildbots that have Version N installed. To me that sounds like we are claiming that Tahoe-LAFS will work with Version N of Package X, but making that claim without reproducible evidence that it is true. On the other hand we can just rely on the absence of any bug reports from the field which can be traced to a bug or incompatibility in some version of one of our dependencies.

One possible future improvement would be to automate the testing of Tahoe-LAFS combined with different versions of dependencies. The exocet tool looks like it might facilitate that sort of work: Exocet: A Second Look

In the meantime, let's write down a policy on our Wiki and close this ticket.

Another option would be to choose a set of operating systems/distributions that we think we should support and specify the versions of the dependencies which are provided in those. Currently we have the following operating systems/distributions in the Supported Builders category of our buildbot herd: Windows 7 x86_64, Debian 5 "Lenny" ("obsolete stable") x86_64, Ubuntu 10.04 "Lucid" x86_64, Ubuntu 10.10 "Maverick" x86_64, Debian 5 "Lenny" ("obsolete stable") x86, FreeBSD 8.2 x86_64, OpenBSD 4.8 x86_64, NetBSD 5 x86, CentOS 5 (RedHat 5.6) x86, Windows XP 5.1 SP 3 x86.

There are conflicting ideas about how tight to make the requirements Tahoe-LAFS should place on dependencies (in source:src/allmydata/_auto_deps.py). One way to do it would to be require versions of dependencies which we know work with Tahoe-LAFS (i.e. they don't have any critical bugs that are triggered by the way Tahoe-LAFS uses them nor fatal incompatibilities with Tahoe-LAFS or other code that Tahoe-LAFS uses). This would be the tightest policy. The other extreme—the loosest policy—is to allow any version unless we know for sure that particular version has a critical bug or incompatibility. I personally find it a bit dubious if we specify that Tahoe-LAFS requires a version of `Package X` greater than or equal to `Version N`, if we don't have any buildbots that have `Version N` installed. To me that sounds like we are claiming that Tahoe-LAFS will work with `Version N` of `Package X`, but making that claim without reproducible evidence that it is true. On the other hand we can just rely on the absence of any bug reports from the field which can be traced to a bug or incompatibility in some version of one of our dependencies. One possible future improvement would be to automate the testing of Tahoe-LAFS combined with different versions of dependencies. The exocet tool looks like it might facilitate that sort of work: [Exocet: A Second Look](http://washort.blogspot.com/2011/03/exocet-second-look.html) In the meantime, let's write down a policy on our Wiki and close this ticket. Another option would be to choose a set of operating systems/distributions that we think we should support and specify the versions of the dependencies which are provided in those. Currently we have the following operating systems/distributions in the Supported Builders category of our buildbot herd: Windows 7 x86_64, Debian 5 "Lenny" ("obsolete stable") x86_64, Ubuntu 10.04 "Lucid" x86_64, Ubuntu 10.10 "Maverick" x86_64, Debian 5 "Lenny" ("obsolete stable") x86, FreeBSD 8.2 x86_64, OpenBSD 4.8 x86_64, NetBSD 5 x86, CentOS 5 (RedHat 5.6) x86, Windows XP 5.1 SP 3 x86.

zooko added the

labels 2011-07-28 04:07:58 +00:00

zooko added this to the soon (release n/a) milestone 2011-07-28 04:07:58 +00:00

zooko commented

2011-07-28 04:16:56 +00:00

I posted about this to tahoe-dev: http://tahoe-lafs.org/pipermail/tahoe-dev/2011-July/006594.html

I posted about this to tahoe-dev: <http://tahoe-lafs.org/pipermail/tahoe-dev/2011-July/006594.html>

gdt commented

2011-07-28 11:15:43 +00:00

This ticket conflates two issues: how to get test/buildbot coverage over all possible set of legitimate dependencies, and what to express in policy. I think the policy issues are pretty standard and straightforward:

A dependency package's version has two separate constraints:

minimum version M, below which reports of trouble are summarily closed as WONTFIX
frequently tested version T, below which there might be trouble because of lack of testing
(possibly) highest version H believed to work, because >H is too bleeding edge

The response to a problem with x, M <= x < T, would be to take a clean patch if it doesn't break anything, maybe to spend a few minutes thinking about it, and maybe to incrnease M to higher than X.

M should always be a version that's been released for a year, absent special circumstances, one of which is that it's comaintained with trac and not used by much of anything else.

Beyond that, sure it would be great if there were buildbots that install various versions of dependencies.

This ticket conflates two issues: how to get test/buildbot coverage over all possible set of legitimate dependencies, and what to express in policy. I think the policy issues are pretty standard and straightforward: A dependency package's version has two separate constraints: * minimum version M, below which reports of trouble are summarily closed as WONTFIX * frequently tested version T, below which there might be trouble because of lack of testing * (possibly) highest version H believed to work, because >H is too bleeding edge The response to a problem with x, M <= x < T, would be to take a clean patch if it doesn't break anything, maybe to spend a few minutes thinking about it, and maybe to incrnease M to higher than X. M should always be a version that's been released for a year, absent special circumstances, one of which is that it's comaintained with trac and not used by much of anything else. Beyond that, sure it would be great if there were buildbots that install various versions of dependencies.

zooko added

major

and removed

critical

labels 2012-03-29 16:11:32 +00:00

zooko commented

2013-10-10 19:34:25 +00:00

Okay, I believe that what we're going to do is this:

We don't change the dependency version requirements (in [_auto_deps.py]source:trunk/src/allmydata/_auto_deps.py?annotate=blame&rev=7bb07fb5e28756fa13ba5190e6c39003c84d3e1e) unless something forces us to change them. That's because we're lazy, and not making changes is easier and less disruptive than making changes.

However, this means that what is written in _auto_deps.py is not an assertion on our part that these versions of the deps will function and be secure. Instead, it is just a record of the fact that nobody has shown us a problem with them that forced us to update them. In particular, we aren't making any particular effort to test with the older versions that we list as the minimum required versions.

So, to close this ticket, write this policy down somewhere, possibly just in a comment in _auto_deps.py, and then link to the patch that documents our policy and close this ticket.

Okay, I believe that what we're going to do is this: We don't change the dependency version requirements (in [_auto_deps.py]source:trunk/src/allmydata/_auto_deps.py?annotate=blame&rev=7bb07fb5e28756fa13ba5190e6c39003c84d3e1e) unless something forces us to change them. That's because we're lazy, and not making changes is easier and less disruptive than making changes. However, this means that what is written in _auto_deps.py is not an assertion on our part that these versions of the deps will function and be secure. Instead, it is just a record of the fact that nobody has shown us a problem with them that forced us to update them. In particular, we aren't making any particular effort to test with the older versions that we list as the minimum required versions. So, to close this ticket, write this policy down somewhere, possibly just in a comment in _auto_deps.py, and then link to the patch that documents our policy and close this ticket.

gdt commented

2013-10-10 20:16:36 +00:00

I second zooko's position in comment:84513. An important point is that various packaging systems have various versions of dependencies, and it's really awkward to insist on an exact particular version, because multiple packages making such demands simply does not work in the general case, and leads to having to version many things.

Another point is that if "X >= Y.Z" is not ok in general, than arguably package X is buggy.

By providing a "make check" facility, packaging system maintainers can be encouraged to run the tests when updating, and to allow the user to do so when building. That's not as good as a buildbot, but requires fewer resources.

I second zooko's position in [comment:84513](/tahoe-lafs/trac-2024-07-25/issues/1452#issuecomment-84513). An important point is that various packaging systems have various versions of dependencies, and it's really awkward to insist on an exact particular version, because multiple packages making such demands simply does not work in the general case, and leads to having to version many things. Another point is that if "X >= Y.Z" is not ok in general, than arguably package X is buggy. By providing a "make check" facility, packaging system maintainers can be encouraged to run the tests when updating, and to allow the user to do so when building. That's not as good as a buildbot, but requires fewer resources.

daira commented

2013-10-17 15:07:56 +00:00

Testing with the earliest allowed versions (i.e. replacing the ">=" requirements for that test with "==" requirements) would be a good test to do on the buildbots and/or Travis. It wouldn't catch problems (including some that have actually occurred) with one dependency no longer being compatible with another, but it would catch cases where Tahoe itself introduces an incompatibility.