BuildBot step to run tests from package #348

New Issue

zooko · 2008-03-13T01:49:31Z

zooko commented

2008-03-13 01:49:31 +00:00

Patches that change packaging can't be conveniently tested by trial tests inside the package. A better way to test such things is to have a BuildStep which, after a package has been built (such as a source tarball or a py2exe or py2app package), it installs that package and then runs the unit tests from the version thus installed.

This would help test patches like #335 (OSError when trying to access !http://localhost:8123/provisioning/), it would catch errors in the build process which cause packages to be built incorrectly (such as the one that distracted several of us from more useful labor Monday morning), and if implemented in re-usable BuildBot code, it would help test similar issues in other projects such as Nevow #2527, Nevow #857, pyflakes #2535, and twisted #2308.

Patches that change packaging can't be conveniently tested by trial tests inside the package. A better way to test such things is to have a BuildStep which, after a package has been built (such as a source tarball or a py2exe or py2app package), it installs that package and then runs the unit tests from the version thus installed. This would help test patches like #335 (OSError when trying to access !<http://localhost:8123/provisioning/>), it would catch errors in the build process which cause packages to be built incorrectly (such as the one that distracted several of us from more useful labor Monday morning), and if implemented in re-usable BuildBot code, it would help test similar issues in other projects such as [Nevow #2527](http://divmod.org/trac/ticket/2527), [Nevow #857](http://divmod.org/trac/ticket/857), [pyflakes #2535](http://divmod.org/trac/ticket/2535), and [twisted #2308](http://twistedmatrix.com/trac/ticket/2308).

zooko added the

labels 2008-03-13 01:49:31 +00:00

zooko added this to the eventually milestone 2008-03-13 01:49:31 +00:00

zooko added

enhancement

and removed

defect

labels 2008-03-13 17:53:23 +00:00

zooko commented

2008-03-13 18:18:47 +00:00

I added a ticket on buildbot #199 for this.

I added a ticket on [buildbot #199](http://buildbot.net/trac/ticket/199) for this.

zooko commented

2008-03-17 21:03:05 +00:00

I updated buildbot #199 to suggest installing into a local, newly created subdirectory and testing that install by setting the PYTHONPATH to point to that subdirectory.

I updated [buildbot #199](http://buildbot.net/trac/ticket/199) to suggest installing into a local, newly created subdirectory and testing that install by setting the PYTHONPATH to point to that subdirectory.

warner modified the milestone from eventually to undecided

2008-06-01 20:43:52 +00:00

zooko commented

2008-08-12 21:21:37 +00:00

Nevow #2630 (installation of Nevow doesn't have automated tests) has been fixed -- the Nevow buildbot now tests installation of Nevow. We could do similarly for Tahoe.

[Nevow #2630 (installation of Nevow doesn't have automated tests)](http://divmod.org/trac/ticket/2630) has been fixed -- the Nevow buildbot now tests installation of Nevow. We could do similarly for Tahoe.

zooko commented

2008-08-28 20:43:55 +00:00

See also #434 (automate testing of installation) which is perhaps going to be fixed by the same patch that fixes this ticket -- if that patch makes it so that the buildbot both installs the package and runs the unit tests from the installed package.