"python ./setup.py test" rebuilds packages #657

New Issue

zooko · 2009-03-09T16:47:40Z

zooko commented

2009-03-09 16:47:40 +00:00

If you build with python ./setup.py build, it will automatically build required libraries such as pycryptopp. On Zandr's NAS device, building pycryptopp takes about an hour. Then if you test with python ./setup.py test, it will rebuild pycryptopp again. I investigated and I see that this is because the initial build puts the resulting package into ./support/lib/python2.5/site-packages, but the test puts it into the current working directory. So a good fix would be to configure it so that the test step uses ./support/ as its "target directory".

You can work-around this issue by installing with python ./setup.py install, which will copy all libraries such as pycryptopp into your system directory (e.g. /usr/lib/python2.5/site-packages, or by manually prepending the test step with PYTHONPATH=./support/lib/python2.5/site-packages.

(Note that if you install with python ./setup.py install then it is very easy to uninstall. All you do is /bin/rm -rf the one item that got added into your /usr/lib/python2.5/site-packages/ directory for each library that got installed. For example, for pycryptopp that item is named pycryptopp-$VERSION_NUMBER-py2.5.egg.)

If you build with `python ./setup.py build`, it will automatically build required libraries such as pycryptopp. On Zandr's NAS device, building pycryptopp takes about an hour. Then if you test with `python ./setup.py test`, it will rebuild pycryptopp *again*. I investigated and I see that this is because the initial `build` puts the resulting package into `./support/lib/python2.5/site-packages`, but the `test` puts it into the current working directory. So a good fix would be to configure it so that the `test` step uses `./support/` as its "target directory". You can work-around this issue by installing with `python ./setup.py install`, which will copy all libraries such as pycryptopp into your system directory (e.g. `/usr/lib/python2.5/site-packages`, or by manually prepending the `test` step with `PYTHONPATH=./support/lib/python2.5/site-packages`. (Note that if you install with `python ./setup.py install` then it is *very easy to uninstall*. All you do is `/bin/rm -rf` the *one* item that got added into your `/usr/lib/python2.5/site-packages/` directory for each library that got installed. For example, for pycryptopp that item is named `pycryptopp-$VERSION_NUMBER-py2.5.egg`.)

zooko added the

labels 2009-03-09 16:47:40 +00:00

zooko added this to the eventually milestone 2009-03-09 16:47:40 +00:00

zooko commented

2009-03-09 16:50:47 +00:00

I added a ticket for setuptools_trial for a way that it could help with this: http://allmydata.org/trac/setuptools_trial/ticket/6 # accept a --prefix option for where to put install requirements (or to find them)

I added a ticket for `setuptools_trial` for a way that it could help with this: <http://allmydata.org/trac/setuptools_trial/ticket/6> # accept a --prefix option for where to put install requirements (or to find them)

zooko added

packaging

and removed

unknown

labels 2009-03-09 16:50:58 +00:00

warner commented

2009-03-09 21:14:10 +00:00

ah, this matches what I was experiencing on OS-X last week.

I guess I'm not surprised that the setuptools 'test' target is doing a compile (I can see it helping people who forget/decline to do a 'build' first, and I can imagine the setuptools folks believing that their is-a-library-already-installed logic working fast enough and correctly enough to avoid repeating a lot of work on each test pass). What I am surprised about is that it appears to compile+copy things into the working directory (which I assume is the root of the source tree). I would expect it to compile+copy things into build/lib, and then add that directory to PYTHONPATH before running the tests. Isn't that what build/ is for?

Thanks for the note about how easy it is to manually uninstall things that you've allowed setuptools to write to system directories.. that makes me feel like I have more control over my system if I had to resort to 'sudo setup.py install'.

ah, this matches what I was experiencing on OS-X last week. I guess I'm not surprised that the setuptools 'test' target is doing a compile (I can see it helping people who forget/decline to do a 'build' first, and I can imagine the setuptools folks believing that their is-a-library-already-installed logic working fast enough and correctly enough to avoid repeating a lot of work on each test pass). What I am surprised about is that it appears to compile+copy things into the working directory (which I assume is the root of the source tree). I would expect it to compile+copy things into build/lib, and then add that directory to PYTHONPATH before running the tests. Isn't that what build/ is for? Thanks for the note about how easy it is to manually uninstall things that you've allowed setuptools to write to system directories.. that makes me feel like I have more control over my system if I had to resort to 'sudo setup.py install'.

zooko commented

2009-08-06 18:30:56 +00:00

This may be related to #717 (unnecessary rebuild of dependencies when tahoe-deps/ is present).

zooko commented

2009-10-14 15:09:25 +00:00

It looks like this may be related to http://bugs.python.org/setuptools/issue65 (setuptools bug,inconsistent replacement of eggs when installing from file:// urls.) which is fixed in setuptools-0.6c10 prerelease. setuptools 65 is related to http://bugs.python.org/setuptools/issue17 (easy_install will install a package that is already there) and http://bugs.python.org/setuptools/issue20 (package required at build time seems to be not fully present at install time?).

Maybe someone (cgalvan?) could test this issue with the setuptools v0.6c10.

It looks like this may be related to <http://bugs.python.org/setuptools/issue65> (setuptools bug,inconsistent replacement of eggs when installing from file:// urls.) which is fixed in setuptools-0.6c10 prerelease. setuptools 65 is related to <http://bugs.python.org/setuptools/issue17> (easy_install will install a package that is already there) and <http://bugs.python.org/setuptools/issue20> (package required at build time seems to be not fully present at install time?). Maybe someone (cgalvan?) could test this issue with the setuptools v0.6c10.

zooko modified the milestone from eventually to 1.7.0

2010-04-28 20:42:13 +00:00

zooko modified the milestone from 1.7.0 to eventually

2010-06-16 04:41:03 +00:00

warner commented

2016-03-26 23:21:55 +00:00

We've removed the setup.py test command (which was an alias for "build a bunch of stuff and then run trial"). And we've switched to tox for running unit tests, where our tox.ini just runs trial in its virtualenv.

So I think we can close this out now.

We've removed the `setup.py test` command (which was an alias for "build a bunch of stuff and then run `trial`"). And we've switched to `tox` for running unit tests, where our `tox.ini` just runs `trial` in its virtualenv. So I think we can close this out now.

warner added the

fixed

label 2016-03-26 23:21:55 +00:00

warner modified the milestone from eventually to 1.11.0

2016-03-26 23:21:55 +00:00

warner closed this issue

2016-03-26 23:21:55 +00:00

Sign in to join this conversation.