setuptools console scripts vs. manually installed dependencies #141

New Issue

zooko · 2007-09-21T22:44:07Z

zooko commented

2007-09-21 22:44:07 +00:00

The setuptools console scripts feature is useful for making allmydata-tahoe.exe on Windows.

When it runs, it checks for the presence of all of the required dependencies by looking in its easy-install.pth. This means that if the dependency is present but not registered in easy-install.pth (because it was installed other than by setuptools/easy_install) then the check fails. One can easily fix this by running "easy_install ${PKGNAME}", which will not change anything except to register the already-installed package in easy-install.pth.

The setuptools console scripts feature is useful for making allmydata-tahoe.exe on Windows. When it runs, it checks for the presence of all of the required dependencies by looking in its easy-install.pth. This means that if the dependency is present but not registered in easy-install.pth (because it was installed other than by setuptools/easy_install) then the check fails. One can easily fix this by running "easy_install ${PKGNAME}", which will not change anything except to register the already-installed package in easy-install.pth.

zooko added the

labels 2007-09-21 22:44:07 +00:00

zooko added this to the 0.6.1 milestone 2007-09-21 22:44:07 +00:00

zooko added

and removed

labels 2007-09-26 13:38:35 +00:00

zooko commented

2007-09-26 13:39:18 +00:00

merging in #149

zooko commented

2007-09-26 18:19:12 +00:00

Note that there is potentially value in this check, because it checks for version and not just name. So for example if there was a version of Nevow older than v0.6 installed, then with this check will give a clear error message to the user instead of whatever unexpected behavior results from trying to use that older version of nevow.

Another workaround would be to use the allmydata-tahoe script that we wrote instead of the wrapper that setuptools generated.

We could use our bare unwrapped script on systems other than Windows.
We could always use our script and add our own "build an executable on Windows" feature.
We could offer the option to the user.

Note that there is potentially value in this check, because it checks for version and not just name. So for example if there was a version of Nevow older than v0.6 installed, then with this check will give a clear error message to the user instead of whatever unexpected behavior results from trying to use that older version of nevow. Another workaround would be to use the allmydata-tahoe script that we wrote instead of the wrapper that setuptools generated. 1. We could use our bare unwrapped script on systems other than Windows. 2. We could always use our script and add our own "build an executable on Windows" feature. 3. We could offer the option to the user.

zooko commented

2007-09-26 18:23:08 +00:00

Actually, I was wrong about how the setuptools-ified script tests for the presence of dependencies. According to this thread:

http://mail.python.org/pipermail/distutils-sig/2007-September/008225.html

It is the .egg-info that the script is testing for. Hm... That doesn't seem consistent with my earlier experiment... I'll investigate.

By the way, as of Python 2.5, distutils-packaged packages are supposed to come with .egg-info metadata. If they do, then this helps with this problem.

Actually, I was wrong about how the setuptools-ified script tests for the presence of dependencies. According to this thread: <http://mail.python.org/pipermail/distutils-sig/2007-September/008225.html> It is the .egg-info that the script is testing for. Hm... That doesn't seem consistent with my earlier experiment... I'll investigate. By the way, as of Python 2.5, distutils-packaged packages are supposed to come with .egg-info metadata. If they do, then this helps with this problem.

warner commented

2007-09-26 19:44:32 +00:00

I just tried doing 'easy_install nevow' on a feisty system that had
Nevow-0.9.0 installed from a .deb . This downloaded and built the latest
Nevow (0.9.18) and wrote it to
/usr/lib/python2.5/site-packages/Nevow-0.9.18-py2.5.egg/* . It also modified
the easy-install.pth file there.

I suppose that it did this rather than just touching easy-install.pth because
it saw an older version in there.

I'm uncomfortable about modifying /usr/lib/ on a debian system outside the
control of the debian package manager, so I'd like to find a different
solution for this. At the moment I'm in favor of your option number 1 (use
our own bin/allmydata-tahoe on non-windows systems).

We could also add a couple of version checks at the point-of-import to help
discover too-old versions. That would be slightly better than requiring them
to run allmydata-tahoe at all, for example you don't actually need Nevow
unless you turn on the webport feature (storage-only nodes would not need
it).

I just tried doing 'easy_install nevow' on a feisty system that had Nevow-0.9.0 installed from a .deb . This downloaded and built the latest Nevow (0.9.18) and wrote it to /usr/lib/python2.5/site-packages/Nevow-0.9.18-py2.5.egg/* . It also modified the easy-install.pth file there. I suppose that it did this rather than just touching easy-install.pth because it saw an older version in there. I'm uncomfortable about modifying /usr/lib/ on a debian system outside the control of the debian package manager, so I'd like to find a different solution for this. At the moment I'm in favor of your option number 1 (use our own bin/allmydata-tahoe on non-windows systems). We could also add a couple of version checks at the point-of-import to help discover too-old versions. That would be slightly better than requiring them to run allmydata-tahoe at all, for example you don't actually need Nevow unless you turn on the webport feature (storage-only nodes would not need it).

zooko commented

2007-09-26 22:24:00 +00:00

So it turns out that this is an issue with setuptools's handling of "scripts=" just as it is with its handling of the 'console_scripts' entry point. The two ways of doing it do different things (details below), but both of them run into this problem deploying on a system that has dependencies without .egg-info's.

Here is the script produced by setuptools with the "script=" option:

# EASY-INSTALL-SCRIPT: 'allmydata-tahoe==0.6.0-13','allmydata-tahoe'
__requires__ = 'allmydata-tahoe==0.6.0-13'
import pkg_resources
pkg_resources.run_script('allmydata-tahoe==0.6.0-13', 'allmydata-tahoe')

And here is the script produced by setuptools with the console_scripts option:

# EASY-INSTALL-ENTRY-SCRIPT: 'allmydata-tahoe==0.6.0-13','console_scripts','allmydata-tahoe'
__requires__ = 'allmydata-tahoe==0.6.0-13'
import sys
from pkg_resources import load_entry_point

sys.exit(
   load_entry_point('allmydata-tahoe==0.6.0-13', 'console_scripts', 'allmydata-tahoe')()
)

So it turns out that this is an issue with setuptools's handling of "scripts=" just as it is with its handling of the 'console_scripts' entry point. The two ways of doing it do different things (details below), but both of them run into this problem deploying on a system that has dependencies without .egg-info's. Here is the script produced by setuptools with the "script=" option: ```/usr/local/bin/python # EASY-INSTALL-SCRIPT: 'allmydata-tahoe==0.6.0-13','allmydata-tahoe' __requires__ = 'allmydata-tahoe==0.6.0-13' import pkg_resources pkg_resources.run_script('allmydata-tahoe==0.6.0-13', 'allmydata-tahoe') ``` And here is the script produced by setuptools with the console_scripts option: ```/usr/local/bin/python # EASY-INSTALL-ENTRY-SCRIPT: 'allmydata-tahoe==0.6.0-13','console_scripts','allmydata-tahoe' __requires__ = 'allmydata-tahoe==0.6.0-13' import sys from pkg_resources import load_entry_point sys.exit( load_entry_point('allmydata-tahoe==0.6.0-13', 'console_scripts', 'allmydata-tahoe')() ) ```

zooko commented

2007-10-01 19:37:48 +00:00

So the fix Brian and I agreed on is that the Makefile targets to build debian packages will manually cp our allmydata-tahoe script over the setuptools-generated allmydata-tahoe wrapper script.

zooko removed their assignment 2007-10-01 19:37:48 +00:00

warner was assigned by zooko

2007-10-01 19:37:48 +00:00

zooko commented

2007-10-02 20:35:42 +00:00

fixed by changeset:49813c28abc92356

zooko added the

fixed

label 2007-10-02 20:35:42 +00:00

zooko closed this issue

2007-10-02 20:35:42 +00:00

Sign in to join this conversation.