"one-file" executable for non-developers #1951

New Issue

warner · 2013-04-22T22:23:23Z

warner commented

2013-04-22 22:23:23 +00:00

I'd like it if Tahoe users (not developers) could download a single
file and run it, rather than pointing them at "quickstart"
instructions that begin with a source download/checkout and then
some build steps. And I'd love it if this meant we could reduce
complexity (by reducing functionality) of our setup.py, because we'd
no longer need to support a single simple command to
acquire/build/locally-install dependencies.

For the one-file executable, I'm thinking bbfreeze, about or if that
doesn't work then something homemade. I recently learned that you
can concatenate a /usr/bin/env python line with a zipfile
that contains a directory with an *init*.py inside it, and
running the result will cause python to execute that
__init__.py. This sounds like a better-supported version of
what bbfreeze does (or maybe this is what bbfreeze uses these days,
I dunno). We'd probably publish one version for linux, and a
different one for windows. Maybe one per platform if their binary
dependencies are different. I'd want each one to be self-contained:
no dependency upon system-supplied libraries.

For the remaining setup.py, I'm thinking that maybe virtualenvs are
the way to go. Developer instructions could look like:

create and activate a virtualenv somewhere
run python setup.py develop from the tahoe source tree,
which will get python from the virtualenv, which will read
requirements.txt and install everything therein to the
virtualenv, then will pseudo-symlink tahoe itself into the
virtualenv
run ./bin/tahoe, which has a /usr/bin/env python,
and will thus use the virtualenv's python. Or, use the
virtualenv's bin/tahoe.

I believe packagers usually build OS packages by running
python setup.py install in a fakeroot environment, possibly with
additional options to install the results into a local directory
instead of /usr. And that they expect this install to not drag
in any dependencies. So I'd like that to work for packagers.

I don't know how long-term installation should work, but I'd like
some variant on sudo python setup.py install to work, maybe in
the form of two separate commands (one to install dependencies, the
second to install tahoe itself). Another option is to have
developer/installers construct their virtualenv as above, then
simply symlink the virtualenv's bin/tahoe into /usr/local/bin or
something.

I think it's ok to require that developers get virtualenv and
whatever it requires (pip, setuptools, etc) installed first, and
that we shouldn't feel obligated to provide additional source code
to install those for them.

I'd like it if Tahoe users (not developers) could download a single file and run it, rather than pointing them at "quickstart" instructions that begin with a source download/checkout and then some build steps. And I'd love it if this meant we could reduce complexity (by reducing functionality) of our setup.py, because we'd no longer need to support a single simple command to acquire/build/locally-install dependencies. For the one-file executable, I'm thinking bbfreeze, about or if that doesn't work then something homemade. I recently learned that you can concatenate a `/usr/bin/env python` line with a zipfile that contains a directory with an `*init*.py` inside it, and running the result will cause python to execute that `__init__.py`. This sounds like a better-supported version of what bbfreeze does (or maybe this is what bbfreeze uses these days, I dunno). We'd probably publish one version for linux, and a different one for windows. Maybe one per platform if their binary dependencies are different. I'd want each one to be self-contained: no dependency upon system-supplied libraries. For the remaining setup.py, I'm thinking that maybe virtualenvs are the way to go. Developer instructions could look like: * create and activate a virtualenv somewhere * run `python setup.py develop` from the tahoe source tree, which will get `python` from the virtualenv, which will read `requirements.txt` and install everything therein to the virtualenv, then will pseudo-symlink tahoe itself into the virtualenv * run `./bin/tahoe`, which has a `/usr/bin/env python`, and will thus use the virtualenv's python. Or, use the virtualenv's bin/tahoe. I believe packagers usually build OS packages by running `python setup.py install` in a fakeroot environment, possibly with additional options to install the results into a local directory instead of /usr. And that they expect this `install` to not drag in any dependencies. So I'd like that to work for packagers. I don't know how long-term installation should work, but I'd like some variant on `sudo python setup.py install` to work, maybe in the form of two separate commands (one to install dependencies, the second to install tahoe itself). Another option is to have developer/installers construct their virtualenv as above, then simply symlink the virtualenv's bin/tahoe into /usr/local/bin or something. I think it's ok to require that developers get virtualenv and whatever it requires (pip, setuptools, etc) installed first, and that we shouldn't feel obligated to provide additional source code to install those for them.

warner added the

labels 2013-04-22 22:23:23 +00:00

warner added this to the undecided milestone 2013-04-22 22:23:23 +00:00

warner commented

2013-04-22 22:24:12 +00:00

(https://pypi.python.org/pypi/bbfreeze/) is the recently-updated bbfreeze package, which apparently knows how to copy system-supplied shared libraries and DLLs into the generated executable.

zooko commented

2014-07-16 00:54:16 +00:00

wiki/BBFreeze

[wiki/BBFreeze](wiki/BBFreeze)

daira commented

2014-07-29 20:55:02 +00:00

I'm pretty skeptical about this. We already got bbfreeze working once (#585), and it didn't really solve any problem that we actually had. Almost all of the setuptools bugs/misdesign that cause problems with a normal build, also cause problems with a bbfreeze build, except that the latter is actually worse because no developer uses it, and so it bitrots. Also, the result doesn't in practice work unless you have one per platform ABI variant.

I think we should be using virtualenv for everything. A single-file build cannot possibly meet developer requirements, and so it's always going to be a second-class citizen. A virtualenv build can meet both developer and end-user requirements.

I'm pretty skeptical about this. We already got bbfreeze working once (#585), and it didn't really solve any problem that we actually had. Almost all of the setuptools bugs/misdesign that cause problems with a normal build, also cause problems with a bbfreeze build, except that the latter is actually worse because no developer uses it, and so it bitrots. Also, the result doesn't in practice work unless you have one per platform ABI variant. I think we should be using virtualenv for everything. A single-file build cannot possibly meet developer requirements, and so it's always going to be a second-class citizen. A virtualenv build *can* meet both developer and end-user requirements.

daira commented

2014-07-29 21:00:32 +00:00

virtualenv-based packaging is #2255.

daira commented

2015-01-13 18:04:43 +00:00

A one-file package for Mac OS X is #182.

A one-file package for Windows is #195.

I don't think a one-file package for other Unix platforms in general is feasible, but I'm open to counterarguments.

A one-file package for Mac OS X is #182. A one-file package for Windows is #195. I don't think a one-file package for other Unix platforms in general is feasible, but I'm open to counterarguments.

warner commented

2015-05-08 19:11:06 +00:00

One note: I think I should have said *main*.py or main.py instead of *init*.py. See the last section (about the <script> argument) in https://docs.python.org/2.7/using/cmdline.html#interface-options .

One note: I think I should have said `*main*.py` or `main.py` instead of `*init*.py`. See the last section (about the `<script>` argument) in <https://docs.python.org/2.7/using/cmdline.html#interface-options> .

exarkun commented

2020-01-17 14:28:08 +00:00

There is a PyInstaller package for Tahoe-LAFS now. This is almost a single-file executable. It's also tested on CI and has the advantage of being minimally weird compared to normal installations.

Since it is very similar to bbfreeze and satisfies the "download one thing to run tahoe" I think this can be considered fixed.

There is a [PyInstaller](wiki/PyInstaller) package for Tahoe-LAFS now. This is almost a single-file executable. It's also tested on CI and has the advantage of being _minimally_ weird compared to normal installations. Since it is very similar to bbfreeze and satisfies the "download one thing to run tahoe" I think this can be considered fixed.

exarkun added the

was already fixed

label 2020-01-17 14:28:08 +00:00

exarkun closed this issue

2020-01-17 14:28:08 +00:00

Sign in to join this conversation.