Experiment with virtualenv based packaging in the petmail style. #2255

New Issue

nejucomo · 2014-07-02T02:10:18Z

nejucomo commented

2014-07-02 02:10:18 +00:00

No description provided.

nejucomo added the

labels 2014-07-02 02:10:18 +00:00

nejucomo added this to the undecided milestone 2014-07-02 02:10:18 +00:00

nejucomo commented

2014-07-02 03:15:03 +00:00

Some notes on how nejucomo generated ./requirements.txt:

Edited ./setup.py to rename support to legacysupport.
renamed support to legacysupport in .gitignore.
recursively copied ./support from petmail
spliced in the SafeDevelop and run_command from petmail, but renamed the latter to avoid a collision from default tahoe setup.py.
Ran python ./setup.py safe_develop mainly to initialize ./venv (but perhaps it would be simpler to run python ./support/virtualenv.py ./venv)
Ran ./venv/bin/pip install . to use the existing ./setup.* to install dependencies into the new ./venv.
Generated requirements.txt using the existing ./venv by running ./venv/bin/python ./venv/bin/peep.py freeze > ./requirements.txt
manually removed wsgiref and allmydata-tahoe from requirements.txt
deactivated ./venv then ran python ./setup.py safe_develop to trigger peep's display of package hashes.
overwrote requirements.txt with the output of peep.

Some notes on how nejucomo generated `./requirements.txt`: 1. Edited `./setup.py` to rename `support` to `legacysupport`. 1. renamed `support` to `legacysupport` in `.gitignore`. 1. recursively copied `./support` from petmail 1. spliced in the `SafeDevelop` and `run_command` from petmail, but renamed the latter to avoid a collision from default tahoe `setup.py`. 1. Ran `python ./setup.py safe_develop` mainly to initialize `./venv` (but perhaps it would be simpler to run `python ./support/virtualenv.py ./venv`) 1. Ran `./venv/bin/pip install .` to use the existing `./setup.*` to install dependencies into the new `./venv`. 1. Generated `requirements.txt` using the existing `./venv` by running `./venv/bin/python ./venv/bin/peep.py freeze > ./requirements.txt` 1. manually removed `wsgiref` and `allmydata-tahoe` from `requirements.txt` 1. deactivated `./venv` then ran `python ./setup.py safe_develop` to trigger peep's display of package hashes. 1. overwrote `requirements.txt` with the output of peep.

nejucomo commented

2014-07-02 03:29:24 +00:00

I've just pushed an implementation of the above steps to https://github.com/nejucomo/tahoe-lafs/commits/2255-virtualenv-packaging.messy

I've just pushed an implementation of the above steps to <https://github.com/nejucomo/tahoe-lafs/commits/2255-virtualenv-packaging.messy>

nejucomo commented

2014-07-02 03:34:04 +00:00

Warner has an independent implementation of this approach which was created in "parallel sprint race mode": https://github.com/warner/tahoe-lafs/tree/venv

That implementation was more liberal in hacking out most of ./setup.py.

Warner has an independent implementation of this approach which was created in "parallel sprint race mode": <https://github.com/warner/tahoe-lafs/tree/venv> That implementation was more liberal in hacking out most of `./setup.py`.

daira commented

2014-07-24 01:17:14 +00:00

So, I see how this works for source packages, or if the requirements.txt is created on the same platform as the build in the virtualenv. I don't see how it works for binary eggs -- wouldn't we need to ship different requirements.txt files for each platform, containing different hashes?

Perhaps it is sufficient to have a requirements.txt for Windows 32-bit w/ Python 2.7, one for Windows 64-bit w/ Python 2.7, and one for every other platform that assumes a compiler and builds everything from source?

So, I see how this works for source packages, or if the `requirements.txt` is created on the same platform as the build in the virtualenv. I don't see how it works for binary eggs -- wouldn't we need to ship different `requirements.txt` files for each platform, containing different hashes? Perhaps it is sufficient to have a `requirements.txt` for Windows 32-bit w/ Python 2.7, one for Windows 64-bit w/ Python 2.7, and one for every other platform that assumes a compiler and builds everything from source?

tahoe-lafs added

packaging

and removed

unknown

labels 2014-07-29 21:00:02 +00:00

tahoe-lafs modified the milestone from undecided to 1.12.0

2014-07-29 21:00:02 +00:00

daira commented

2014-08-19 17:47:06 +00:00

See also #2077 (pip packaging plan).

warner commented

2016-03-22 05:02:25 +00:00

Milestone renamed

warner modified the milestone from 1.12.0 to 1.13.0

2016-03-22 05:02:25 +00:00

warner commented

2016-03-26 23:14:58 +00:00

We've switched to virtualenv-based installation. Not quite in the style that Nathan and I were exploring (where a new command creates+populates a virtualenv for you), but I think the existing #2055 build-it-safely ticket can cover the distance between what we've got now (virtualenv; pip install) and a peep/requirements.txt -based safe-build step on top of it. So I think we can close this one out.

warner added the

fixed

label 2016-03-26 23:14:58 +00:00

warner modified the milestone from 1.13.0 to 1.11.0

2016-03-26 23:14:58 +00:00

warner closed this issue

2016-03-26 23:14:58 +00:00

Sign in to join this conversation.