rewrite our Makefile in Python instead of GNUmake #179

New Issue

zooko · 2007-10-15T18:13:53Z

zooko commented

2007-10-15 18:13:53 +00:00

I just committed changeset:bf3f629a683fd4cb (and the preceding related patches) which moves one kludgey little system-compatibility function from Makefile to misc/find_trial.py. This reminds me once again that we seem to make our build system more maintainable by rewriting parts of it from GNUmake into Python, and perhaps we ought to go all the way, eliminate Makefile entirely, remove the manual dependency on the GNU make tool, and write each of our build features into a logically separate setuptools plugin.

By the way, setuptools plugins like that can be in separate files and could even be separately packaged if we want, such as the setuptools_darcs_plugin that I recently wrote.

I just committed changeset:bf3f629a683fd4cb (and the preceding related patches) which moves one kludgey little system-compatibility function from Makefile to misc/find_trial.py. This reminds me once again that we seem to make our build system more maintainable by rewriting parts of it from GNUmake into Python, and perhaps we ought to go all the way, eliminate Makefile entirely, remove the manual dependency on the GNU make tool, and write each of our build features into a logically separate setuptools plugin. By the way, setuptools plugins like that can be in separate files and could even be separately packaged if we want, such as [the setuptools_darcs_plugin](http://pypi.python.org/pypi/setuptools_darcs_plugin) that I recently wrote.

zooko added the

labels 2007-10-15 18:13:53 +00:00

zooko added this to the eventually milestone 2007-10-15 18:13:53 +00:00

zooko commented

2007-10-19 23:12:51 +00:00

I think the Next Step for this one is to pick the next build feature which would look better in Python as a setuptools plugin than it currently looks as a GNUmake target, and replace that one.

zooko added

0.6.1

and removed

0.6.0

labels 2007-10-19 23:12:51 +00:00

warner commented

2008-01-26 00:06:37 +00:00

FYI, it looks like the 'Python Buildutils' project has similar goals:

http://buildutils.lesscode.org/

However I don't see any actual overlap between the commands they offer and
the tasks we perform in our makefile.

FYI, it looks like the 'Python Buildutils' project has similar goals: <http://buildutils.lesscode.org/> However I don't see any actual overlap between the commands they offer and the tasks we perform in our makefile.

zooko commented

2008-01-28 04:28:38 +00:00

I found some other setuptools plugins that might be useful. I wish I'd known about bbfreeze a month ago, before we put so much effort into persuading py2exe. (Of course, bbfreeze might not have been less effort, but it sounds like it might have made some of our issues easier.)

http://mail.python.org/pipermail/distutils-sig/2008-January/008724.html

I found some other setuptools plugins that might be useful. I wish I'd known about bbfreeze a month ago, before we put so much effort into persuading py2exe. (Of course, bbfreeze might not have been less effort, but it sounds like it might have made some of our issues easier.) <http://mail.python.org/pipermail/distutils-sig/2008-January/008724.html>

warner modified the milestone from eventually to undecided

2008-06-01 20:39:52 +00:00

zooko commented

2008-06-02 16:55:29 +00:00

My long-term goal is to make the source:Makefile entirely redundant so that it consists of only a list of targets like this:

install:
	./setup.py install

test:
	./setup.py test

build:
	./setup.py build

deb-feisty:
	./setup.py sdist_deb feisty

etc.

Currently our install.html instructs the user to invoke make check-deps, make, and then make test.

Soon we'll be able to change this doc by removing make check-deps entirely, and by instructing users to run python ./setup.py build and python ./setup.py test. At that point we can remove GNU make from the list of manual dependencies.

My long-term goal is to make the source:Makefile entirely redundant so that it consists of only a list of targets like this: ``` install: ./setup.py install test: ./setup.py test build: ./setup.py build deb-feisty: ./setup.py sdist_deb feisty ``` etc. Currently our [install.html](http://allmydata.org/source/tahoe/trunk/docs/install.html) instructs the user to invoke `make check-deps`, `make`, and then `make test`. Soon we'll be able to change this doc by removing `make check-deps` entirely, and by instructing users to run `python ./setup.py build` and `python ./setup.py test`. At that point we can remove `GNU make` from the list of manual dependencies.

zooko commented

2008-08-06 22:14:43 +00:00

Brian likes to use a makefile as a way to find out which commands the maintainer of the build system chose to offer. (I guess by something like grep ^.*: Makefile.) You can run ./setup.py --help-commands to get a list of commands that ./setup.py knows how to run, but this is answering a different question -- it isn't telling what commands the maintainer of the build system chose to add, because one of the points of the setuptools plugin approach is that the maintainer of the build system for package X doesn't have to do anything to add commands, nor does he even know what commands are available. For example, if someone created a setuptools plugin that runs trial on your packages, and you've installed that plugin on your system, then package_X/setup.py --help-commands will list that command as one of the ones that it can do, even though the maintainer of package X has never heard of it.

Even if I ever get around to implementing this ticket, there may still be a place for a makefile that lists likely targets, per the example above, or else maybe a text README document, or maybe a special ./setup.py --help-tahoe-commands...

Brian likes to use a makefile as a way to find out which commands the maintainer of the build system chose to offer. (I guess by something like `grep ^.*: Makefile`.) You can run `./setup.py --help-commands` to get a list of commands that `./setup.py` knows how to run, but this is answering a different question -- it isn't telling what commands the maintainer of the build system chose to add, because one of the points of the setuptools plugin approach is that the maintainer of the build system for package X doesn't have to do anything to add commands, nor does he even know what commands are available. For example, if someone created a setuptools plugin that runs trial on your packages, and you've installed that plugin on your system, then `package_X/setup.py --help-commands` will list that command as one of the ones that it can do, even though the maintainer of package X has never heard of it. Even if I ever get around to implementing this ticket, there may still be a place for a makefile that lists likely targets, per the example above, or else maybe a text README document, or maybe a special `./setup.py --help-tahoe-commands`...

warner commented

2008-09-12 05:48:07 +00:00

I've just pushed a number of patches that move a lot of functionality from the Makefile into our setup.py, via a number of helper subcommands. This has removed all of the platform-dependent code and pretty much all of the gnarly shell-quoting goop from the Makefile.

zooko commented

2008-09-17 17:17:23 +00:00

Here is a related discussion on tahoe-dev: http://allmydata.org/pipermail/tahoe-dev/2008-September/000799.html

Here is a related discussion on tahoe-dev: <http://allmydata.org/pipermail/tahoe-dev/2008-September/000799.html>

zooko commented

2009-01-23 23:49:16 +00:00

The fact that http://allmydata.org/source/tahoe/trunk/docs/install.html process now works without requiring GNU make and that this is tested by buildbot (#434 (automate testing of installation)) means that I'm satisfied with the status of this ticket.

There is still functionality in Makefile, including make quicktest, the Windows and Mac package building, and more, but let's close this ticket and hopefully in the future people will gradually write more and more in setuptools and less and less in ```GNU make}}].

The fact that <http://allmydata.org/source/tahoe/trunk/docs/install.html> process now works without requiring GNU make and that this is tested by buildbot (#434 (automate testing of installation)) means that I'm satisfied with the status of this ticket. There is still functionality in Makefile, including `make quicktest`, the Windows and Mac package building, and more, but let's close this ticket and hopefully in the future people will gradually write more and more in `setuptools` and less and less in ```GNU make}}].

zooko added the

fixed

label 2009-01-23 23:49:16 +00:00

zooko closed this issue

2009-01-23 23:49:16 +00:00

Sign in to join this conversation.