clean up after yourself when told to "make clean" #10

New Issue

zooko · 2007-04-27T00:10:19Z

zooko commented

2007-04-27 00:10:19 +00:00

make "./setup.py clean --all" really clean up properly

zooko added the

major

defect

labels 2007-04-27 00:10:19 +00:00

zooko commented

2007-04-27 23:45:18 +00:00

I'd like to add a test to be executed by the buildslave that does "make ; make clean" and then asserts that "darcs whatsnew -s -l" shows no new files.

warner added the

unknown

label 2007-04-28 19:15:20 +00:00

zooko added

minor

and removed

major

labels 2007-04-30 05:32:36 +00:00

zooko commented

2007-05-02 02:51:41 +00:00

#6 merged into here

zooko changed title from ~~clean up after yourself~~ to clean up after yourself when told to "make clean"

2007-05-06 20:37:32 +00:00

zooko added

packaging

and removed

unknown

labels 2007-05-07 00:30:01 +00:00

warner commented

2007-05-24 00:35:36 +00:00

I'll work on this. I noticed that 'make; darcs whatsnew -s -l' shows no new files: i.e., our .darcs-boringfile is doing a great job of ignoring generated files, which makes darcs less-than-useful to detect generated files :-).

I think that I'll build two tests. One will verify that .darcs-boringfile is doing its job, and will pass if 'make; darcs whatsnew -s -l' says 'No changes!'. The second will verify that we clean up after ourselves, and will be something like:

find . -ls |sort >allfiles.tmp.old
make
make clean
find . -ls |grep -v allfiles.tmp |sort >allfiles.tmp.new
diff allfiles.tmp.old allfiles.tmp.new

and see if there are any differences.

I'll work on this. I noticed that 'make; darcs whatsnew -s -l' shows no new files: i.e., our .darcs-boringfile is doing a great job of ignoring generated files, which makes darcs less-than-useful to detect generated files :-). I think that I'll build two tests. One will verify that .darcs-boringfile is doing its job, and will pass if 'make; darcs whatsnew -s -l' says 'No changes!'. The second will verify that we clean up after ourselves, and will be something like: ``` find . -ls |sort >allfiles.tmp.old make make clean find . -ls |grep -v allfiles.tmp |sort >allfiles.tmp.new diff allfiles.tmp.old allfiles.tmp.new ``` and see if there are any differences.

warner commented

2007-05-24 01:17:42 +00:00

I've added these tests, in changeset:319516a5a212cdf3, changeset:13e2af80339eceeb, and changeset:8b9d7d6306137b86. The test-darcs-boringfile test should be passing now, but the test-clean check is failing. The problems fall into the following categories:

src/allmydata/version.py
build-in-place .pyc files for foolscap
src/zfec/ez_setup.pyc
src/zfec/zfec.egg-info

I've added these tests, in changeset:319516a5a212cdf3, changeset:13e2af80339eceeb, and changeset:8b9d7d6306137b86. The test-darcs-boringfile test should be passing now, but the test-clean check is failing. The problems fall into the following categories: * src/allmydata/version.py * build-in-place .pyc files for foolscap * src/zfec/ez_setup.pyc * src/zfec/zfec.egg-info

warner commented

2007-05-30 02:08:25 +00:00

I spent a few minutes trying to figure out why foolscap has .pyc files in place, with no luck. 'make build-foolscap' is sufficient to cause src/foolscap/foolscap/pb.pyc to be created. We're not doing a 'setup.py build -i' anywhere, and I can't find anywhere that might be adding src/foolscap to PYTHONPATH and then importing foolscap modules. I see two copies of pb.pyc created: the expected one in instdir/lib/foolscap/pb.pyc and the unwanted one in src/foolscap/foolscap/pb.pyc. The instdir one is easy to clean up, but the src/foolscap one isn't (I don't want to add a 'find *.pyc -exec rm' statement to the clean target, since I'd much prefer to put all the generated files in one place rather than go traipsing through source directories looking for them).

src/allmydata/version.py is generated by all build targets, and should probably be deleted by the clean target. I have no idea what zfec and eggs are up to.

I spent a few minutes trying to figure out why foolscap has .pyc files in place, with no luck. 'make build-foolscap' is sufficient to cause src/foolscap/foolscap/pb.pyc to be created. We're not doing a 'setup.py build -i' anywhere, and I can't find anywhere that might be adding src/foolscap to PYTHONPATH and then importing foolscap modules. I see two copies of pb.pyc created: the expected one in `instdir/lib/foolscap/pb.pyc` and the unwanted one in `src/foolscap/foolscap/pb.pyc`. The instdir one is easy to clean up, but the src/foolscap one isn't (I don't want to add a 'find *.pyc -exec rm' statement to the clean target, since I'd much prefer to put all the generated files in one place rather than go traipsing through source directories looking for them). src/allmydata/version.py is generated by all build targets, and should probably be deleted by the clean target. I have no idea what zfec and eggs are up to.

zooko commented

2007-07-26 03:33:04 +00:00

Note to self: I really want the buildbot status to be all green when we release v0.5.

This isn't a "blocker" -- it isn't that important -- but I'm putting it in the v0.5 milestone to remind myself when I look at this link tomorrow:

http://allmydata.org/trac/tahoe/query?status=new&status=assigned&status=reopened&groupdesc=1&group=priority&component=code&milestone=0.5.0&order=priority

Note to self: I really want the buildbot status to be all green when we release v0.5. This isn't a "blocker" -- it isn't that important -- but I'm putting it in the v0.5 milestone to remind myself when I look at this link tomorrow: <http://allmydata.org/trac/tahoe/query?status=new&status=assigned&status=reopened&groupdesc=1&group=priority&component=code&milestone=0.5.0&order=priority>

zooko added this to the 0.5.0 milestone 2007-07-26 03:33:04 +00:00

zooko commented

2007-07-26 21:26:56 +00:00

I have asked for help from the setuptools experts at the distutils-sig mailing list.

My current hypothesis is that setuptools ought to be amended to not product the .egg-info directory when the --single-version-externally-managed flag is present.

I have asked for help from the setuptools experts at the distutils-sig mailing list. My current hypothesis is that setuptools ought to be amended to *not* product the `.egg-info` directory when the `--single-version-externally-managed` flag is present.

zooko commented

2007-08-16 21:53:48 +00:00

bumping this to v0.6

zooko modified the milestone from 0.5.0 to 0.6.0

2007-08-16 21:53:48 +00:00

warner commented

2007-08-18 00:06:57 +00:00

With changeset:8088044bf3a7dc5d, our 'make clean' target restores the tree to its original state, and the buildbot 'clean' builder is now green. This uses clean-zfec and clean-setuptools targets that explicitly delete all of the random .pyc and egg.info and setuptools.egg files that are created during their building, which may not be as satisfactory a solution as changing the setup.py files to do this in the 'setup.py clean' target.. I'll leave that decision up to zooko.

zooko modified the milestone from 0.6.0 to 0.6.1

2007-09-19 23:07:20 +00:00

zooko added the

0.6.0

label 2007-10-01 22:13:46 +00:00

warner commented

2007-10-11 22:42:56 +00:00

are we happy with how this works now?

zooko commented

2007-10-13 20:14:01 +00:00

Yep! It is all good. Now, for the record, we no longer have clean-zfec and clean-setuptools targets, but we do explicitly delete random .pyc and such, as per this excerpt from the Makefile:

clean:
	rm -rf build _trial_temp _test_memory .checked-deps .built
	rm -f debian
	rm -f `find src/allmydata -name '*.so' -or -name '*.pyc'`
	rm -rf tahoe_deps.egg-info allmydata_tahoe.egg-info
	rm -rf support dist
	rm -rf setuptools*.egg *.pyc

Yep! It is all good. Now, for the record, we no longer have clean-zfec and clean-setuptools targets, but we do explicitly delete random .pyc and such, as per this excerpt from the Makefile: ``` clean: rm -rf build _trial_temp _test_memory .checked-deps .built rm -f debian rm -f `find src/allmydata -name '*.so' -or -name '*.pyc'` rm -rf tahoe_deps.egg-info allmydata_tahoe.egg-info rm -rf support dist rm -rf setuptools*.egg *.pyc ```

zooko added the

fixed

label 2007-10-13 20:14:01 +00:00

zooko closed this issue

2007-10-13 20:14:01 +00:00

Sign in to join this conversation.