test-from-egg and test-from-prefixdir are not testing the right code #1137
Labels
No Label
0.2.0
0.3.0
0.4.0
0.5.0
0.5.1
0.6.0
0.6.1
0.7.0
0.8.0
0.9.0
1.0.0
1.1.0
1.10.0
1.10.1
1.10.2
1.10a2
1.11.0
1.12.0
1.12.1
1.13.0
1.14.0
1.15.0
1.15.1
1.2.0
1.3.0
1.4.1
1.5.0
1.6.0
1.6.1
1.7.0
1.7.1
1.7β
1.8.0
1.8.1
1.8.2
1.8.3
1.8β
1.9.0
1.9.0-s3branch
1.9.0a1
1.9.0a2
1.9.0b1
1.9.1
1.9.2
1.9.2a1
LeastAuthority.com automation
blocker
cannot reproduce
cloud-branch
code
code-dirnodes
code-encoding
code-frontend
code-frontend-cli
code-frontend-ftp-sftp
code-frontend-magic-folder
code-frontend-web
code-mutable
code-network
code-nodeadmin
code-peerselection
code-storage
contrib
critical
defect
dev-infrastructure
documentation
duplicate
enhancement
fixed
invalid
major
minor
n/a
normal
operational
packaging
somebody else's problem
supercritical
task
trivial
unknown
was already fixed
website
wontfix
worksforme
No Milestone
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Reference: tahoe-lafs/trac-2024-07-25#1137
Loading…
Reference in New Issue
No description provided.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The test-from-egg and test-from-prefixdir buildbot steps are supposed to be testing the same code that was built, but they may actually test a previous version that is installed in site-packages.
For example,
are supposed to be testing [1.7.1-r4600 on the ticket1074 branch]source:ticket1074@4600, but are actually testing the 1.7.1 release.
I wrote in the original description:
Actually this wouldn't be enough. We're not just supposed to be testing the right version, we're supposed to be testing a specific egg or installation, so that we know that it is usable (not missing any needed files, for example). So we need to:
We should also consider checking constraints on the versions of our dependencies, but I think that's a separate ticket.
Zooko wrote on IRC reformatted:
where source:misc/build_helpers/run_trial.py@4434 is:
The code that is messing up the sys.path is from the
site.py
installed by setuptools. The breakage is essentially the same as described in this message.I don't know what to think about setuptools making a global change to Python's import semantics, that no-one but Phillip Eby (setuptools' maintainer) seems to think is a good idea, every time a setuptools package is installed. It seems like borderline malicious behaviour to me.
In any case, this may break the other places we run a program with a modified PYTHONPATH, in [run-with-pythonpath.py]source:misc/build_helpers/run-with-pythonpath.py@4434 and in the
bin/tahoe
script (generated from source:bin/tahoe-script.template).See also #145.
In [4607/ticket1074]:
In [4609/ticket1074]:
The current ticket1074 branch has a couple of interesting failures on my Mac OS 10.6 "zomp":
Replying to zooko:
Fixed in build 67 (which correctly complains that we're testing the wrong code, and says where it came from):
This was fixed by [4619/ticket1074].
In [4624/ticket798]:
Tests fixed on trunk by changeset:3af6f19cb0f02fb4 and 4683..4687.
Now at least we know when we're testing the wrong code :-)
Replying to davidsarah:
See #1165 for how we can more quickly know that we're testing the wrong code.
Equivalent bugs for pycryptopp and zfec:
Are there any other projects we need this for?
Replying to davidsarah:
There are no other projects that have run-tests-from-egg and run-tests-from-prefix-installdir steps.
Do you know how to fix it to run the right code?
Replying to zooko:
No, but I think I know why this is happening (a bug in the
site.py
installed by setuptools that causes the sys.path entries to be added in the wrong order). I will file a bug against setuptools.If I'm correct, it can't be fixed in the
TestFromEgg
andTestFromPrefixDir
code, because they are corrrectly setting up PYTHONPATH, and the bug is in the sys.path setup in the invoked process.Replying to [davidsarah]comment:14:
That bug was probably not relevant to this ticket.
The fixes to #1190 seem to have fixed this issue as well; at least we're no longer getting any buildslave failures from the tests listed in comment:78886.
Replying to [davidsarah]comment:15:
OTOH, I'm not convinced that the
line in
TestFromEgg
(and similar inTestFromPrefixDir
) are looking for the eggs in the right directory.You were right. I changed the buildmaster config for
TestFromEgg
so now it finds the eggs. I didn't extendTestFromPrefixDir
to do that because the idea ofTestFromPrefixDir
is basically to see how things would work in a world without eggs.source:NEWS entry not needed because this is a buildslave issue (the user-visible bug was #1190).
Okay I just changed the buildmaster so that failing test-from-prefixdir does not cause the build as a whole to be an orange warning (nor a red failure), since test-from-prefixdir is not something that we expect to work (except on systems where the system administrator has manually installed sufficient versions of all deps).
Replying to [davidsarah]comment:15:
Correction: #1246 is such a failure. (Since it has its own ticket, I'm not going to reopen this one.)