SandboxViolation: mkdir('/usr/lib/python2.7/site-packages/cryptography/hazmat/bindings/__pycache__', 511) {} #2217

New Issue

tahoe-lafs · 2014-04-11T20:31:28Z

aredridel commented

2014-04-11 20:31:28 +00:00

Searching for Nevow>=0.6.0
Reading <http://pypi.python.org/simple/Nevow/>
Best match: Nevow 0.10.0
Processing Nevow-0.10.0.tar.gz
Running Nevow-0.10.0/setup.py -q bdist_egg --dist-dir /home/users/aredridel/tmp/easy_install-5DXjsN/Nevow-0.10.0/egg-dist-tmp-_Y38Ne
/usr/lib/python2.7/site-packages/zope/interface/interface.py:548: [UserWarning](wiki/UserWarning): Hashing uninitialized [InterfaceClass](wiki/InterfaceClass) instance
error: [SandboxViolation](wiki/SandboxViolation): mkdir('/usr/lib/python2.7/site-packages/cryptography/hazmat/bindings/*pycache*', 511) {}

The package setup script has attempted to modify files on your system
that are not within the [EasyInstall](wiki/EasyInstall) build area, and has been aborted.

This package cannot be safely installed by [EasyInstall](wiki/EasyInstall), and may not
support alternate installation locations even if you run its setup
script by hand.  Please inform the package's author and the [EasyInstall](wiki/EasyInstall)
maintainers to find out if a fix or workaround is available.

``` Searching for Nevow>=0.6.0 Reading <http://pypi.python.org/simple/Nevow/> Best match: Nevow 0.10.0 Processing Nevow-0.10.0.tar.gz Running Nevow-0.10.0/setup.py -q bdist_egg --dist-dir /home/users/aredridel/tmp/easy_install-5DXjsN/Nevow-0.10.0/egg-dist-tmp-_Y38Ne /usr/lib/python2.7/site-packages/zope/interface/interface.py:548: [UserWarning](wiki/UserWarning): Hashing uninitialized [InterfaceClass](wiki/InterfaceClass) instance error: [SandboxViolation](wiki/SandboxViolation): mkdir('/usr/lib/python2.7/site-packages/cryptography/hazmat/bindings/*pycache*', 511) {} The package setup script has attempted to modify files on your system that are not within the [EasyInstall](wiki/EasyInstall) build area, and has been aborted. This package cannot be safely installed by [EasyInstall](wiki/EasyInstall), and may not support alternate installation locations even if you run its setup script by hand. Please inform the package's author and the [EasyInstall](wiki/EasyInstall) maintainers to find out if a fix or workaround is available. ```

tahoe-lafs added the

labels 2014-04-11 20:31:28 +00:00

tahoe-lafs added this to the undecided milestone 2014-04-11 20:31:28 +00:00

zooko commented

2014-04-11 22:06:28 +00:00

Oookay. Let's see… possibly related to http://bugs.python.org/setuptools/issue23 which was ported into Distribute in https://bitbucket.org/tarek/distribute/issue/13 . Your "show-tool-versions" output from a previous bug report ((@@https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2216#comment:-1@@)) indicates that you have "setuptools: [0.6.36 (/usr/share/python2.7/site-packages)]distribute". So, I wonder if distribute 0.6.36 has the patch from https://bitbucket.org/tarek/distribute/issue/13 ?

Oookay. Let's see… possibly related to <http://bugs.python.org/setuptools/issue23> which was ported into Distribute in <https://bitbucket.org/tarek/distribute/issue/13> . Your "show-tool-versions" output from a previous bug report ((@@https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2216#[comment:-1](/tahoe-lafs/trac-2024-07-25/issues/2217#issuecomment--1)@@)) indicates that you have "setuptools: [0.6.36 (/usr/share/python2.7/site-packages)]distribute". So, I wonder if distribute 0.6.36 has the patch from <https://bitbucket.org/tarek/distribute/issue/13> ?

zooko changed title from ~~Build error with Nevow~~ to SandboxViolation: mkdir('/usr/lib/python2.7/site-packages/cryptography/hazmat/bindings/__pycache__', 511) {}

2014-04-11 22:06:28 +00:00

zooko commented

2014-04-11 22:07:45 +00:00

Hm, yeah, according to https://bitbucket.org/tarek/distribute/src/3bf82d0cbc6e0a6d383f177c4def1222f3c3ba46/CHANGES.txt?at=default that should have been fixed as far back as Distribute 0.6. So I guess it isn't exactly that. Maybe we could open a new ticket on Distribute?

Hm, yeah, according to <https://bitbucket.org/tarek/distribute/src/3bf82d0cbc6e0a6d383f177c4def1222f3c3ba46/CHANGES.txt?at=default> that should have been fixed as far back as Distribute 0.6. So I guess it isn't *exactly* that. Maybe we could open a new ticket on Distribute?

zooko commented

2014-04-11 22:33:59 +00:00

Hm, you know, that error message actually seems kind of legit:

"The package setup script has attempted to modify files on your system
that are not within the EasyInstall build area, and has been aborted."

Did you authorize some process to write into your system directories (e.g. /usr/lib/python2.7/site-packages/) when you ran the command that resulted in the above error message? What was that command? Could you please paste in the command and the entire output from it?

Hm, you know, that error message actually seems kind of legit: ``` "The package setup script has attempted to modify files on your system that are not within the EasyInstall build area, and has been aborted." ``` Did you authorize some process to write into your system directories (e.g. `/usr/lib/python2.7/site-packages/`) when you ran the command that resulted in the above error message? What *was* that command? Could you please paste in the command and the entire output from it?

daira commented

2014-04-11 23:47:56 +00:00

Despite appearances, I think this has nothing to do with Nevow; it's just that there isn't a clear boundary in the output between building Nevow, and building cryptography.

daira commented

2014-04-11 23:59:33 +00:00

Ah, possibly the reason why this appears in the Nevow build section is that cryptography is a dependency of pyOpenSSL (>= 0.14), which is a dependency of Twisted, which is a dependency of Nevow.

daira commented

2014-04-12 00:11:59 +00:00

daira: I really feel like we should require pyOpenSSL <= 0.13 until this whole CFFI thing has been more worked out.

okay, in that case we need exactly pyOpenSSL==0.13

because the API to get the OpenSSL version [order to fix #2215]in was only added in 0.13

[...]

note that this line will need to be changed to include pyOpenSSL:

https://tahoe-lafs.org/trac/tahoe-lafs/browser/trunk/setup.py#L134

it should be:

setup_requires += [for req in install_requires if req.startswith('Twisted') or req.startswith('pyOpenSSL') or req.startswith('zope.interface')]req

otherwise setuptools/distribute might attempt to satisfy the setup-time dependency of Nevow->Twisted->pyOpenSSL using pyOpenSSL >= 0.14

<zooko> daira: I really feel like we should require pyOpenSSL <= 0.13 until this whole CFFI thing has been more worked out. <daira> okay, in that case we need exactly pyOpenSSL==0.13 <daira> because the API to get the OpenSSL version [order to fix #2215]in was only added in 0.13 [...] <daira> note that this line will need to be changed to include pyOpenSSL: <daira> <https://tahoe-lafs.org/trac/tahoe-lafs/browser/trunk/setup.py#L134> <daira> it should be: <daira> setup_requires += [for req in install_requires if req.startswith('Twisted') or req.startswith('pyOpenSSL') or req.startswith('zope.interface')]req <daira> otherwise setuptools/distribute might attempt to satisfy the setup-time dependency of Nevow->Twisted->pyOpenSSL using pyOpenSSL >= 0.14

zooko commented

2014-04-12 02:10:02 +00:00

Judging from https://twistedmatrix.com/trac/browser/trunk/setup.py, Twisted doesn't currently depend on pyOpenSSL.

Judging from <https://twistedmatrix.com/trac/browser/trunk/setup.py>, Twisted doesn't currently depend on pyOpenSSL.

daira commented

2014-04-12 10:12:10 +00:00

Replying to zooko:

Judging from https://twistedmatrix.com/trac/browser/trunk/setup.py, Twisted doesn't currently depend on pyOpenSSL

So it doesn't. Then I'm confused: why wasn't there any output between the building of Nevow and the error due to attempting to build cryptography?

Replying to [zooko](/tahoe-lafs/trac-2024-07-25/issues/2217#issuecomment-94798): > Judging from <https://twistedmatrix.com/trac/browser/trunk/setup.py>, Twisted doesn't currently depend on pyOpenSSL So it doesn't. Then I'm confused: why wasn't there any output between the building of Nevow and the error due to attempting to build cryptography?

daira commented

2014-04-14 21:12:13 +00:00

Oh! I bet the thing that is failing is the generation of cffi bindings. But then how does installing (as opposed to building) a package with cffi bindings using setuptools ever work?

Oh! I bet the thing that is failing is the generation of cffi bindings. But then how does installing (as opposed to building) a package with cffi bindings using setuptools *ever* work?

daira commented

2014-04-14 21:14:14 +00:00

Replying to zooko:

Did you authorize some process to write into your system directories (e.g. /usr/lib/python2.7/site-packages/) when you ran the command that resulted in the above error message? What was that command? Could you please paste in the command and the entire output from it?

aredridel, we really need this information in order to progress further with this bug.

Replying to [zooko](/tahoe-lafs/trac-2024-07-25/issues/2217#issuecomment-94794): > Did you authorize some process to write into your system directories (e.g. `/usr/lib/python2.7/site-packages/`) when you ran the command that resulted in the above error message? What *was* that command? Could you please paste in the command and the entire output from it? aredridel, we really need this information in order to progress further with this bug.

aredridel commented

2014-04-14 21:15:26 +00:00

Complete output including command:

:; python setup.py build
running update_version
no version-control data found, leaving _version.py alone
running develop
Not found: tahoe-deps
Not found: ../tahoe-deps
running egg_info
writing requirements to src/allmydata_tahoe.egg-info/requires.txt
writing src/allmydata_tahoe.egg-info/PKG-INFO
writing top-level names to src/allmydata_tahoe.egg-info/top_level.txt
writing dependency_links to src/allmydata_tahoe.egg-info/dependency_links.txt
writing entry points to src/allmydata_tahoe.egg-info/entry_points.txt
writing requirements to src/allmydata_tahoe.egg-info/requires.txt
writing src/allmydata_tahoe.egg-info/PKG-INFO
writing top-level names to src/allmydata_tahoe.egg-info/top_level.txt
writing dependency_links to src/allmydata_tahoe.egg-info/dependency_links.txt
writing entry points to src/allmydata_tahoe.egg-info/entry_points.txt
package init file 'src/allmydata/web/static/__init__.py' not found (or not a regular file)
package init file 'src/allmydata/web/static/css/__init__.py' not found (or not a regular file)
reading manifest file 'src/allmydata_tahoe.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '*~' found anywhere in distribution
writing manifest file 'src/allmydata_tahoe.egg-info/SOURCES.txt'
running build_ext
Processing setuptools-0.6c16dev4.egg
removing '/home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/setuptools-0.6c16dev4.egg' (and everything under it)
Copying setuptools-0.6c16dev4.egg to /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages
setuptools 0.6c16dev4 is already the active version in easy-install.pth
Installing easy_install_z-2.6 script to support/bin
Installing easy_install_z script to support/bin

Installed /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/setuptools-0.6c16dev4.egg
Creating /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/allmydata-tahoe.egg-link (link to src)
allmydata-tahoe 1.10.0 is already the active version in easy-install.pth
Installing tahoe script to support/bin

Installed /home/users/aredridel/allmydata-tahoe-1.10.0/src
Processing dependencies for allmydata-tahoe==1.10.0
Searching for Nevow>=0.6.0
Reading https://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-sdists/
Reading https://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-eggs/
Best match: Nevow 0.10.0
Downloading https://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-eggs/Nevow-0.10.0.tar.bz2
Processing Nevow-0.10.0.tar.bz2
Running Nevow-0.10.0/setup.py -q bdist_egg --dist-dir /home/users/aredridel/tmp/easy_install-XS2kO7/Nevow-0.10.0/egg-dist-tmp-wGVHsQ
/usr/lib/python2.7/site-packages/zope/interface/interface.py:548: UserWarning: Hashing uninitialized InterfaceClass instance
error: SandboxViolation: mkdir('/usr/lib/python2.7/site-packages/cryptography/hazmat/bindings/__pycache__', 511) {}

The package setup script has attempted to modify files on your system
that are not within the EasyInstall build area, and has been aborted.

This package cannot be safely installed by EasyInstall, and may not
support alternate installation locations even if you run its setup
script by hand.  Please inform the package's author and the EasyInstall
maintainers to find out if a fix or workaround is available.

Complete output including command: ``` :; python setup.py build running update_version no version-control data found, leaving _version.py alone running develop Not found: tahoe-deps Not found: ../tahoe-deps running egg_info writing requirements to src/allmydata_tahoe.egg-info/requires.txt writing src/allmydata_tahoe.egg-info/PKG-INFO writing top-level names to src/allmydata_tahoe.egg-info/top_level.txt writing dependency_links to src/allmydata_tahoe.egg-info/dependency_links.txt writing entry points to src/allmydata_tahoe.egg-info/entry_points.txt writing requirements to src/allmydata_tahoe.egg-info/requires.txt writing src/allmydata_tahoe.egg-info/PKG-INFO writing top-level names to src/allmydata_tahoe.egg-info/top_level.txt writing dependency_links to src/allmydata_tahoe.egg-info/dependency_links.txt writing entry points to src/allmydata_tahoe.egg-info/entry_points.txt package init file 'src/allmydata/web/static/__init__.py' not found (or not a regular file) package init file 'src/allmydata/web/static/css/__init__.py' not found (or not a regular file) reading manifest file 'src/allmydata_tahoe.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' warning: no previously-included files matching '*~' found anywhere in distribution writing manifest file 'src/allmydata_tahoe.egg-info/SOURCES.txt' running build_ext Processing setuptools-0.6c16dev4.egg removing '/home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/setuptools-0.6c16dev4.egg' (and everything under it) Copying setuptools-0.6c16dev4.egg to /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages setuptools 0.6c16dev4 is already the active version in easy-install.pth Installing easy_install_z-2.6 script to support/bin Installing easy_install_z script to support/bin Installed /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/setuptools-0.6c16dev4.egg Creating /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/allmydata-tahoe.egg-link (link to src) allmydata-tahoe 1.10.0 is already the active version in easy-install.pth Installing tahoe script to support/bin Installed /home/users/aredridel/allmydata-tahoe-1.10.0/src Processing dependencies for allmydata-tahoe==1.10.0 Searching for Nevow>=0.6.0 Reading https://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-sdists/ Reading https://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-eggs/ Best match: Nevow 0.10.0 Downloading https://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-eggs/Nevow-0.10.0.tar.bz2 Processing Nevow-0.10.0.tar.bz2 Running Nevow-0.10.0/setup.py -q bdist_egg --dist-dir /home/users/aredridel/tmp/easy_install-XS2kO7/Nevow-0.10.0/egg-dist-tmp-wGVHsQ /usr/lib/python2.7/site-packages/zope/interface/interface.py:548: UserWarning: Hashing uninitialized InterfaceClass instance error: SandboxViolation: mkdir('/usr/lib/python2.7/site-packages/cryptography/hazmat/bindings/__pycache__', 511) {} The package setup script has attempted to modify files on your system that are not within the EasyInstall build area, and has been aborted. This package cannot be safely installed by EasyInstall, and may not support alternate installation locations even if you run its setup script by hand. Please inform the package's author and the EasyInstall maintainers to find out if a fix or workaround is available. ```

aredridel commented

2014-04-14 21:15:49 +00:00

This is as an unprivileged user, without write access to the system install locations.

daira commented

2014-04-14 21:21:49 +00:00

Thanks.

daira commented

2014-04-14 21:36:48 +00:00

It's possible that using a more recent version of setuptools at build time would help (#2044).

zooko commented

2014-04-14 21:43:40 +00:00

It has to do with cffi, so a sufficient work-around for this, and probably many other issues, is to limit our dependency on pyOpenSSL to be ≤ 0.13. (Because pyOpenSSL v0.13 didn't use cffi and the cryptography.io library and many other things.)

aredridel: would you be so kind as to try this for us? You'd need to patch your Tahoe-LAFS source distribution by editing source:src/allmydata/_auto_deps.py and change "pyOpenSSL" to "pyOpenSSL <= 0.13".

It has to do with cffi, so a sufficient work-around for this, and probably many other issues, is to limit our dependency on pyOpenSSL to be ≤ 0.13. (Because pyOpenSSL v0.13 didn't use cffi and the cryptography.io library and many other things.) aredridel: would you be so kind as to try this for us? You'd need to patch your Tahoe-LAFS source distribution by editing source:src/allmydata/_auto_deps.py and change `"pyOpenSSL"` to `"pyOpenSSL <= 0.13"`.

aredridel commented

2014-04-14 23:59:12 +00:00

Hm. No dice. I started from a clean unpack of allmydata-tahoe-1.10.0, changed "pyOpenSSL" to "pyOpenSSL <= 0.13" and got this:

:; python setup.py build
Not found: tahoe-deps
Not found: ../tahoe-deps

Installed /home/users/aredridel/allmydata-tahoe-1.10.0/Twisted-12.2.0-py2.7-linux-i686.egg
running update_version
no version-control data found, leaving _version.py alone
running develop
Not found: tahoe-deps
Not found: ../tahoe-deps
running egg_info
writing requirements to src/allmydata_tahoe.egg-info/requires.txt
writing src/allmydata_tahoe.egg-info/PKG-INFO
writing top-level names to src/allmydata_tahoe.egg-info/top_level.txt
writing dependency_links to src/allmydata_tahoe.egg-info/dependency_links.txt
writing entry points to src/allmydata_tahoe.egg-info/entry_points.txt
writing requirements to src/allmydata_tahoe.egg-info/requires.txt
writing src/allmydata_tahoe.egg-info/PKG-INFO
writing top-level names to src/allmydata_tahoe.egg-info/top_level.txt
writing dependency_links to src/allmydata_tahoe.egg-info/dependency_links.txt
writing entry points to src/allmydata_tahoe.egg-info/entry_points.txt
package init file 'src/allmydata/web/static/__init__.py' not found (or not a regular file)
package init file 'src/allmydata/web/static/css/__init__.py' not found (or not a regular file)
reading manifest file 'src/allmydata_tahoe.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '*~' found anywhere in distribution
writing manifest file 'src/allmydata_tahoe.egg-info/SOURCES.txt'
running build_ext
Creating /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/site.py
Processing setuptools-0.6c16dev4.egg
Copying setuptools-0.6c16dev4.egg to /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages
Adding setuptools 0.6c16dev4 to easy-install.pth file
Installing easy_install_z-2.6 script to support/bin
Installing easy_install_z script to support/bin

Installed /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/setuptools-0.6c16dev4.egg
Creating /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/allmydata-tahoe.egg-link (link to src)
Adding allmydata-tahoe 1.10.0 to easy-install.pth file
Installing tahoe script to support/bin

Installed /home/users/aredridel/allmydata-tahoe-1.10.0/src
Processing dependencies for allmydata-tahoe==1.10.0
Searching for pycryptopp>=0.6.0
Reading https://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-sdists/
Reading https://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-eggs/
Best match: pycryptopp 0.6.0.1206569328141510525648634803928199668821045408958
Downloading https://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-eggs/pycryptopp-0.6.0.1206569328141510525648634803928199668821045408958-py2.7-linux-i686.egg
Processing pycryptopp-0.6.0.1206569328141510525648634803928199668821045408958-py2.7-linux-i686.egg
creating /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/pycryptopp-0.6.0.1206569328141510525648634803928199668821045408958-py2.7-linux-i686.egg
Extracting pycryptopp-0.6.0.1206569328141510525648634803928199668821045408958-py2.7-linux-i686.egg to /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages
Adding pycryptopp 0.6.0.1206569328141510525648634803928199668821045408958 to easy-install.pth file

Installed /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/pycryptopp-0.6.0.1206569328141510525648634803928199668821045408958-py2.7-linux-i686.egg
Searching for mock>=0.8.0
Reading http://pypi.python.org/simple/mock/
Best match: mock 1.0.1
Downloading https://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-sdists/mock-1.0.1.tar.gz
Processing mock-1.0.1.tar.gz
Running mock-1.0.1/setup.py -q bdist_egg --dist-dir /home/users/aredridel/tmp/easy_install-UxieV4/mock-1.0.1/egg-dist-tmp-rmnR94
warning: no files found matching '*.png' under directory 'docs'
warning: no files found matching '*.css' under directory 'docs'
warning: no files found matching '*.html' under directory 'docs'
warning: no files found matching '*.js' under directory 'docs'
zip_safe flag not set; analyzing archive contents...
Adding mock 1.0.1 to easy-install.pth file

Installed /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/mock-1.0.1-py2.7.egg
Searching for pyasn1>=0.0.8a
Reading http://pypi.python.org/simple/pyasn1/
Best match: pyasn1 0.1.7
Downloading https://pypi.python.org/packages/2.7/p/pyasn1/pyasn1-0.1.7-py2.7.egg#md5=15f079cabee01402bf86ca8b83356469
Processing pyasn1-0.1.7-py2.7.egg
creating /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/pyasn1-0.1.7-py2.7.egg
Extracting pyasn1-0.1.7-py2.7.egg to /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages
Adding pyasn1 0.1.7 to easy-install.pth file

Installed /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/pyasn1-0.1.7-py2.7.egg
Searching for Nevow>=0.6.0
Reading http://pypi.python.org/simple/Nevow/
Best match: Nevow 0.10.0
Downloading https://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-eggs/Nevow-0.10.0.tar.bz2
Processing Nevow-0.10.0.tar.bz2
Running Nevow-0.10.0/setup.py -q bdist_egg --dist-dir /home/users/aredridel/tmp/easy_install-S4fHff/Nevow-0.10.0/egg-dist-tmp-nPwwPz
/usr/lib/python2.7/site-packages/zope/interface/interface.py:548: UserWarning: Hashing uninitialized InterfaceClass instance
error: SandboxViolation: mkdir('/usr/lib/python2.7/site-packages/cryptography/hazmat/bindings/__pycache__', 511) {}

The package setup script has attempted to modify files on your system
that are not within the EasyInstall build area, and has been aborted.

This package cannot be safely installed by EasyInstall, and may not
support alternate installation locations even if you run its setup
script by hand.  Please inform the package's author and the EasyInstall
maintainers to find out if a fix or workaround is available.

Hm. No dice. I started from a clean unpack of `allmydata-tahoe-1.10.0`, changed `"pyOpenSSL"` to `"pyOpenSSL <= 0.13"` and got this: ``` :; python setup.py build Not found: tahoe-deps Not found: ../tahoe-deps Installed /home/users/aredridel/allmydata-tahoe-1.10.0/Twisted-12.2.0-py2.7-linux-i686.egg running update_version no version-control data found, leaving _version.py alone running develop Not found: tahoe-deps Not found: ../tahoe-deps running egg_info writing requirements to src/allmydata_tahoe.egg-info/requires.txt writing src/allmydata_tahoe.egg-info/PKG-INFO writing top-level names to src/allmydata_tahoe.egg-info/top_level.txt writing dependency_links to src/allmydata_tahoe.egg-info/dependency_links.txt writing entry points to src/allmydata_tahoe.egg-info/entry_points.txt writing requirements to src/allmydata_tahoe.egg-info/requires.txt writing src/allmydata_tahoe.egg-info/PKG-INFO writing top-level names to src/allmydata_tahoe.egg-info/top_level.txt writing dependency_links to src/allmydata_tahoe.egg-info/dependency_links.txt writing entry points to src/allmydata_tahoe.egg-info/entry_points.txt package init file 'src/allmydata/web/static/__init__.py' not found (or not a regular file) package init file 'src/allmydata/web/static/css/__init__.py' not found (or not a regular file) reading manifest file 'src/allmydata_tahoe.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' warning: no previously-included files matching '*~' found anywhere in distribution writing manifest file 'src/allmydata_tahoe.egg-info/SOURCES.txt' running build_ext Creating /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/site.py Processing setuptools-0.6c16dev4.egg Copying setuptools-0.6c16dev4.egg to /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages Adding setuptools 0.6c16dev4 to easy-install.pth file Installing easy_install_z-2.6 script to support/bin Installing easy_install_z script to support/bin Installed /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/setuptools-0.6c16dev4.egg Creating /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/allmydata-tahoe.egg-link (link to src) Adding allmydata-tahoe 1.10.0 to easy-install.pth file Installing tahoe script to support/bin Installed /home/users/aredridel/allmydata-tahoe-1.10.0/src Processing dependencies for allmydata-tahoe==1.10.0 Searching for pycryptopp>=0.6.0 Reading https://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-sdists/ Reading https://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-eggs/ Best match: pycryptopp 0.6.0.1206569328141510525648634803928199668821045408958 Downloading https://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-eggs/pycryptopp-0.6.0.1206569328141510525648634803928199668821045408958-py2.7-linux-i686.egg Processing pycryptopp-0.6.0.1206569328141510525648634803928199668821045408958-py2.7-linux-i686.egg creating /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/pycryptopp-0.6.0.1206569328141510525648634803928199668821045408958-py2.7-linux-i686.egg Extracting pycryptopp-0.6.0.1206569328141510525648634803928199668821045408958-py2.7-linux-i686.egg to /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages Adding pycryptopp 0.6.0.1206569328141510525648634803928199668821045408958 to easy-install.pth file Installed /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/pycryptopp-0.6.0.1206569328141510525648634803928199668821045408958-py2.7-linux-i686.egg Searching for mock>=0.8.0 Reading http://pypi.python.org/simple/mock/ Best match: mock 1.0.1 Downloading https://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-sdists/mock-1.0.1.tar.gz Processing mock-1.0.1.tar.gz Running mock-1.0.1/setup.py -q bdist_egg --dist-dir /home/users/aredridel/tmp/easy_install-UxieV4/mock-1.0.1/egg-dist-tmp-rmnR94 warning: no files found matching '*.png' under directory 'docs' warning: no files found matching '*.css' under directory 'docs' warning: no files found matching '*.html' under directory 'docs' warning: no files found matching '*.js' under directory 'docs' zip_safe flag not set; analyzing archive contents... Adding mock 1.0.1 to easy-install.pth file Installed /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/mock-1.0.1-py2.7.egg Searching for pyasn1>=0.0.8a Reading http://pypi.python.org/simple/pyasn1/ Best match: pyasn1 0.1.7 Downloading https://pypi.python.org/packages/2.7/p/pyasn1/pyasn1-0.1.7-py2.7.egg#md5=15f079cabee01402bf86ca8b83356469 Processing pyasn1-0.1.7-py2.7.egg creating /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/pyasn1-0.1.7-py2.7.egg Extracting pyasn1-0.1.7-py2.7.egg to /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages Adding pyasn1 0.1.7 to easy-install.pth file Installed /home/users/aredridel/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/pyasn1-0.1.7-py2.7.egg Searching for Nevow>=0.6.0 Reading http://pypi.python.org/simple/Nevow/ Best match: Nevow 0.10.0 Downloading https://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-eggs/Nevow-0.10.0.tar.bz2 Processing Nevow-0.10.0.tar.bz2 Running Nevow-0.10.0/setup.py -q bdist_egg --dist-dir /home/users/aredridel/tmp/easy_install-S4fHff/Nevow-0.10.0/egg-dist-tmp-nPwwPz /usr/lib/python2.7/site-packages/zope/interface/interface.py:548: UserWarning: Hashing uninitialized InterfaceClass instance error: SandboxViolation: mkdir('/usr/lib/python2.7/site-packages/cryptography/hazmat/bindings/__pycache__', 511) {} The package setup script has attempted to modify files on your system that are not within the EasyInstall build area, and has been aborted. This package cannot be safely installed by EasyInstall, and may not support alternate installation locations even if you run its setup script by hand. Please inform the package's author and the EasyInstall maintainers to find out if a fix or workaround is available. ```

daira commented

2014-04-15 01:17:45 +00:00

Replying to aredridel:

This is as an unprivileged user, without write access to the system install locations.

Hmm. It's not clear what would have happened had the setuptools sandbox not prevented the access; i.e. whether it would have failed anyway due to lack of OS-level file permissions, or whether cffi would have fallen back to just not using the cache.

Replying to [aredridel](/tahoe-lafs/trac-2024-07-25/issues/2217#issuecomment-94803): > This is as an unprivileged user, without write access to the system install locations. Hmm. It's not clear what would have happened had the setuptools sandbox not prevented the access; i.e. whether it would have failed anyway due to lack of OS-level file permissions, or whether cffi would have fallen back to just not using the cache.

daira commented

2014-04-15 01:19:55 +00:00

The fact that this still happens with pyOpenSSL <= 0.13 as the requirement is really confusing. I don't know what else could be depending on cryptography.

The fact that this still happens with `pyOpenSSL <= 0.13` as the requirement is really confusing. I don't know what else could be depending on `cryptography`.

zooko commented

2014-04-15 12:07:07 +00:00

Yeah, I wonder if aredridel accidentally built a copy that didn't have the pyOpenSSL dependency fixed to <= 0.13? Because other than pyOpenSSL 0.14, I don't think anything else depends on the cryptography.io library.

Yeah, I wonder if aredridel accidentally built a copy that *didn't* have the pyOpenSSL dependency fixed to <= 0.13? Because other than pyOpenSSL 0.14, I don't think anything else depends on the cryptography.io library.

zooko commented

2014-04-15 12:13:56 +00:00

Here is a way to find out what package(s) depend on the cryptography library:

python -c 'import pkg_resources;print pkg_resources.require("pyOpenSSL <= 0.13")'

That should print out all the packages that "pyOpenSSL <= 0.13" depends on. You can also do the same thing for the other packages from [src/allmydata/_auto_deps.py]source:trunk/src/allmydata/_auto_deps.py?annotate=blame&rev=7bb07fb5e28756fa13ba5190e6c39003c84d3e1e, like this:

python -c 'import pkg_resources;print pkg_resources.require("Twisted >= 11.0.0")'

and

python -c 'import pkg_resources;print pkg_resources.require("foolscap >= 0.6.3")'

I'd guess that Twisted and foolscap are two other libraries that might somehow depend on the "cryptography.io" library.

Here is a way to find out what package(s) depend on the cryptography library: ``` python -c 'import pkg_resources;print pkg_resources.require("pyOpenSSL <= 0.13")' ``` That should print out all the packages that `"pyOpenSSL <= 0.13"` depends on. You can also do the same thing for the other packages from [src/allmydata/_auto_deps.py]source:trunk/src/allmydata/_auto_deps.py?annotate=blame&rev=7bb07fb5e28756fa13ba5190e6c39003c84d3e1e, like this: ``` python -c 'import pkg_resources;print pkg_resources.require("Twisted >= 11.0.0")' ``` and ``` python -c 'import pkg_resources;print pkg_resources.require("foolscap >= 0.6.3")' ``` I'd guess that Twisted and foolscap are two other libraries that *might* somehow depend on the "cryptography.io" library.

aredridel commented

2014-04-15 13:29:13 +00:00

:;  python -c 'import pkg_resources;print pkg_resources.require("pyOpenSSL <= 0.13")'
Traceback (most recent call last):
> File "<string>", line 1, in <module>
> File "/usr/share/python2.7/site-packages/pkg_resources.py", line 696, in require
> File "/usr/share/python2.7/site-packages/pkg_resources.py", line 598, in resolve
pkg_resources.VersionConflict: (pyOpenSSL 0.14 (/usr/share/python2.7/site-packages), Requirement.parse('pyOpenSSL<=0.13'))

:;  python -c 'import pkg_resources;print pkg_resources.require("Twisted >= 11.0.0")'
[12.2.0 (/home/users/aredridel/allmydata-tahoe-1.10.0/Twisted-12.2.0-py2.7-linux-i686.egg), zope.interface 4.0.3 (/usr/lib/python2.7/site-packages), distribute 0.6.36 (/usr/share/python2.7/site-packages)]Twisted

aredridel@mizar:allmydata-tahoe-1.10.0
:;  python -c 'import pkg_resources;print pkg_resources.require("foolscap >= 0.6.3")'
Traceback (most recent call last):
> File "<string>", line 1, in <module>
> File "/usr/share/python2.7/site-packages/pkg_resources.py", line 696, in require
> File "/usr/share/python2.7/site-packages/pkg_resources.py", line 594, in resolve
pkg_resources.DistributionNotFound: foolscap>=0.6.3

``` :; python -c 'import pkg_resources;print pkg_resources.require("pyOpenSSL <= 0.13")' Traceback (most recent call last): > File "<string>", line 1, in <module> > File "/usr/share/python2.7/site-packages/pkg_resources.py", line 696, in require > File "/usr/share/python2.7/site-packages/pkg_resources.py", line 598, in resolve pkg_resources.VersionConflict: (pyOpenSSL 0.14 (/usr/share/python2.7/site-packages), Requirement.parse('pyOpenSSL<=0.13')) :; python -c 'import pkg_resources;print pkg_resources.require("Twisted >= 11.0.0")' [12.2.0 (/home/users/aredridel/allmydata-tahoe-1.10.0/Twisted-12.2.0-py2.7-linux-i686.egg), zope.interface 4.0.3 (/usr/lib/python2.7/site-packages), distribute 0.6.36 (/usr/share/python2.7/site-packages)]Twisted aredridel@mizar:allmydata-tahoe-1.10.0 :; python -c 'import pkg_resources;print pkg_resources.require("foolscap >= 0.6.3")' Traceback (most recent call last): > File "<string>", line 1, in <module> > File "/usr/share/python2.7/site-packages/pkg_resources.py", line 696, in require > File "/usr/share/python2.7/site-packages/pkg_resources.py", line 594, in resolve pkg_resources.DistributionNotFound: foolscap>=0.6.3 ```

zooko commented

2014-04-15 15:15:00 +00:00

Replying to aredridel:

pkg_resources.VersionConflict: (pyOpenSSL 0.14 (/usr/share/python2.7/site-packages), Requirement.parse('pyOpenSSL<=0.13'))

Urrrrgh. Okay, this explains it. I think. Actually only partially. The thing is, this appears to be saying that you already have pyOpenSSL 0.14 installed in your system. Okay, fine, that explains why builds of the cryptography.io library/hazmat library are triggered when you build Tahoe-LAFS. Except, actually it totally doesn't because:

Why is it trying to write a cache file into your read-only system-wide filesystem if it is already installed?
Why is it doing anything with pyOpenSSL 0.14 when we've already specified that pyOpenSSL 0.14 is unacceptable for our version requirements?

Sigh, okay, so this suggests a workaround that you can do: remove pyOpenSSL (0.14) from your system and see if that gets past your problem. However, I will feel guilty if we do not report these issues to someone who can do something about them, to help the next user who comes after you. I'm not 100% sure who that would be, though. Who can do something about Python packaging madness? ☹

Replying to [aredridel](/tahoe-lafs/trac-2024-07-25/issues/2217#issuecomment-94812): > pkg_resources.VersionConflict: (pyOpenSSL 0.14 (/usr/share/python2.7/site-packages), Requirement.parse('pyOpenSSL<=0.13')) Urrrrgh. Okay, this explains it. I think. Actually only partially. The thing is, this appears to be saying that you already have pyOpenSSL 0.14 installed in your system. Okay, fine, that explains why builds of the cryptography.io library/hazmat library are triggered when you build Tahoe-LAFS. Except, actually it totally doesn't because: 1. Why is it trying to write a cache file into your read-only system-wide filesystem if it is already installed? 2. Why is it doing *anything* with pyOpenSSL 0.14 when we've already specified that pyOpenSSL 0.14 is unacceptable for our version requirements? Sigh, okay, so this suggests a workaround that you can do: remove pyOpenSSL (0.14) from your system and see if that gets past your problem. However, I will feel guilty if we do not report these issues to *someone* who can do something about them, to help the next user who comes after you. I'm not 100% sure who that would be, though. Who can do something about Python packaging madness? ☹

zooko commented

2014-04-15 16:27:10 +00:00

Okay I asked dstufft and others on IRC and I think I understand answers to these:

Replying to [zooko]comment:23:

Why is it trying to write a cache file into your read-only system-wide filesystem if it is already installed?

I suspect that this is because cffi tries to build hazmat as a side-effect of someone trying to import pyOpenSSL.

Why is it doing anything with pyOpenSSL 0.14 when we've already specified that pyOpenSSL 0.14 is unacceptable for our version requirements?

I suspect that this is because pkg_resources tries to import "OpenSSL" as a side-effect of trying to check what version(s) of pyOpenSSL are available.

aridredel: please go ahead and remove pyOpenSSL 0.14 from your system, and test the patch with "pyOpenSSL <= 0.13". Thank you very much for your help!

Okay I asked dstufft and others on IRC and I think I understand answers to these: Replying to [zooko]comment:23: > > 1. Why is it trying to write a cache file into your read-only system-wide filesystem if it is already installed? I suspect that this is because cffi tries to build hazmat as a side-effect of someone trying to import pyOpenSSL. > 2. Why is it doing *anything* with pyOpenSSL 0.14 when we've already specified that pyOpenSSL 0.14 is unacceptable for our version requirements? I suspect that this is because pkg_resources tries to import "OpenSSL" as a side-effect of trying to check what version(s) of pyOpenSSL are available. aridredel: please go ahead and remove pyOpenSSL 0.14 from your system, and test the patch with `"pyOpenSSL <= 0.13"`. Thank you very much for your help!

zooko commented

2014-04-15 16:36:08 +00:00

Replying to [zooko]comment:24:

Okay I asked dstufft and others on IRC and I think I understand answers to these:

Replying to [zooko]comment:23:

Why is it trying to write a cache file into your read-only system-wide filesystem if it is already installed?

I suspect that this is because cffi tries to build hazmat as a side-effect of someone trying to import pyOpenSSL.

They intend to change cffi for the upcoming cffi v1.0 to stop doing this -- doing "build" stuff automatically at import-time: https://groups.google.com/d/msg/python-cffi/L9ngHcpRJMU/7Y3CrZg-tn0J

Replying to [zooko]comment:24: > Okay I asked dstufft and others on IRC and I think I understand answers to these: > > Replying to [zooko]comment:23: > > > > 1. Why is it trying to write a cache file into your read-only system-wide filesystem if it is already installed? > > I suspect that this is because cffi tries to build hazmat as a side-effect of someone trying to import pyOpenSSL. They intend to change cffi for the upcoming cffi v1.0 to stop doing this -- doing "build" stuff automatically at import-time: <https://groups.google.com/d/msg/python-cffi/L9ngHcpRJMU/7Y3CrZg-tn0J>

aredridel commented

2014-04-15 17:19:22 +00:00

:; sudo rpm -e python-pyOpenSSL-0.14-1.i686 python-TwistedCore-ssl-13.0.0-1.i686

coming up!

``` :; sudo rpm -e python-pyOpenSSL-0.14-1.i686 python-TwistedCore-ssl-13.0.0-1.i686 ``` coming up!

aredridel commented

2014-04-15 17:19:59 +00:00

And Huzzah! Success!

:; make
.
.
.
.
touch .built

And Huzzah! Success! ``` :; make . . . . touch .built ```

aredridel commented

2014-04-15 17:20:47 +00:00

:; bin/tahoe --version
allmydata-tahoe: 1.10.0
foolscap: 0.6.4
pycryptopp: 0.6.0.1206569328141510525648634803928199668821045408958
zfec: 1.4.24
Twisted: 12.2.0
Nevow: 0.10.0
zope.interface: unknown
python: 2.7.6
platform: Linux-PLD_3.0-i686-32bit_ELF
pyOpenSSL: 0.13
simplejson: 3.0.8
pycrypto: 2.6.1
pyasn1: 0.1.7
mock: 1.0.1
setuptools: 0.6c16dev4

``` :; bin/tahoe --version allmydata-tahoe: 1.10.0 foolscap: 0.6.4 pycryptopp: 0.6.0.1206569328141510525648634803928199668821045408958 zfec: 1.4.24 Twisted: 12.2.0 Nevow: 0.10.0 zope.interface: unknown python: 2.7.6 platform: Linux-PLD_3.0-i686-32bit_ELF pyOpenSSL: 0.13 simplejson: 3.0.8 pycrypto: 2.6.1 pyasn1: 0.1.7 mock: 1.0.1 setuptools: 0.6c16dev4 ```

zooko commented

2014-04-15 17:23:35 +00:00

Replying to [zooko]comment:25:

They intend to change cffi for the upcoming cffi v1.0 to stop doing this -- doing "build" stuff automatically at import-time: https://groups.google.com/d/msg/python-cffi/L9ngHcpRJMU/7Y3CrZg-tn0J

Here's a ticket to track the progress of that change to cffi: https://bitbucket.org/cffi/cffi/issue/109/enable-sane-packaging-for-cffi

Replying to [zooko]comment:25: > > They intend to change cffi for the upcoming cffi v1.0 to stop doing this -- doing "build" stuff automatically at import-time: <https://groups.google.com/d/msg/python-cffi/L9ngHcpRJMU/7Y3CrZg-tn0J> Here's a ticket to track the progress of that change to cffi: <https://bitbucket.org/cffi/cffi/issue/109/enable-sane-packaging-for-cffi>

zooko commented

2014-04-15 17:24:14 +00:00

Yay! Thank you for your help.

zooko commented

2014-04-15 17:37:48 +00:00

(https://bitbucket.org/cffi/cffi/issue/149/sandboxviolation-mkdir-usr-lib-python27)

zooko added the

somebody else's problem

label 2014-04-15 17:37:48 +00:00

zooko closed this issue

2014-04-15 17:37:48 +00:00

daira commented

2014-04-16 13:44:23 +00:00

Replying to [zooko]comment:24:

Replying to [zooko]comment:23:

Why is it trying to write a cache file into your read-only system-wide filesystem if it is already installed?

I suspect that this is because cffi tries to build hazmat as a side-effect of someone trying to import pyOpenSSL.

No, the OpenSSL module is not imported at build time. (It would be easy to verify this.) The OpenSSL package is certainly imported by Tahoe-LAFS code at run-time (quite early, in order to check the version). I suspect it's because cffi installs some hook that is invoked when a package depending on it is built. But if that hook attempts to write to system areas of the filesystem, it is just completely broken and cannot work for builds done as an unprivileged user.

Replying to [zooko]comment:24: > Replying to [zooko]comment:23: > > > > 1. Why is it trying to write a cache file into your read-only system-wide filesystem if it is already installed? > > I suspect that this is because cffi tries to build hazmat as a side-effect of someone trying to import pyOpenSSL. No, the `OpenSSL` module is not imported at build time. (It would be easy to verify this.) The `OpenSSL` package is certainly imported by Tahoe-LAFS code at run-time (quite early, in order to check the version). I suspect it's because cffi installs some hook that is invoked when a package depending on it is built. But if that hook attempts to write to system areas of the filesystem, it is just completely broken and *cannot* work for builds done as an unprivileged user.

Sign in to join this conversation.