I don't like pyopenssl #11
Labels
No Label
0.2.0
0.3.0
0.4.0
0.5.0
0.5.1
0.6.0
0.6.1
0.7.0
0.8.0
0.9.0
1.0.0
1.1.0
1.10.0
1.10.1
1.10.2
1.10a2
1.11.0
1.12.0
1.12.1
1.13.0
1.14.0
1.15.0
1.15.1
1.2.0
1.3.0
1.4.1
1.5.0
1.6.0
1.6.1
1.7.0
1.7.1
1.7β
1.8.0
1.8.1
1.8.2
1.8.3
1.8β
1.9.0
1.9.0-s3branch
1.9.0a1
1.9.0a2
1.9.0b1
1.9.1
1.9.2
1.9.2a1
LeastAuthority.com automation
blocker
cannot reproduce
cloud-branch
code
code-dirnodes
code-encoding
code-frontend
code-frontend-cli
code-frontend-ftp-sftp
code-frontend-magic-folder
code-frontend-web
code-mutable
code-network
code-nodeadmin
code-peerselection
code-storage
contrib
critical
defect
dev-infrastructure
documentation
duplicate
enhancement
fixed
invalid
major
minor
n/a
normal
operational
packaging
somebody else's problem
supercritical
task
trivial
unknown
was already fixed
website
wontfix
worksforme
No Milestone
No Assignees
3 Participants
Notifications
Due Date
No due date set.
Reference: tahoe-lafs/trac-2024-07-25#11
Loading…
Reference in New Issue
Block a user
No description provided.
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
There is no known way (to me) way to get pyopenssl running on Windows without finding a binary that somebody has uploaded somewhere (i.e., not an actual supported package on an official project's web site or anything). For example, right now the best way for our Windows users to get pyOpenSSL is to get this binary:
http://allmydata.org/source/pyOpenSSL-0.6.win32-py2.5
Which I copied from webcleaner.sf.net.
I wouldn't know how to rebuild this if we needed to, for example if someone discovered a critical security hole in it.
Also, pyopenssl's web site hasn't been updated since 2004.
Also, it apparently uses lots of memory for secure connections, although Brian might want to add more accurate and precise notes about that issue.
All in all, I would be really happy to find a well maintained, easy to compile alternative.
The guy who made the tracdarcs plugin work is K. S. Sreeram. I was idly
looking for alternate python crypto modules when I found ncrypt by K. S.
Sreeram. Then I saw that ncrypt is sponsored by a p2p company, tachyon.in,
which also makes a secure decentralized (?) instant messaging protocol:
It has a very nice straightforward explanation up front:
http://cspace.in/
I was thinking that it might be a nice optional underlay protocol for Foolscap.
Unfortunately it is GPL'ed, so it is a non-starter for Allmydata unless
tachyon.in wants to give us a more permissive licence.
But the OpenSSL Python wrappers that they wrote are permissively licensed:
http://tachyon.in/ncrypt/
And it works well on Windows:
http://tachyon.in/pipermail/ncrypt-users/2007-February/000016.html
Here's K. S. Sreeram's page:
http://sreeram.cc/
So all Python crypto libraries that I know of that do TLS and that have
compatible licences:
I vaguely remember that Brian Warner investigated tlslite and had trouble with
it. I've heard bad things about M2Crypto. I would be interested in trying
ncrypt.
By the way, I was reminded while doing this browsing that we need to add "the
OpenSSL+GPL exception" to our licence.
http://en.wikipedia.org/wiki/OpenSSL#The_exception
I don't like pycrypto either. The official version's ctr mode is too slow, and AMK didn't accept my patch to speed it up. I never received a rejection letter either.
Actually, I do kind of like pycrypto, but it doesn't do TLS, and so we can probably replace both of our libraries -- pyOpenSSL and pycrypto -- with one which provides all the crypto functions we need and which is also portable and maintained and so on.
I don't like pyopensslto I don't like pyopenssl. or pycryptoWe've added the OpenSSL exception to our licence.
There appear to be people hacking on pycrypto other than AMK, as discovered by launchpad. I reported bugs on launchpad.
https://bugs.launchpad.net/pycrypto/+bug/112423
https://bugs.launchpad.net/pycrypto/+bug/112422
fwiw the allmydata.com 'ext' repository has a py24 native build of pyopenssl which was also found from trawling the web
(which I suspect means that we don't have a py25 requirement on windows because of this dependency)
See also:
http://twistedmatrix.com/trac/ticket/2293
The Twisted folks are planning to fork pyOpenSSL.
You know, we could probably ditch pycrypto altogether if we just copied implementations of AES-CTR and SHA-256 into our tree. We'll want RSA sooner or later but that can't be all that big. We certainly don't need any of the other block ciphers or hash algorithms that pycrypto offers.
And if we required python2.5 (which I'm !!!not!!! advocating) then it comes with SHA-256 in the batteries-included 'hashlib' module..
I've copied AES-CTR and SHA-256 into our tree, and removed pycrypto (and src/Crypto) altogether.
I also copied RSA in there too, but I've disabled it in setup.py because it requires the GMP package, and I don't want to add another dependency to tahoe until we actually need it (say, when we need RSA for distributed dirnodes and SSK files).
Our new mutable file design (source:docs/mutable.txt) requires public key cryptography.
The aforementioned (comment:59446) thing about someone maintaing pycrypto hasn't panned out -- nobody responded to the patches I submitted.
Oh boy, and now I see that the current version of pycrypto -- v2.0.1 -- has another bug which causes SHA-256 to give incorrect results:
http://sourceforge.net/tracker/index.php?func=detail&aid=1740198&group_id=20937&atid=120937
This bug report and accompanying patch has been open since June. This is another demonstration that pycrypto is unmaintained.
This also raises the question: why are we copying our hash function code from pycrypto ? Let's copy hashlib from python 2.5 instead.
Likewise, I'm a bit reluctant to depend on the RSA implementation from pycrypto.
I would be delighted if someone would make a Python wrapper around Crypto++.
I might try it myself.
Crypto++ has the following features:
It has the following drawback:
There are so many ways to make Python wrappers nowadays:
Apparently the OLPC project has created Python wrappers around libtomcrypt (which is the upstream source for both the pycrypto sha256 and the Python standard library hashlib sha256), but hasn't really packaged or publicized these wrappers:
http://wiki.laptop.org/go/Software_projects/lang-ja#pyltc
Also some person named Larry contributed incomplete python wrappers for libtomcrypt in March of this year:
http://libtom.org/?page=index&newsitems=10000&whatfile=crypt
The OLPC wrapper code is here. Any idea what
the license is?
Also, it looks like they've got ECC wrappers..
Oh and just for completeness, there is also a 5th way to wrap C++ code in Python code -- boost.python. Truly, we enjoy an abundance of ways to wrap C/C++ in Python...
Oh and just for more complete completeness, there is also cython.
So that's seven Ways To Do It.
But I'm using the hand-rolled technique, as per
http://allmydata.org/pipermail/tahoe-dev/2007-October/000215.html
See also ticket #199.
Itamar pointed out that M2crypto has been integrated with Twisted, but on the other hand, Guido van Rossum had bad experiences with M2crypto:
http://www.artima.com/forums/flat.jsp?forum=106&thread=95863
Removing "pycrypto" from the subject line of this ticket, since we have removed our dependency on it by switching to pycryptopp. (This fix isn't committed to trunk yet, but I want to point people at this ticket who are interested in pyOpenSSL specifically.)
I don't like pyopenssl. or pycryptoto I don't like pyopensslReplying to zooko:
I'd like to point out that Guido wrote that almost three years ago! As far as I know, all the issues he experienced were fixed long time ago.
Itamar is also correct: M2Crypto has a Twisted protocol wrapper, which can be used to do SSL instead of pyOpenSSL. It was modeled after similar thing in TLS Lite. We use Twisted in Chandler, and M2Crypto does the SSL part using this wrapper. More on Chandler at http://chandlerproject.org.
If you try M2Crypto and run into any issues, please let me know. I can't fix issues I don't know about. You can find the mailing list and bugzilla info on the M2Crypto homepage at http://chandlerproject.org/Projects/MeTooCrypto.
Heikki Toivonen - M2Crypto maintainer
Heikki:
Thanks for the post. If M2Crypto is actively maintained, then this is a big advantage that it has over pyOpenSSL!
We could use M2Crypto for both our SSL needs and our filesystem crypto needs, thus removing the need for pyOpenSSL and removing the need for pycryptopp. (This latter part makes me a little sad because I like pycryptopp -- it is my newest baby.)
Brian: are you interested in using M2Crypto for foolscap's SSL layer?
[http://yassl.com/]yassl is an implementation of SSL written on top of Crypto++. It is licensed under the traditional dual-licence model of "You can use it under the GPL, or you can ask us for a special licence.". I have already asked them for a special licence, and I'm waiting to hear back from them.
Heikki Toivonen, the M2Crypto maintainer, posted the following comment in a discussion: "I suspect most
people use something more robust than plain M2Crypto for a server
application (like Apache or Twisted)."
This makes me think that M2Crypto might not be best for allmydata.org.
The aforementioned discussion leads to two further options:
yet another openssl wrapper
This is a backport of the SSL implementation that is intended to be standard in Python >= v2.6.
Also, allmydata.com has decided that it would be okay to go ahead and use GPL'ed source code such as yassl, so that opens up some more options.
Oh, by the way, I forgot to mention that the reason I am looking at this ticket is that I can't compile pyOpenSSL 0.6 on Mac OS X against openssl-0.9.8g. There is a callback defined in openssl that takes (const SSL*, int, int), and pyOpenSSL passes a function that takes (SSL*, int, int). Patching pyOpenSSL to add the const keyword makes the gcc warning/error go away, but when I try to import it I still get:
Replying to zooko:
It really depends on what you need. If your website is not a high traffic site, M2Crypto is probably ok. But if you need traffic shaping, load balancing, guaranteed high availability etc. then I believe none of the simple libraries will be robust enough for you. That is why I mentioned Apache etc. which certainly can handle high traffic sites.
FYI, I started playing with a port of Foolscap to M2Crypto. The Twisted interface seems pretty well implemented, but the way that you get access to the certificate (and the way you control validation) is pretty different, so I haven't gotten it working yet. When I get back next week I'll post my results and maybe beg Heikki for some help :).
Most of what we need is just reactor.connectTCP, transport.startTLS(), and access to the certificate so we can validate it according to Foolscap's rules instead of the whole CA business (basically: the hash of the cert must match the server you were intending to connect to, no more, no less). My biggest concern about pyopenssl is an apparent memory consumption problem (something like 100kB to 1MB per open connection, although I haven't re-run those tests in a few years). If M2Crypto has comparable speed to pyopenssl, and it isn't too hard for users to obtain/build/install, then I'll seriously consider moving foolscap over to it.
Replying to warner:
M2Crypto is used in Chandler, and since the certificates are stored in a database in Chandler, the validation is a bit different. Also, Chandler will present the user with a dialog if there are errors in the SSL connection (the users can choose to ignore these), so this presents additional challenges for the SSL implementation. You can see how it is done in here:
http://svn.osafoundation.org/chandler/trunk/chandler/parcels/osaf/framework/certstore/ssl.py
Rejoice -- exarkun and bigdog are working on pyOpenSSL. That makes the path of least resistance for us (continuing to use pyOpenSSL) also be the path of future promise, since exarkun is an excellent engineer.
binary builds from exarkun and company:
http://buildbot.twistedmatrix.com/builds/
Okay, I like
pyOpenSSL
now since exarkun et alia are maintaining it. Closing as, um, "fixed".Re-opening this since I get unit test failures when I try to use the current pyOpenSSL-0.7 with Tahoe, and different unit test failures when I run pyOpenSSL-0.7's own unit tests. Here's the bug report for the pyOpenSSL project:
https://sourceforge.net/tracker/index.php?func=detail&aid=1978729&group_id=31249&atid=401758
Another reason why I still don't like pyOpenSSL, and therefore this ticket should remain open, is ticket #402 (bug in Twisted, triggered by pyOpenSSL-0.7).
Okay, now I like pyOpenSSL. See also #456 (it would be nice if the dependency on OpenSSL could be automatically resolved), but basically ticket #11 can finally be closed, thanks to JP Calderone and bigdog's stewardship of pyOpenSSL.
Updating Launchpad bug reference