"tahoe --version" shouldn't show scary diagnostic warnings #2436

Closed
opened 2015-05-27 21:14:16 +00:00 by warner · 13 comments

It is very common for tahoe --version to report diagnostics warnings about the difference between what pkg_resources claims to provide, and what the actual imported modules (and their *version*) reports. These warnings have convinced several users that their tahoe installation has failed, when in fact everything will would work normally apart from those warnings.

Zooko and I agreed on IRC to change tahoe --version to report the versions of Tahoe itself and its dependencies (foolscap, pycryptopp, etc), but to remove the import-vs-pkg_resources warnings. We'll add a separate tahoe debug versions command that emits complete diagnostics.

For reference, this is what the problematic output looks like:

% bin/tahoe -V
allmydata-tahoe: 1.10.0
foolscap: 0.8.0
pycryptopp: 0.6.0.1206569328141510525648634803928199668821045408958
zfec: 1.4.24
Twisted: 13.0.0
Nevow: 0.11.1
zope.interface: unknown
python: 2.7.9
platform: Linux-fedora_22-x86_64-64bit_ELF
pyOpenSSL: 0.14
simplejson: 3.7.2
pycrypto: 2.6.1
pyasn1: 0.1.7
mock: 1.0.1
setuptools: 0.6c16dev4

Warning: dependency 'pycparser' (version '2.10') found by pkg_resources not found by import.
Warning: dependency 'six' (version '1.9.0') found by pkg_resources not found by import.
Warning: dependency 'cryptography' (version '0.8.2') found by pkg_resources not found by import.
Warning: dependency 'cffi' (version '0.8.6') found by pkg_resources not found by import.
Warning: dependency 'pyasn1-modules' (version '0.0.5') found by pkg_resources not found by import.
Warning: dependency 'enum34' (version '1.0') found by pkg_resources not found by import.
Warning: dependency 'characteristic' (version '14.3.0') found by pkg_resources not found by import.
Warning: dependency 'service-identity' (version '14.0.0') found by pkg_resources not found by import.

For debugging purposes, the PYTHONPATH was
  '/home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages'
install_requires was
  ['setuptools >= 0.6c6', 'zfec >= 1.1.0', 'simplejson >= 1.4', 'zope.interface == 3.6.0, == 3.6.1, == 3.6.2, >= 3.6.5', 'Twisted >= 11.0.0', 'foolscap >= 0.6.3', 'pyOpenSSL', 'Nevow >= 0.6.0', 'pycrypto == 2.1.0, == 2.3, >= 2.4.1', 'pyasn1 >= 0.0.8a', 'mock >= 0.8.0', 'pycryptopp >= 0.6.0']
sys.path after importing pkg_resources was
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/bin:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/setuptools-0.6c16dev4.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/src:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/pycryptopp-0.6.0.1206569328141510525648634803928199668821045408958-py2.7-linux-x86_64.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/mock-1.0.1-py2.7.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/Nevow-0.11.1-py2.7.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/foolscap-0.8.0-py2.7.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/zfec-1.4.24-py2.7-linux-x86_64.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/service_identity-14.0.0-py2.7.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/pyutil-2.0.0-py2.7.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/pyasn1_modules-0.0.5-py2.7.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/characteristic-14.3.0-py2.7.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/zbase32-1.1.5-py2.7.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/Twisted-13.0.0-py2.7-linux-x86_64.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/zope.interface-4.1.2-py2.7-linux-x86_64.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages:
  /usr/lib64/python27.zip:
  /usr/lib64/python2.7:
  /usr/lib64/python2.7/plat-linux2:
  /usr/lib64/python2.7/lib-tk:
  /usr/lib64/python2.7/lib-old:
  /usr/lib64/python2.7/lib-dynload:
  /usr/lib64/python2.7/site-packages:
  /usr/lib64/python2.7/site-packages/gst-0.10:
  /usr/lib64/python2.7/site-packages/gtk-2.0:
  /usr/lib/python2.7/site-packages

The output is large enough that, depending upon the size of your screen, you may wind up with the scary "Warning:... not found by import" lines at the top of the screen, which sounds an awful lot like an ImportError.

It is very common for `tahoe --version` to report diagnostics warnings about the difference between what `pkg_resources` claims to provide, and what the actual imported modules (and their `*version*`) reports. These warnings have convinced several users that their tahoe installation has failed, when in fact everything will would work normally apart from those warnings. Zooko and I agreed on IRC to change `tahoe --version` to report the versions of Tahoe itself and its dependencies (foolscap, pycryptopp, etc), but to remove the import-vs-pkg_resources warnings. We'll add a separate `tahoe debug versions` command that emits complete diagnostics. For reference, this is what the problematic output looks like: ``` % bin/tahoe -V allmydata-tahoe: 1.10.0 foolscap: 0.8.0 pycryptopp: 0.6.0.1206569328141510525648634803928199668821045408958 zfec: 1.4.24 Twisted: 13.0.0 Nevow: 0.11.1 zope.interface: unknown python: 2.7.9 platform: Linux-fedora_22-x86_64-64bit_ELF pyOpenSSL: 0.14 simplejson: 3.7.2 pycrypto: 2.6.1 pyasn1: 0.1.7 mock: 1.0.1 setuptools: 0.6c16dev4 Warning: dependency 'pycparser' (version '2.10') found by pkg_resources not found by import. Warning: dependency 'six' (version '1.9.0') found by pkg_resources not found by import. Warning: dependency 'cryptography' (version '0.8.2') found by pkg_resources not found by import. Warning: dependency 'cffi' (version '0.8.6') found by pkg_resources not found by import. Warning: dependency 'pyasn1-modules' (version '0.0.5') found by pkg_resources not found by import. Warning: dependency 'enum34' (version '1.0') found by pkg_resources not found by import. Warning: dependency 'characteristic' (version '14.3.0') found by pkg_resources not found by import. Warning: dependency 'service-identity' (version '14.0.0') found by pkg_resources not found by import. For debugging purposes, the PYTHONPATH was '/home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages' install_requires was ['setuptools >= 0.6c6', 'zfec >= 1.1.0', 'simplejson >= 1.4', 'zope.interface == 3.6.0, == 3.6.1, == 3.6.2, >= 3.6.5', 'Twisted >= 11.0.0', 'foolscap >= 0.6.3', 'pyOpenSSL', 'Nevow >= 0.6.0', 'pycrypto == 2.1.0, == 2.3, >= 2.4.1', 'pyasn1 >= 0.0.8a', 'mock >= 0.8.0', 'pycryptopp >= 0.6.0'] sys.path after importing pkg_resources was /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/bin: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/setuptools-0.6c16dev4.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/src: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/pycryptopp-0.6.0.1206569328141510525648634803928199668821045408958-py2.7-linux-x86_64.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/mock-1.0.1-py2.7.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/Nevow-0.11.1-py2.7.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/foolscap-0.8.0-py2.7.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/zfec-1.4.24-py2.7-linux-x86_64.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/service_identity-14.0.0-py2.7.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/pyutil-2.0.0-py2.7.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/pyasn1_modules-0.0.5-py2.7.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/characteristic-14.3.0-py2.7.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/zbase32-1.1.5-py2.7.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/Twisted-13.0.0-py2.7-linux-x86_64.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/zope.interface-4.1.2-py2.7-linux-x86_64.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages: /usr/lib64/python27.zip: /usr/lib64/python2.7: /usr/lib64/python2.7/plat-linux2: /usr/lib64/python2.7/lib-tk: /usr/lib64/python2.7/lib-old: /usr/lib64/python2.7/lib-dynload: /usr/lib64/python2.7/site-packages: /usr/lib64/python2.7/site-packages/gst-0.10: /usr/lib64/python2.7/site-packages/gtk-2.0: /usr/lib/python2.7/site-packages ``` The output is large enough that, depending upon the size of your screen, you may wind up with the scary "Warning:... not found by import" lines at the top of the screen, which sounds an awful lot like an ImportError.
warner added the
code
normal
defect
1.10.0
labels 2015-05-27 21:14:16 +00:00
warner added this to the 1.10.1 milestone 2015-05-27 21:14:16 +00:00

On IRC Daira said that she thinks all of the warnings shown in the user-supplied output above have been fixed in trunk. I'll ask the user to try the 10.1a1post9 tarball.

On IRC Daira said that she thinks all of the warnings shown in the user-supplied output above have been fixed in trunk. I'll ask the user to try the 10.1a1post9 tarball.

(https://twitter.com/olabini/status/604300869832597504)

"the printing of sys.path REALLY looks like a stack trace btw."

I think a lesson here is never to show output to a user that looks like a stack trace, i.e. has a list of lines where the beginning of each line is a filename/pathname. I've seen this before: if a user sees something that looks like a stack trace, they immediately avert their gaze and assume that the software is malfunctioning.

(https://twitter.com/olabini/status/604300869832597504) "the printing of sys.path REALLY looks like a stack trace btw." I think a lesson here is never to show output to a user that looks like a stack trace, i.e. has a list of lines where the beginning of each line is a filename/pathname. I've seen this before: if a user sees something that looks like a stack trace, they immediately avert their gaze and assume that the software is malfunctioning.
Author

What if tahoe --version emitted just the versions and a single line that said "warning: not all dependency versions matched expectations. Run 'tahoe debug versions' for more info." ? And then we move all that other (scary) stuff to the other command?

What if `tahoe --version` emitted just the versions and a single line that said "warning: not all dependency versions matched expectations. Run 'tahoe debug versions' for more info." ? And then we move all that other (scary) stuff to the other command?
Author

Moving this out of 1.10.1, insufficient consensus to make a last-minute change.

Moving this out of 1.10.1, insufficient consensus to make a last-minute change.
warner modified the milestone from 1.10.1 to 1.11.0 2015-06-01 05:34:22 +00:00
tahoe-lafs modified the milestone from 1.11.0 to 1.10.2 2015-07-17 21:41:02 +00:00
daira commented 2015-07-17 21:43:06 +00:00
Owner

The issue I have with not displaying a warning is that the printed versions may not be correct. However, I'm okay with printing a short warning on the line that displays each offending version (in the "comment" field), and suppressing the long debugging information in cases where that is the only problem.

The issue I have with not displaying a warning is that the printed versions *may not be correct*. However, I'm okay with printing a short warning on the line that displays each offending version (in the "comment" field), and suppressing the long debugging information in cases where that is the only problem.
daira commented 2015-07-17 21:49:10 +00:00
Owner

So in the example in the description, for instance, the output would be:

allmydata-tahoe: 1.10.0
foolscap: 0.8.0
pycryptopp: 0.6.0.1206569328141510525648634803928199668821045408958
zfec: 1.4.24
Twisted: 13.0.0
Nevow: 0.11.1
zope.interface: unknown
python: 2.7.9
platform: Linux-fedora_22-x86_64-64bit_ELF
pyOpenSSL: 0.14
simplejson: 3.7.2
pycrypto: 2.6.1
pyasn1: 0.1.7
mock: 1.0.1
setuptools: 0.6c16dev4
pycparser: 2.10 [according to pkg_resources]
six: 1.9.0 [according to pkg_resources]
cryptography: 0.8.2 [according to pkg_resources]
cffi: 0.8.6 [according to pkg_resources]
pyasn1-modules: 0.0.5 [according to pkg_resources]
enum34: 1.0 [according to pkg_resources]
characteristic: 14.3.0 [according to pkg_resources]
service-identity: 14.0.0 [according to pkg_resources]
So in the example in the description, for instance, the output would be: ``` allmydata-tahoe: 1.10.0 foolscap: 0.8.0 pycryptopp: 0.6.0.1206569328141510525648634803928199668821045408958 zfec: 1.4.24 Twisted: 13.0.0 Nevow: 0.11.1 zope.interface: unknown python: 2.7.9 platform: Linux-fedora_22-x86_64-64bit_ELF pyOpenSSL: 0.14 simplejson: 3.7.2 pycrypto: 2.6.1 pyasn1: 0.1.7 mock: 1.0.1 setuptools: 0.6c16dev4 pycparser: 2.10 [according to pkg_resources] six: 1.9.0 [according to pkg_resources] cryptography: 0.8.2 [according to pkg_resources] cffi: 0.8.6 [according to pkg_resources] pyasn1-modules: 0.0.5 [according to pkg_resources] enum34: 1.0 [according to pkg_resources] characteristic: 14.3.0 [according to pkg_resources] service-identity: 14.0.0 [according to pkg_resources] ```
tahoe-lafs added
packaging
and removed
code
labels 2015-07-17 21:51:23 +00:00
warner was assigned by tahoe-lafs 2015-07-17 21:51:23 +00:00
Author

That sounds like a good compromise to me. I think I'll use () instead of [].

That sounds like a good compromise to me. I think I'll use `()` instead of `[]`.
daira commented 2015-07-18 01:23:32 +00:00
Owner

[] is already used for comments; () is used for paths when --version-and-path is given. I want the output to remain consistently parsable.

`[]` is already used for comments; `()` is used for paths when `--version-and-path` is given. I want the output to remain consistently parsable.
daira commented 2015-07-21 17:57:00 +00:00
Owner
Review needed for <https://github.com/tahoe-lafs/tahoe-lafs/pull/176>.
warner was unassigned by zooko 2015-07-21 17:57:55 +00:00
zooko self-assigned this 2015-07-21 17:57:55 +00:00
Daira Hopwood <daira@jacaranda.org> commented 2015-07-21 17:59:34 +00:00
Owner

In /tahoe-lafs/trac-2024-07-25/commit/431728f8f854e02d9ab2f731675f12ce18cda122:

Don't show scary diagnostic warnings from --version[-and-path]
if the only problem is that extra packages are found by pkg_resources.
fixes ticket:2436

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
In [/tahoe-lafs/trac-2024-07-25/commit/431728f8f854e02d9ab2f731675f12ce18cda122](/tahoe-lafs/trac-2024-07-25/commit/431728f8f854e02d9ab2f731675f12ce18cda122): ``` Don't show scary diagnostic warnings from --version[-and-path] if the only problem is that extra packages are found by pkg_resources. fixes ticket:2436 Signed-off-by: Daira Hopwood <daira@jacaranda.org> ```
tahoe-lafs added the
fixed
label 2015-07-21 17:59:34 +00:00
daira commented 2015-07-21 19:46:56 +00:00
Owner

Oops, I just spotted a bug. get_package_versions_string extends _vers_and_locs_list each time it is called (assuming not hasattr(sys, 'frozen')), but it may be called more than once.

I will fix it.

Oops, I just spotted a bug. `get_package_versions_string` extends `_vers_and_locs_list` each time it is called (assuming `not hasattr(sys, 'frozen')`), but it may be called more than once. I will fix it.
tahoe-lafs removed the
fixed
label 2015-07-21 19:46:56 +00:00
Daira Hopwood <daira@jacaranda.org> commented 2015-07-29 00:40:35 +00:00
Owner

In /tahoe-lafs/trac-2024-07-25/commit/2c99294a4faa0e0f94fdb53ed60954e1e0724955:

Don't show scary diagnostic warnings from --version[-and-path] (corrected). refs ticket:2436

The previous version would incorrectly add to the output of
get_package_versions_string each time it was called.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
In [/tahoe-lafs/trac-2024-07-25/commit/2c99294a4faa0e0f94fdb53ed60954e1e0724955](/tahoe-lafs/trac-2024-07-25/commit/2c99294a4faa0e0f94fdb53ed60954e1e0724955): ``` Don't show scary diagnostic warnings from --version[-and-path] (corrected). refs ticket:2436 The previous version would incorrectly add to the output of get_package_versions_string each time it was called. Signed-off-by: Daira Hopwood <daira@jacaranda.org> ```
Author

Landed in 28399b6.

Landed in 28399b6.
warner added the
fixed
label 2015-07-29 00:41:33 +00:00
Sign in to join this conversation.
No Milestone
No Assignees
3 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Reference: tahoe-lafs/trac-2024-07-25#2436
No description provided.