os.path.expanduser (and therefore abspath_expanduser_unicode) fails on Windows if username or home directory contains non-ASCII characters #1674

New Issue

tahoe-lafs · 2012-02-14T10:02:35Z

vikarti commented

2012-02-14 10:02:35 +00:00

attempt to perform setup.py test results in

argparse 0.9.1 is already the active version in easy-install.pth                                                                                                                                                16:34

Using c:\tahoe\allmydata-tahoe-1.9.1\support\lib\site-packages\argparse-0.9.1-py2.7-win32.egg
Searching for zbase32==1.1.3
Best match: zbase32 1.1.3
Processing zbase32-1.1.3-py2.7.egg
zbase32 1.1.3 is already the active version in easy-install.pth

Using c:\tahoe\allmydata-tahoe-1.9.1\support\lib\site-packages\zbase32-1.1.3-py2.7.egg
Finished processing dependencies for allmydata-tahoe==1.9.1

Setting up environment to run scripts for the current user...
Already done: associate the filetype .pyscript with Python.File for the current user.
Already done: add .pyscript to the user environment variable PATHEXT.
Already done: add .pyw to the user environment variable PATHEXT.
running make_executable
running build
running build_py

Setting up environment to run scripts for the current user...
Already done: associate the filetype .pyscript with Python.File for the current user.
Already done: add .pyscript to the user environment variable PATHEXT.
Already done: add .pyw to the user environment variable PATHEXT.
running trial
Traceback (most recent call last):
  File "C:\Tahoe\allmydata-tahoe-1.9.1\support\Scripts\tahoe.pyscript", line 16, in <module>
    load_entry_point('allmydata-tahoe==1.9.1', 'console_scripts', 'tahoe')()
  File "C:\Tahoe\allmydata-tahoe-1.9.1\support\Lib\site-packages\setuptools-0.6c16dev3.egg\pkg_resources.py", line 318, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "C:\Tahoe\allmydata-tahoe-1.9.1\support\Lib\site-packages\setuptools-0.6c16dev3.egg\pkg_resources.py", line 2245, in load_entry_point
    return ep.load()
  File "C:\Tahoe\allmydata-tahoe-1.9.1\support\Lib\site-packages\setuptools-0.6c16dev3.egg\pkg_resources.py", line 1978, in load
    entry = __import__(self.module_name, globals(),globals(), ['__name__'])
  File "c:\tahoe\allmydata-tahoe-1.9.1\src\allmydata\scripts\runner.py", line 7, in <module>
    from allmydata.scripts.common import BaseOptions
  File "c:\tahoe\allmydata-tahoe-1.9.1\src\allmydata\scripts\common.py", line 19, in <module>
    path = abspath_expanduser_unicode(u"~/.tahoe")
  File "c:\tahoe\allmydata-tahoe-1.9.1\src\allmydata\util\fileutil.py", line 291, in abspath_expanduser_unicode
    path = os.path.expanduser(path)
  File "C:\Python27\lib\ntpath.py", line 301, in expanduser
    return userhome + path[i:]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc0 in position 9: ordinal not in range(128)

C:\Tahoe\allmydata-tahoe-1.9.1>                                                                                                                                                                                     ^

Host:Windows 7,Python 2.7(same issue - with 2.6.6)
This problem happens if current username is not ASCII (it is by default on some international localizations of Windows
is this bug with python on windows?

attempt to perform setup.py test results in ``` argparse 0.9.1 is already the active version in easy-install.pth 16:34 Using c:\tahoe\allmydata-tahoe-1.9.1\support\lib\site-packages\argparse-0.9.1-py2.7-win32.egg Searching for zbase32==1.1.3 Best match: zbase32 1.1.3 Processing zbase32-1.1.3-py2.7.egg zbase32 1.1.3 is already the active version in easy-install.pth Using c:\tahoe\allmydata-tahoe-1.9.1\support\lib\site-packages\zbase32-1.1.3-py2.7.egg Finished processing dependencies for allmydata-tahoe==1.9.1 Setting up environment to run scripts for the current user... Already done: associate the filetype .pyscript with Python.File for the current user. Already done: add .pyscript to the user environment variable PATHEXT. Already done: add .pyw to the user environment variable PATHEXT. running make_executable running build running build_py Setting up environment to run scripts for the current user... Already done: associate the filetype .pyscript with Python.File for the current user. Already done: add .pyscript to the user environment variable PATHEXT. Already done: add .pyw to the user environment variable PATHEXT. running trial Traceback (most recent call last): File "C:\Tahoe\allmydata-tahoe-1.9.1\support\Scripts\tahoe.pyscript", line 16, in <module> load_entry_point('allmydata-tahoe==1.9.1', 'console_scripts', 'tahoe')() File "C:\Tahoe\allmydata-tahoe-1.9.1\support\Lib\site-packages\setuptools-0.6c16dev3.egg\pkg_resources.py", line 318, in load_entry_point return get_distribution(dist).load_entry_point(group, name) File "C:\Tahoe\allmydata-tahoe-1.9.1\support\Lib\site-packages\setuptools-0.6c16dev3.egg\pkg_resources.py", line 2245, in load_entry_point return ep.load() File "C:\Tahoe\allmydata-tahoe-1.9.1\support\Lib\site-packages\setuptools-0.6c16dev3.egg\pkg_resources.py", line 1978, in load entry = __import__(self.module_name, globals(),globals(), ['__name__']) File "c:\tahoe\allmydata-tahoe-1.9.1\src\allmydata\scripts\runner.py", line 7, in <module> from allmydata.scripts.common import BaseOptions File "c:\tahoe\allmydata-tahoe-1.9.1\src\allmydata\scripts\common.py", line 19, in <module> path = abspath_expanduser_unicode(u"~/.tahoe") File "c:\tahoe\allmydata-tahoe-1.9.1\src\allmydata\util\fileutil.py", line 291, in abspath_expanduser_unicode path = os.path.expanduser(path) File "C:\Python27\lib\ntpath.py", line 301, in expanduser return userhome + path[i:] UnicodeDecodeError: 'ascii' codec can't decode byte 0xc0 in position 9: ordinal not in range(128) C:\Tahoe\allmydata-tahoe-1.9.1> ^ ``` Host:Windows 7,Python 2.7(same issue - with 2.6.6) This problem happens if current username is not ASCII (it is by default on some international localizations of Windows is this bug with python on windows?

tahoe-lafs added the

labels 2012-02-14 10:02:35 +00:00

tahoe-lafs added this to the undecided milestone 2012-02-14 10:02:35 +00:00

davidsarah commented

2012-02-14 19:44:42 +00:00

The bug might be in os.path.expanduser or we might be calling it incorrectly; I'll have to check its specification.

The bug might be in `os.path.expanduser` or we might be calling it incorrectly; I'll have to check its specification.

tahoe-lafs added

code

and removed

packaging

labels 2012-02-14 19:44:42 +00:00

tahoe-lafs modified the milestone from undecided to 1.10.0

2012-02-14 19:44:42 +00:00

tahoe-lafs changed title from ~~python setup.py test fails on windows with non-ASCII chars~~ to abspath_expanduser_unicode fails on Windows if username or home directory contains with non-ASCII chars

2012-02-14 19:44:42 +00:00

tahoe-lafs changed title from ~~abspath_expanduser_unicode fails on Windows if username or home directory contains with non-ASCII chars~~ to abspath_expanduser_unicode fails on Windows if username or home directory contains non-ASCII characters

2012-02-14 19:45:30 +00:00

davidsarah commented

2012-02-14 20:23:49 +00:00

os.path.expanduser appears to be hopelessly buggy when the username is non-ASCII (it returns the wrong answer even when it doesn't fail). We were already working around two bugs in the path functions (http://bugs.python.org/issue1669539 and http://bugs.python.org/issue5827), so it might be simpler just to write our own implementation of expanduser from scratch.

`os.path.expanduser` appears to be hopelessly buggy when the username is non-ASCII (it returns the wrong answer even when it doesn't fail). We were already working around two bugs in the path functions (<http://bugs.python.org/issue1669539> and <http://bugs.python.org/issue5827>), so it might be simpler just to write our own implementation of `expanduser` from scratch.

davidsarah commented

2012-02-14 20:42:44 +00:00

Replying to davidsarah:

We were already working around two bugs in the path functions ...

Actually three, the other is http://bugs.python.org/issue3426 .

Replying to [davidsarah](/tahoe-lafs/trac-2024-07-25/issues/1674#issuecomment-87483): > We were already working around two bugs in the path functions ... Actually three, the other is <http://bugs.python.org/issue3426> .

daira commented

2014-08-21 09:30:21 +00:00

See also #2235: by making all user-expansions go through fileutil.abspath_expanduser_unicode, we'll only need to fix this problem in that function.

See also #2235: by making all user-expansions go through `fileutil.abspath_expanduser_unicode`, we'll only need to fix this problem in that function.

tahoe-lafs changed title from ~~abspath_expanduser_unicode fails on Windows if username or home directory contains non-ASCII characters~~ to os.path.expanduser (and therefore abspath_expanduser_unicode) fails on Windows if username or home directory contains non-ASCII characters

2014-08-21 09:30:21 +00:00

tahoe-lafs modified the milestone from soon to 1.12.0

2014-08-21 09:32:12 +00:00

daira commented

2014-09-10 09:44:12 +00:00

I'm working on this in the branch for #2235.

tahoe-lafs modified the milestone from 1.12.0 to 1.11.0

2014-09-10 09:44:55 +00:00

daira commented

2015-01-29 19:33:08 +00:00

See https://github.com/tahoe-lafs/tahoe-lafs/pull/138.

See <https://github.com/tahoe-lafs/tahoe-lafs/pull/138>.

daira commented

2015-02-05 02:09:29 +00:00

Fixed on the pr138 branch which was merged in [3afe827ad0cbdb41b2928d17c8bfbbaf5102acc8/trunk].

(Woohoo! This set of bugs has been irking me for some time.)

Fixed on the pr138 branch which was merged in [3afe827ad0cbdb41b2928d17c8bfbbaf5102acc8/trunk]. (Woohoo! This set of bugs has been irking me for some time.)

tahoe-lafs added the

fixed

label 2015-02-05 02:09:29 +00:00

daira closed this issue

2015-02-05 02:09:29 +00:00

Sign in to join this conversation.