os.path.abspath() sometimes returns str instead of unicode? #1108

New Issue

zooko · 2010-07-06T05:46:36Z

zooko commented

2010-07-06 05:46:36 +00:00

Python 2.7 just came out, and reading through the changes led me to:

http://bugs.python.org/issue3426

We use os.path.abspath() in a few places where its argument is unicode.

I also noticed:

http://bugs.python.org/issue5827

but we don't currently use os.path.normpath().

How come our unit tests don't show this problem happening to us?

Python 2.7 just came out, and reading through the changes led me to: <http://bugs.python.org/issue3426> We use `os.path.abspath()` in a few places where its argument is unicode. I also noticed: <http://bugs.python.org/issue5827> but we don't currently use `os.path.normpath()`. How come our unit tests don't show this problem happening to us?

zooko added the

labels 2010-07-06 05:46:36 +00:00

zooko added this to the undecided milestone 2010-07-06 05:46:36 +00:00

tahoe-lafs modified the milestone from undecided to 1.8.0

2010-07-17 05:35:35 +00:00

tahoe-lafs added

code

and removed

unknown

labels 2010-07-17 05:35:47 +00:00

davidsarah commented

2010-07-22 00:35:25 +00:00

Attachment abspath-and-unicode-basedirs.dpatch (33158 bytes) added

util.fileutil, test.test_util: add abspath_expanduser_unicode function, to work around <http://bugs.python.org/issue3426>. util.encodingutil: add a convenience function argv_to_abspath. * Replace uses of os.path.abspath with abspath_expanduser_unicode where necessary. This makes basedir paths consistently represented as Unicode.

**Attachment** abspath-and-unicode-basedirs.dpatch (33158 bytes) added * util.fileutil, test.test_util: add abspath_expanduser_unicode function, to work around <<http://bugs.python.org/issue3426>>. util.encodingutil: add a convenience function argv_to_abspath. * Replace uses of os.path.abspath with abspath_expanduser_unicode where necessary. This makes basedir paths consistently represented as Unicode.

abspath-and-unicode-basedirs.dpatch

32 KiB

davidsarah commented

2010-07-22 00:50:44 +00:00

Our tests might not show this for any of the following reasons:

We don't test or support Unicode paths for basedirs, which account for most of our uses of os.path.abspath.
Our Unicode tests for the CLI are a bit patchy; they only test Unicode paths for some commands.
When running the test suite, the current directory is the _trial_temp directory, which is at an ASCII path even if the test creates files and subdirectories at non-ASCII paths.
The bug in os.path.abspath only occurs when ~~none of the resulting path components come from the input string~~ the input path is relative and the current directory path is non-ASCII.
Even when os.path.abspath returns a bytestring, it will often get coerced to a Unicode string (not necessarily the correct one!) by the default Python coercions.

Note that while abspath-and-unicode-basedirs.dpatch makes most of the changes needed to support Unicode paths for basedirs, you still can't, at least on Windows, build or test a Tahoe source tree at an arbitrary Unicode path. (That would require changes to Python, darcs, setuptools, Twisted trial, etc., and is probably not a realistic aim.)

Our tests might not show this for any of the following reasons: * We don't test or support Unicode paths for basedirs, which account for most of our uses of `os.path.abspath`. * Our Unicode tests for the CLI are a bit patchy; they only test Unicode paths for some commands. * When running the test suite, the current directory is the `_trial_temp` directory, which is at an ASCII path even if the test creates files and subdirectories at non-ASCII paths. * The bug in `os.path.abspath` only occurs when ~~none of the resulting path components come from the input string~~ the input path is relative and the current directory path is non-ASCII. * Even when `os.path.abspath` returns a bytestring, it will often get coerced to a Unicode string (not necessarily the correct one!) by the default Python coercions. Note that while [abspath-and-unicode-basedirs.dpatch](/tahoe-lafs/trac-2024-07-25/attachments/000078ac-282c-0a20-f052-64bdbd1c759d) makes most of the changes needed to support Unicode paths for basedirs, you still can't, at least on Windows, build or test a Tahoe source tree at an arbitrary Unicode path. (That would require changes to Python, darcs, setuptools, Twisted trial, etc., and is probably not a realistic aim.)

davidsarah commented

2010-08-02 01:31:34 +00:00

The changesets relevant to this ticket on the ticket798 branch are:
[4598/ticket798]
[4599/ticket798]
[4601/ticket798]
[4604/ticket798]
[4606/ticket798]
[4607/ticket798]

The changesets relevant to this ticket on the ticket798 branch are: [4598/ticket798] [4599/ticket798] [4601/ticket798] [4604/ticket798] [4606/ticket798] [4607/ticket798]

davidsarah commented

2010-08-07 21:34:50 +00:00

The changesets relevant to this ticket on trunk are changeset:11b18824c7ff3237 changeset:618db4867c68a6f9 changeset:f036dfaa4bfb9f90 changeset:2b33704d613da8d1 changeset:013954c2659db71f changeset:d3a8ef762f74b925 changeset:02199db615047ea1.