Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #93119 > unrolled thread

windows and file names > 256 bytes

Back to article view | Back to comp.lang.python

Contents

  windows and file names > 256 bytes Albert-Jan Roskam <sjeik_appie@hotmail.com> - 2015-06-25 08:00 +0000
    Re: windows and file names > 256 bytes Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-06-25 19:16 +1000
      Re: windows and file names > 256 bytes Chris Angelico <rosuav@gmail.com> - 2015-06-25 19:23 +1000
      Re: windows and file names > 256 bytes Tim Golden <mail@timgolden.me.uk> - 2015-06-25 11:10 +0100
      Re: windows and file names > 256 bytes Chris Angelico <rosuav@gmail.com> - 2015-06-25 20:23 +1000
      Re: windows and file names > 256 bytes Terry Reedy <tjreedy@udel.edu> - 2015-06-25 13:03 -0400

#93119 — windows and file names > 256 bytes

Hi,

Consider the following calls, where very_long_path is more than 256 bytes:
[1] os.mkdir(very_long_path)
[2] os.getsize(very_long_path)
[3] shutil.rmtree(very_long_path)

I am using Python 2.7 and [1] and [2] fail under Windows XP [3] fails 
under Win7 (not sure about XP). It throws: “WindowsError: [Error 206] The 
filename or extension is too long” This is even when I use the "special" 
notations \\?\c:\dir\file or \\?\UNC\server\share\file, e.g.
os.path.getsize("\\\\?\\" + "c:\\dir\\file")
(Oddly, os.path.getsize(os.path.join("\\\\?", "c:\\dir\\file")) will 
truncate the prefix)

My questions:
1. How can I get the file size of very long paths under XP?
2. Is this a bug in Python? I would prefer if Python dealt with the gory 
details of Windows' silly behavior.

Regards,
Albert-Jan

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

[toc] | [next] | [standalone]

#93120

On Thursday 25 June 2015 18:00, Albert-Jan Roskam wrote:

> Hi,
> 
> Consider the following calls, where very_long_path is more than 256 bytes:
> [1] os.mkdir(very_long_path)
> [2] os.getsize(very_long_path)
> [3] shutil.rmtree(very_long_path)
> 
> I am using Python 2.7 and [1] and [2] fail under Windows XP [3] fails
> under Win7 (not sure about XP). It throws: “WindowsError: [Error 206] The
> filename or extension is too long” 

I don't think this is a bug. It seems to be a limitation of Windows.

https://msdn.microsoft.com/en-
us/library/windows/desktop/aa365247%28v=vs.85%29.aspx#maxpath

> This is even when I use the "special"
> notations \\?\c:\dir\file or \\?\UNC\server\share\file, e.g.
> os.path.getsize("\\\\?\\" + "c:\\dir\\file")

However, that may be a bug.

What happens if you use a Unicode string?

path = u"\\\\?\\c:a\\very\\long\\path"
os.mkdir(path)


Can you open an existing file?

open(u"\\\\?\\c:a\\very\\long\\path\\file.txt")



> (Oddly, os.path.getsize(os.path.join("\\\\?", "c:\\dir\\file")) will
> truncate the prefix)

That's worth reporting as a bug.


> My questions:
> 1. How can I get the file size of very long paths under XP?

If all else fails:

last = os.getcwd()
try:
    os.chdir('C:/a/very/long')
    os.chdir('path/with/many')
    os.chdir('nested/folders')
    os.path.getsize('/and/even/more/file.txt')
finally:
    os.chdir(last)


> 2. Is this a bug in Python? I would prefer if Python dealt with the gory
> details of Windows' silly behavior.

I would say that it is a bug that it doesn't work with extended-length paths 
(those starting with \\?\) but may or may not be a bug with regular paths.


-- 
Steve

[toc] | [prev] | [next] | [standalone]

#93121

On Thu, Jun 25, 2015 at 7:16 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
>> 2. Is this a bug in Python? I would prefer if Python dealt with the gory
>> details of Windows' silly behavior.
>
> I would say that it is a bug that it doesn't work with extended-length paths
> (those starting with \\?\) but may or may not be a bug with regular paths.

I'd go further and say that the OP is right in expecting Python to
deal with the gory details. Would it break anything for Python to
prepend \\?\ to all file names before giving them to certain APIs?
Then the current behaviour of stripping off that prefix would be fine.

Are there any times when you *don't* want Windows to use the
extended-length path?

ChrisA

[toc] | [prev] | [next] | [standalone]

#93129

On 25/06/2015 10:23, Chris Angelico wrote:
> On Thu, Jun 25, 2015 at 7:16 PM, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> wrote:
>>> 2. Is this a bug in Python? I would prefer if Python dealt with the gory
>>> details of Windows' silly behavior.
>>
>> I would say that it is a bug that it doesn't work with extended-length paths
>> (those starting with \\?\) but may or may not be a bug with regular paths.
> 
> I'd go further and say that the OP is right in expecting Python to
> deal with the gory details. Would it break anything for Python to
> prepend \\?\ to all file names before giving them to certain APIs?
> Then the current behaviour of stripping off that prefix would be fine.
> 
> Are there any times when you *don't* want Windows to use the
> extended-length path?

Yes: when you're passing a relative filepath. Which could pretty much be
any time. As you might imagine, this has come up before -- there's an
issue on the tracker for it somewhere. I just don't think it's simple
enough for Python to know when and when not to use the extended path
syntax without danger of breaking something.

Bear in mind that the \\?\ prefix doesn't just extend the length: it
also allows otherwise special-cased characters such as "." or "..". It's
a general-purpose mechanism for handing something straight to the file
system without parsing it first.

TJG

[toc] | [prev] | [next] | [standalone]

#93132

On Thu, Jun 25, 2015 at 8:10 PM, Tim Golden <mail@timgolden.me.uk> wrote:
>> Are there any times when you *don't* want Windows to use the
>> extended-length path?
>
> Yes: when you're passing a relative filepath. Which could pretty much be
> any time. As you might imagine, this has come up before -- there's an
> issue on the tracker for it somewhere. I just don't think it's simple
> enough for Python to know when and when not to use the extended path
> syntax without danger of breaking something.

Oh blah. So I suppose that means there's fundamentally no way to use a
long (>256 byte) relative path on Windows?

> Bear in mind that the \\?\ prefix doesn't just extend the length: it
> also allows otherwise special-cased characters such as "." or "..". It's
> a general-purpose mechanism for handing something straight to the file
> system without parsing it first.

Ohh. So... hmm. So what this really means is that a path could get
\\?\ prepended when, and ONLY when, it becomes absolute. Windows can
be a real pest...

ChrisA

[toc] | [prev] | [next] | [standalone]

#93154

On 6/25/2015 5:16 AM, Steven D'Aprano wrote:
> On Thursday 25 June 2015 18:00, Albert-Jan Roskam wrote:
>
>> Hi,
>>
>> Consider the following calls, where very_long_path is more than 256 bytes:
>> [1] os.mkdir(very_long_path)
>> [2] os.getsize(very_long_path)
>> [3] shutil.rmtree(very_long_path)
>>
>> I am using Python 2.7 and [1] and [2] fail under Windows XP [3] fails
>> under Win7 (not sure about XP). It throws: “WindowsError: [Error 206] The
>> filename or extension is too long”
>
> I don't think this is a bug. It seems to be a limitation of Windows.
>
> https://msdn.microsoft.com/en-
> us/library/windows/desktop/aa365247%28v=vs.85%29.aspx#maxpath
>
>> This is even when I use the "special"
>> notations \\?\c:\dir\file or \\?\UNC\server\share\file, e.g.
>> os.path.getsize("\\\\?\\" + "c:\\dir\\file")
>
> However, that may be a bug.
>
> What happens if you use a Unicode string?
>
> path = u"\\\\?\\c:a\\very\\long\\path"
> os.mkdir(path)
>
>
> Can you open an existing file?
>
> open(u"\\\\?\\c:a\\very\\long\\path\\file.txt")
>
>
>
>> (Oddly, os.path.getsize(os.path.join("\\\\?", "c:\\dir\\file")) will
>> truncate the prefix)
>
> That's worth reporting as a bug.

If possible, please try the same operations with Python 3.4 or .5 before 
making a report


-- 
Terry Jan Reedy

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web