Path: csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From: eryk sun <eryksun@gmail.com>
Newsgroups: comp.lang.python
Subject: Re: Remove directory tree without following symlinks
Date: Sat, 23 Apr 2016 15:22:35 -0500
Lines: 53
Message-ID: <mailman.10.1461442997.32212.python-list@python.org>
References: <571a3ba2$0$1597$c3e8da3$5496439d@news.astraweb.com> <1461337766.365000.586700849.0DDBDB0B@webmail.messagingengine.com> <mailman.16.1461337768.2861.python-list@python.org> <571a5be6$0$1590$c3e8da3$5496439d@news.astraweb.com> <DUB123-W22A605538041FB2749C64B836F0@phx.gbl> <CACL+1aueH_scSTYW1Q6pGsZ92+8OWHB6tks3SqAAaU-wb6Ac4Q@mail.gmail.com> <DUB123-W362E7C64F347B92A74E34B83600@phx.gbl> <CACL+1avhu9YcM5qbFaAdhq9ZUoizP5Dc8p99LMrCcREmcxBNWw@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
In-Reply-To: <DUB123-W362E7C64F347B92A74E34B83600@phx.gbl>
Precedence: list
Xref: csiph.com comp.lang.python:107518

On Sat, Apr 23, 2016 at 4:34 AM, Albert-Jan Roskam
<sjeik_appie@hotmail.com> wrote:
>
>> From: eryksun@gmail.com
>> Date: Fri, 22 Apr 2016 13:28:01 -0500
>> On Fri, Apr 22, 2016 at 12:39 PM, Albert-Jan Roskam
>> <sjeik_appie@hotmail.com> wrote:
>> > FYI, Just today I found out that shutil.rmtree raises a WindowsError if
>> > the dir is read-only (or its contents). Using 'ignore_errors', won't help.
>> > Sure, no error is raised, but the dir is not deleted either! A 'force' option
>> > would be a nice improvement.
>>
>> Use the onerror handler to call os.chmod(path, stat.S_IWRITE). For
>> example, see pip's rmtree_errorhandler:
>>
>> https://github.com/pypa/pip/blob/8.1.1/pip/utils/__init__.py#L105
>
> Thanks, that looks useful indeed. I thought about os.chmod, but with
> os.walk. That seemed expensive. So I used subprocess.call('rmdir "%s" /s /q'
> % dirname). That's Windows only, of course, but aside of that, is using
> subprocess less preferable?

I assume you used shell=True in the above call, and not an external
rmdir.exe. There are security concerns with using the shell if you're
not in complete control of the command line.

As to performance, cmd's rmdir wins without question, not only because
it's implemented in C, but also because it uses the stat data from the
WIN32_FIND_DATA returned by FindFirstFile/FindNextFile to check for
FILE_ATTRIBUTE_DIRECTORY and FILE_ATTRIBUTE_READONLY.

On the other hand, Python wins when it comes to working with deeply
nested directories. Paths in cmd are limited to MAX_PATH characters.
rmdir uses DOS 8.3 short names (i.e. cAlternateFileName in
WIN32_FIND_DATA), but that could still exceed MAX_PATH for a deeply
nested tree, or the volume may not even have 8.3 DOS filenames.
shutil.rmtree allows you to work around the DOS limit by prefixing the
path with "\\?\". For example:

    >>> subprocess.call(r'rmdir /q/s Z:\Temp\long', shell=True)
    The path Z:\Temp\long\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    aaaaa is too long.
    0

    >>> shutil.rmtree(r'\\?\Z:\Temp\long')
    >>> os.path.exists(r'Z:\Temp\long')
    False

Using "\\?\" requires a path that's fully qualified, normalized
(backslash only), and unicode (i.e. decode a Python 2 str).