Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #107649 > unrolled thread

Why does pathlib not have is_readable() & things like that?

Back to article view | Back to comp.lang.python

Contents

  Why does pathlib not have is_readable() & things like that? Adam Funk <a24061@ducksburg.com> - 2016-04-26 14:30 +0100
    Re: Why does pathlib not have is_readable() & things like that? Random832 <random832@fastmail.com> - 2016-04-26 09:57 -0400
      Re: Why does pathlib not have is_readable() & things like that? Adam Funk <a24061@ducksburg.com> - 2016-04-28 14:02 +0100
        Re: Why does pathlib not have is_readable() & things like that? Jussi Piitulainen <jussi.piitulainen@helsinki.fi> - 2016-04-28 16:24 +0300
        Re: Why does pathlib not have is_readable() & things like that? Grant Edwards <grant.b.edwards@gmail.com> - 2016-04-28 13:34 +0000
          Re: Why does pathlib not have is_readable() & things like that? Adam Funk <a24061@ducksburg.com> - 2016-04-29 10:49 +0100
            Re: Why does pathlib not have is_readable() & things like that? Jussi Piitulainen <jussi.piitulainen@helsinki.fi> - 2016-04-29 14:51 +0300
              Re: Why does pathlib not have is_readable() & things like that? eryk sun <eryksun@gmail.com> - 2016-04-29 10:09 -0500
              Re: Why does pathlib not have is_readable() & things like that? Grant Edwards <grant.b.edwards@gmail.com> - 2016-04-29 23:53 +0000
    Re: Why does pathlib not have is_readable() & things like that? Steven D'Aprano <steve@pearwood.info> - 2016-04-27 01:46 +1000
      Re: Why does pathlib not have is_readable() & things like that? Adam Funk <a24061@ducksburg.com> - 2016-04-28 13:57 +0100

#107649 — Why does pathlib not have is_readable() & things like that?

I recently discovered pathlib in the Python 3 standard library, & find
it very useful, but I'm a bit surprised that it doesn't offer things
like is_readable() and is_writable.  Is there a good reason for that?

I've been improvising with things like this:

import pathlib, os

path = pathlib.Path('some/directory')
writable = os.access(str(path), os.W_OK | os.X_OK) 

Is that the best way to do it?


-- 
Unix is a user-friendly operating system. It's just very choosy about
its friends.

[toc] | [next] | [standalone]

#107651

On Tue, Apr 26, 2016, at 09:30, Adam Funk wrote:
> I recently discovered pathlib in the Python 3 standard library, & find
> it very useful, but I'm a bit surprised that it doesn't offer things
> like is_readable() and is_writable.  Is there a good reason for that?

Well, one reason would be EAFP. Just try to open the file and see if it
gives you a PermissionError.

[toc] | [prev] | [next] | [standalone]

#107785

On 2016-04-26, Random832 wrote:

> On Tue, Apr 26, 2016, at 09:30, Adam Funk wrote:
>> I recently discovered pathlib in the Python 3 standard library, & find
>> it very useful, but I'm a bit surprised that it doesn't offer things
>> like is_readable() and is_writable.  Is there a good reason for that?
>
> Well, one reason would be EAFP. Just try to open the file and see if it
> gives you a PermissionError.

I understand that in general, but the tool I'm working on here takes a
command-line option to specify an output directory, & I'd rather not
start processing the data (which involves GETting from a REST service,
processing, and PUTting back modifications to the data) only to crash
after the first batch because of a user error.


-- 
Specifications are for the weak & timid!
          --- Klingon Programmer's Guide

[toc] | [prev] | [next] | [standalone]

#107787

Adam Funk writes:

> On 2016-04-26, Random832 wrote:
>
>> On Tue, Apr 26, 2016, at 09:30, Adam Funk wrote:
>>> I recently discovered pathlib in the Python 3 standard library, & find
>>> it very useful, but I'm a bit surprised that it doesn't offer things
>>> like is_readable() and is_writable.  Is there a good reason for that?
>>
>> Well, one reason would be EAFP. Just try to open the file and see if it
>> gives you a PermissionError.
>
> I understand that in general, but the tool I'm working on here takes a
> command-line option to specify an output directory, & I'd rather not
> start processing the data (which involves GETting from a REST service,
> processing, and PUTting back modifications to the data) only to crash
> after the first batch because of a user error.

Open an output file first. Crash before reading the first batch.

[toc] | [prev] | [next] | [standalone]

#107789

On 2016-04-28, Adam Funk <a24061@ducksburg.com> wrote:
> On 2016-04-26, Random832 wrote:
>
>> On Tue, Apr 26, 2016, at 09:30, Adam Funk wrote:
>>> I recently discovered pathlib in the Python 3 standard library, & find
>>> it very useful, but I'm a bit surprised that it doesn't offer things
>>> like is_readable() and is_writable.  Is there a good reason for that?
>>
>> Well, one reason would be EAFP. Just try to open the file and see if it
>> gives you a PermissionError.
>
> I understand that in general, but the tool I'm working on here takes a
> command-line option to specify an output directory, & I'd rather not
> start processing the data (which involves GETting from a REST service,
> processing, and PUTting back modifications to the data) only to crash
> after the first batch because of a user error.

Then open the output file before you do the GET.

Or just do os.access("directory/where/you/want/to/open/a/file",os.W_OK)

-- 
Grant Edwards               grant.b.edwards        Yow! Jesus is my POSTMASTER
                                  at               GENERAL ...
                              gmail.com            

[toc] | [prev] | [next] | [standalone]

#107849

On 2016-04-28, Grant Edwards wrote:

> On 2016-04-28, Adam Funk <a24061@ducksburg.com> wrote:
>> On 2016-04-26, Random832 wrote:
>>
>>> On Tue, Apr 26, 2016, at 09:30, Adam Funk wrote:
>>>> I recently discovered pathlib in the Python 3 standard library, & find
>>>> it very useful, but I'm a bit surprised that it doesn't offer things
>>>> like is_readable() and is_writable.  Is there a good reason for that?
>>>
>>> Well, one reason would be EAFP. Just try to open the file and see if it
>>> gives you a PermissionError.
>>
>> I understand that in general, but the tool I'm working on here takes a
>> command-line option to specify an output directory, & I'd rather not
>> start processing the data (which involves GETting from a REST service,
>> processing, and PUTting back modifications to the data) only to crash
>> after the first batch because of a user error.
>
> Then open the output file before you do the GET.

I guess I could, but fetching the data actually involves a whole lot
of GET requests (the first one includes cross-references to the URLs
where the rest of the data is found), some BeautifulSoup processing, &
a lot of other processing to produce a big dict, which I then write
out as json using what I think is the best way (output_file is an
instance of pathlib.Path):

    with output_file.open(mode='w', encoding='UTF-8', errors='replace')  as f:
        json.dump(output, f, sort_keys=True, indent=2)

> Or just do os.access("directory/where/you/want/to/open/a/file",os.W_OK)

That's what I'm doing now, but I prefer to give the user the error
message early on.

[toc] | [prev] | [next] | [standalone]

#107852

Adam Funk writes:

> On 2016-04-28, Grant Edwards wrote:
>>
>> Then open the output file before you do the GET.
>
> I guess I could, but fetching the data actually involves a whole lot
> of GET requests (the first one includes cross-references to the URLs
> where the rest of the data is found), some BeautifulSoup processing, &
> a lot of other processing to produce a big dict, which I then write
> out as json using what I think is the best way (output_file is an
> instance of pathlib.Path):
>
>     with output_file.open(mode='w', encoding='UTF-8', errors='replace')  as f:
>         json.dump(output, f, sort_keys=True, indent=2)

It doesn't matter how much work it actually is. Make it conceptually a
single unit:

def get_output(): ...

with output_file.open(mode='w',
                      encoding='UTF-8',
                      errors='replace')  as f:
    output = get_output()
    json.dump(output, f,
              sort_keys=True,
              indent=2)

>> Or just do os.access("directory/where/you/want/to/open/a/file",os.W_OK)
>
> That's what I'm doing now, but I prefer to give the user the error
> message early on.

Then do that early on.

[toc] | [prev] | [next] | [standalone]

#107858

On Fri, Apr 29, 2016 at 6:51 AM, Jussi Piitulainen
<jussi.piitulainen@helsinki.fi> wrote:
> Adam Funk writes:
>> On 2016-04-28, Grant Edwards wrote:
>
>>> Or just do os.access("directory/where/you/want/to/open/a/file",os.W_OK)
>>
>> That's what I'm doing now, but I prefer to give the user the error
>> message early on.
>
> Then do that early on.

Note that on Windows os.access only checks if a file is read-only.
There's a proposed update to check the file security. The patch needs
to be updated and needs to be reworked, but it's a low priority. For
the most part calling os.access isn't recommended.

[toc] | [prev] | [next] | [standalone]

#107872

On 2016-04-29, eryk sun <eryksun@gmail.com> wrote:
> On Fri, Apr 29, 2016 at 6:51 AM, Jussi Piitulainen
><jussi.piitulainen@helsinki.fi> wrote:
>> Adam Funk writes:
>>> On 2016-04-28, Grant Edwards wrote:
>>
>>>> Or just do os.access("directory/where/you/want/to/open/a/file",os.W_OK)
>>>
>>> That's what I'm doing now, but I prefer to give the user the error
>>> message early on.
>>
>> Then do that early on.
>
> Note that on Windows os.access only checks if a file is read-only.
> There's a proposed update to check the file security. The patch needs
> to be updated and needs to be reworked, but it's a low priority. For
> the most part calling os.access isn't recommended.

I'm sure there are probably other ways that it fails also.  That's why
it's almost always simpler and better to just open the file in write
mode and see if it works.

--
Grant

[toc] | [prev] | [next] | [standalone]

#107660

On Tue, 26 Apr 2016 11:30 pm, Adam Funk wrote:

> I recently discovered pathlib in the Python 3 standard library, & find
> it very useful, but I'm a bit surprised that it doesn't offer things
> like is_readable() and is_writable.  Is there a good reason for that?

Maybe nobody thought of it. Why don't you propose a feature request on the
bug tracker?

> I've been improvising with things like this:
> 
> import pathlib, os
> 
> path = pathlib.Path('some/directory')
> writable = os.access(str(path), os.W_OK | os.X_OK)
> 
> Is that the best way to do it?

No. All you have learned is that the directory is writable *now*. In a
millisecond, or a minute, when you actually go to write to it, it may no
longer be writable -- it may not even exist.

There is a whole class of serious security vulnerabilities and bugs caused
by the difference between the time you check something and the time you
actually use it. "Time of check to time of use" bugs can be best avoided by
not checking ahead of time whether the directory is writable, but just
*attempting to write to it*, and catching the error if you can't.




-- 
Steven

[toc] | [prev] | [next] | [standalone]

#107784

On 2016-04-26, Steven D'Aprano wrote:

> On Tue, 26 Apr 2016 11:30 pm, Adam Funk wrote:

>> I've been improvising with things like this:
>> 
>> import pathlib, os
>> 
>> path = pathlib.Path('some/directory')
>> writable = os.access(str(path), os.W_OK | os.X_OK)
>> 
>> Is that the best way to do it?
>
> No. All you have learned is that the directory is writable *now*. In a
> millisecond, or a minute, when you actually go to write to it, it may no
> longer be writable -- it may not even exist.
>
> There is a whole class of serious security vulnerabilities and bugs caused
> by the difference between the time you check something and the time you
> actually use it. "Time of check to time of use" bugs can be best avoided by
> not checking ahead of time whether the directory is writable, but just
> *attempting to write to it*, and catching the error if you can't.

I appreciate the general principle, but the situation here is a tool
that iterates over a loop of the following: GET some chunks of data
from a REST service, process them, PUT something back to existing
documents through the REST service, & save the result in a directory
specified as a command-line option.  I don't want the tool to modify
the first batch of data in the REST service, be unable to store the
results locally, & crash as a result.


-- 
A mathematical formula should never be "owned" by anybody! Mathematics
belongs to God.                                       --- Donald Knuth

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web