Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #27429 > unrolled thread

Why doesn't Python remember the initial directory?

Back to article view | Back to comp.lang.python

Contents

  Why doesn't Python remember the initial directory? kj <no.email@please.post> - 2012-08-19 20:42 +0000
    Re: Why doesn't Python remember the initial directory? Giacomo Alzetta <giacomo.alzetta@gmail.com> - 2012-08-19 14:01 -0700
      Re: Why doesn't Python remember the initial directory? Nobody <nobody@nowhere.com> - 2012-08-20 01:45 +0100
        Re: Why doesn't Python remember the initial directory? Neil Hodgson <nhodgson@iinet.net.au> - 2012-08-21 11:39 +1000
    Re: Why doesn't Python remember the initial directory? Roy Smith <roy@panix.com> - 2012-08-19 17:03 -0400
      Re: Why doesn't Python remember the initial directory? kj <no.email@please.post> - 2012-08-20 01:57 +0000
        Re: Why doesn't Python remember the initial directory? Jerry Hill <malaclypse2@gmail.com> - 2012-08-19 22:05 -0400
        Re: Why doesn't Python remember the initial directory? alex23 <wuwei23@gmail.com> - 2012-08-19 19:57 -0700
          Re: Why doesn't Python remember the initial directory? alex23 <wuwei23@gmail.com> - 2012-08-19 20:04 -0700
            Re: Why doesn't Python remember the initial directory? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2012-08-20 08:11 +0100
          Re: Why doesn't Python remember the initial directory? alex23 <wuwei23@gmail.com> - 2012-08-19 20:03 -0700
          Re: Why doesn't Python remember the initial directory? alex23 <wuwei23@gmail.com> - 2012-08-19 19:59 -0700
          Re: Why doesn't Python remember the initial directory? Paul Rubin <no.email@nospam.invalid> - 2012-08-20 21:20 -0700
        Re: Why doesn't Python remember the initial directory? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-08-20 03:38 +0000
          Re: Why doesn't Python remember the initial directory? alex23 <wuwei23@gmail.com> - 2012-08-19 20:43 -0700
        Re: Why doesn't Python remember the initial directory? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2012-08-20 08:10 +0100
        Re: Why doesn't Python remember the initial directory? andrea crotti <andrea.crotti.0@gmail.com> - 2012-08-20 12:56 +0100
          Re: Why doesn't Python remember the initial directory? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-08-20 13:14 +0000
            Re: Why doesn't Python remember the initial directory? Walter Hurry <walterhurry@lavabit.com> - 2012-08-20 13:49 +0000
              Re: Why doesn't Python remember the initial directory? Roy Smith <roy@panix.com> - 2012-08-20 09:58 -0400
                Re: Why doesn't Python remember the initial directory? andrea crotti <andrea.crotti.0@gmail.com> - 2012-08-20 15:15 +0100
        Re: Why doesn't Python remember the initial directory? Jean-Michel Pichavant <jeanmichel@sequans.com> - 2012-08-20 16:39 +0200
        Re: Why doesn't Python remember the initial directory? Grant Edwards <invalid@invalid.invalid> - 2012-08-20 16:28 +0000
        Re: Why doesn't Python remember the initial directory? Piet van Oostrum <piet@vanoostrum.org> - 2012-08-20 16:06 -0400
        Re: Why doesn't Python remember the initial directory? John Roth <johnroth1@gmail.com> - 2012-08-21 13:35 -0700
    Re: Why doesn't Python remember the initial directory? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2012-08-19 22:18 +0100
    Re: Why doesn't Python remember the initial directory? Laszlo Nagy <gandalf@shopzeus.com> - 2012-08-19 23:05 +0200
    Re: Why doesn't Python remember the initial directory? 88888 Dihedral <dihedral88888@googlemail.com> - 2012-08-19 17:54 -0700

Page 1 of 2  [1] 2  Next page →

#27429 — Why doesn't Python remember the initial directory?

As far as I've been able to determine, Python does not remember
(immutably, that is) the working directory at the program's start-up,
or, if it does, it does not officially expose this information.

Does anyone know why this is?  Is there a PEP stating the rationale
for it?

Thanks!

[toc] | [next] | [standalone]

#27430

Il giorno domenica 19 agosto 2012 22:42:16 UTC+2, kj ha scritto:
> As far as I've been able to determine, Python does not remember
> 
> (immutably, that is) the working directory at the program's start-up,
> 
> or, if it does, it does not officially expose this information.
> 
> 
> 
> Does anyone know why this is?  Is there a PEP stating the rationale
> 
> for it?
> 
> 
> 
> Thanks!

You can obtain the working directory with os.getcwd().

giacomo@jack-laptop:~$ echo 'import os; print os.getcwd()' > testing-dir.py
giacomo@jack-laptop:~$ python testing-dir.py 
/home/giacomo
giacomo@jack-laptop:~$ cd Documenti
giacomo@jack-laptop:~/Documenti$ python ../testing-dir.py 
/home/giacomo/Documenti
giacomo@jack-laptop:~/Documenti$ 

Obviously using os.chdir() will change the working directory, and the os.getcwd() will not be the start-up working directory, but if you need the start-up working directory you can get it at start-up and save it in some constant.

[toc] | [prev] | [next] | [standalone]

#27446

On Sun, 19 Aug 2012 14:01:15 -0700, Giacomo Alzetta wrote:

> You can obtain the working directory with os.getcwd().

Maybe. On Unix, it's possible that the current directory no longer
has a pathname. As with files, directories can be "deleted" (i.e.
unlinked) even while they're still in use.

Similarly, a directory can be renamed while it's in use, so the current
directory's pathname may have changed even while the current directory
itself hasn't.

[toc] | [prev] | [next] | [standalone]

#27543

Nobody:

> Maybe. On Unix, it's possible that the current directory no longer
> has a pathname.

    Its also possible that you do not have permission to successfully 
call getcwd. One example of this I have experienced is the OS X sandbox 
where you can run Python starting in a directory where you have only 
limited permissions.

    getcwd works by calling readdir and lstat and looping up from the 
current directory to the root to build the whole path so will break 
without read permissions on directories:
http://www.opensource.apple.com/source/Libc/Libc-763.13/gen/FreeBSD/getcwd.c

    Neil

[toc] | [prev] | [next] | [standalone]

#27431

In article <k0rj38$2gc$1@reader1.panix.com>, kj <no.email@please.post> 
wrote:

> As far as I've been able to determine, Python does not remember
> (immutably, that is) the working directory at the program's start-up,
> or, if it does, it does not officially expose this information.

Why would you expect that it would?  What would it (or you) do with this 
information?

More to the point, doing a chdir() is not something any library code 
would do (at least not that I'm aware of), so if the directory changed, 
it's because some application code did it.  In which case, you could 
have just stored the working directory yourself.

[toc] | [prev] | [next] | [standalone]

#27449

In <roy-CA6D77.17031119082012@news.panix.com> Roy Smith <roy@panix.com> writes:

>In article <k0rj38$2gc$1@reader1.panix.com>, kj <no.email@please.post> 
>wrote:

>> As far as I've been able to determine, Python does not remember
>> (immutably, that is) the working directory at the program's start-up,
>> or, if it does, it does not officially expose this information.

>Why would you expect that it would?  What would it (or you) do with this 
>information?

>More to the point, doing a chdir() is not something any library code 
>would do (at least not that I'm aware of), so if the directory changed, 
>it's because some application code did it.  In which case, you could 
>have just stored the working directory yourself.

This means that no library code can ever count on, for example,
being able to reliably find the path to the file that contains the
definition of __main__.  That's a weakness, IMO.  One manifestation
of this weakness is that os.chdir breaks inspect.getmodule, at
least on Unix.  If you have some Unix system handy, you can try
the following.  First change the argument to os.chdir below to some
valid directory other than your working directory.  Then, run the
script, making sure that you refer to it using a relative path.
When I do this on my system (OS X + Python 2.7.3), the script bombs
at the last print statement, because the second call to inspect.getmodule
(though not the first one) returns None.

import inspect
import os

frame = inspect.currentframe()

print inspect.getmodule(frame).__name__

os.chdir('/some/other/directory') # where '/some/other/directory' is
                                  # different from the initial directory

print inspect.getmodule(frame).__name__

...............

% python demo.py
python demo.py
__main__
Traceback (most recent call last):
  File "demo.py", line 11, in <module>
    print inspect.getmodule(frame).__name__
AttributeError: 'NoneType' object has no attribute '__name__'



I don't know of any way to fix inspect.getmodule that does not
involve, directly or indirectly, keeping a stable record of the
starting directory.

But, who am I kidding?  What needs fixing, right?  That's not a
bug, that's a feature!  Etc.

By now I have learned to expect that 99.99% of Python programmers
will find that there's nothing wrong with behavior like the one
described above, that it is in fact exactly As It Should Be, because,
you see, since Python is the epitome of perfection, it follows
inexorably that any flaw or shortcoming one may *perceive* in Python
is only an *illusion*: the flaw or shortcoming is really in the
benighted programmer, for having stupid ideas about programming
(i.e. any idea that may entail that Python is not *gasp* perfect).
Pardon my cynicism, but the general vibe from the replies I've
gotten to my post (i.e. "if Python ain't got it, it means you don't
need it") is entirely in line with these expectations.

[toc] | [prev] | [next] | [standalone]

#27450

On Sun, Aug 19, 2012 at 9:57 PM, kj <no.email@please.post> wrote:
> By now I have learned to expect that 99.99% of Python programmers
> will find that there's nothing wrong with behavior like the one
> described above, that it is in fact exactly As It Should Be, because,
> you see, since Python is the epitome of perfection, it follows
> inexorably that any flaw or shortcoming one may *perceive* in Python
> is only an *illusion*: the flaw or shortcoming is really in the
> benighted programmer, for having stupid ideas about programming
> (i.e. any idea that may entail that Python is not *gasp* perfect).
> Pardon my cynicism, but the general vibe from the replies I've
> gotten to my post (i.e. "if Python ain't got it, it means you don't
> need it") is entirely in line with these expectations.

Since you have no respect for the people you're writing to, why
bother?  I know I certainly have no desire to spend any time at all on
your problem when you say things like that.  Perhaps you're looking
for for the argument clinic instead?

http://www.youtube.com/watch?v=RDjCqjzbvJY

-- 
Jerry

[toc] | [prev] | [next] | [standalone]

#27451

On Monday, 20 August 2012 11:57:46 UTC+10, kj  wrote:
> This means that no library code can ever count on, for example,
> being able to reliably find the path to the file that contains the
> definition of __main__.  That's a weakness, IMO.

No, it's not. It's a _strength_. If you've written a library that requires absolute knowledge of its installed location in order for its internals to work, then I'm not installing your library.

> When I do this on my system (OS X + Python 2.7.3), the script bombs
> at the last print statement, because the second call to inspect.getmodule
> (though not the first one) returns None.

So, uh, do something sane like test for the result of inspect.getmodule _before_ trying to do something invalid to it?

> I don't know of any way to fix inspect.getmodule that does not
> involve, directly or indirectly, keeping a stable record of the
> starting directory.

Then _that is the answer_. YOU need to keep "a stable record":

    import inspect 
    import os 
    
    THIS_FILE = os.path.join(os.getcwd(), '<this_module_name>.py')
    
    frame = inspect.currentframe() 
    print inspect.getmodule(frame).__name__ 
    
    os.chdir('/some/other/directory')
    
    print inspect.getmodule(frame, _filename=THIS_FILE).__name__ 

> But, who am I kidding?  What needs fixing, right?  That's not a
> bug, that's a feature!  Etc.

Right. Because that sort of introspection of objects is rare, why burden the _entire_ language with an obligation that is only required in a few places?

> By now I have learned to expect that 99.99% of Python programmers
> will find that [blah blah blah, whine whine whine]. 
> Pardon my cynicism, but the general vibe from the replies I've
> gotten to my post (i.e. "if Python ain't got it, it means you don't
> need it") is entirely in line with these expectations.

Oh my god, how DARE people with EXPERIENCE in a language challenge the PRECONCEPTIONS of an AMATEUR!!! HOW DARE THEY?!?!

[toc] | [prev] | [next] | [standalone]

#27452

My apologies for any double-ups and bad formatting. The new Google Groups interface seems to have effectively shat away decades of UX for something that I can only guess was generated randomly.

[toc] | [prev] | [next] | [standalone]

#27468

On 20/08/2012 04:04, alex23 wrote:
> My apologies for any double-ups and bad formatting. The new Google Groups interface seems to have effectively shat away decades of UX for something that I can only guess was generated randomly.
>

It's very useful for reporting spam.  Otherwise Thunderbird is go :)

-- 
Cheers.

Mark Lawrence.

[toc] | [prev] | [next] | [standalone]

#27453

On Monday, 20 August 2012 12:57:44 UTC+10, alex23  wrote:
> a library that requires absolute knowledge of its 
> installed location in order for its internals to work

That should read: "a library that requires derivation of its installed location from the current working directory in order" etc

[toc] | [prev] | [next] | [standalone]

#27455

On Monday, 20 August 2012 12:57:44 UTC+10, alex23  wrote:
> If you've written a library that requires absolute 
> knowledge of its installed location in order for its 
> internals to work, then I'm not installing your library.

That should read: "requires the ability to derive absolute knowledge of its installed location for an application's current working directory".

There's nothing wrong with a library knowing where shit is. But again, that should in NO WAY be bound to the application working directory.

[toc] | [prev] | [next] | [standalone]

#27548

alex23 <wuwei23@gmail.com> writes:
> Oh my god, how DARE people with EXPERIENCE in a language challenge the
> PRECONCEPTIONS of an AMATEUR!!! HOW DARE THEY?!?!

+1 QOTW :)

[toc] | [prev] | [next] | [standalone]

#27454

On Mon, 20 Aug 2012 01:57:46 +0000, kj wrote:

> In <roy-CA6D77.17031119082012@news.panix.com> Roy Smith <roy@panix.com>
> writes:
> 
>>In article <k0rj38$2gc$1@reader1.panix.com>, kj <no.email@please.post>
>>wrote:
> 
>>> As far as I've been able to determine, Python does not remember
>>> (immutably, that is) the working directory at the program's start-up,
>>> or, if it does, it does not officially expose this information.
[...]
> This means that no library code can ever count on, for example, being
> able to reliably find the path to the file that contains the definition
> of __main__.

Possibly not in the face of chmod.


> That's a weakness, IMO.

Certainly.


> One manifestation of this
> weakness is that os.chdir breaks inspect.getmodule, at least on Unix.

At the very least, it can do so under some circumstances.

By the way, inspect.currentframe is quite problematic. Although it is 
given a public name, it is just a thin wrapper around sys._getframe which 
is not just *named* as a private function, but also states clearly in the 
documentation:

"This function should be used for internal and specialized purposes only."

So I'm not sure that fixing problems with inspect.currentframe will be 
high on anyone's priority.


[...]
> I don't know of any way to fix inspect.getmodule that does not involve,
> directly or indirectly, keeping a stable record of the starting
> directory.

I'm not convinced that this is necessary for the example you give. 
Perhaps modules need to store absolute pathnames rather than relative 
ones?

os.chdir probably breaks a lot of things. Python operates under the 
assumption that the working directory does not change. If you change it, 
it's your responsibility to deal with it.

Apart from shell scripting languages, do other languages cope well with 
having the working directory changed?

 
> But, who am I kidding?  What needs fixing, right?  That's not a bug,
> that's a feature!  Etc.

Not so much. More of, "yes, that's a bug, but it's not an important bug. 
If you have a patch for fixing it, we'll consider it, but if you don't, 
we've got about two thousand more important bugs and features to get 
through first."

Your chances of getting this fixed will be *much* improved if you can 
demonstrate a genuine problem that can't easily be fixed by just telling 
the programmer "don't change working directories, or if you do, you need 
to change back again before doing X, Y or Z".

Personally, I think that trying to fix all the things that break if you 
chdir is not worthwhile, but having Python record the directory you 
started with in (say) sys.startingdirectory might be a worthwhile feature 
request.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#27456

On Monday, 20 August 2012 13:38:03 UTC+10, Steven D'Aprano  wrote:
> Not so much. More of, "yes, that's a bug, but it's not an important bug. 
> If you have a patch for fixing it, we'll consider it, but if you don't, 
> we've got about two thousand more important bugs and features to get 
> through first."

I'm not really convinced it's a bug, though. The issue only happens if the module is the main application, so wrapping this behind a launcher will resolve the issue. Given that you can manually provide the filepath to inspect, I'd say it's a known limitation with a workaround.

[toc] | [prev] | [next] | [standalone]

#27467

On 20/08/2012 02:57, kj wrote:
> In <roy-CA6D77.17031119082012@news.panix.com> Roy Smith <roy@panix.com> writes:
>
>> In article <k0rj38$2gc$1@reader1.panix.com>, kj <no.email@please.post>
>> wrote:
>
>>> As far as I've been able to determine, Python does not remember
>>> (immutably, that is) the working directory at the program's start-up,
>>> or, if it does, it does not officially expose this information.
>
>> Why would you expect that it would?  What would it (or you) do with this
>> information?
>
>> More to the point, doing a chdir() is not something any library code
>> would do (at least not that I'm aware of), so if the directory changed,
>> it's because some application code did it.  In which case, you could
>> have just stored the working directory yourself.
>
> This means that no library code can ever count on, for example,
> being able to reliably find the path to the file that contains the
> definition of __main__.  That's a weakness, IMO.  One manifestation
> of this weakness is that os.chdir breaks inspect.getmodule, at
> least on Unix.  If you have some Unix system handy, you can try
> the following.  First change the argument to os.chdir below to some
> valid directory other than your working directory.  Then, run the
> script, making sure that you refer to it using a relative path.
> When I do this on my system (OS X + Python 2.7.3), the script bombs
> at the last print statement, because the second call to inspect.getmodule
> (though not the first one) returns None.
>
> import inspect
> import os
>
> frame = inspect.currentframe()
>
> print inspect.getmodule(frame).__name__
>
> os.chdir('/some/other/directory') # where '/some/other/directory' is
>                                    # different from the initial directory
>
> print inspect.getmodule(frame).__name__
>
> ...............
>
> % python demo.py
> python demo.py
> __main__
> Traceback (most recent call last):
>    File "demo.py", line 11, in <module>
>      print inspect.getmodule(frame).__name__
> AttributeError: 'NoneType' object has no attribute '__name__'
>
>
>
> I don't know of any way to fix inspect.getmodule that does not
> involve, directly or indirectly, keeping a stable record of the
> starting directory.
>
> But, who am I kidding?  What needs fixing, right?  That's not a
> bug, that's a feature!  Etc.
>
> By now I have learned to expect that 99.99% of Python programmers
> will find that there's nothing wrong with behavior like the one
> described above, that it is in fact exactly As It Should Be, because,
> you see, since Python is the epitome of perfection, it follows
> inexorably that any flaw or shortcoming one may *perceive* in Python
> is only an *illusion*: the flaw or shortcoming is really in the
> benighted programmer, for having stupid ideas about programming
> (i.e. any idea that may entail that Python is not *gasp* perfect).
> Pardon my cynicism, but the general vibe from the replies I've
> gotten to my post (i.e. "if Python ain't got it, it means you don't
> need it") is entirely in line with these expectations.
>

I see, I see, I get the picture.  You're in the "The OS is flawed so I 
expect the Python core developers to do my work for me by producing code 
that gets around every known flaw in every supported OS" club.  Somehow 
I don't think that will happen.

-- 
Cheers.

Mark Lawrence.

[toc] | [prev] | [next] | [standalone]

#27482

2012/8/20 kj <no.email@please.post>:
> In <roy-CA6D77.17031119082012@news.panix.com> Roy Smith <roy@panix.com> writes> This means that no library code can ever count on, for example,
> being able to reliably find the path to the file that contains the
> definition of __main__.  That's a weakness, IMO.  One manifestation
> of this weakness is that os.chdir breaks inspect.getmodule, at
> least on Unix.  If you have some Unix system handy, you can try
> the following.  First change the argument to os.chdir below to some
> valid directory other than your working directory.  Then, run the
> script, making sure that you refer to it using a relative path.
> When I do this on my system (OS X + Python 2.7.3), the script bombs
> at the last print statement, because the second call to inspect.getmodule
> (though not the first one) returns None.
>
> import inspect
> import os
>
> frame = inspect.currentframe()
>
> print inspect.getmodule(frame).__name__
>
> os.chdir('/some/other/directory') # where '/some/other/directory' is
>                                   # different from the initial directory
>
> print inspect.getmodule(frame).__name__
>
> ...............
>
> % python demo.py
> python demo.py
> __main__
> Traceback (most recent call last):
>   File "demo.py", line 11, in <module>
>     print inspect.getmodule(frame).__name__
> AttributeError: 'NoneType' object has no attribute '__name__'
>
>..

As in many other cases the programming language can't possibly act
safely on all the possible stupid things that the programmer wants to
do, and not understanding how an operating system works doesn't help
either..

In the specific case there is absolutely no use of os.chdir, since you
can:
- use absolute paths
- things like subprocess.Popen accept a cwd argument
- at worst you can chdir back to the previous position right after the
broken thing that require a certain path that you are calling is run

[toc] | [prev] | [next] | [standalone]

#27487

On Mon, 20 Aug 2012 12:56:42 +0100, andrea crotti wrote:

> In the specific case there is absolutely no use of os.chdir, since you
> can:
> - use absolute paths
> - things like subprocess.Popen accept a cwd argument - at worst you can
> chdir back to the previous position right after the broken thing that
> require a certain path that you are calling is run


I wouldn't say so much that there's "absolutely no use", it's more that 
there are other, safer, ways to get the same result.

As I understand it, os.chdir is more for the convenience of sys admins 
who want to write quick scripts in Python than a function intended to be 
used in libraries or major applications.

An interesting question is, what do other languages do in this case?



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#27492

On Mon, 20 Aug 2012 13:14:02 +0000, Steven D'Aprano wrote:

> On Mon, 20 Aug 2012 12:56:42 +0100, andrea crotti wrote:
> 
>> In the specific case there is absolutely no use of os.chdir, since you
>> can:
>> - use absolute paths - things like subprocess.Popen accept a cwd
>> argument - at worst you can chdir back to the previous position right
>> after the broken thing that require a certain path that you are calling
>> is run
> 
> 
> I wouldn't say so much that there's "absolutely no use", it's more that
> there are other, safer, ways to get the same result.
> 
> As I understand it, os.chdir is more for the convenience of sys admins
> who want to write quick scripts in Python than a function intended to be
> used in libraries or major applications.
> 

I don't disagree, but in my experience few sysadmins use Python; they use 
shell scripts. And these seldom do a 'cd' anyway - they will normally 
operate irrespective of the current working directory.

It is difficult to think of a sensible use for os.chdir, IMHO.

> An interesting question is, what do other languages do in this case?

[toc] | [prev] | [next] | [standalone]

#27494

In article <k0tf8g$adc$1@news.albasani.net>,
 Walter Hurry <walterhurry@lavabit.com> wrote:

> It is difficult to think of a sensible use for os.chdir, IMHO.

It is true that you can mostly avoid chdir() by building absolute 
pathnames, but it's often more convenient to just cd somewhere and use 
names relative to that.  Fabric (a very cool tool for writing remote 
sysadmin scripts), gives you a cd() command which is a context manager, 
making it extra convenient.

Also, core files get created in the current directory.  Sometimes 
daemons will cd to some fixed location to make sure that if they dump 
core, it goes in the right place.

On occasion, you run into (poorly designed, IMHO) utilities which insist 
of reading or writing a file in the current directory.  If you're 
invoking one of those, you may have no choice but to chdir() to the 
right place before running them.

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.python

csiph-web