Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #10859 > unrolled thread

PyWhich

Back to article view | Back to comp.lang.python

Contents

  PyWhich Billy Mays <81282ed9a88799d21e77957df2d84bd6514d9af6@myhashismyemail.com> - 2011-08-04 08:43 -0400
    Re: PyWhich Chris Rebert <clp2@rebertia.com> - 2011-08-04 10:22 -0700
    Re: PyWhich Tim Chase <python.list@tim.thechases.com> - 2011-08-04 13:24 -0500
    Re: PyWhich Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-08-05 10:34 +1000
      Re: PyWhich Tim Chase <python.list@tim.thechases.com> - 2011-08-04 20:04 -0500
        Re: PyWhich Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-08-05 23:51 +1000
          Re: PyWhich Tim Golden <mail@timgolden.me.uk> - 2011-08-05 15:03 +0100
          Re: PyWhich John Gordon <gordon@panix.com> - 2011-08-05 14:33 +0000
      Re: PyWhich Chris Angelico <rosuav@gmail.com> - 2011-08-05 03:03 +0100
        Re: PyWhich Billy Mays <81282ed9a88799d21e77957df2d84bd6514d9af6@myhashismyemail.com> - 2011-08-05 10:43 -0400
          Re: PyWhich Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-08-06 00:56 +1000
    Re: PyWhich Web Dreamer <webdreamer@nospam.fr> - 2011-08-05 16:34 +0200

#10859 — PyWhich

Hey c.l.p.,

I wrote a little python script that finds the file that a python module 
came from.  Does anyone see anything wrong with this script?


#!/usr/bin/python

import sys
if __name__ == '__main__':
     if len(sys.argv) > 1:
         try:
             m = __import__(sys.argv[1])
             sys.stdout.write(m.__file__ + '\n')
             sys.stdout.flush()
             sys.exit(0)
         except ImportError:
             sys.stderr.write("No such module '%s'\n" % sys.argv[1])
             sys.stderr.flush()
             sys.exit(1)
     else:
         sys.stderr.write("Usage: pywhich <module>\n")
         sys.stderr.flush()
         sys.exit(0)


--
Bill

[toc] | [next] | [standalone]

#10865

On Thu, Aug 4, 2011 at 5:43 AM, Billy Mays
<81282ed9a88799d21e77957df2d84bd6514d9af6@myhashismyemail.com> wrote:
> Hey c.l.p.,
>
> I wrote a little python script that finds the file that a python module came
> from.  Does anyone see anything wrong with this script?
>
>
> #!/usr/bin/python
>
> import sys
> if __name__ == '__main__':
>    if len(sys.argv) > 1:
>        try:
>            m = __import__(sys.argv[1])
>            sys.stdout.write(m.__file__ + '\n')
>            sys.stdout.flush()
>            sys.exit(0)
>        except ImportError:
>            sys.stderr.write("No such module '%s'\n" % sys.argv[1])
>            sys.stderr.flush()
>            sys.exit(1)
>    else:
>        sys.stderr.write("Usage: pywhich <module>\n")
>        sys.stderr.flush()
>        sys.exit(0)

Nothing wrong per se, but the flush()es seem unnecessary, and why do
stdout.write() when you can just print()?

Cheers,
Chris
--
I can't not think of the pitchman when I read your posts...
http://rebertia.com

[toc] | [prev] | [next] | [standalone]

#10866

On 08/04/2011 07:43 AM, Billy Mays wrote:
> Hey c.l.p.,
>
> I wrote a little python script that finds the file that a python module
> came from.  Does anyone see anything wrong with this script?
>
> #!/usr/bin/python
>
> import sys
> if __name__ == '__main__':
>       if len(sys.argv)>  1:
>           try:
>               m = __import__(sys.argv[1])
>               sys.stdout.write(m.__file__ + '\n')

For a quick script in a controlled environment, not bad.  In a 
hostile environment, I'd be nervous about running arbitrary 
module code triggered by the import.  Even if non-malicious, some 
imports (like PyCrypto) may have some initialization lag which 
would be nice to avoid.  I think I'd make use of imp.find_module 
to write it something like this (untested)

   from sys import argv, stderr
   import imp
   type_map = {
     imp.PY_SOURCE: "Source file",
     imp.PY_COMPILED: "Compiled code object",
     imp.C_EXTENSION: "Dynamically loadabld shared library",
     imp.PKG_DIRECTORY: "Package directory",
     imp.C_BUILTIN: "Built-in",
     imp.PY_FROZEN: "Frozen module",
     }
   if __name__ == '__main__':
     if len(argv) > 1:
       for modname in argv[1:]:
         try:
           fp, pth, desc = imp.find_module(modname)
           (suffix, mode, type_) = desc
           if fp is not None: fp.close()
           print("%s\t[%s]" % (
             pth,
             type_map.get(type_, "UNKNOWN")
             ))
         except ImportError:
           stderr.write("No such module '%s'\n" % modname)
     else:
       stderr.write("Usage: pywhich <module> [<module>...]\n")

I don't know a good way to tap into other import hooks (such as 
the zipfile import) to augment that type_map dictionary.

-tkc

[toc] | [prev] | [next] | [standalone]

#10885

Billy Mays wrote:

> Hey c.l.p.,
> 
> I wrote a little python script that finds the file that a python module
> came from.  Does anyone see anything wrong with this script?


Yes -- the most screamingly obvious question has to be, why are you writing
directly to sys.stdout instead of just using print?


> #!/usr/bin/python

I believe the recommended, platform independent hash-bang line is

#!/usr/bin/which python


> import sys
> if __name__ == '__main__':
>      if len(sys.argv) > 1:
>          try:
>              m = __import__(sys.argv[1])

The major risk here is whether or not you trust the module enough to import
it and run arbitrary code. The alternative would be a lot more work: you
would have to duplicate the logic of the import statement, search the
PYTHONPATH, look for packages, inside zip files, etc. 


>              sys.stdout.write(m.__file__ + '\n')

Built-in modules don't have a __file__ attribute. The most obvious example:

>>> import builtins  # spelled __builtin__ in Python 2.x
>>> builtins.__file__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute '__file__'

Also, __file__ is not always entirely honest. In Python 3, module.__file__
may report a .py file when it is actually loaded from a .pyc file, even if
the .py file doesn't exist. So if you care about the distinction
between .py, .pyc, .pyo etc. files, looking at __file__ alone will be
inadequate.



>          except ImportError:
>              sys.stderr.write("No such module '%s'\n" % sys.argv[1])

That's not a given -- it may be that the module exists, but importing fails
for some other reason. I would recommend not catching ImportError at all,
and just let the standard Python error handling print the traceback.
Especially for a tool aimed at programmers (who else would be interested in
PyWhich?), hiding useful diagnostic errors and replacing them with a
generic, and potentially wrong, message, is very bad.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#10887

On 08/04/2011 07:34 PM, Steven D'Aprano wrote:
> Billy Mays wrote:
>> #!/usr/bin/python
>
> I believe the recommended, platform independent hash-bang line is
>
> #!/usr/bin/which python

I think you mean

   #!/usr/bin/env python

-tkc

[toc] | [prev] | [next] | [standalone]

#10900

Tim Chase wrote:

> On 08/04/2011 07:34 PM, Steven D'Aprano wrote:
>> Billy Mays wrote:
>>> #!/usr/bin/python
>>
>> I believe the recommended, platform independent hash-bang line is
>>
>> #!/usr/bin/which python
> 
> I think you mean
> 
>    #!/usr/bin/env python


Doh! I *always* conflate env and which. Thank you for the correction.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#10901

On 05/08/2011 14:51, Steven D'Aprano wrote:
> Tim Chase wrote:
>
>> On 08/04/2011 07:34 PM, Steven D'Aprano wrote:
>>> Billy Mays wrote:
>>>> #!/usr/bin/python
>>>
>>> I believe the recommended, platform independent hash-bang line is
>>>
>>> #!/usr/bin/which python
>>
>> I think you mean
>>
>>     #!/usr/bin/env python
>
>
> Doh! I *always* conflate env and which. Thank you for the correction.

And there I was thinking you were making a sly and ironic point
about using a utility to find something on the path in order to
run a utility which finds something on the path...

TJG

[toc] | [prev] | [next] | [standalone]

#10902

In <4e3bf554$0$29976$c3e8da3$5496439d@news.astraweb.com> Steven D'Aprano <steve+comp.lang.python@pearwood.info> writes:

> Doh! I *always* conflate env and which. Thank you for the correction.

Way to say "conflate"!  :-)

-- 
John Gordon                   A is for Amy, who fell down the stairs
gordon@panix.com              B is for Basil, assaulted by bears
                                -- Edward Gorey, "The Gashlycrumb Tinies"

[toc] | [prev] | [next] | [standalone]

#10889

On Fri, Aug 5, 2011 at 1:34 AM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> Especially for a tool aimed at programmers (who else would be interested in
> PyWhich?)

The use that first springs to my mind is debugging import paths etc.
If you have multiple pythons installed and aren't sure that they're
finding the right modules, you could fire up PyWhich on an innocuous
module like math or sys, and see if it's loading it from the right
path. People doing this might not necessarily be programmers, they
might be sysadmins; but you're right that it's most likely this will
be used by competent Python programmers.

ChrisA

[toc] | [prev] | [next] | [standalone]

#10905

On 08/04/2011 10:03 PM, Chris Angelico wrote:
> On Fri, Aug 5, 2011 at 1:34 AM, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info>  wrote:
>> Especially for a tool aimed at programmers (who else would be interested in
>> PyWhich?)
>
> The use that first springs to my mind is debugging import paths etc.
> If you have multiple pythons installed and aren't sure that they're
> finding the right modules, you could fire up PyWhich on an innocuous
> module like math or sys, and see if it's loading it from the right
> path. People doing this might not necessarily be programmers, they
> might be sysadmins; but you're right that it's most likely this will
> be used by competent Python programmers.
>
> ChrisA

I am trying to do debugging.  I have had some trouble with multiple 
python installs with virtualenv, and I was trying to see where given 
modules came from.  I knew about the code execution, but I couldn't 
think of a clean way to just find out the location rather than load it.

The reason I used stdout was because I was going to be using it in a 
tool chain where the stdout might need to be formatted for another 
program to read in.  Thats also why I was catching ImportError since a 
later version of this script might need to do something special with it.

This is also useful to see if python is really using the module you 
think it is.

--
Bill

[toc] | [prev] | [next] | [standalone]

#10907

Billy Mays wrote:

> The reason I used stdout was because I was going to be using it in a
> tool chain where the stdout might need to be formatted for another
> program to read in.

print writes to sys.stdout unless you tell it different.

>>> import sys
>>> import StringIO
>>> capture = StringIO.StringIO()
>>> sys.stdout = capture
>>> print "spam"
>>> sys.stdout = sys.__stdout__  # Restore the real file.
>>> capture.getvalue()
'spam\n'

Syntax for printing elsewhere is ugly as sin in Python 2, but it works:

>>> print >>sys.stderr, "spam"
spam
>>> print >>capture, "ham"
>>> capture.getvalue()
'spam\nham\n'




> Thats also why I was catching ImportError since a 
> later version of this script might need to do something special with it.

Well you better also catch SyntaxError, because a later version of your
script might need to do something special with it too :)

Also RuntimeError, ValueError, TypeError... *wink*


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#10903

Billy Mays a écrit ce jeudi 4 août 2011 14:43 dans 
<j1e45l$jtg$1@speranza.aioe.org> :

> Hey c.l.p.,
> 
> I wrote a little python script that finds the file that a python module
> came from.  Does anyone see anything wrong with this script?
> 
> 
> #!/usr/bin/python
> 
> import sys
> if __name__ == '__main__':
>      if len(sys.argv) > 1:
>          try:
>              m = __import__(sys.argv[1])
>              sys.stdout.write(m.__file__ + '\n')
>              sys.stdout.flush()
>              sys.exit(0)
>          except ImportError:
>              sys.stderr.write("No such module '%s'\n" % sys.argv[1])
>              sys.stderr.flush()
>              sys.exit(1)
>      else:
>          sys.stderr.write("Usage: pywhich <module>\n")
>          sys.stderr.flush()
>          sys.exit(0)

It does not work for all modules:

$ python pywhich.py os
/usr/lib/python2.6/os.pyc

$ python pywhich.py sys
Traceback (most recent call last):
  File "pywhich.py", line 8, in <module>
    sys.stdout.write(m.__file__ + '\n')
AttributeError: 'module' object has no attribute '__file__'

Some modules do not have the __file__ attribute
(usually modules in C)


-- 
Web Dreamer

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web