Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #47421 > unrolled thread

Listing modules from all installed packages

Back to article view | Back to comp.lang.python

Contents

  Listing modules from all installed packages Julien Phalip <jphalip@gmail.com> - 2013-06-08 21:30 -0700
    RE: Listing modules from all installed packages Carlos Nepomuceno <carlosnepomuceno@outlook.com> - 2013-06-09 08:23 +0300
      Re: Listing modules from all installed packages 88888 Dihedral <dihedral88888@gmail.com> - 2013-06-09 11:57 -0700
    RE: Listing modules from all installed packages Carlos Nepomuceno <carlosnepomuceno@outlook.com> - 2013-06-09 09:22 +0300
      Re: Listing modules from all installed packages cclauss@bluewin.ch - 2013-06-09 05:51 -0700
      Re: Listing modules from all installed packages Julien Phalip <jphalip@gmail.com> - 2013-06-14 12:18 -0700

#47421 — Listing modules from all installed packages

Hi,

I'm trying to write a function that programmatically obtains and returns the exact location of all first-level modules for all installed packages.

For example, if the packages named 'django' and 'django-debug-toolbar' are installed, I'd like this function to return something like:
>>> installed_modules()
/Users/my_user/.virtualenvs/my_venv/lib/python2.6/site-packages/django
/Users/my_user/.virtualenvs/my_venv/src/debug_toolbar

That is, this function needs to consider all installed packages, including those that have been installed in "edit" mode (i.e. in the src/ folder). Note also that the main module for the 'django-debug-toolbar' is in fact named 'debug_toolbar'.

So far the closest I've been to retrieving the list of first-level modules is as follows:

    import os
    import pkg_resources
    import setuptools

    pkgs = set()

    for dist in pkg_resources.working_set:
        if os.path.isdir(dist.location):
            for pkg in setuptools.find_packages(dist.location):
                if '.' not in pkg:
                    pkgs.add(pkg)

The idea is then to loop through that list of modules, import them and get their exact locations by fetching their __file__ attribute values.

However, this feels very hackish and I don't think it's actually quite correct either. I'm sure there must be a better way. If possible I'd also like to avoid having to use setuptools.

Do you have any tips on how to achieve this?

Many thanks!

Julien

[toc] | [next] | [standalone]

#47424

[Multipart message — attachments visible in raw view] — view raw

print '\n'.join([re.findall("from '(.*)'",str(v))[0] for k,v in sys.modules.items() if str(v).find('from')>-1])


> Date: Sat, 8 Jun 2013 21:30:48 -0700
> Subject: Listing modules from all installed packages
> From: jphalip@gmail.com
> To: python-list@python.org
> 
> Hi,
> 
> I'm trying to write a function that programmatically obtains and returns the exact location of all first-level modules for all installed packages.
> 
> For example, if the packages named 'django' and 'django-debug-toolbar' are installed, I'd like this function to return something like:
> >>> installed_modules()
> /Users/my_user/.virtualenvs/my_venv/lib/python2.6/site-packages/django
> /Users/my_user/.virtualenvs/my_venv/src/debug_toolbar
> 
> That is, this function needs to consider all installed packages, including those that have been installed in "edit" mode (i.e. in the src/ folder). Note also that the main module for the 'django-debug-toolbar' is in fact named 'debug_toolbar'.
> 
> So far the closest I've been to retrieving the list of first-level modules is as follows:
> 
>     import os
>     import pkg_resources
>     import setuptools
> 
>     pkgs = set()
> 
>     for dist in pkg_resources.working_set:
>         if os.path.isdir(dist.location):
>             for pkg in setuptools.find_packages(dist.location):
>                 if '.' not in pkg:
>                     pkgs.add(pkg)
> 
> The idea is then to loop through that list of modules, import them and get their exact locations by fetching their __file__ attribute values.
> 
> However, this feels very hackish and I don't think it's actually quite correct either. I'm sure there must be a better way. If possible I'd also like to avoid having to use setuptools.
> 
> Do you have any tips on how to achieve this?
> 
> Many thanks!
> 
> Julien
> -- 
> http://mail.python.org/mailman/listinfo/python-list
 		 	   		  

[toc] | [prev] | [next] | [standalone]

#47481

Carlos Nepomuceno於 2013年6月9日星期日UTC+8下午1時23分15秒寫道：
> print '\n'.join([re.findall("from '(.*)'",str(v))[0] for k,v in sys.modules.items() if str(v).find('from')>-1])
> 
> 
> 
> > Date: Sat, 8 Jun 2013 21:30:48 -0700
> > Subject: Listing modules from all installed packages
> > From: jph...@gmail.com
> > To: pytho...@python.org
> > 
> > Hi,
> > 
> > I'm trying to write a function that programmatically obtains and returns the exact location of all first-level modules for all installed packages.
> > 
> > For example, if the packages named 'django' and 'django-debug-toolbar' are installed, I'd like this function to return something like:
> > >>> installed_modules()
> > /Users/my_user/.virtualenvs/my_venv/lib/python2.6/site-packages/django
> > /Users/my_user/.virtualenvs/my_venv/src/debug_toolbar
> > 
> > That is, this function needs to consider all installed packages, including those that have been installed in "edit" mode (i.e. in the src/ folder). Note also that the main module for the 'django-debug-toolbar' is in fact named 'debug_toolbar'.
> > 
> > So far the closest I've been to retrieving the list of first-level modules is as follows:
> > 
> >     import os
> >     import pkg_resources
> >     import setuptools
> > 
> >     pkgs = set()
> > 
> >     for dist in pkg_resources.working_set:
> >         if os.path.isdir(dist.location):
> >             for pkg in setuptools.find_packages(dist.location):
> >                 if '.' not in pkg:
> >                     pkgs.add(pkg)
> > 
> > The idea is then to loop through that list of modules, import them and get their exact locations by fetching their __file__ attribute values.
> > 
> > However, this feels very hackish and I don't think it's actually quite correct either. I'm sure there must be a better way. If possible I'd also like to avoid having to use setuptools.
> > 
> > Do you have any tips on how to achieve this?
> > 
> > Many thanks!
> > 
> > Julien
> > -- 
> > http://mail.python.org/mailman/listinfo/python-list

Please use a dictionary to store a tree first.
Then it is trivial to walk through all nodes of the tree.

[toc] | [prev] | [next] | [standalone]

#47426

[Multipart message — attachments visible in raw view] — view raw

Just realized that you've asked for installed packages. Perhaps the following will do the trick. I don't know why the 'lib-tk' isn't included. Why not?

toplevel_packages = ['%s\\%s'%(ml.path,name)for ml,name,ispkg in pkgutil.iter_modules() if ispkg]
print '\n'.join(toplevel_packages)

> Date: Sat, 8 Jun 2013 21:30:48 -0700
> Subject: Listing modules from all installed packages
> From: jphalip@gmail.com
> To: python-list@python.org
> 
> Hi,
> 
> I'm trying to write a function that programmatically obtains and returns the exact location of all first-level modules for all installed packages.
> 
> For example, if the packages named 'django' and 'django-debug-toolbar' are installed, I'd like this function to return something like:
> >>> installed_modules()
> /Users/my_user/.virtualenvs/my_venv/lib/python2.6/site-packages/django
> /Users/my_user/.virtualenvs/my_venv/src/debug_toolbar
> 
> That is, this function needs to consider all installed packages, including those that have been installed in "edit" mode (i.e. in the src/ folder). Note also that the main module for the 'django-debug-toolbar' is in fact named 'debug_toolbar'.
> 
> So far the closest I've been to retrieving the list of first-level modules is as follows:
> 
>     import os
>     import pkg_resources
>     import setuptools
> 
>     pkgs = set()
> 
>     for dist in pkg_resources.working_set:
>         if os.path.isdir(dist.location):
>             for pkg in setuptools.find_packages(dist.location):
>                 if '.' not in pkg:
>                     pkgs.add(pkg)
> 
> The idea is then to loop through that list of modules, import them and get their exact locations by fetching their __file__ attribute values.
> 
> However, this feels very hackish and I don't think it's actually quite correct either. I'm sure there must be a better way. If possible I'd also like to avoid having to use setuptools.
> 
> Do you have any tips on how to achieve this?
> 
> Many thanks!
> 
> Julien
> -- 
> http://mail.python.org/mailman/listinfo/python-list
 		 	   		  

[toc] | [prev] | [next] | [standalone]

#47460

Adding : python -c 'help("modules") to the other two suggestions:

#!/usr/bin/env python

import commands, pkgutil, re, sys

print('sys.modules.items()...')
print('\n'.join(sorted([re.findall("from '(.*)'",str(v))[0] for k,v in sys.modules.items() if str(v).find('from')>-1])))

print('\npkgutil.iter_modules()...')
toplevel_packages = ['%s\\%s'%(ml.path,name)for ml,name,ispkg in sorted(pkgutil.iter_modules()) if ispkg]
print '\n'.join(toplevel_packages)

theCommand = "python -c 'help(\"modules\")'"
print('\n{} # this may take a few seconds...'.format(theCommand))
print(commands.getstatusoutput(theCommand)[1]) # help() only works in the python interpreter...

[toc] | [prev] | [next] | [standalone]

#48212

On Saturday, June 8, 2013 11:22:16 PM UTC-7, Carlos Nepomuceno wrote:
> Just realized that you've asked for installed packages. Perhaps the following will do the trick. I don't know why the 'lib-tk' isn't included. Why not?
> 
> toplevel_packages = ['%s\\%s'%(ml.path,name)for ml,name,ispkg in pkgutil.iter_modules() if ispkg]
> print '\n'.join(toplevel_packages)

Thanks a lot Carlos, this gives me exactly what I needed!

Best wishes,

Julien

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web