Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #27742 > unrolled thread

Looking for duplicate modules

Back to article view | Back to comp.lang.python

Contents

  Looking for duplicate modules Roy Smith <roy@panix.com> - 2012-08-23 11:30 -0400
    Re: Looking for duplicate modules rusi <rustompmody@gmail.com> - 2012-08-24 09:22 -0700
    Re: Looking for duplicate modules Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-08-26 12:04 +0000

#27742 — Looking for duplicate modules

We got burned yesterday by a scenario which has burned us before.  We had multiple copies of a module in sys.path.  One (the one we wanted) was in our deployed code tree, the other was in /usr/local/lib/python/ or some such.  It was a particularly confusing situation because we *knew* we had uninstalled the copy in /usr/local/lib.  What we didn't know was that puppet was set up to check to make sure it was there and reinstalled it automagically if it ever disappeared!

What often adds to the confusion in cases like this is that if you use a package manager (such as apt-get) to manage your module deployments, when you uninstall, the .py files get removed, but the .pyc's stay behind.

I'm working on a tool which scans all the directories in sys.path and finds any modules which appear multiple times in the path.  It'll also call out any .pyc's it finds without matching py's.

I know those are not strictly errors, but both of them are, for lack of a better term, "deployment smells" and something to be warned about.  Before I invest too much effort into building it, does such a tool already exist?

We're slowly moving more of our deployment to using virtualenv (with --no-site-packages), but we're not fully there yet.  And even when we get there, there will still be opportunities to make these sorts of mistakes.

---
Roy Smith
roy@panix.com

[toc] | [next] | [standalone]

#27812

On Aug 23, 8:30 pm, Roy Smith <r...@panix.com> wrote:
> We got burned yesterday by a scenario which has burned us before.  We had multiple copies of a module in sys.path.  One (the one we wanted) was in our deployed code tree, the other was in /usr/local/lib/python/ or some such.  It was a particularly confusing situation because we *knew* we had uninstalled the copy in /usr/local/lib.  What we didn't know was that puppet was set up to check to make sure it was there and reinstalled it automagically if it ever disappeared!
>
> What often adds to the confusion in cases like this is that if you use a package manager (such as apt-get) to manage your module deployments, when you uninstall, the .py files get removed, but the .pyc's stay behind.
>
> I'm working on a tool which scans all the directories in sys.path and finds any modules which appear multiple times in the path.  It'll also call out any .pyc's it finds without matching py's.
>
> I know those are not strictly errors, but both of them are, for lack of a better term, "deployment smells" and something to be warned about.  Before I invest too much effort into building it, does such a tool already exist?

Not remotely an answer.
emacs has a list-load-path-shadows for the same/similar issue in
elisp.
As a language moves to being a system (programming) language, it seems
to me necessary that such a functionality is built into the language
and gets called as part of standard installation procedure.

>
> We're slowly moving more of our deployment to using virtualenv (with --no-site-packages), but we're not fully there yet.  And even when we get there, there will still be opportunities to make these sorts of mistakes.
>
> ---
> Roy Smith
> r...@panix.com

[toc] | [prev] | [next] | [standalone]

#27915

On Thu, 23 Aug 2012 11:30:05 -0400, Roy Smith wrote:

> I'm working on a tool which scans all the directories in sys.path and
> finds any modules which appear multiple times in the path.  It'll also
> call out any .pyc's it finds without matching py's.

This sounds like a useful tool. Please consider releasing it under an 
appropriate free software licence (e.g. GPL, MIT, or similar compatible 
licence).



-- 
Steven

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web