Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #19612 > unrolled thread

Disable use of pyc file with no matching py file

Back to article view | Back to comp.lang.python

Contents

  Disable use of pyc file with no matching py file Roy Smith <roy@panix.com> - 2012-01-30 13:30 -0800
    Re: Disable use of pyc file with no matching py file Miki Tebeka <miki.tebeka@gmail.com> - 2012-01-30 14:13 -0800
    Re: Disable use of pyc file with no matching py file Terry Reedy <tjreedy@udel.edu> - 2012-01-30 17:43 -0500
      Re: Disable use of pyc file with no matching py file John Roth <johnroth1@gmail.com> - 2012-01-31 12:20 -0800
        Re: Disable use of pyc file with no matching py file Terry Reedy <tjreedy@udel.edu> - 2012-01-31 18:43 -0500
          Re: Disable use of pyc file with no matching py file John Roth <johnroth1@gmail.com> - 2012-02-01 05:11 -0800
            Re: Disable use of pyc file with no matching py file Terry Reedy <tjreedy@udel.edu> - 2012-02-01 14:57 -0500
    Re: Disable use of pyc file with no matching py file Jean-Michel Pichavant <jeanmichel@sequans.com> - 2012-01-31 11:26 +0100
      Re: Disable use of pyc file with no matching py file Ben Finney <ben+python@benfinney.id.au> - 2012-01-31 22:56 +1100
      Re: Disable use of pyc file with no matching py file Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-01-31 12:01 +0000
        Re: Disable use of pyc file with no matching py file Jean-Michel Pichavant <jeanmichel@sequans.com> - 2012-01-31 15:19 +0100
        Re: Disable use of pyc file with no matching py file Terry Reedy <tjreedy@udel.edu> - 2012-01-31 18:55 -0500
          Re: Disable use of pyc file with no matching py file Roy Smith <roy@panix.com> - 2012-01-31 23:14 -0500
            Re: Disable use of pyc file with no matching py file Terry Reedy <tjreedy@udel.edu> - 2012-02-01 05:28 -0500
        Re: Disable use of pyc file with no matching py file Jean-Michel Pichavant <jeanmichel@sequans.com> - 2012-02-01 11:12 +0100
        Re: Disable use of pyc file with no matching py file Devin Jeanpierre <jeanpierreda@gmail.com> - 2012-02-01 06:14 -0500
        Re: Disable use of pyc file with no matching py file Terry Reedy <tjreedy@udel.edu> - 2012-02-01 14:53 -0500
        Re: Disable use of pyc file with no matching py file Devin Jeanpierre <jeanpierreda@gmail.com> - 2012-02-02 01:42 -0500
        Re: Disable use of pyc file with no matching py file Terry Reedy <tjreedy@udel.edu> - 2012-02-02 04:21 -0500
          Re: Disable use of pyc file with no matching py file Brian <not_here@no-where.net> - 2012-02-12 10:53 -0800
    Re: Disable use of pyc file with no matching py file Andrea Crotti <andrea.crotti.0@gmail.com> - 2012-01-31 11:18 +0000

Page 1 of 2  [1] 2  Next page →

#19612 — Disable use of pyc file with no matching py file

Every so often (typically when refactoring), I'll remove a .py file and forget to remove the corresponding .pyc file.  If I then import the module, python finds the orphaned .pyc and happily imports it.  Usually leading to confusing and hard to debug failures.

Is there some way to globally tell python, "Never import a .pyc unless the corresponding .py file exits"?  Perhaps some environment variable I could set in my .login file?

[toc] | [next] | [standalone]

#19614

Not that I'm aware of.

I have a script that run the test suite, one of the first commands (before calling nosetests) is:
    find . -name '*.py[co]' -exec rm {} \;

This makes sure the tests run in a "clean" environment.

[toc] | [prev] | [next] | [standalone]

#19615

On 1/30/2012 4:30 PM, Roy Smith wrote:
> Every so often (typically when refactoring), I'll remove a .py file
> and forget to remove the corresponding .pyc file.  If I then import
> the module, python finds the orphaned .pyc and happily imports it.
> Usually leading to confusing and hard to debug failures.
>
> Is there some way to globally tell python, "Never import a .pyc
> unless the corresponding .py file exits"?

Upgrade to 3.2.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#19657

On Jan 30, 3:43 pm, Terry Reedy <tjre...@udel.edu> wrote:
> On 1/30/2012 4:30 PM, Roy Smith wrote:
>
> > Every so often (typically when refactoring), I'll remove a .py file
> > and forget to remove the corresponding .pyc file.  If I then import
> > the module, python finds the orphaned .pyc and happily imports it.
> > Usually leading to confusing and hard to debug failures.
>
> > Is there some way to globally tell python, "Never import a .pyc
> > unless the corresponding .py file exits"?
>
> Upgrade to 3.2.
>
> --
> Terry Jan Reedy

Terry,

I've noticed that the tutorial (section 6.1.3) hasn't been updated for
PEP 3147; there's no way of telling that this is the behavior from
reading the tutorial. The development doc for 3.3 hasn't been updated
either.

[toc] | [prev] | [next] | [standalone]

#19663

On 1/31/2012 3:20 PM, John Roth wrote:
> On Jan 30, 3:43 pm, Terry Reedy<tjre...@udel.edu>  wrote:
>> On 1/30/2012 4:30 PM, Roy Smith wrote:
>>
>>> Every so often (typically when refactoring), I'll remove a .py file
>>> and forget to remove the corresponding .pyc file.  If I then import
>>> the module, python finds the orphaned .pyc and happily imports it.
>>> Usually leading to confusing and hard to debug failures.
>>
>>> Is there some way to globally tell python, "Never import a .pyc
>>> unless the corresponding .py file exits"?
>>
>> Upgrade to 3.2.

I tested before writing this. The caveat is that x.pyc in the same 
directly as x.py will not be ignored (for back compatibility). However, 
this only happens intentionally as .pyc files are put in __pycache__/ 
with name x.<version>.pyc, where <version> is 'cpython-32' or something 
similar for another version or implementation.

> I've noticed that the tutorial (section 6.1.3) hasn't been updated for
> PEP 3147; there's no way of telling that this is the behavior from
> reading the tutorial. The development doc for 3.3 hasn't been updated
> either.

You are right. An oversight. Thanks for noticing.
http://bugs.python.org/issue13915
Suggested rewrites are welcome.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#19709

On Jan 31, 4:43 pm, Terry Reedy <tjre...@udel.edu> wrote:
> On 1/31/2012 3:20 PM, John Roth wrote:
>
> > On Jan 30, 3:43 pm, Terry Reedy<tjre...@udel.edu>  wrote:
> >> On 1/30/2012 4:30 PM, Roy Smith wrote:
>
> >>> Every so often (typically when refactoring), I'll remove a .py file
> >>> and forget to remove the corresponding .pyc file.  If I then import
> >>> the module, python finds the orphaned .pyc and happily imports it.
> >>> Usually leading to confusing and hard to debug failures.
>
> >>> Is there some way to globally tell python, "Never import a .pyc
> >>> unless the corresponding .py file exits"?
>
> >> Upgrade to 3.2.
>
> I tested before writing this. The caveat is that x.pyc in the same
> directly as x.py will not be ignored (for back compatibility). However,
> this only happens intentionally as .pyc files are put in __pycache__/
> with name x.<version>.pyc, where <version> is 'cpython-32' or something
> similar for another version or implementation.
>
> > I've noticed that the tutorial (section 6.1.3) hasn't been updated for
> > PEP 3147; there's no way of telling that this is the behavior from
> > reading the tutorial. The development doc for 3.3 hasn't been updated
> > either.
>
> You are right. An oversight. Thanks for noticing.http://bugs.python.org/issue13915
> Suggested rewrites are welcome.
>
> --
> Terry Jan Reedy

I'll see if I can put a suggestion in the bug report.

One other point: I'm unclear if a compiled module in the source
directory would be named spam.pyc or spam.cpython-32.pyc. I'd think
the latter to allow two versions of a compiled-only distribution.

John Roth

[toc] | [prev] | [next] | [standalone]

#19737

On 2/1/2012 8:11 AM, John Roth wrote:

> One other point: I'm unclear if a compiled module in the source
> directory would be named spam.pyc or spam.cpython-32.pyc. I'd think
> the latter to allow two versions of a compiled-only distribution.

By test, it has to be spam.pyc, as before.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#19626

Terry Reedy wrote:
> On 1/30/2012 4:30 PM, Roy Smith wrote:
>> Every so often (typically when refactoring), I'll remove a .py file
>> and forget to remove the corresponding .pyc file.  If I then import
>> the module, python finds the orphaned .pyc and happily imports it.
>> Usually leading to confusing and hard to debug failures.
>>
>> Is there some way to globally tell python, "Never import a .pyc
>> unless the corresponding .py file exits"?
>
> Upgrade to 3.2.
>
No.

JM

[toc] | [prev] | [next] | [standalone]

#19629

Jean-Michel Pichavant <jeanmichel@sequans.com> writes:

> Terry Reedy wrote:
> > On 1/30/2012 4:30 PM, Roy Smith wrote:
> >> Is there some way to globally tell python, "Never import a .pyc
> >> unless the corresponding .py file exits"?
> >
> > Upgrade to 3.2.
> >
> No.

Terry's response was an answer to “Is there some way to [do what Roy
Smith asked how to do]?”. Are you saying that upgrading to Python 3.2
doesn't work for that?

-- 
 \        “Faith is belief without reason. I think as soon as you turn |
  `\          your back on reason, you turn your back on what makes us |
_o__)                               human.” —Iain M. Banks, 2010-10-07 |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#19630

On Tue, 31 Jan 2012 11:26:10 +0100, Jean-Michel Pichavant wrote:

> Terry Reedy wrote:
>> On 1/30/2012 4:30 PM, Roy Smith wrote:
>>> Every so often (typically when refactoring), I'll remove a .py file
>>> and forget to remove the corresponding .pyc file.  If I then import
>>> the module, python finds the orphaned .pyc and happily imports it.
>>> Usually leading to confusing and hard to debug failures.
>>>
>>> Is there some way to globally tell python, "Never import a .pyc unless
>>> the corresponding .py file exits"?
>>
>> Upgrade to 3.2.
>>
> No.


Is that intended as "No, I won't upgrade" or "No, Python 3.2 doesn't do 
the job"?

If the first one, then that's your decision, of course, but there really 
is no other straight-forward way to have Python ignore left-over .pyc 
files. Like it or not, there is no way to disable the use of .pyc files 
in older versions of Python. 

(I suppose you could write a patch for Python, and use your custom 
version, but that's it.)

The ability to use .pyc files without providing source code is considered 
by some people a feature, not a bug, and in fact even in Python 3.2 the 
same applies. The difference in 3.2 is that you can't *accidentally* use 
old left-over .pyc files, you have to move and rename them before they 
can be used as sourceless modules.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#19639

Steven D'Aprano wrote:
> On Tue, 31 Jan 2012 11:26:10 +0100, Jean-Michel Pichavant wrote:
>
>   
>> Terry Reedy wrote:
>>     
>>> On 1/30/2012 4:30 PM, Roy Smith wrote:
>>>       
>>>> Every so often (typically when refactoring), I'll remove a .py file
>>>> and forget to remove the corresponding .pyc file.  If I then import
>>>> the module, python finds the orphaned .pyc and happily imports it.
>>>> Usually leading to confusing and hard to debug failures.
>>>>
>>>> Is there some way to globally tell python, "Never import a .pyc unless
>>>> the corresponding .py file exits"?
>>>>         
>>> Upgrade to 3.2.
>>>
>>>       
>> No.
>>     
>
>
> Is that intended as "No, I won't upgrade" or "No, Python 3.2 doesn't do 
> the job"?
>   
To answer Ben's mail as well, the "No" would be more of the "don't do 
it". My answer was as argued as Terry's one anyway (it was quite intended).
Steven, you often use analogies/similarities, here's one:

A: "My wheel is flat"
B: "Buy a new car"

Buying a new car would solve A's problem : yes
Should A buy a new car : probably no

Python 3.2 is fine, but someone could run into several issues while 
migrating. This is quite a pretty huge decision to make (I dedicate this 
sentence to Rick, I hope he's trolling fine).

JM

[toc] | [prev] | [next] | [standalone]

#19664

On 1/31/2012 9:19 AM, Jean-Michel Pichavant wrote:

> A: "My wheel is flat"
> B: "Buy a new car"

A better analogy would be

Q. "How do I make my old model car do something (it cannot do)?"
A. "Get the free new model that has that feature added."

Of course, there is a cost to giving up the old and familiar and 
learning and adjusting to the new, even when it is available gratis. A 
husband wearing an old sweater after his wife gives him a new one, and 
even retrieving it from the trash when she tosses it out, is a classic 
(and true) cartoon joke.

But I am sure that 95% of readers here will be using 3.x withing 10 
years. The only question for them is "When?". This not-well-known new 
feature is one straw that some will put on the 'sooner' side of the balance.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#19685

In article <mailman.5264.1328054130.27778.python-list@python.org>,
 Terry Reedy <tjreedy@udel.edu> wrote:

> But I am sure that 95% of readers here will be using 3.x withing 10 
> years. The only question for them is "When?". This not-well-known new 
> feature is one straw that some will put on the 'sooner' side of the balance.

We would love to move to 3.x, for the better unicode support, if nothing 
else.  What's keeping us from doing so is the host of third-party 
modules and tools we depend on that don't yet support 3.x.  Off the top 
of my head (it's possible some of these already have support):

django (I understand it's been done, if not yet officially supported)
pymongo
mongoengine
tornado
pybeanstalk
dateutils
requests
lxml
blinker
gunicorn
I'm sure I've missed a few

It's getting to the point where any open-source python project needs to 
either jump on the 3.x bandwagon or get consigned to obscurity.  I'm 
guessing that's going to start happening in a big way in 2012.

[toc] | [prev] | [next] | [standalone]

#19702

On 1/31/2012 11:14 PM, Roy Smith wrote:

> We would love to move to 3.x, for the better unicode support, if nothing
> else.  What's keeping us from doing so is the host of third-party
> modules and tools we depend on that don't yet support 3.x.

Tell that to the authors of packages you use so they no longer say that 
they have not converted for lack of demand ;-)

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#19701

Terry Reedy wrote:
> On 1/31/2012 9:19 AM, Jean-Michel Pichavant wrote:
>
>> A: "My wheel is flat"
>> B: "Buy a new car"
>
> A better analogy would be
>
> Q. "How do I make my old model car do something (it cannot do)?"
> A. "Get the free new model that has that feature added."
>
> Of course, there is a cost to giving up the old and familiar and 
> learning and adjusting to the new, even when it is available gratis. A 
> husband wearing an old sweater after his wife gives him a new one, and 
> even retrieving it from the trash when she tosses it out, is a classic 
> (and true) cartoon joke.
>
> But I am sure that 95% of readers here will be using 3.x withing 10 
> years. The only question for them is "When?". This not-well-known new 
> feature is one straw that some will put on the 'sooner' side of the 
> balance.
>
A simple solution to that problem has been provided by Miki. Someone 
should read at least http://wiki.python.org/moin/Python2orPython3 before 
migrating to python 3.

Speaking for myself, I'm stuck with python 2.5 :-/

JM

[toc] | [prev] | [next] | [standalone]

#19705

On Tue, Jan 31, 2012 at 6:55 PM, Terry Reedy <tjreedy@udel.edu> wrote:
> Q. "How do I make my old model car do something (it cannot do)?"
> A. "Get the free new model that has that feature added."
>
> Of course, there is a cost to giving up the old and familiar and learning
> and adjusting to the new, even when it is available gratis. A husband
> wearing an old sweater after his wife gives him a new one, and even
> retrieving it from the trash when she tosses it out, is a classic (and true)
> cartoon joke.

It really bothers me that you imagine that there are no other problems
than the newness. It's disheartening, because the problems are not
that trivial and the world would be better if people were less callous
about it, and realized that they exist. Python 3 is not very different
from Python 2, as far as humans are concerned
semantically/syntactically -- but, hell, just pick any project that
uses PyPy, or Jython, or IronPython, or Twisted, or Zope, etc. -- it
can be a lot of effort (sometimes infeasibly much) to port something
dependent on these things, and it's taken years to get the (smallish)
set of dependencies ported that we have now [and we literally paid
people to do it, too!], and still many large projects haven't made the
transition, and many small projects never will.

Anyone that relies on those projects is stuck, and your "free car"
metaphor completely ignores the true cost of wasting that much time
porting everything for a tiny little feature. Evaluating only the
monetary amounts can be misleading as to what the rational decision is
(in particular when there are no monetary amounts). The only true
notion of cost is the alternatives you sacrifice in making a decision:
opportunity cost. The car is not free.

-- Devin

[toc] | [prev] | [next] | [standalone]

#19736

On 2/1/2012 6:14 AM, Devin Jeanpierre wrote:

> It really bothers me that you imagine that there are no other problems
> than the newness.

And it bothers me that you imput such ignorance to me. You made what I 
think was a bad analogy and I made a better one of the same type, though 
still imperfect. I acknowledged that the transition will take years.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#19770

On Wed, Feb 1, 2012 at 2:53 PM, Terry Reedy <tjreedy@udel.edu> wrote:
> And it bothers me that you imput such ignorance to me. You made what I think
> was a bad analogy and I made a better one of the same type, though still
> imperfect. I acknowledged that the transition will take years.

Ah. It is a common attitude among those that make these sorts of
comments about Python 3, and I hadn't read anything in what you said
that made me think that you were considering more than the superficial
costs of moving. I am sorry that I did not give you the benefit of the
doubt.

-- Devin

[toc] | [prev] | [next] | [standalone]

#19782

On 2/2/2012 1:42 AM, Devin Jeanpierre wrote:
> On Wed, Feb 1, 2012 at 2:53 PM, Terry Reedy<tjreedy@udel.edu>  wrote:
>> And it bothers me that you imput such ignorance to me. You made what I think
>> was a bad analogy and I made a better one of the same type, though still
>> imperfect. I acknowledged that the transition will take years.
>
> Ah. It is a common attitude among those that make these sorts of
> comments about Python 3, and I hadn't read anything in what you said
> that made me think that you were considering more than the superficial
> costs of moving.

I thought '95% in 10 years' would be a hint that I know upgrading is not 
trivial for every one ;-).

> I am sorry that I did not give you the benefit of the doubt.

Apology accepted.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#20287

On 2/2/2012 1:21 AM, Terry Reedy wrote:
> On 2/2/2012 1:42 AM, Devin Jeanpierre wrote:
>> On Wed, Feb 1, 2012 at 2:53 PM, Terry Reedy<tjreedy@udel.edu>
>> wrote:
>>> And it bothers me that you imput such ignorance to me. You
>>> made what I think
>>> was a bad analogy and I made a better one of the same type,
>>> though still
>>> imperfect. I acknowledged that the transition will take years.
>>
>> Ah. It is a common attitude among those that make these sorts of
>> comments about Python 3, and I hadn't read anything in what
>> you said
>> that made me think that you were considering more than the
>> superficial
>> costs of moving.
>
> I thought '95% in 10 years' would be a hint that I know
> upgrading is not trivial for every one ;-).
>
>> I am sorry that I did not give you the benefit of the doubt.
>
> Apology accepted.

This is OT, but relevant to the side discussion. Also note that 
the community respects and appreciates the T.J. Reedy 
contributions and hard work.

Reality. It was a monumental task to convert ATE driver dev and 
related factory automation stuff from C to Python starting in 
2000. Most was done guerrilla-style. My employer now appreciates 
the resultant infrastructure that even the Mexico factory 
engineering team can use, and is good for their pride because 
they are no longer totally dependent on gringo engineers.

Am currently being disruptive to their little world with my 
recent (1Q 2011) switch to 3.x. The corporate natives are 
restless and there is talk of an armed insurrection. Humans, at 
the tribal level, do not adapt to change. Expect a long series 
of battles that are ruthless, bloody, and have a high body 
count. Vive la Revolution.

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.python

csiph-web