Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #103952 > unrolled thread

A mistake which almost went me mad

Back to article view | Back to comp.lang.python

Contents

  A mistake which almost went me mad "ast" <nomail@invalid.com> - 2016-03-03 11:21 +0100
    Re: A mistake which almost went me mad Chris Angelico <rosuav@gmail.com> - 2016-03-03 21:31 +1100
    Re: A mistake which almost went me mad Nick Sarbicki <nick.a.sarbicki@gmail.com> - 2016-03-03 10:43 +0000
    Re: A mistake which almost went me mad Tim Golden <mail@timgolden.me.uk> - 2016-03-03 10:48 +0000
      Re: A mistake which almost went me mad Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2016-03-04 19:19 +1300
    Re: A mistake which almost went me mad Steven D'Aprano <steve@pearwood.info> - 2016-03-03 23:57 +1100
    Re: A mistake which almost went me mad Tim Chase <python.list@tim.thechases.com> - 2016-03-03 05:48 -0600
    Re: A mistake which almost went me mad Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2016-03-03 16:29 +0000
      Re: A mistake which almost went me mad Rob Gaddi <rgaddi@highlandtechnology.invalid> - 2016-03-03 17:53 +0000
    Re: A mistake which almost went me mad Tim Chase <python.list@tim.thechases.com> - 2016-03-03 12:50 -0600
    Re: A mistake which almost went me mad Ian Kelly <ian.g.kelly@gmail.com> - 2016-03-07 09:19 -0700
    Re: A mistake which almost went me mad Random832 <random832@fastmail.com> - 2016-03-07 11:28 -0500
    Re: A mistake which almost went me mad Chris Angelico <rosuav@gmail.com> - 2016-03-08 03:40 +1100
    Re: A mistake which almost went me mad Rick Johnson <rantingrickjohnson@gmail.com> - 2016-03-09 08:34 -0800
      Re: A mistake which almost went me mad Rustom Mody <rustompmody@gmail.com> - 2016-03-09 22:13 -0800
        Re: A mistake which almost went me mad Rick Johnson <rantingrickjohnson@gmail.com> - 2016-03-10 16:54 -0800

#103952 — A mistake which almost went me mad

Hello

This has to be told

I created a file pickle.py in order to test some files
read/write with objects and put it in a directory
which is on my python path. 

Then the nightmare began

- Idle no longer works, window no longer opens 
when double clicked, no errors messsages

- python -m pip list doesnt work, crash with an 
error message related to pickle

I uninstalled python34 and reinstalled it, same problem
I uninstalled python34 and instaled 3.5, same problem

...

I finally understood that pickle.py is the name of the file
containing the official pickle module.

This module is probably used by various python programs,
IDLE, pip ...

Instead of reading official pickle, python read my file ...

[toc] | [next] | [standalone]

#103953

On Thu, Mar 3, 2016 at 9:21 PM, ast <nomail@invalid.com> wrote:
> - python -m pip list doesnt work, crash with an error message related to
> pickle

At this point, you could have come to this list, asking for help - and
posting the *entire* traceback. It may have mentioned a file name,
which would give a strong clue; otherwise, there'd be an error like
this:

rosuav@sikorsky:~/tmp$ echo 'print("Hello, world!")' >pickle.py
rosuav@sikorsky:~/tmp$ python3 -m pip list
Hello, world!
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/runpy.py", line 174, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/usr/local/lib/python3.6/runpy.py", line 133, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/usr/local/lib/python3.6/runpy.py", line 109, in _get_module_details
    __import__(pkg_name)
  File "/usr/local/lib/python3.6/site-packages/pip/__init__.py", line
15, in <module>
    from pip.vcs import git, mercurial, subversion, bazaar  # noqa
  File "/usr/local/lib/python3.6/site-packages/pip/vcs/subversion.py",
line 9, in <module>
    from pip.index import Link
  File "/usr/local/lib/python3.6/site-packages/pip/index.py", line 29,
in <module>
    from pip.wheel import Wheel, wheel_ext
  File "/usr/local/lib/python3.6/site-packages/pip/wheel.py", line 6,
in <module>
    import compileall
  File "/usr/local/lib/python3.6/compileall.py", line 20, in <module>
    from concurrent.futures import ProcessPoolExecutor
  File "/usr/local/lib/python3.6/concurrent/futures/__init__.py", line
17, in <module>
    from concurrent.futures.process import ProcessPoolExecutor
  File "/usr/local/lib/python3.6/concurrent/futures/process.py", line
55, in <module>
    from multiprocessing.connection import wait
  File "/usr/local/lib/python3.6/multiprocessing/connection.py", line
23, in <module>
    from . import reduction
  File "/usr/local/lib/python3.6/multiprocessing/reduction.py", line
32, in <module>
    class ForkingPickler(pickle.Pickler):
AttributeError: module 'pickle' has no attribute 'Pickler'

And someone would have suggested changing to another directory, or
running something like this:

rosuav@sikorsky:~/tmp$ python3 -c 'import pickle; print(pickle.__file__)'
Hello, world!
/home/rosuav/tmp/pickle.py

which shows up the issue.

Don't get mad - get even! Or even better, get help! But show us the
entire exception traceback, not just "got an exception".

ChrisA

[toc] | [prev] | [next] | [standalone]

#103956

On Thu, Mar 3, 2016 at 10:26 AM ast <nomail@invalid.com> wrote:

> Hello
>
> This has to be told
>
> I created a file pickle.py
>

 You could stop there.

The number of times I've had to correct a student for naming their script
"turtle.py".

And the number of times I've caught myself doing it...


............................................________
....................................,.-'"...................``~.,
.............................,.-"..................................."-.,
.........................,/...............................................":,
.....................,?......................................................,
.................../...........................................................,}
................./......................................................,:`^`..}
.............../...................................................,:"........./
..............?.....__.........................................:`.........../
............./__.(....."~-,_..............................,:`........../
.........../(_...."~,_........"~,_....................,:`........_/
..........{.._$;_......"=,_......."-,_.......,.-~-,},.~";/....}
...........((.....*~_......."=-._......";,,./`..../"............../
...,,,___.`~,......"~.,....................`.....}............../
............(....`=-,,.......`........................(......;_,,-"
............/.`~,......`-...................................../
.............`~.*-,.....................................|,./.....,__
,,_..........}.>-._...................................|..............`=~-,
.....`=~-,__......`,.................................
...................`=~-,,.,...............................
................................`:,,...........................`..............__
.....................................`=-,...................,%`>--==``
........................................_..........._,-%.......`
...................................,

[toc] | [prev] | [next] | [standalone]

#103958

On 03/03/2016 10:43, Nick Sarbicki wrote:
> On Thu, Mar 3, 2016 at 10:26 AM ast <nomail@invalid.com> wrote:
> 
>> Hello
>>
>> This has to be told
>>
>> I created a file pickle.py
>>
> 
>  You could stop there.
> 
> The number of times I've had to correct a student for naming their script
> "turtle.py".

A few teachers recently were discussing this on Twitter. One suggested
that his pupils always add their initials to whatever filename they use.
That tends to avoid this issue (as well as helping work out later whose
file is whose on things like Raspberry Pi).

TJG

[toc] | [prev] | [next] | [standalone]

#104014

Tim Golden wrote:
> A few teachers recently were discussing this on Twitter. One suggested
> that his pupils always add their initials to whatever filename they use.

Works well until Lawrence Ian Bernstein writes his own
module called "url"...

-- 
Greg

[toc] | [prev] | [next] | [standalone]

#103962

On Thu, 3 Mar 2016 09:21 pm, ast wrote:

> Hello
> 
> This has to be told
> 
> I created a file pickle.py in order to test some files
> read/write with objects and put it in a directory
> which is on my python path.

[...] 

> I uninstalled python34 and reinstalled it, same problem
> I uninstalled python34 and instaled 3.5, same problem


And now you have learned a very important lesson:

Do not waste your time uninstalling and reinstalling Python until everything
else has failed.


> This module is probably used by various python programs,
> IDLE, pip ...
> 
> Instead of reading official pickle, python read my file ...

Correct.

To diagnose these sorts of problems, start by opening your operating
system's command interpreter shell (PowerShell, command.com, com.exe,
terminal, or whatever it is called in your OS), and launch IDLE from the
shell. If something is broken, you will see the error message.

This is on Linux, but it will work the same way under Windows:



[steve@ando ~]$ touch time.py  # Accidentally shadow the module
[steve@ando ~]$ idle  # you may need to give the full path
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python2.7/idlelib/run.py", line 7, in <module>
    import threading
  File "/usr/local/lib/python2.7/threading.py", line 13, in <module>
    from time import time as _time, sleep as _sleep
ImportError: cannot import name time


This tells you that there is a problem with the module time.py.

It is much easier to debug problems when you can see the error messages!




-- 
Steven

[toc] | [prev] | [next] | [standalone]

#103965

On 2016-03-03 10:43, Nick Sarbicki wrote:
> The number of times I've had to correct a student for naming their
> script "turtle.py".
> 
> And the number of times I've caught myself doing it...

I'm surprised at the number of times I find myself creating an
"email.py" DESPITE KNOWING BETTER EVERY SINGLE TIME.

-tkc

[toc] | [prev] | [next] | [standalone]

#103977

On 3 March 2016 at 11:48, Tim Chase <python.list@tim.thechases.com> wrote:
> On 2016-03-03 10:43, Nick Sarbicki wrote:
>> The number of times I've had to correct a student for naming their
>> script "turtle.py".
>>
>> And the number of times I've caught myself doing it...
>
> I'm surprised at the number of times I find myself creating an
> "email.py" DESPITE KNOWING BETTER EVERY SINGLE TIME.

This mistake is too easy to make and should be fixed in the language
somehow. There's no way that a novice user can know which module names
are implicitly "reserved" by being used somewhere in the stdlib or the
collection of 3rd party modules that they may happen to have
installed.

--
Oscar

[toc] | [prev] | [next] | [standalone]

#103981

Oscar Benjamin wrote:

> On 3 March 2016 at 11:48, Tim Chase <python.list@tim.thechases.com> wrote:
>> On 2016-03-03 10:43, Nick Sarbicki wrote:
>>> The number of times I've had to correct a student for naming their
>>> script "turtle.py".
>>>
>>> And the number of times I've caught myself doing it...
>>
>> I'm surprised at the number of times I find myself creating an
>> "email.py" DESPITE KNOWING BETTER EVERY SINGLE TIME.
>
> This mistake is too easy to make and should be fixed in the language
> somehow. There's no way that a novice user can know which module names
> are implicitly "reserved" by being used somewhere in the stdlib or the
> collection of 3rd party modules that they may happen to have
> installed.
>

Inside of modules it is, under Python 3.  Inside of modules there's a
clear distinction of:
  import thingfromstdlib
  from . import thingfromlocal

A bit of a nuisance at first, but once you get used to it everything
just makes unambiguous sense.

The problem is that this same distinction doesn't get made for
"programs", only for "modules".  So, among other issues, you wind up
unable to run tests inside of module directories because the syntax
becomes wrong.

-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com

Email address domain is currently out of order.  See above to fix.

[toc] | [prev] | [next] | [standalone]

#103994

On 2016-03-03 16:29, Oscar Benjamin wrote:
> On 3 March 2016 at 11:48, Tim Chase <python.list@tim.thechases.com>
> wrote:
> > On 2016-03-03 10:43, Nick Sarbicki wrote:
> >> The number of times I've had to correct a student for naming
> >> their script "turtle.py".
> >>
> >> And the number of times I've caught myself doing it...
> >
> > I'm surprised at the number of times I find myself creating an
> > "email.py" DESPITE KNOWING BETTER EVERY SINGLE TIME.
> 
> This mistake is too easy to make and should be fixed in the language
> somehow. There's no way that a novice user can know which module
> names are implicitly "reserved" by being used somewhere in the
> stdlib or the collection of 3rd party modules that they may happen
> to have installed.

I think that relative imports should ameliorate this, as I usually
hit it when I'm using smtplib which in turn imports "email" (and, in
2.x when it found my local email.py would crash and burn). If it used
a relative import that forced it to find the one in the stdlib, it
should(?) prevent it from finding my local version first.

-tkc

[toc] | [prev] | [next] | [standalone]

#104221

On Thu, Mar 3, 2016 at 11:50 AM, Tim Chase
<python.list@tim.thechases.com> wrote:
> I think that relative imports should ameliorate this, as I usually
> hit it when I'm using smtplib which in turn imports "email" (and, in
> 2.x when it found my local email.py would crash and burn). If it used
> a relative import that forced it to find the one in the stdlib, it
> should(?) prevent it from finding my local version first.

Relative imports only work inside packages. You can't use a relative
import to import one top-level module from another.

Besides, the relative import doesn't help to disambiguate in this
case. The absolute path of the stdlib email module is "email". The
absolute path of the module in your CWD is also "email". Why should a
relative import prefer one over the other? So I would think that even
if it worked, it would still just end up importing the first one it
finds on your sys.path.

[toc] | [prev] | [next] | [standalone]

#104223

On Mon, Mar 7, 2016, at 11:19, Ian Kelly wrote:
> Relative imports only work inside packages. You can't use a relative
> import to import one top-level module from another.
> 
> Besides, the relative import doesn't help to disambiguate in this
> case. The absolute path of the stdlib email module is "email". The
> absolute path of the module in your CWD is also "email". Why should a
> relative import prefer one over the other? So I would think that even
> if it worked, it would still just end up importing the first one it
> finds on your sys.path.

Maybe what we need is a system smarter than sys.path - a clear "import
from stdlib" statement, and a way to designate what directories *contain
stdlib modules*, would be useful.

[toc] | [prev] | [next] | [standalone]

#104226

On Tue, Mar 8, 2016 at 3:19 AM, Ian Kelly <ian.g.kelly@gmail.com> wrote:
> On Thu, Mar 3, 2016 at 11:50 AM, Tim Chase
> <python.list@tim.thechases.com> wrote:
>> I think that relative imports should ameliorate this, as I usually
>> hit it when I'm using smtplib which in turn imports "email" (and, in
>> 2.x when it found my local email.py would crash and burn). If it used
>> a relative import that forced it to find the one in the stdlib, it
>> should(?) prevent it from finding my local version first.
>
> Relative imports only work inside packages. You can't use a relative
> import to import one top-level module from another.
>
> Besides, the relative import doesn't help to disambiguate in this
> case. The absolute path of the stdlib email module is "email". The
> absolute path of the module in your CWD is also "email". Why should a
> relative import prefer one over the other? So I would think that even
> if it worked, it would still just end up importing the first one it
> finds on your sys.path.

So the solution is to treat the current directory as a pseudo-package.
It'd be a backward-incompatible change, so it would need to be
explicitly invoked. Something like:

python3 -p somefile.py

which would pretend to create an __init__.py in the current directory,
change to the parent, and "from dirname import somefile". I mentioned
this one time before, but it got kinda missed; I think I'll reraise it
on python-ideas. It's a protection on par with not having the current
directory in $PATH.

ChrisA

[toc] | [prev] | [next] | [standalone]

#104430

On Thursday, March 3, 2016 at 4:22:07 AM UTC-6, ast wrote:
> Hello
> 
> This has to be told
> 
> I created a file pickle.py in order to test some files
> read/write with objects and put it in a directory
> which is on my python path. 
> 
> Then the nightmare began
> 
> - Idle no longer works, window no longer opens 
> when double clicked, no errors messsages
> 
> - python -m pip list doesnt work, crash with an 
> error message related to pickle
> 
> I uninstalled python34 and reinstalled it, same problem
> I uninstalled python34 and instaled 3.5, same problem
> 
> ...
> 
> I finally understood that pickle.py is the name of the file
> containing the official pickle module.
> 
> This module is probably used by various python programs,
> IDLE, pip ...
> 
> Instead of reading official pickle, python read my file ...

This is a design flaw of the python language. If all standard library modules would have been protected from the global namespace by a single symbol -- say "stdlib" or "py" or whatever -- we would only need to avoid *ONE* symbol, instead of hundreds of them!!! Third party modules should have their own namespace as well. 

 from stdlib import re
 from stdlib.re import search
 from extlib import PIL
 from extlib.PIL import Image, ImageTk

Since those responsible for this flaw are unable, or unwilling, to fix it, the only solution for *YOU* is to (1) memorize every module name that exists in the python standard library, and also those that exist in your "site-packages" directory, or (2) hide all your personal modules behind a single symbol by utilizing a package.
 
 +mylib
   __init__.py
   pickle.py

import mylib.pickle as mypickle
mypickle.do_something()

 

[toc] | [prev] | [next] | [standalone]

#104473

On Wednesday, March 9, 2016 at 10:04:35 PM UTC+5:30, Rick Johnson wrote:
> On Thursday, March 3, 2016 at 4:22:07 AM UTC-6, ast wrote:
> > Hello
> > 
> > This has to be told
> > 
> > I created a file pickle.py in order to test some files
> > read/write with objects and put it in a directory
> > which is on my python path. 
> > 
> > Then the nightmare began
> > 
> > - Idle no longer works, window no longer opens 
> > when double clicked, no errors messsages
> > 
> > - python -m pip list doesnt work, crash with an 
> > error message related to pickle
> > 
> > I uninstalled python34 and reinstalled it, same problem
> > I uninstalled python34 and instaled 3.5, same problem
> > 
> > ...
> > 
> > I finally understood that pickle.py is the name of the file
> > containing the official pickle module.
> > 
> > This module is probably used by various python programs,
> > IDLE, pip ...
> > 
> > Instead of reading official pickle, python read my file ...
> 
> This is a design flaw of the python language. If all standard library modules would have been protected from the global namespace by a single symbol -- say "stdlib" or "py" or whatever -- we would only need to avoid *ONE* symbol, instead of hundreds of them!!! Third party modules should have their own namespace as well. 
> 
>  from stdlib import re
>  from stdlib.re import search
>  from extlib import PIL
>  from extlib.PIL import Image, ImageTk
> 
> Since those responsible for this flaw are unable, or unwilling, to fix it, the only solution for *YOU* is to (1) memorize every module name that exists in the python standard library, and also those that exist in your "site-packages" directory, or (2) hide all your personal modules behind a single symbol by utilizing a package.
>  
>  +mylib
>    __init__.py
>    pickle.py
> 
> import mylib.pickle as mypickle
> mypickle.do_something()

As usual Rick I find myself agreeing with your direction [also it seems
Random832's direction] though not with the ranty tone.

Ive been collecting factoids about how programming pedagogy and professional
programming practice cannot be conflated under one roof

[See http://blog.languager.org/2015/06/functional-programming-moving-target.html ]

One argument against that is -- in some guise or other -- to use namespaces.
eg If you dont need a language-feature dont use it
  If you dont need a library dont use it

These arguments neglect that the namespacing -- semantics, intricacies, headaches, etc -- ITSELF may be significantly noob-unfriendly.

For cars one needs: 
- the hood securely fastened when teaching driving
- easily openable when teaching automobile engineering

[toc] | [prev] | [next] | [standalone]

#104569

On Thursday, March 10, 2016 at 12:13:39 AM UTC-6, Rustom Mody wrote: 
> As usual Rick I find myself agreeing with your direction [also it seems
> Random832's direction]

Somehow i missed Random's remark... Hmm, he does have a good idea! Introducing a new "import statement" would not break anything in the way that rearranging the stdlib structure would. I think it's a great idea, and *LONG* overdue.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web