Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #83616 > unrolled thread

I must be missing something obvious in installing Python 3.4.2...

Back to article view | Back to comp.lang.python

Contents

  I must be missing something obvious in installing Python 3.4.2... Andrew Koenig <ark@acm.org> - 2015-01-12 10:02 -0800
    Re: I must be missing something obvious in installing Python 3.4.2... Chris Angelico <rosuav@gmail.com> - 2015-01-13 05:36 +1100
      Re: I must be missing something obvious in installing Python 3.4.2... Andrew Koenig <ark@acm.org> - 2015-01-12 12:51 -0800
    Re: I must be missing something obvious in installing Python 3.4.2... Tim Golden <mail@timgolden.me.uk> - 2015-01-12 20:52 +0000
      Re: I must be missing something obvious in installing Python 3.4.2... Andrew Koenig <ark@acm.org> - 2015-01-12 13:45 -0800
        Re: I must be missing something obvious in installing Python 3.4.2... Tim Golden <mail@timgolden.me.uk> - 2015-01-12 22:00 +0000
          Re: I must be missing something obvious in installing Python 3.4.2... Andrew Koenig <ark@acm.org> - 2015-01-12 14:35 -0800
            Re: I must be missing something obvious in installing Python 3.4.2... Andrew Koenig <ark@acm.org> - 2015-01-12 15:12 -0800
              Re: I must be missing something obvious in installing Python 3.4.2... Tim Golden <mail@timgolden.me.uk> - 2015-01-13 06:08 +0000
                Re: I must be missing something obvious in installing Python 3.4.2... cjgohlke@gmail.com - 2015-01-12 23:05 -0800
                  Re: I must be missing something obvious in installing Python 3.4.2... Tim Golden <mail@timgolden.me.uk> - 2015-01-13 08:18 +0000

#83616 — I must be missing something obvious in installing Python 3.4.2...

Downloaded and installed 64-bit Python 3.4 and pywin32-219. Both installed smoothly on my 64-bit Win7 machine. I added C:\Python34 to the search path.

If I launch a Windows command window and run

        python -m ensurepip

I get the following:

Ignoring indexes: https://pypi.python.org/simple/
Downloading/unpacking setuptools
Cleaning up...
Exception:
Traceback (most recent call last):
  File "C:\Users\ark\AppData\Local\Temp\tmpe3ckts9f\pip-1.5.6-py2.py3-none-any.w
hl\pip\basecommand.py", line 122, in main
    status = self.run(options, args)
  File "C:\Users\ark\AppData\Local\Temp\tmpe3ckts9f\pip-1.5.6-py2.py3-none-any.w
hl\pip\commands\install.py", line 278, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundl
e=self.bundle)
  File "C:\Users\ark\AppData\Local\Temp\tmpe3ckts9f\pip-1.5.6-py2.py3-none-any.w
hl\pip\req.py", line 1177, in prepare_files
    url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
  File "C:\Users\ark\AppData\Local\Temp\tmpe3ckts9f\pip-1.5.6-py2.py3-none-any.w
hl\pip\index.py", line 209, in find_requirement
    file_locations, url_locations = self._sort_locations(locations)
  File "C:\Users\ark\AppData\Local\Temp\tmpe3ckts9f\pip-1.5.6-py2.py3-none-any.w
hl\pip\index.py", line 128, in _sort_locations
    sort_path(os.path.join(path, item))
  File "C:\Users\ark\AppData\Local\Temp\tmpe3ckts9f\pip-1.5.6-py2.py3-none-any.w
hl\pip\index.py", line 109, in sort_path
    if mimetypes.guess_type(url, strict=False)[0] == 'text/html':
  File "C:\Python34\lib\mimetypes.py", line 287, in guess_type
    init()
  File "C:\Python34\lib\mimetypes.py", line 348, in init
    db.read_windows_registry()
  File "C:\Python34\lib\mimetypes.py", line 255, in read_windows_registry
    with _winreg.OpenKey(hkcr, subkeyname) as subkey:
TypeError: OpenKey() argument 2 must be str without null characters or None, not
 str

The pip.log file contains the same text.

I'm sure I'm doing something obviously wrong here--but what?

[toc] | [next] | [standalone]

#83622

On Tue, Jan 13, 2015 at 5:02 AM, Andrew Koenig <ark@acm.org> wrote:
> Downloaded and installed 64-bit Python 3.4 and pywin32-219. Both installed smoothly on my 64-bit Win7 machine. I added C:\Python34 to the search path.
>
> If I launch a Windows command window and run
>
>         python -m ensurepip

Do you have any other Pythons in PATH? Try:

python --version

Is it what you expect? If it is, see if there's a PYTHONPATH or
somesuch from a different version of Python.

ChrisA

[toc] | [prev] | [next] | [standalone]

#83633

python --version reports 3.4.2, which is what I expected. I see no PYTHONPATH variable, or any environment variable with a name beginning PY (either upper or lower case).

[toc] | [prev] | [next] | [standalone]

#83634

On 12/01/2015 18:02, Andrew Koenig wrote:
> Downloaded and installed 64-bit Python 3.4 and pywin32-219. Both installed smoothly on my 64-bit Win7 machine. I added C:\Python34 to the search path.
>
> If I launch a Windows command window and run
>
>          python -m ensurepip
>
> I get the following:
>
> Ignoring indexes: https://pypi.python.org/simple/
> Downloading/unpacking setuptools
> Cleaning up...
> Exception:
> Traceback (most recent call last):
>    File "C:\Users\ark\AppData\Local\Temp\tmpe3ckts9f\pip-1.5.6-py2.py3-none-any.w
> hl\pip\basecommand.py", line 122, in main
>      status = self.run(options, args)
>    File "C:\Users\ark\AppData\Local\Temp\tmpe3ckts9f\pip-1.5.6-py2.py3-none-any.w
> hl\pip\commands\install.py", line 278, in run
>      requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundl
> e=self.bundle)
>    File "C:\Users\ark\AppData\Local\Temp\tmpe3ckts9f\pip-1.5.6-py2.py3-none-any.w
> hl\pip\req.py", line 1177, in prepare_files
>      url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
>    File "C:\Users\ark\AppData\Local\Temp\tmpe3ckts9f\pip-1.5.6-py2.py3-none-any.w
> hl\pip\index.py", line 209, in find_requirement
>      file_locations, url_locations = self._sort_locations(locations)
>    File "C:\Users\ark\AppData\Local\Temp\tmpe3ckts9f\pip-1.5.6-py2.py3-none-any.w
> hl\pip\index.py", line 128, in _sort_locations
>      sort_path(os.path.join(path, item))
>    File "C:\Users\ark\AppData\Local\Temp\tmpe3ckts9f\pip-1.5.6-py2.py3-none-any.w
> hl\pip\index.py", line 109, in sort_path
>      if mimetypes.guess_type(url, strict=False)[0] == 'text/html':
>    File "C:\Python34\lib\mimetypes.py", line 287, in guess_type
>      init()
>    File "C:\Python34\lib\mimetypes.py", line 348, in init
>      db.read_windows_registry()
>    File "C:\Python34\lib\mimetypes.py", line 255, in read_windows_registry
>      with _winreg.OpenKey(hkcr, subkeyname) as subkey:
> TypeError: OpenKey() argument 2 must be str without null characters or None, not
>   str

Ouch! That looks like another variation on a bug of which we've fixed at 
least two variations to do with reading mimetypes from the Windows 
registry. Can you see what the following code produces, please:

<code>
import winreg as _winreg

hkcr = _winreg.OpenKey(_winreg.HKEY_CLASSES_ROOT, "")

with open("classes.txt", "w") as f:
     i = 0
     while True:
         try:
             ctype = _winreg.EnumKey(hkcr, i)
         except EnvironmentError:
             break
         else:
             f.write("%s -> %r\n" % (type(ctype), ctype))

         i += 1

</code>

TJG

[toc] | [prev] | [next] | [standalone]

#83636

It runs and creates a classes.txt file with 803 lines. The first few:

<class 'str'> -> '$cpfile12'
<class 'str'> -> '$crfile12'
<class 'str'> -> '$cxfile12'
<class 'str'> -> '*'
<class 'str'> -> '.$cp'
<class 'str'> -> '.$cr'
<class 'str'> -> '.$cx'
<class 'str'> -> '.386'
<class 'str'> -> '.3ds'

A few lines in the middle that might be relevant:

<class 'str'> -> '.py'
<class 'str'> -> '.pyc'
<class 'str'> -> '.pyo'
<class 'str'> -> '.pys'
<class 'str'> -> '.pyw'

And the last few:

<class 'str'> -> '{C7A40493-BF23-4B53-AB2A-4A923B3EE34B}\x00'
<class 'str'> -> '{E14E55A7-29C8-4389-8E5A-3EF964510FCA}\x00'
<class 'str'> -> '{F5E30566-7C8F-4037-A8FF-A7382E251C56}\x00'

Anything in particular I should be looking for?

[toc] | [prev] | [next] | [standalone]

#83639

On 12/01/2015 21:45, Andrew Koenig wrote:
> It runs and creates a classes.txt file with 803 lines. The first few:
>
> <class 'str'> -> '$cpfile12'
> <class 'str'> -> '$crfile12'
> <class 'str'> -> '$cxfile12'
> <class 'str'> -> '*'
> <class 'str'> -> '.$cp'
> <class 'str'> -> '.$cr'
> <class 'str'> -> '.$cx'
> <class 'str'> -> '.386'
> <class 'str'> -> '.3ds'
>
> A few lines in the middle that might be relevant:
>
> <class 'str'> -> '.py'
> <class 'str'> -> '.pyc'
> <class 'str'> -> '.pyo'
> <class 'str'> -> '.pys'
> <class 'str'> -> '.pyw'
>
> And the last few:
>
> <class 'str'> -> '{C7A40493-BF23-4B53-AB2A-4A923B3EE34B}\x00'
> <class 'str'> -> '{E14E55A7-29C8-4389-8E5A-3EF964510FCA}\x00'
> <class 'str'> -> '{F5E30566-7C8F-4037-A8FF-A7382E251C56}\x00'
>
> Anything in particular I should be looking for?
>

I was actually expecting to see something like a byestring, but in fact 
those last few items look suspicious with their trailing NULs. The 
mimetypes code tries to open each key to find its associated mimetype(s) 
and would probably fail on those.

The PyArg_ParseTupleAndKeywords in OpenKey is using the "Z" pattern for 
its first parameter which disallows embedded NULs. Obviously we can 
trap-and-skip within the mimetypes code but that won't help you 
immediately (short of your patching your own lib/mimetypes).

Is it possible to temporarily rename those keys out of the way, or to 
strip their trailing NULs to see if the problem goes away?

TJG

[toc] | [prev] | [next] | [standalone]

#83643

Not sure that would be a good idea: There are 22 such keys, as opposed to only two keys with Windows ID strings that don't end in nulls. I found this article:

http://www.swarley.me.uk/blog/2014/04/23/python-pip-and-windows-registry-corruption/

with the comment "If you are happy to completely remove any problematic keys, and accept any associated breakages that may occur, then this is the easiest solution."

I am concerned that if I change these keys, something else might break and I have no easy way of figuring out what.

[toc] | [prev] | [next] | [standalone]

#83647

Fixed it!

The aforementioned article is correct. I downloaded the RegDelNull program mentioned in the article (http://technet.microsoft.com/en-us/sysinternals/bb897448.aspx) and ran it on hkcr, hkcu, hklm, hku, and hkcc (short for HKEY_CLASSES_ROOT, HKEY_CURRENT_USER, HKEY_LOCAL_MACHINE, HKEY_USERS, and HKEY_CURRENT_CONFIG), respectively. It deleted a bunch of keys. Rerunning the program you posted earlier revealed no keys with embedded nulls in their names, and ensurepip now works.

I have no idea how these keys got there. For all I know they are the result of malware.

I think it would be worthwhile changing the Python code to detect nulls and perhaps issue a warning that directs people to this article or something like it.

[toc] | [prev] | [next] | [standalone]

#83663

On 12/01/2015 23:12, Andrew Koenig wrote:
> Fixed it!
>
> The aforementioned article is correct. I downloaded the RegDelNull
> program mentioned in the article
> (http://technet.microsoft.com/en-us/sysinternals/bb897448.aspx) and
> ran it on hkcr, hkcu, hklm, hku, and hkcc (short for
> HKEY_CLASSES_ROOT, HKEY_CURRENT_USER, HKEY_LOCAL_MACHINE, HKEY_USERS,
> and HKEY_CURRENT_CONFIG), respectively. It deleted a bunch of keys.
> Rerunning the program you posted earlier revealed no keys with
> embedded nulls in their names, and ensurepip now works.
>
> I have no idea how these keys got there. For all I know they are the
> result of malware.
>
> I think it would be worthwhile changing the Python code to detect
> nulls and perhaps issue a warning that directs people to this article
> or something like it.
>

Thanks, yes. Unfortunate that no-one's actually reported this before as 
a bug -- as far as I'm aware. Would you be in a position to report this 
at bugs.python.org? If not, I'll do it when I get a chance.

TJG

[toc] | [prev] | [next] | [standalone]

#83669

On Monday, January 12, 2015 at 10:09:03 PM UTC-8, Tim Golden wrote:
> On 12/01/2015 23:12, Andrew Koenig wrote:
> > Fixed it!
> >
> > The aforementioned article is correct. I downloaded the RegDelNull
> > program mentioned in the article
> > (http://technet.microsoft.com/en-us/sysinternals/bb897448.aspx) and
> > ran it on hkcr, hkcu, hklm, hku, and hkcc (short for
> > HKEY_CLASSES_ROOT, HKEY_CURRENT_USER, HKEY_LOCAL_MACHINE, HKEY_USERS,
> > and HKEY_CURRENT_CONFIG), respectively. It deleted a bunch of keys.
> > Rerunning the program you posted earlier revealed no keys with
> > embedded nulls in their names, and ensurepip now works.
> >
> > I have no idea how these keys got there. For all I know they are the
> > result of malware.
> >
> > I think it would be worthwhile changing the Python code to detect
> > nulls and perhaps issue a warning that directs people to this article
> > or something like it.
> >
> 
> Thanks, yes. Unfortunate that no-one's actually reported this before as 
> a bug -- as far as I'm aware. Would you be in a position to report this 
> at bugs.python.org? If not, I'll do it when I get a chance.
> 
> TJG

<http://bugs.python.org/issue22028>

[toc] | [prev] | [next] | [standalone]

#83674

On 13/01/2015 07:05, cjgohlke@gmail.com wrote:
> On Monday, January 12, 2015 at 10:09:03 PM UTC-8, Tim Golden wrote:
>> On 12/01/2015 23:12, Andrew Koenig wrote:
>>> Fixed it!
>>>
>>> The aforementioned article is correct. I downloaded the RegDelNull
>>> program mentioned in the article
>>> (http://technet.microsoft.com/en-us/sysinternals/bb897448.aspx) and
>>> ran it on hkcr, hkcu, hklm, hku, and hkcc (short for
>>> HKEY_CLASSES_ROOT, HKEY_CURRENT_USER, HKEY_LOCAL_MACHINE, HKEY_USERS,
>>> and HKEY_CURRENT_CONFIG), respectively. It deleted a bunch of keys.
>>> Rerunning the program you posted earlier revealed no keys with
>>> embedded nulls in their names, and ensurepip now works.
>>>
>>> I have no idea how these keys got there. For all I know they are the
>>> result of malware.
>>>
>>> I think it would be worthwhile changing the Python code to detect
>>> nulls and perhaps issue a warning that directs people to this article
>>> or something like it.
>>>
>>
>> Thanks, yes. Unfortunate that no-one's actually reported this before as 
>> a bug -- as far as I'm aware. Would you be in a position to report this 
>> at bugs.python.org? If not, I'll do it when I get a chance.
>>
>> TJG
> 
> <http://bugs.python.org/issue22028>
> 

(Sigh). Thanks -- and I'm even listed on the call. In my defence it was
logged while I was away over the summer and I did a bit of a bulk delete
when I came back.

Thanks

TJG

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web