Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #78022 > unrolled thread

hashlib suddenly broken

Back to article view | Back to comp.lang.python

Contents

  hashlib suddenly broken Larry Martell <larry.martell@gmail.com> - 2014-09-18 10:27 -0600
    Re: hashlib suddenly broken John Gordon <gordon@panix.com> - 2014-09-18 16:47 +0000
      Re: hashlib suddenly broken Larry Martell <larry.martell@gmail.com> - 2014-09-18 13:18 -0600
        Re: hashlib suddenly broken John Gordon <gordon@panix.com> - 2014-09-18 20:21 +0000
          Re: hashlib suddenly broken Larry Martell <larry.martell@gmail.com> - 2014-09-18 15:30 -0600
    Re: hashlib suddenly broken Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-09-19 03:07 +1000
      Re: hashlib suddenly broken Chris Angelico <rosuav@gmail.com> - 2014-09-19 03:18 +1000
      Re: hashlib suddenly broken Larry Martell <larry.martell@gmail.com> - 2014-09-18 13:22 -0600
      Re: hashlib suddenly broken Larry Martell <larry.martell@gmail.com> - 2014-09-18 13:23 -0600
      Re: hashlib suddenly broken Larry Martell <larry.martell@gmail.com> - 2014-09-18 13:46 -0600
      Re: hashlib suddenly broken Ned Deily <nad@acm.org> - 2014-09-18 13:44 -0700
      Re: hashlib suddenly broken Christian Heimes <christian@python.org> - 2014-09-18 22:49 +0200
      Re: hashlib suddenly broken Larry Martell <larry.martell@gmail.com> - 2014-09-18 15:38 -0600
      Re: hashlib suddenly broken Larry Martell <larry.martell@gmail.com> - 2014-09-18 15:39 -0600
      Re: hashlib suddenly broken Christian Heimes <christian@python.org> - 2014-09-19 00:17 +0200
      Re: hashlib suddenly broken Ned Deily <nad@acm.org> - 2014-09-18 15:19 -0700
        Re: hashlib suddenly broken Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-09-19 15:00 +1000
          Re: hashlib suddenly broken Larry Martell <larry.martell@gmail.com> - 2014-09-19 09:09 -0600

#78022 — hashlib suddenly broken

I am on a mac running 10.8.5, python 2.7

Suddenly, many of my scripts started failing with:

ValueError: unsupported hash type sha1

Googling this showed that it's an issue with hashlib with a common
cause being a file called hashlib.py that gets in the way of the
interpreter finding the standard hashlib module, but that doesn't seem
to be the case:

>>> import hashlib
ERROR:root:code for hash sha1 was not found.
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py",
line 139, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py",
line 103, in __get_openssl_constructor
    return __get_builtin_constructor(name)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py",
line 91, in __get_builtin_constructor
    raise ValueError('unsupported hash type %s' % name)
ValueError: unsupported hash type sha1

And that file has not changed any time recently:

$ ls -l /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py
-rw-r--r--  1 root  wheel  5013 Apr 12  2013
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py

This just started happening yesterday, and I cannot think of anything
that I've done that could cause this.

[toc] | [next] | [standalone]

#78025

In <mailman.14109.1411057681.18130.python-list@python.org> Larry Martell <larry.martell@gmail.com> writes:

> Googling this showed that it's an issue with hashlib with a common
> cause being a file called hashlib.py that gets in the way of the
> interpreter finding the standard hashlib module, but that doesn't seem
> to be the case:

Perhaps hashlib imports some other module which has a local module of the
same name?

SHA1 has been deprecated for some time.  Maybe a recent OS update finally
got rid of it altogether?

-- 
John Gordon         Imagine what it must be like for a real medical doctor to
gordon@panix.com    watch 'House', or a real serial killer to watch 'Dexter'.

[toc] | [prev] | [next] | [standalone]

#78037

On Thu, Sep 18, 2014 at 10:47 AM, John Gordon <gordon@panix.com> wrote:
> In <mailman.14109.1411057681.18130.python-list@python.org> Larry Martell <larry.martell@gmail.com> writes:
>
>> Googling this showed that it's an issue with hashlib with a common
>> cause being a file called hashlib.py that gets in the way of the
>> interpreter finding the standard hashlib module, but that doesn't seem
>> to be the case:
>
> Perhaps hashlib imports some other module which has a local module of the
> same name?

It's failing on the 'import _sha' in hashlib.py:

 66   def __get_builtin_constructor(name):
 67        try:
 68          if name in ('SHA1', 'sha1'):
 69   ->        import _sha
 70              return _sha.new

(Pdb) s
ImportError: 'No module named _sha'



>
> SHA1 has been deprecated for some time.  Maybe a recent OS update finally
> got rid of it altogether?

I did not do an OS, or any other upgrade or install.

[toc] | [prev] | [next] | [standalone]

#78042

In <mailman.14119.1411067922.18130.python-list@python.org> Larry Martell <larry.martell@gmail.com> writes:

> It's failing on the 'import _sha' in hashlib.py:

>  66   def __get_builtin_constructor(name):
>  67        try:
>  68          if name in ('SHA1', 'sha1'):
>  69   ->        import _sha
>  70              return _sha.new

> (Pdb) s
> ImportError: 'No module named _sha'

This appears to differ from the error you originally reported:

>   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py",
> line 91, in __get_builtin_constructor
>     raise ValueError('unsupported hash type %s' % name)
> ValueError: unsupported hash type sha1

Could there be two different versions of hashlib.py on your system?

-- 
John Gordon         Imagine what it must be like for a real medical doctor to
gordon@panix.com    watch 'House', or a real serial killer to watch 'Dexter'.

[toc] | [prev] | [next] | [standalone]

#78046

On Thu, Sep 18, 2014 at 2:21 PM, John Gordon <gordon@panix.com> wrote:
> In <mailman.14119.1411067922.18130.python-list@python.org> Larry Martell <larry.martell@gmail.com> writes:
>
>> It's failing on the 'import _sha' in hashlib.py:
>
>>  66   def __get_builtin_constructor(name):
>>  67        try:
>>  68          if name in ('SHA1', 'sha1'):
>>  69   ->        import _sha
>>  70              return _sha.new
>
>> (Pdb) s
>> ImportError: 'No module named _sha'
>
> This appears to differ from the error you originally reported:
>
>>   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py",
>> line 91, in __get_builtin_constructor
>>     raise ValueError('unsupported hash type %s' % name)
>> ValueError: unsupported hash type sha1

It's the lower level error that triggers the initial error I reported.
The ImportError is caught and the ValueError is reported.

> Could there be two different versions of hashlib.py on your system?

No, I checked and there is only the ones for the various python
versions. And none that were recently installed or modified. And you
can see the full path reported by python is the expected one.

[toc] | [prev] | [next] | [standalone]

#78029

Larry Martell wrote:

> I am on a mac running 10.8.5, python 2.7
> 
> Suddenly, many of my scripts started failing with:
> 
> ValueError: unsupported hash type sha1
[...]
> This just started happening yesterday, and I cannot think of anything
> that I've done that could cause this.

Ah, the ol' "I didn't change anything, I swear!" excuse *wink*

But seriously... did you perhaps upgrade Python prior to yesterday? Or
possibly an automatic update ran?

Check the creation/last modified dates on:

/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py


but I expect that's probably not where the problem lies. My *wild guess* is
that your system updated SSL, and removed some underlying SHA-1 library
needed by hashlib. SHA-1 is pretty old, and there is now a known attack on
it, so some over-zealous security update may have removed it.

If that's the case, it really is over-zealous, for although SHA-1 is
deprecated, the threat is still some years away. Microsoft, Google and
Mozilla have all announced that they will continue accepting it until 2017.
I can't imagine why Apple would removed it so soon.

-- 
Steven

[toc] | [prev] | [next] | [standalone]

#78030

On Fri, Sep 19, 2014 at 3:07 AM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> but I expect that's probably not where the problem lies. My *wild guess* is
> that your system updated SSL, and removed some underlying SHA-1 library
> needed by hashlib. SHA-1 is pretty old, and there is now a known attack on
> it, so some over-zealous security update may have removed it.

Or, more likely, the actual code for sha1 is imported from somewhere
else, and *that* module is what's been shadowed. What happens if you
change directory to something with absolutely no .py files in it, then
start interactive Python and try importing hashlib? Maybe you have an
openssl.py or something.

ChrisA

[toc] | [prev] | [next] | [standalone]

#78038

On Thu, Sep 18, 2014 at 11:07 AM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> Larry Martell wrote:
>
>> I am on a mac running 10.8.5, python 2.7
>>
>> Suddenly, many of my scripts started failing with:
>>
>> ValueError: unsupported hash type sha1
> [...]
>> This just started happening yesterday, and I cannot think of anything
>> that I've done that could cause this.
>
> Ah, the ol' "I didn't change anything, I swear!" excuse *wink*
>
> But seriously... did you perhaps upgrade Python prior to yesterday? Or
> possibly an automatic update ran?

No, I did not upgrade or install anything.

> Check the creation/last modified dates on:
>
> /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py

That was in my original post:

$ ls -l /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py
-rw-r--r--  1 root  wheel  5013 Apr 12  2013
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py


> but I expect that's probably not where the problem lies. My *wild guess* is
> that your system updated SSL, and removed some underlying SHA-1 library
> needed by hashlib. SHA-1 is pretty old, and there is now a known attack on
> it, so some over-zealous security update may have removed it.
>
> If that's the case, it really is over-zealous, for although SHA-1 is
> deprecated, the threat is still some years away. Microsoft, Google and
> Mozilla have all announced that they will continue accepting it until 2017.
> I can't imagine why Apple would removed it so soon.


So you know how I could check and see if I have SHA-1 and when my SSL
was updated?

[toc] | [prev] | [next] | [standalone]

#78039

On Thu, Sep 18, 2014 at 11:18 AM, Chris Angelico <rosuav@gmail.com> wrote:
> On Fri, Sep 19, 2014 at 3:07 AM, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> wrote:
>> but I expect that's probably not where the problem lies. My *wild guess* is
>> that your system updated SSL, and removed some underlying SHA-1 library
>> needed by hashlib. SHA-1 is pretty old, and there is now a known attack on
>> it, so some over-zealous security update may have removed it.
>
> Or, more likely, the actual code for sha1 is imported from somewhere
> else, and *that* module is what's been shadowed. What happens if you
> change directory to something with absolutely no .py files in it, then
> start interactive Python and try importing hashlib? Maybe you have an
> openssl.py or something.

I still get the same error.

[toc] | [prev] | [next] | [standalone]

#78040

On Thu, Sep 18, 2014 at 1:22 PM, Larry Martell <larry.martell@gmail.com> wrote:
> On Thu, Sep 18, 2014 at 11:07 AM, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> wrote:
>> Larry Martell wrote:
>>
>>> I am on a mac running 10.8.5, python 2.7
>>>
>>> Suddenly, many of my scripts started failing with:
>>>
>>> ValueError: unsupported hash type sha1
>> [...]
>>> This just started happening yesterday, and I cannot think of anything
>>> that I've done that could cause this.
>>
>> Ah, the ol' "I didn't change anything, I swear!" excuse *wink*
>>
>> But seriously... did you perhaps upgrade Python prior to yesterday? Or
>> possibly an automatic update ran?
>
> No, I did not upgrade or install anything.
>
>> Check the creation/last modified dates on:
>>
>> /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py
>
> That was in my original post:
>
> $ ls -l /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py
> -rw-r--r--  1 root  wheel  5013 Apr 12  2013
> /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py
>
>
>> but I expect that's probably not where the problem lies. My *wild guess* is
>> that your system updated SSL, and removed some underlying SHA-1 library
>> needed by hashlib. SHA-1 is pretty old, and there is now a known attack on
>> it, so some over-zealous security update may have removed it.
>>
>> If that's the case, it really is over-zealous, for although SHA-1 is
>> deprecated, the threat is still some years away. Microsoft, Google and
>> Mozilla have all announced that they will continue accepting it until 2017.
>> I can't imagine why Apple would removed it so soon.
>
>
> So you know how I could check and see if I have SHA-1 and when my SSL
> was updated?

Nothing appears to have been recently changed:

$ ls -la /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/OpenSSL
total 224
drwxr-xr-x  12 root  wheel     408 Jun 20  2012 .
drwxr-xr-x  41 root  wheel    1394 Apr 13  2013 ..
-rwxr-xr-x   1 root  wheel  124736 Apr 12  2013 SSL.so
-rw-r--r--   1 root  wheel     965 Apr 12  2013 __init__.py
-rw-r--r--   1 root  wheel     991 Apr 12  2013 __init__.pyc
-rwxr-xr-x   1 root  wheel  168544 Apr 12  2013 crypto.so
-rwxr-xr-x   1 root  wheel   40864 Apr 12  2013 rand.so
drwxr-xr-x  12 root  wheel     408 Jun 20  2012 test
-rw-r--r--   1 root  wheel    1010 Apr 12  2013 tsafe.py
-rw-r--r--   1 root  wheel    1775 Apr 12  2013 tsafe.pyc
-rw-r--r--   1 root  wheel     176 Apr 12  2013 version.py
-rw-r--r--   1 root  wheel     293 Apr 12  2013 version.pyc

[toc] | [prev] | [next] | [standalone]

#78044

In article 
<CACwCsY7YfqRL-08qeywmYox8oQh5iwTcx_LCx5maaDzwsMDUeQ@mail.gmail.com>,
 Larry Martell <larry.martell@gmail.com> wrote:
> On Thu, Sep 18, 2014 at 1:22 PM, Larry Martell <larry.martell@gmail.com> 
> wrote:
> > On Thu, Sep 18, 2014 at 11:07 AM, Steven D'Aprano
> > <steve+comp.lang.python@pearwood.info> wrote:
> >> Larry Martell wrote:
> >>> I am on a mac running 10.8.5, python 2.7
> >>> Suddenly, many of my scripts started failing with:
> >>>
> >>> ValueError: unsupported hash type sha1
> >> [...]
> >>> This just started happening yesterday, and I cannot think of anything
> >>> that I've done that could cause this.
[...]
> > So you know how I could check and see if I have SHA-1 and when my SSL
> > was updated?

IIRC, the _sha1 extension module is only built for Python 2.7 if the 
necessary OpenSSL libraries (libssl and libcrypto) are not available 
when Python is built.  They are available on OS X so, normally, you 
won't see an _sha1.so with Pythons there.  hashlib.py first tries to 
import _hashlib.so and check that if it was built with the corresponding 
OpenSSL API and then calls it.  On OS X many Python builds, including 
the Apple system Pythons and the python.org Pythons, are dynamically 
linked to the system OpenSSL libs in /usr/lib.  From your original post, 
I'm assuming you are using the Apple-supplied system Python 2.7 on OS X 
10.8.5.  If so, you should see something like this:

$ sw_vers
ProductName:   Mac OS X
ProductVersion:   10.8.5
BuildVersion:  12F45 
$ /usr/bin/python2.7
Python 2.7.2 (default, Oct 11 2012, 20:14:37)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on 
darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import _hashlib
>>> dir(_hashlib)
['__doc__', '__file__', '__name__', '__package__', 'new', 'openssl_md5', 
'openssl_sha1', 'openssl_sha224', 'openssl_sha256', 'openssl_sha384', 
'openssl_sha512']
>>> _hashlib.__file__
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/l
ib-dynload/_hashlib.so'
>>> ^D
$ otool -L 
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/l
ib-dynload/_hashlib.so'
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/li
b-dynload/_hashlib.so:
   /usr/lib/libssl.0.9.8.dylib (compatibility version 0.9.8, current 
version 47.0.0)
   /usr/lib/libcrypto.0.9.8.dylib (compatibility version 0.9.8, current 
version 47.0.0)
   /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current 
version 169.3.0)
$ ls -l /usr/lib/libssl.0.9.8.dylib
-rwxr-xr-x  1 root  wheel  620848 Sep 18 13:13 
/usr/lib/libssl.0.9.8.dylib
$ ls -l /usr/lib/libcrypto.0.9.8.dylib
-rwxr-xr-x  1 root  wheel  2712368 Sep 18 13:13 
/usr/lib/libcrypto.0.9.8.dylib

Note that this was taken *after* installing the latest 10.8.5 Security 
Update for 10.8 (Security Update 2014-004, 
http://support.apple.com/kb/ht6443) which was just released today; that 
includes an updated OpenSSL.  But, I tried this today just before 
installing the update and it worked the same way, with older 
modification dates.  The python.org Python 2.7.x should look very 
similar but with /Library/Frameworks paths instead of 
/System/Library/Frameworks.  Other Pythons (e.g. MacPorts or Homebrew) 
may be using their own copies of OpenSSL libraries.

-- 
 Ned Deily,
 nad@acm.org

[toc] | [prev] | [next] | [standalone]

#78045

On 18.09.2014 21:23, Larry Martell wrote:
> On Thu, Sep 18, 2014 at 11:18 AM, Chris Angelico <rosuav@gmail.com> wrote:
>> On Fri, Sep 19, 2014 at 3:07 AM, Steven D'Aprano
>> <steve+comp.lang.python@pearwood.info> wrote:
>>> but I expect that's probably not where the problem lies. My *wild guess* is
>>> that your system updated SSL, and removed some underlying SHA-1 library
>>> needed by hashlib. SHA-1 is pretty old, and there is now a known attack on
>>> it, so some over-zealous security update may have removed it.
>>
>> Or, more likely, the actual code for sha1 is imported from somewhere
>> else, and *that* module is what's been shadowed. What happens if you
>> change directory to something with absolutely no .py files in it, then
>> start interactive Python and try importing hashlib? Maybe you have an
>> openssl.py or something.
> 
> I still get the same error.

The Python's implementation of SHA-1 either comes from _hashlib (which
wraps OpenSSL) or from _sha (which uses code from LibTomCrypt and
doesn't require external dependencies. Python 2.7 doesn't have a _sha
module if OpenSSL is available at compile time.

Please try to import _hashlib and see what happens. On Linux:

>>> import _hashlib
>>> _hashlib.__file__
'/usr/lib/python2.7/lib-dynload/_hashlib.x86_64-linux-gnu.so'
>>> _hashlib.openssl_sha1()
<sha1 HASH object @ 0x7eff3e5a8300>
>>> _hashlib.openssl_sha1().hexdigest()
'da39a3ee5e6b4b0d3255bfef95601890afd80709'

[toc] | [prev] | [next] | [standalone]

#78047

On Thu, Sep 18, 2014 at 2:44 PM, Ned Deily <nad@acm.org> wrote:
> In article
> <CACwCsY7YfqRL-08qeywmYox8oQh5iwTcx_LCx5maaDzwsMDUeQ@mail.gmail.com>,
>  Larry Martell <larry.martell@gmail.com> wrote:
>> On Thu, Sep 18, 2014 at 1:22 PM, Larry Martell <larry.martell@gmail.com>
>> wrote:
>> > On Thu, Sep 18, 2014 at 11:07 AM, Steven D'Aprano
>> > <steve+comp.lang.python@pearwood.info> wrote:
>> >> Larry Martell wrote:
>> >>> I am on a mac running 10.8.5, python 2.7
>> >>> Suddenly, many of my scripts started failing with:
>> >>>
>> >>> ValueError: unsupported hash type sha1
>> >> [...]
>> >>> This just started happening yesterday, and I cannot think of anything
>> >>> that I've done that could cause this.
> [...]
>> > So you know how I could check and see if I have SHA-1 and when my SSL
>> > was updated?
>
> IIRC, the _sha1 extension module is only built for Python 2.7 if the
> necessary OpenSSL libraries (libssl and libcrypto) are not available
> when Python is built.  They are available on OS X so, normally, you
> won't see an _sha1.so with Pythons there.  hashlib.py first tries to
> import _hashlib.so and check that if it was built with the corresponding
> OpenSSL API and then calls it.  On OS X many Python builds, including
> the Apple system Pythons and the python.org Pythons, are dynamically
> linked to the system OpenSSL libs in /usr/lib.  From your original post,
> I'm assuming you are using the Apple-supplied system Python 2.7 on OS X
> 10.8.5.

Yes, I am using the Apple-supplied system Python 2.7 on OS X 10.8.5.

> If so, you should see something like this:
>
> $ sw_vers
> ProductName:   Mac OS X
> ProductVersion:   10.8.5
> BuildVersion:  12F45
> $ /usr/bin/python2.7
> Python 2.7.2 (default, Oct 11 2012, 20:14:37)
> [GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on
> darwin
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import _hashlib
>>>> dir(_hashlib)
> ['__doc__', '__file__', '__name__', '__package__', 'new', 'openssl_md5',
> 'openssl_sha1', 'openssl_sha224', 'openssl_sha256', 'openssl_sha384',
> 'openssl_sha512']
>>>> _hashlib.__file__
> '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/l
> ib-dynload/_hashlib.so'
>>>> ^D
> $ otool -L
> '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/l
> ib-dynload/_hashlib.so'
> /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/li
> b-dynload/_hashlib.so:
>    /usr/lib/libssl.0.9.8.dylib (compatibility version 0.9.8, current
> version 47.0.0)
>    /usr/lib/libcrypto.0.9.8.dylib (compatibility version 0.9.8, current
> version 47.0.0)
>    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
> version 169.3.0)
> $ ls -l /usr/lib/libssl.0.9.8.dylib
> -rwxr-xr-x  1 root  wheel  620848 Sep 18 13:13
> /usr/lib/libssl.0.9.8.dylib
> $ ls -l /usr/lib/libcrypto.0.9.8.dylib
> -rwxr-xr-x  1 root  wheel  2712368 Sep 18 13:13
> /usr/lib/libcrypto.0.9.8.dylib

I get identical output, with the exception of the mod dates on those 2 files:

$ ls -l /usr/lib/libssl.0.9.8.dylib
-rwxr-xr-x  1 root  wheel  620768 Sep 19  2013 /usr/lib/libssl.0.9.8.dylib
$ ls -l /usr/lib/libcrypto.0.9.8.dylib
-rwxr-xr-x  1 root  wheel  2724720 Sep 19  2013 /usr/lib/libcrypto.0.9.8.dylib

> Note that this was taken *after* installing the latest 10.8.5 Security
> Update for 10.8 (Security Update 2014-004,
> http://support.apple.com/kb/ht6443) which was just released today; that
> includes an updated OpenSSL.

Do you think I should install this update? Perhaps that would restore
whatever is missing.

> But, I tried this today just before
> installing the update and it worked the same way, with older
> modification dates.  The python.org Python 2.7.x should look very
> similar but with /Library/Frameworks paths instead of
> /System/Library/Frameworks.  Other Pythons (e.g. MacPorts or Homebrew)
> may be using their own copies of OpenSSL libraries.

[toc] | [prev] | [next] | [standalone]

#78048

On Thu, Sep 18, 2014 at 2:49 PM, Christian Heimes <christian@python.org> wrote:
> On 18.09.2014 21:23, Larry Martell wrote:
>> On Thu, Sep 18, 2014 at 11:18 AM, Chris Angelico <rosuav@gmail.com> wrote:
>>> On Fri, Sep 19, 2014 at 3:07 AM, Steven D'Aprano
>>> <steve+comp.lang.python@pearwood.info> wrote:
>>>> but I expect that's probably not where the problem lies. My *wild guess* is
>>>> that your system updated SSL, and removed some underlying SHA-1 library
>>>> needed by hashlib. SHA-1 is pretty old, and there is now a known attack on
>>>> it, so some over-zealous security update may have removed it.
>>>
>>> Or, more likely, the actual code for sha1 is imported from somewhere
>>> else, and *that* module is what's been shadowed. What happens if you
>>> change directory to something with absolutely no .py files in it, then
>>> start interactive Python and try importing hashlib? Maybe you have an
>>> openssl.py or something.
>>
>> I still get the same error.
>
> The Python's implementation of SHA-1 either comes from _hashlib (which
> wraps OpenSSL) or from _sha (which uses code from LibTomCrypt and
> doesn't require external dependencies. Python 2.7 doesn't have a _sha
> module if OpenSSL is available at compile time.
>
> Please try to import _hashlib and see what happens. On Linux:
>
>>>> import _hashlib
>>>> _hashlib.__file__
> '/usr/lib/python2.7/lib-dynload/_hashlib.x86_64-linux-gnu.so'
>>>> _hashlib.openssl_sha1()
> <sha1 HASH object @ 0x7eff3e5a8300>
>>>> _hashlib.openssl_sha1().hexdigest()
> 'da39a3ee5e6b4b0d3255bfef95601890afd80709'


$ python
Python 2.7.2 (default, Oct 11 2012, 20:14:37)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import _hashlib
>>> _hashlib.__file__
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_hashlib.so'
>>> _hashlib.openssl_sha1()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: unsupported hash type
>>> _hashlib.openssl_sha1().hexdigest()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: unsupported hash type

[toc] | [prev] | [next] | [standalone]

#78049

On 18.09.2014 23:39, Larry Martell wrote:
> $ python
> Python 2.7.2 (default, Oct 11 2012, 20:14:37)
> [GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import _hashlib
>>>> _hashlib.__file__
> '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_hashlib.so'
>>>> _hashlib.openssl_sha1()
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> ValueError: unsupported hash type
>>>> _hashlib.openssl_sha1().hexdigest()
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> ValueError: unsupported hash type
> 

For unknown reasions your OpenSSL version doesn't support SHA-1. Please
try these two commands on the command line to check version and digest
support of your OpenSSL:

  $ echo -n '' | openssl dgst -sha1 -hex
  (stdin)= da39a3ee5e6b4b0d3255bfef95601890afd80709

  $ openssl version
  OpenSSL 1.0.1f 6 Jan 2014


Please also check which OpenSSL libcrypto is used by the _hashlib.so
shared library. On OSX otool -L should give a similar output as ldd on
Linux:

  $ otool -L
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_hashlib.so

Christian

[toc] | [prev] | [next] | [standalone]

#78050

In article 
<CACwCsY4qQ1tYStHukSooeftGe=J-ug2LR8sz6JR0qP58yKKPuQ@mail.gmail.com>,
 Larry Martell <larry.martell@gmail.com> wrote:
> Do you think I should install this update? Perhaps that would restore
> whatever is missing.

Yes. You should install the update in any case and it's unlikely to make 
the hashlib situation worse :=)

-- 
 Ned Deily,
 nad@acm.org

[toc] | [prev] | [next] | [standalone]

#78056

Ned Deily wrote:

> In article
> <CACwCsY4qQ1tYStHukSooeftGe=J-ug2LR8sz6JR0qP58yKKPuQ@mail.gmail.com>,
>  Larry Martell <larry.martell@gmail.com> wrote:
>> Do you think I should install this update? Perhaps that would restore
>> whatever is missing.
> 
> Yes. You should install the update in any case and it's unlikely to make
> the hashlib situation worse :=)

However, it is likely to make it impossible to diagnose the problem and stop
it from happening again.

It's not normal behaviour to have functionality just disappear overnight
like this. If Larry is telling the truth that there were no updates
running, *how did the sha-1 library disappear*?

Larry, I recommend that you try Christian's suggestions before upgrading:

  $ echo -n '' | openssl dgst -sha1 -hex
  (stdin)= da39a3ee5e6b4b0d3255bfef95601890afd80709

  $ openssl version
  OpenSSL 1.0.1f 6 Jan 2014



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#78080

On Thu, Sep 18, 2014 at 11:00 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> Ned Deily wrote:
>
>> In article
>> <CACwCsY4qQ1tYStHukSooeftGe=J-ug2LR8sz6JR0qP58yKKPuQ@mail.gmail.com>,
>>  Larry Martell <larry.martell@gmail.com> wrote:
>>> Do you think I should install this update? Perhaps that would restore
>>> whatever is missing.
>>
>> Yes. You should install the update in any case and it's unlikely to make
>> the hashlib situation worse :=)
>
> However, it is likely to make it impossible to diagnose the problem and stop
> it from happening again.
>
> It's not normal behaviour to have functionality just disappear overnight
> like this. If Larry is telling the truth that there were no updates
> running, *how did the sha-1 library disappear*?
>
> Larry, I recommend that you try Christian's suggestions before upgrading:
>
>   $ echo -n '' | openssl dgst -sha1 -hex
>   (stdin)= da39a3ee5e6b4b0d3255bfef95601890afd80709
>
>   $ openssl version
>   OpenSSL 1.0.1f 6 Jan 2014

I download the update and rebooted to install it, and my machine would
not reboot. It was just spinning and spinning. I powered it down and
rebooted it and it said "There are updates to install. Do you want
install them or just reboot without installing them?" I chose the
latter. It rebooted and now hashlib is working again. That made me
think the updates were in fact installed, but when I run the software
update utility it says the updates have not been installed. It's all
very weird. But it's working again. Thanks everyone for the help!!

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web