Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #11882 > unrolled thread

Re: python-urllib3 1.25.6 uploaded to experimental (closes CVE-2019-11236) but fails build tests

Back to article view | Back to linux.debian.maint.python

Contents

  Re: python-urllib3 1.25.6 uploaded to experimental (closes  CVE-2019-11236) but fails build tests Drew Parsons <dparsons@debian.org> - 2019-10-13 16:40 +0200
    Re: python-urllib3 1.25.6 uploaded to experimental (closes  CVE-2019-11236) but fails build tests Daniele Tricoli <eriol@debian.org> - 2019-10-27 16:30 +0100
      Re: python-urllib3 1.25.6 uploaded to experimental (closes  CVE-2019-11236) but fails build tests Drew Parsons <dparsons@debian.org> - 2019-10-27 17:30 +0100
        Re: python-urllib3 1.25.6 uploaded to experimental (closes  CVE-2019-11236) but fails build tests Michael Kesper <mkesper@schokokeks.org> - 2019-10-29 13:30 +0100
          Re: python-urllib3 1.25.6 uploaded to experimental (closes  CVE-2019-11236) but fails build tests Ondrej Novy <novy@ondrej.org> - 2019-10-29 14:20 +0100
          Re: python-urllib3 1.25.6 uploaded to experimental (closes  CVE-2019-11236) but fails build tests Jeremy Stanley <fungi@yuggoth.org> - 2019-10-29 14:40 +0100
            Re: python-urllib3 1.25.6 uploaded to experimental (closes  CVE-2019-11236) but fails build tests Michael Kesper <mkesper@schokokeks.org> - 2019-10-29 15:40 +0100

#11882 — Re: python-urllib3 1.25.6 uploaded to experimental (closes CVE-2019-11236) but fails build tests

Daniele wrote:
> I hope to have the time to investigate also this: 
> urllib3/contrib/pyopenssl.py
> contains code to have SSL with SNI_-support for Python 2 and it depends 
> on
> pyOpenSSL, cryptography and idna. Maybe looking at them can give us 
> more clues.

> Also, could you see if using Python3 the connection to 
> https://pub.orcid.org work?

It conditionally works.  Using curl, I found that TLSv1_0 or TLSv1_1 
will support a successful connection, but only if the maximum 
SSL_VERSION is constrained to TLSv1_0 or TLSv1_1 (e.g. curl -v --tlsv1.1 
--tls-max 1.1  https://pub.orcid.org). Without the max, the connection 
fails:
$ curl --tlsv1.1  https://pub.orcid.org
curl: (35) error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert 
handshake failure

The urllib3 failure was similar, but I do not know how to set tls-max 
with urllib3. I could only find the option with curl.  I could set up a 
custom HTTPAdapter as suggested at 
https://requests.readthedocs.io/en/master/user/advanced/#example-specific-ssl-version 
to set ssl_version=ssl.PROTOCOL_TLSv1_1 but the ssl module doesn't have 
the SSLVERSION_MAX_TLSv1_1 value that curl has. I could solve it with 
pycurl using c.setopt(pycurl.SSLVERSION, pycurl.SSLVERSION_TLSv1_1 | 
pycurl.SSLVERSION_MAX_TLSv1_1)

Evidently the orcid server only supports TLSv1.0 and TLSv1.1 and no 
higher (why haven't they activated TLSv1.3 yet?!), while curl and 
urllib3 without tls-max first test TLSv1.3 and then quit without 
cascading downwards once they receive the TLSv1.3 handshake failure.  
Which is rather odd behaviour when I think about it.  The whole point of 
supporting multiple protocol versions is to try the next available 
version if the first one doesn't work.



>> Th package build was successful on my system but gives build-time 
>> errors in
>> chroot (on buildd).  I'm not sure why that's failing.
> I will look at them during this weekend, I already had a look at build 
> log from
> the phone, but it's better to look from a PC.

I had a closer look.  The failing tests were in python2 only, coming 
from the non-ascii (Gërman http://Königsgäßchen.de/straße and Japanese 
http://ヒ:キ@ヒ.abc.ニ/ヒ?キ#ワ) unicode url tests. So from one perspective we 
don't need to worry so much about them, we could just disable them (e.g. 
prepend @onlyPy3 to test_parse_url and test_url_vulnerabilities in 
test_util.py). We'll be dropping python2 any way in the near future.

On the other hand, given the nature of the vulnerabilities and the 
possible uses of urllib3, it's probably best not to leave python2 
untested, especially since they are known to pass on python2 anyway in 
the right conditions.  Probably there is some package that should be 
added to Build-Depends to enable python2 tests to pass, though I have no 
idea which that package might be.

Drew

[toc] | [next] | [standalone]

#11952

[Multipart message — attachments visible in raw view] — view raw

On Sun, Oct 13, 2019 at 10:31:31PM +0800, Drew Parsons wrote:
> It conditionally works.  Using curl, I found that TLSv1_0 or TLSv1_1 will
> support a successful connection, but only if the maximum SSL_VERSION is
> constrained to TLSv1_0 or TLSv1_1 (e.g. curl -v --tlsv1.1 --tls-max 1.1
> https://pub.orcid.org). Without the max, the connection fails:
> $ curl --tlsv1.1  https://pub.orcid.org
> curl: (35) error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake
> failure
> 
> The urllib3 failure was similar, but I do not know how to set tls-max with
> urllib3. I could only find the option with curl.  I could set up a custom
> HTTPAdapter as suggested at https://requests.readthedocs.io/en/master/user/advanced/#example-specific-ssl-version
> to set ssl_version=ssl.PROTOCOL_TLSv1_1 but the ssl module doesn't have the
> SSLVERSION_MAX_TLSv1_1 value that curl has. I could solve it with pycurl
> using c.setopt(pycurl.SSLVERSION, pycurl.SSLVERSION_TLSv1_1 |
> pycurl.SSLVERSION_MAX_TLSv1_1)

For sure I'm missing something, but why not just set TLS version?
I tried the following on both Python2 and Python3:

    >>> import ssl
    >>> from urllib3.poolmanager import PoolManager
    >>> http = PoolManager(ssl_version=ssl.PROTOCOL_TLSv1)
    >>> r = http.request('GET', 'https://pub.orcid.org')
    >>> r.status
    200


> Evidently the orcid server only supports TLSv1.0 and TLSv1.1 and no higher
> (why haven't they activated TLSv1.3 yet?!), while curl and urllib3 without
> tls-max first test TLSv1.3 and then quit without cascading downwards once
> they receive the TLSv1.3 handshake failure.  Which is rather odd behaviour
> when I think about it.  The whole point of supporting multiple protocol
> versions is to try the next available version if the first one doesn't work.

Not an expert here, but I think fallback is not done on purpose due downgrade
attacks: https://en.wikipedia.org/wiki/Downgrade_attack

> I had a closer look.  The failing tests were in python2 only, coming from
> the non-ascii (Gërman http://Königsgäßchen.de/straße and Japanese
> http://ヒ:キ@ヒ.abc.ニ/ヒ?キ#ワ) unicode url tests. So from one perspective
> we don't need to worry so much about them, we could just disable them (e.g.
> prepend @onlyPy3 to test_parse_url and test_url_vulnerabilities in
> test_util.py). We'll be dropping python2 any way in the near future.
> 
> On the other hand, given the nature of the vulnerabilities and the possible
> uses of urllib3, it's probably best not to leave python2 untested,
> especially since they are known to pass on python2 anyway in the right
> conditions.  Probably there is some package that should be added to
> Build-Depends to enable python2 tests to pass, though I have no idea which
> that package might be.

Fixed adding python{,3}-idna on B-D. I had to add python3-idna
because the same tests were failing also on Python3 when I tested
them buinding on DoM.

Kind regards,

-- 
  Daniele Tricoli 'eriol'
  https://mornie.org

[toc] | [prev] | [next] | [standalone]

#11953

On 2019-10-27 23:13, Daniele Tricoli wrote:
> On Sun, Oct 13, 2019 at 10:31:31PM +0800, Drew Parsons wrote:
>> It conditionally works.  Using curl, I found that TLSv1_0 or TLSv1_1 
>> will
>> support a successful connection, but only if the maximum SSL_VERSION 
>> is
>> constrained to TLSv1_0 or TLSv1_1 (e.g. curl -v --tlsv1.1 --tls-max 
>> 1.1
>> https://pub.orcid.org). Without the max, the connection fails:
>> $ curl --tlsv1.1  https://pub.orcid.org
>> curl: (35) error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert 
>> handshake
>> failure
>> 
>> The urllib3 failure was similar, but I do not know how to set tls-max 
>> with
>> urllib3. I could only find the option with curl.  I could set up a 
>> custom
>> HTTPAdapter as suggested at 
>> https://requests.readthedocs.io/en/master/user/advanced/#example-specific-ssl-version
>> to set ssl_version=ssl.PROTOCOL_TLSv1_1 but the ssl module doesn't 
>> have the
>> SSLVERSION_MAX_TLSv1_1 value that curl has. I could solve it with 
>> pycurl
>> using c.setopt(pycurl.SSLVERSION, pycurl.SSLVERSION_TLSv1_1 |
>> pycurl.SSLVERSION_MAX_TLSv1_1)
> 
> For sure I'm missing something, but why not just set TLS version?
> I tried the following on both Python2 and Python3:
> 
>     >>> import ssl
>     >>> from urllib3.poolmanager import PoolManager
>     >>> http = PoolManager(ssl_version=ssl.PROTOCOL_TLSv1)
>     >>> r = http.request('GET', 'https://pub.orcid.org')
>     >>> r.status
>     200


That's a good tip, I missed that permutation. I was originally trying to 
access using the requests module, so didn't think to do it directly with 
urllib.PoolManager


> 
>> Evidently the orcid server only supports TLSv1.0 and TLSv1.1 and no 
>> higher
>> (why haven't they activated TLSv1.3 yet?!), while curl and urllib3 
>> without
>> tls-max first test TLSv1.3 and then quit without cascading downwards 
>> once
>> they receive the TLSv1.3 handshake failure.  Which is rather odd 
>> behaviour
>> when I think about it.  The whole point of supporting multiple 
>> protocol
>> versions is to try the next available version if the first one doesn't 
>> work.
> 
> Not an expert here, but I think fallback is not done on purpose due 
> downgrade
> attacks: https://en.wikipedia.org/wiki/Downgrade_attack


I see. Still an odd kind of protection though.  The attacker can just 
downgrade themselves.


>> I had a closer look.  The failing tests were in python2 only, coming 
>> from
>> the non-ascii (Gërman http://Königsgäßchen.de/straße and Japanese
>> http://ヒ:キ@ヒ.abc.ニ/ヒ?キ#ワ) unicode url tests.
...
> Fixed adding python{,3}-idna on B-D. I had to add python3-idna
> because the same tests were failing also on Python3 when I tested
> them buinding on DoM.


Thanks for that, and thanks again for the PoolManager tip.

Drew

[toc] | [prev] | [next] | [standalone]

#11964

[Multipart message — attachments visible in raw view] — view raw

Hi all,

On 27.10.19 17:27, Drew Parsons wrote:
> On 2019-10-27 23:13, Daniele Tricoli wrote:
>> On Sun, Oct 13, 2019 at 10:31:31PM +0800, Drew Parsons wrote:
>>> It conditionally works.  Using curl, I found that TLSv1_0 or TLSv1_1 will
>>> support a successful connection, but only if the maximum SSL_VERSION is
>>> constrained to TLSv1_0 or TLSv1_1 (e.g. curl -v --tlsv1.1 --tls-max 1.1
>>> https://pub.orcid.org). Without the max, the connection fails:
>>> $ curl --tlsv1.1  https://pub.orcid.org
>>> curl: (35) error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake
>>> failure
>>>
>>> The urllib3 failure was similar, but I do not know how to set tls-max with
>>> urllib3. I could only find the option with curl.  I could set up a custom
>>> HTTPAdapter as suggested at https://requests.readthedocs.io/en/master/user/advanced/#example-specific-ssl-version
>>> to set ssl_version=ssl.PROTOCOL_TLSv1_1 but the ssl module doesn't have the
>>> SSLVERSION_MAX_TLSv1_1 value that curl has. I could solve it with pycurl
>>> using c.setopt(pycurl.SSLVERSION, pycurl.SSLVERSION_TLSv1_1 |
>>> pycurl.SSLVERSION_MAX_TLSv1_1)
>>
>> For sure I'm missing something, but why not just set TLS version?
>> I tried the following on both Python2 and Python3:
>>
>>     >>> import ssl
>>     >>> from urllib3.poolmanager import PoolManager
>>     >>> http = PoolManager(ssl_version=ssl.PROTOCOL_TLSv1)
>>     >>> r = http.request('GET', 'https://pub.orcid.org')
>>     >>> r.status
>>     200
> 
> 
> That's a good tip, I missed that permutation. I was originally trying to access using the requests module, so didn't think to do it directly with urllib.PoolManager
> 
> 
>>
>>> Evidently the orcid server only supports TLSv1.0 and TLSv1.1 and no higher
>>> (why haven't they activated TLSv1.3 yet?!), while curl and urllib3 without
>>> tls-max first test TLSv1.3 and then quit without cascading downwards once
>>> they receive the TLSv1.3 handshake failure.  Which is rather odd behaviour
>>> when I think about it.  The whole point of supporting multiple protocol
>>> versions is to try the next available version if the first one doesn't work.
>>
>> Not an expert here, but I think fallback is not done on purpose due downgrade
>> attacks: https://en.wikipedia.org/wiki/Downgrade_attack
> 
> 
> I see. Still an odd kind of protection though.  The attacker can just downgrade themselves.

No. A sensible server will not talk to you if your requested SSL version is too low.
pub.orcid.org seems to use absolutely outdated and insecure software versions.

Bye
Michael

[toc] | [prev] | [next] | [standalone]

#11965

[Multipart message — attachments visible in raw view] — view raw

Hi,

út 29. 10. 2019 v 13:29 odesílatel Michael Kesper <mkesper@schokokeks.org>
napsal:

> > I see. Still an odd kind of protection though.  The attacker can just
> downgrade themselves.
>
> No. A sensible server will not talk to you if your requested SSL version
> is too low.
> pub.orcid.org seems to use absolutely outdated and insecure software
> versions.
>

right. If you want good security, you need to limit TLS version on both
side (client-urlib3 and server-whatever). Than downgrade attack is not
possible.

-- 
Best regards
 Ondřej Nový

[toc] | [prev] | [next] | [standalone]

#11966

[Multipart message — attachments visible in raw view] — view raw

On 2019-10-29 13:29:02 +0100 (+0100), Michael Kesper wrote:
> On 27.10.19 17:27, Drew Parsons wrote:
> > On 2019-10-27 23:13, Daniele Tricoli wrote:
[...]
> > > Not an expert here, but I think fallback is not done on
> > > purpose due downgrade attacks:
> > > https://en.wikipedia.org/wiki/Downgrade_attack
> > 
> > I see. Still an odd kind of protection though.  The attacker can
> > just downgrade themselves.
> 
> No. A sensible server will not talk to you if your requested SSL
> version is too low. pub.orcid.org seems to use absolutely outdated
> and insecure software versions.

Well, downgrade attacks aren't usually a two-party scenario. The
risk with a downgrade attack is when a victim client attempts
communication with some server, and a third-party attacker tampers
with the communication between the client and server sufficiently to
cause protocol negotiation to fall back to an old enough version
that the attacker can then exploit known flaws to decrypt and/or
proxy ("man in the middle") that communication. Having both the
client and the server be unwilling to use susceptible older protocol
versions helps thwart this attack vector.
-- 
Jeremy Stanley

[toc] | [prev] | [next] | [standalone]

#11967

[Multipart message — attachments visible in raw view] — view raw

Hi all,

On 29.10.19 14:15, Jeremy Stanley wrote:
> On 2019-10-29 13:29:02 +0100 (+0100), Michael Kesper wrote:
>> On 27.10.19 17:27, Drew Parsons wrote:
>>> On 2019-10-27 23:13, Daniele Tricoli wrote:
> [...]
>>>> Not an expert here, but I think fallback is not done on
>>>> purpose due downgrade attacks:
>>>> https://en.wikipedia.org/wiki/Downgrade_attack
>>>
>>> I see. Still an odd kind of protection though.  The attacker can
>>> just downgrade themselves.
>>
>> No. A sensible server will not talk to you if your requested SSL
>> version is too low. pub.orcid.org seems to use absolutely outdated
>> and insecure software versions.
> 
> Well, downgrade attacks aren't usually a two-party scenario. The
> risk with a downgrade attack is when a victim client attempts
> communication with some server, and a third-party attacker tampers
> with the communication between the client and server sufficiently to
> cause protocol negotiation to fall back to an old enough version
> that the attacker can then exploit known flaws to decrypt and/or
> proxy ("man in the middle") that communication. Having both the
> client and the server be unwilling to use susceptible older protocol
> versions helps thwart this attack vector.

Ah, you're right.
So the right fix would be to remove this possibility from urllib3.
This, however, would break applications that need to use these insecure 
connections.
At the least, I don't think build tests should fail if software
refuses to use deprecated and insecure connections.

Bye
Michael

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web