Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #99971 > unrolled thread

urllib2.urlopen() crashes on Windows 2008 Server

Started byUlli Horlacher <framstag@rus.uni-stuttgart.de>
First post2015-12-03 16:32 +0000
Last post2015-12-06 09:38 +0000
Articles 5 — 2 participants

Back to article view | Back to comp.lang.python

Contents

  urllib2.urlopen() crashes on Windows 2008 Server Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2015-12-03 16:32 +0000
    Re: urllib2.urlopen() crashes on Windows 2008 Server Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2015-12-03 20:07 -0500
      Re: urllib2.urlopen() crashes on Windows 2008 Server Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2015-12-04 13:00 +0000
        Re: urllib2.urlopen() crashes on Windows 2008 Server Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2015-12-04 13:16 -0500
          Re: urllib2.urlopen() crashes on Windows 2008 Server Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2015-12-06 09:38 +0000

#99971 — urllib2.urlopen() crashes on Windows 2008 Server

FromUlli Horlacher <framstag@rus.uni-stuttgart.de>
Date2015-12-03 16:32 +0000
Subjecturllib2.urlopen() crashes on Windows 2008 Server
Message-ID<n3pqq9$6sm$1@news2.informatik.uni-stuttgart.de>
I have a Python2 program which runs fine on Windows 7, but
crashes on Windows 2008 Server R2 64 bit:

downloading http://fex.belwue.de/download/7za.exe
Traceback (most recent call last):
  File "<string>", line 1992, in <module>
  File "<string>", line 180, in main
  File "<string>", line 329, in get_ID
  File "<string>", line 1627, in check_7z
  File "C:\Software\Python\lib\urllib2.py", line 154, in urlopen
  File "C:\Software\Python\lib\urllib2.py", line 431, in open
  File "C:\Software\Python\lib\urllib2.py", line 449, in _open
  File "C:\Software\Python\lib\urllib2.py", line 409, in _call_chain
  File "C:\Software\Python\lib\urllib2.py", line 1227, in http_open
  File "C:\Software\Python\lib\urllib2.py", line 1200, in do_open
  File "C:\Software\Python\lib\httplib.py", line 1132, in getresponse
  File "C:\Software\Python\lib\httplib.py", line 485, in begin
  File "C:\Software\Python\lib\mimetools.py", line 25, in __init__
  File "C:\Software\Python\lib\rfc822.py", line 108, in __init__
  File "C:\Software\Python\lib\httplib.py", line 319, in readheaders
  File "C:\Software\Python\lib\socket.py", line 480, in readline
error: [Errno 10054] Eine vorhandene Verbindung wurde vom Remotehost geschlossen

This is the function where the error happens:

if python2: import urllib2 as urllib
if python3: import urllib.request as urllib

def check_7z():
  global sz

  sz = path.join(fexhome,'7za.exe')
  szurl = "http://fex.belwue.de/download/7za.exe"

  if not path.exists(sz) or path.getsize(sz) < 9999:
    try:
      szo = open(sz,'wb')
    except (IOError,OSError) as e:
      die('cannot write %s - %s' % (sz,e.strerror))
    printf("\ndownloading %s\n",szurl)
    try:
      req = urllib.Request(szurl)
      req.add_header('User-Agent',useragent)
      u = urllib.urlopen(req) # line 1627 #
    except urllib.URLError as e:
      die('cannot get %s - %s' % (szurl,e.reason))
    except urllib.HTTPError as e:
      die('cannot get %s - server reply: %d %s' % (szurl,e.code,e.reason))
    if u.getcode() == 200:
      copy_file_obj(u,szo)
      szo.close()
    else:
      die('cannot get %s - server reply: %d' % (szurl,u.getcode()))


The curious thing is: the download was successful, the file 7za.exe is
there in the local directory!

-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum IZUS/TIK         E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart         Tel:    ++49-711-68565868
Allmandring 30a                Fax:    ++49-711-682357
70550 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/

[toc] | [next] | [standalone]

#99990

FromDennis Lee Bieber <wlfraed@ix.netcom.com>
Date2015-12-03 20:07 -0500
Message-ID<mailman.189.1449191259.14615.python-list@python.org>
In reply to#99971
On Thu, 3 Dec 2015 16:32:09 +0000 (UTC), Ulli Horlacher
<framstag@rus.uni-stuttgart.de> declaimed the following:

>I have a Python2 program which runs fine on Windows 7, but
>crashes on Windows 2008 Server R2 64 bit:
>
>downloading http://fex.belwue.de/download/7za.exe
>Traceback (most recent call last):
>  File "<string>", line 1992, in <module>
>  File "<string>", line 180, in main
>  File "<string>", line 329, in get_ID
>  File "<string>", line 1627, in check_7z
>  File "C:\Software\Python\lib\urllib2.py", line 154, in urlopen
>  File "C:\Software\Python\lib\urllib2.py", line 431, in open
>  File "C:\Software\Python\lib\urllib2.py", line 449, in _open
>  File "C:\Software\Python\lib\urllib2.py", line 409, in _call_chain
>  File "C:\Software\Python\lib\urllib2.py", line 1227, in http_open
>  File "C:\Software\Python\lib\urllib2.py", line 1200, in do_open
>  File "C:\Software\Python\lib\httplib.py", line 1132, in getresponse
>  File "C:\Software\Python\lib\httplib.py", line 485, in begin
>  File "C:\Software\Python\lib\mimetools.py", line 25, in __init__
>  File "C:\Software\Python\lib\rfc822.py", line 108, in __init__
>  File "C:\Software\Python\lib\httplib.py", line 319, in readheaders
>  File "C:\Software\Python\lib\socket.py", line 480, in readline
>error: [Errno 10054] Eine vorhandene Verbindung wurde vom Remotehost geschlossen
>

Per MSDN:
"""
WSAECONNRESET
10054

	

Connection reset by peer.

    An existing connection was forcibly closed by the remote host. This
normally results if the peer application on the remote host is suddenly
stopped, the host is rebooted, the host or remote network interface is
disabled, or the remote host uses a hard close (see setsockopt for more
information on the SO_LINGER option on the remote socket). This error may
also result if a connection was broken due to keep-alive activity detecting
a failure while one or more operations are in progress. Operations that
were in progress fail with WSAENETRESET. Subsequent operations fail with
WSAECONNRESET.
"""



>The curious thing is: the download was successful, the file 7za.exe is
>there in the local directory!

	But is it complete? If it is, perhaps you are getting hit by the "hard
close" condition?
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]

#100002

FromUlli Horlacher <framstag@rus.uni-stuttgart.de>
Date2015-12-04 13:00 +0000
Message-ID<n3s2pg$qkp$1@news2.informatik.uni-stuttgart.de>
In reply to#99990
Dennis Lee Bieber <wlfraed@ix.netcom.com> wrote:

> >I have a Python2 program which runs fine on Windows 7, but
> >crashes on Windows 2008 Server R2 64 bit:
> >
> >downloading http://fex.belwue.de/download/7za.exe
> >Traceback (most recent call last):
> >  File "<string>", line 1992, in <module>
> >  File "<string>", line 180, in main
> >  File "<string>", line 329, in get_ID
> >  File "<string>", line 1627, in check_7z
> >  File "C:\Software\Python\lib\urllib2.py", line 154, in urlopen
> >  File "C:\Software\Python\lib\urllib2.py", line 431, in open
> >  File "C:\Software\Python\lib\urllib2.py", line 449, in _open
> >  File "C:\Software\Python\lib\urllib2.py", line 409, in _call_chain
> >  File "C:\Software\Python\lib\urllib2.py", line 1227, in http_open
> >  File "C:\Software\Python\lib\urllib2.py", line 1200, in do_open
> >  File "C:\Software\Python\lib\httplib.py", line 1132, in getresponse
> >  File "C:\Software\Python\lib\httplib.py", line 485, in begin
> >  File "C:\Software\Python\lib\mimetools.py", line 25, in __init__
> >  File "C:\Software\Python\lib\rfc822.py", line 108, in __init__
> >  File "C:\Software\Python\lib\httplib.py", line 319, in readheaders
> >  File "C:\Software\Python\lib\socket.py", line 480, in readline
> >error: [Errno 10054] Eine vorhandene Verbindung wurde vom Remotehost geschlossen
> >
> 
> Per MSDN:
> """
> WSAECONNRESET
> 10054
> 
>         
> 
> Connection reset by peer.
> 
>     An existing connection was forcibly closed by the remote host. 

This is not true.
The server is under my control. Die client has terminated the connection
(or a router between).


How can I trap this within the python program?
I see no exception.


-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum IZUS/TIK         E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart         Tel:    ++49-711-68565868
Allmandring 30a                Fax:    ++49-711-682357
70550 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/

[toc] | [prev] | [next] | [standalone]

#100010

FromDennis Lee Bieber <wlfraed@ix.netcom.com>
Date2015-12-04 13:16 -0500
Message-ID<mailman.203.1449253048.14615.python-list@python.org>
In reply to#100002
On Fri, 4 Dec 2015 13:00:32 +0000 (UTC), Ulli Horlacher
<framstag@rus.uni-stuttgart.de> declaimed the following:

>Dennis Lee Bieber <wlfraed@ix.netcom.com> wrote:

>> 
>> Per MSDN:
>> """
>> WSAECONNRESET
>> 10054
>> 
>>         
>> 
>> Connection reset by peer.
>> 
>>     An existing connection was forcibly closed by the remote host. 
>
>This is not true.
>The server is under my control. Die client has terminated the connection
>(or a router between).
>

http://stackoverflow.com/questions/16888429/how-to-make-a-hard-abortive-close-on-a-tcp-connection
Case 2: linger->l_onoff is non-zero and linger->l_linger is zero:

A close() returns immediately. The underlying stack discards any unsent
data, and, in the case of connection-oriented protocols such as TCP, sends
a RST (reset) to the peer (this is termed a hard or abortive close). All
subsequent attempts by the peer's application to read()/recv() data will
result in an ECONNRESET.

	The odds are still good that something on the server is configured to
not make a clean shutdown of TCP connections (a router should not be
involved as the TCP connection is from client IP to server IP, regardless
of intervening MAC Ethernet routing). The RST is coming from the server.


>
>How can I trap this within the python program?
>I see no exception.

	It's a socket error

https://docs.python.org/2/library/socket.html
"""

exception socket.error

    This exception is raised for socket-related errors. The accompanying
value is either a string telling what went wrong or a pair (errno, string)
representing an error returned by a system call, similar to the value
accompanying os.error. See the module errno, which contains names for the
error codes defined by the underlying operating system.

    Changed in version 2.6: socket.error is now a child class of IOError.
"""
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]

#100055

FromUlli Horlacher <framstag@rus.uni-stuttgart.de>
Date2015-12-06 09:38 +0000
Message-ID<n40vls$4dr$1@news2.informatik.uni-stuttgart.de>
In reply to#100010
Dennis Lee Bieber <wlfraed@ix.netcom.com> wrote:

> >> Connection reset by peer.
> >> 
> >>     An existing connection was forcibly closed by the remote host. 
> >
> >This is not true.
> >The server is under my control. Die client has terminated the connection
> >(or a router between).

>         The odds are still good that something on the server is configured to
> not make a clean shutdown of TCP connections (a router should not be
> involved as the TCP connection is from client IP to server IP, regardless
> of intervening MAC Ethernet routing). The RST is coming from the server.

No.
The server does not reset the connection.
It is the router/firewall.


> >How can I trap this within the python program?
> >I see no exception.
> 
>         It's a socket error
> 
> https://docs.python.org/2/library/socket.html
> """
> 
> exception socket.error

Ok, I have now:

   try:
     req = urllib.Request(szurl)
     req.add_header('User-Agent',useragent)
     u = urllib.urlopen(req)
   except urllib.URLError as e:
     die('cannot get %s - %s' % (szurl,e.reason))
   except urllib.HTTPError as e:
     die('cannot get %s - server reply: %d %s' % (szurl,e.code,e.reason))
   except (IOError,httplib.BadStatusLine,httplib.HTTPException):
     die('cannot get %s - connection reset by router or firewall' % szurl)
   except socket.error as msg:
     die('cannot get %s - %s' % (szurl,msg))


-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum IZUS/TIK         E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart         Tel:    ++49-711-68565868
Allmandring 30a                Fax:    ++49-711-682357
70550 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web