Re: corrupt download with urllib2

Path	csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From	Peter Otten <__peter__@web.de>
Newsgroups	comp.lang.python
Subject	Re: corrupt download with urllib2
Date	Tue, 10 Nov 2015 14:40:59 +0100
Organization	None
Lines	38
Message-ID	<mailman.210.1447162874.16136.python-list@python.org> (permalink)
References	<n1sq82$k3g$1@news2.informatik.uni-stuttgart.de>
Mime-Version	1.0
Content-Type	text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding	7Bit
X-Trace	news.uni-berlin.de uv81GjXhpcJYKKP8gkhmFg/XqrtS/aGTA1RsvdI1NDNA==
Return-Path	<python-python-list@m.gmane.org>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.001
X-Spam-Evidence	'H': 1.00; 'S': 0.00; 'else:': 0.03; 'binary': 0.05; '__future__': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'python': 0.10; 'exception': 0.13; 'occurs.': 0.16; 'received:80.91.229.3': 0.16; 'received:dip0.t-ipconnect.de': 0.16; 'received:io': 0.16; 'received:plane.gmane.org': 0.16; 'received:psf.io': 0.16; 'received:t-ipconnect.de': 0.16; 'subject:download': 0.16; 'surprising': 0.16; 'wrote:': 0.16; 'memory': 0.17; '>>>': 0.20; 'text,': 0.22; 'import': 0.24; 'script': 0.25; 'header:User- Agent:1': 0.26; "doesn't": 0.26; 'header:X-Complaints-To:1': 0.26; 'skip:( 20': 0.28; 'random': 0.29; 'putting': 0.30; 'foo': 0.33; 'impression': 0.33; 'file': 0.34; 'server': 0.34; 'gives': 0.35; 'skip:p 30': 0.35; 'but': 0.36; 'to:addr:python-list': 0.36; 'subject:: ': 0.37; 'received:org': 0.37; 'data': 0.39; 'to:addr:python.org': 0.40; 'subject:with': 0.40; 'received:de': 0.40; 'your': 0.60; 'matter': 0.63; 'more': 0.63; 'limit': 0.65; 'here': 0.66; 'results': 0.66; 'habit': 0.91
X-Injected-Via-Gmane	http://gmane.org/
X-Gmane-NNTP-Posting-Host	p57bd8e07.dip0.t-ipconnect.de
User-Agent	KNode/4.13.3
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.20+
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list/>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Xref	csiph.com comp.lang.python:98598

Show key headers only | View raw

Ulli Horlacher wrote:

>     if u.getcode() == 200:
>       print(u.read(),file=szo,end='')
>       szo.close()
>     else:
>       die('cannot get %s - server reply: %d' % (szurl,u.getcode()))

More random remarks:

- print() gives the impression that you are dealing with text, and using it
  with binary strings will produce surprising results when you migrate to
  Python 3:

Python 2:

>>> from __future__ import print_function
>>> print(b"foo")
foo

Python 3:

>>> print(b"foo")
b'foo'

- with open(...) ensures that the file is closed when an exception occurs.
  It doesn't matter here as your script is going to die() anyway, but using
  with is a got habit to get into.
- consider shutil.copyfileobj to limit memory usage when dealing with data
  of arbitrary size.

Putting it together:

    with open(sz, "wb") as szo:
        shutil.copyfileobj(u, szo)

Thread

corrupt download with urllib2 Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2015-11-10 13:08 +0000
  Re: corrupt download with urllib2 Peter Otten <__peter__@web.de> - 2015-11-10 14:20 +0100
    Re: corrupt download with urllib2 Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2015-11-10 13:43 +0000
  Re: corrupt download with urllib2 Peter Otten <__peter__@web.de> - 2015-11-10 14:40 +0100
    Re: corrupt download with urllib2 Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2015-11-10 13:59 +0000
      Re: corrupt download with urllib2 Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2015-11-10 15:51 +0000
        Re: corrupt download with urllib2 Peter Otten <__peter__@web.de> - 2015-11-10 17:48 +0100
          Re: corrupt download with urllib2 Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2015-11-10 17:21 +0000

csiph-web