Path: csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From: Peter Otten <__peter__@web.de>
Newsgroups: comp.lang.python
Subject: Re: corrupt download with urllib2
Date: Tue, 10 Nov 2015 14:40:59 +0100
Organization: None
Lines: 38
Message-ID: <mailman.210.1447162874.16136.python-list@python.org>
References: <n1sq82$k3g$1@news2.informatik.uni-stuttgart.de>
Mime-Version: 1.0
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 7Bit
X-Trace: news.uni-berlin.de uv81GjXhpcJYKKP8gkhmFg/XqrtS/aGTA1RsvdI1NDNA==
Return-Path: <python-python-list@m.gmane.org>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.001
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'else:': 0.03; 'binary': 0.05; '__future__': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'python': 0.10; 'exception': 0.13; 'occurs.': 0.16; 'received:80.91.229.3': 0.16; 'received:dip0.t-ipconnect.de': 0.16; 'received:io': 0.16; 'received:plane.gmane.org': 0.16; 'received:psf.io': 0.16; 'received:t-ipconnect.de': 0.16; 'subject:download': 0.16; 'surprising': 0.16; 'wrote:': 0.16; 'memory': 0.17; '>>>': 0.20; 'text,': 0.22; 'import': 0.24; 'script': 0.25; 'header:User- Agent:1': 0.26; "doesn't": 0.26; 'header:X-Complaints-To:1': 0.26; 'skip:( 20': 0.28; 'random': 0.29; 'putting': 0.30; 'foo': 0.33; 'impression': 0.33; 'file': 0.34; 'server': 0.34; 'gives': 0.35; 'skip:p 30': 0.35; 'but': 0.36; 'to:addr:python-list': 0.36; 'subject:: ': 0.37; 'received:org': 0.37; 'data': 0.39; 'to:addr:python.org': 0.40; 'subject:with': 0.40; 'received:de': 0.40; 'your': 0.60; 'matter': 0.63; 'more': 0.63; 'limit': 0.65; 'here': 0.66; 'results': 0.66; 'habit': 0.91
X-Injected-Via-Gmane: http://gmane.org/
X-Gmane-NNTP-Posting-Host: p57bd8e07.dip0.t-ipconnect.de
User-Agent: KNode/4.13.3
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.20+
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Xref: csiph.com comp.lang.python:98598

Ulli Horlacher wrote:

>     if u.getcode() == 200:
>       print(u.read(),file=szo,end='')
>       szo.close()
>     else:
>       die('cannot get %s - server reply: %d' % (szurl,u.getcode()))

More random remarks:

- print() gives the impression that you are dealing with text, and using it
  with binary strings will produce surprising results when you migrate to
  Python 3:

Python 2:

>>> from __future__ import print_function
>>> print(b"foo")
foo

Python 3:

>>> print(b"foo")
b'foo'

- with open(...) ensures that the file is closed when an exception occurs.
  It doesn't matter here as your script is going to die() anyway, but using
  with is a got habit to get into.
- consider shutil.copyfileobj to limit memory usage when dealing with data
  of arbitrary size.

Putting it together:

    with open(sz, "wb") as szo:
        shutil.copyfileobj(u, szo)