Re: corrupt download with urllib2

From	Ulli Horlacher <framstag@rus.uni-stuttgart.de>
Newsgroups	comp.lang.python
Subject	Re: corrupt download with urllib2
Date	2015-11-10 17:21 +0000
Organization	University of Stuttgart, FRG
Message-ID	<n1t92k$q1o$1@news2.informatik.uni-stuttgart.de> (permalink)
References	<n1sq82$k3g$1@news2.informatik.uni-stuttgart.de> <mailman.210.1447162874.16136.python-list@python.org> <n1st8h$l75$1@news2.informatik.uni-stuttgart.de> <n1t3p9$nq3$1@news2.informatik.uni-stuttgart.de> <mailman.212.1447174160.16136.python-list@python.org>

Show all headers | View raw

Peter Otten <__peter__@web.de> wrote:

> > I have a problem with it: There is no feedback for the user about the
> > progress of the transfer, which can last several hours.
> > 
> > For small files shutil.copyfileobj() is a good idea, but not for huge
> > ones.
> 
> Indeed. Have a look at the source code:
> 
> def copyfileobj(fsrc, fdst, length=16*1024):
>     """copy data from file-like object fsrc to file-like object fdst"""
>     while 1:
>         buf = fsrc.read(length)
>         if not buf:
>             break
>         fdst.write(buf)
> 
> As simple as can be

Oooops - that's all?!


> I suggested the function as an alternative to writing 
> the loop yourself when your example code basically showed

Good idea :-)


> For the huge downloads that you intend to cater to you probably want your 
> script not just to print a dot on every iteration, you need expected 
> remaining time, checksums, ability to stop and resume a download and 
> whatnot.
> 
> Does the Perl code offer that?

Of course, yes. For download AND upload.



> Then why rewrite?

There is no more a Perl compiler for windows which supports https.



> Or are there Python libraries that do that out of the box?

No.


-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum IZUS/TIK         E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart         Tel:    ++49-711-68565868
Allmandring 30a                Fax:    ++49-711-682357
70550 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/

Back to comp.lang.python | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

corrupt download with urllib2 Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2015-11-10 13:08 +0000
  Re: corrupt download with urllib2 Peter Otten <__peter__@web.de> - 2015-11-10 14:20 +0100
    Re: corrupt download with urllib2 Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2015-11-10 13:43 +0000
  Re: corrupt download with urllib2 Peter Otten <__peter__@web.de> - 2015-11-10 14:40 +0100
    Re: corrupt download with urllib2 Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2015-11-10 13:59 +0000
      Re: corrupt download with urllib2 Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2015-11-10 15:51 +0000
        Re: corrupt download with urllib2 Peter Otten <__peter__@web.de> - 2015-11-10 17:48 +0100
          Re: corrupt download with urllib2 Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2015-11-10 17:21 +0000

csiph-web