Path: csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From: Stephen Hansen <me+python@ixokai.io>
Newsgroups: comp.lang.python
Subject: Re: Fastest way to retrieve and write html contents to file
Date: Sun, 01 May 2016 23:27:32 -0700
Lines: 29
Message-ID: <mailman.308.1462170455.32212.python-list@python.org>
References: <ng6jie$1ap$1@dont-email.me> <85vb2xgj2i.fsf@benfinney.id.au> <mailman.298.1462164614.32212.python-list@python.org> <ng6mn7$8nv$1@dont-email.me> <1462166136.1167243.595273897.291B0865@webmail.messagingengine.com> <mailman.303.1462166138.32212.python-list@python.org> <ng6q67$gnm$1@dont-email.me> <1462170452.1180117.595306673.68B64F02@webmail.messagingengine.com>
Mime-Version: 1.0
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
In-Reply-To: <ng6q67$gnm$1@dont-email.me>
Precedence: list
Xref: csiph.com comp.lang.python:107992

On Sun, May 1, 2016, at 10:59 PM, DFS wrote:
> startTime = time.clock()        
> for i in range(loops):
> 	r = urllib2.urlopen(webpage)
> 	f = open(webfile,"w")
> 	f.write(r.read())
> 	f.close
> endTime = time.clock()          
> print "Finished urllib2 in %.2g seconds" %(endTime-startTime)

Yeah on my system I get 1.8 out of this, amounting to 0.18s. 

I'm again going back to the point of: its fast enough. When comparing
two small numbers, "twice as slow" is meaningless.

You have an assumption you haven't answered, that downloading a 10 meg
file will be twice as slow as downloading this tiny file. You haven't
proven that at all. 

I suspect you have a constant overhead of X, and in this toy example,
that makes it seem twice as slow. But when downloading a file of size,
you'll have the same constant factor, at which point the difference is
irrelevant. 

If you believe otherwise, demonstrate it.

-- 
Stephen Hansen
  m e @ i x o k a i . i o