Path: csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From: Peter Otten <__peter__@web.de>
Newsgroups: comp.lang.python
Subject: Re: Fastest way to retrieve and write html contents to file
Date: Mon, 02 May 2016 10:42:36 +0200
Organization: None
Lines: 42
Message-ID: <mailman.313.1462178582.32212.python-list@python.org>
References: <ng6jie$1ap$1@dont-email.me> <85vb2xgj2i.fsf@benfinney.id.au> <mailman.298.1462164614.32212.python-list@python.org> <ng6mn7$8nv$1@dont-email.me> <5726ee33$0$1617$c3e8da3$5496439d@news.astraweb.com> <ng6sul$o85$1@dont-email.me> <ng73tt$hd2$1@ger.gmane.org>
Mime-Version: 1.0
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 7Bit
X-Trace: news.uni-berlin.de 2YbsOKb/u/ZFkc8Gnv51pAujJIhK1Hzwume3ltYCsaAA==
Return-Path: <python-python-list@m.gmane.org>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.000
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'cache': 0.05; 'subject:file': 0.07; 'true)': 0.07; 'cached': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'specifying': 0.09; 'python': 0.10; '"get",': 0.16; '/nologo': 0.16; 'caching': 0.16; 'dfs': 0.16; 'received:80.91.229.3': 0.16; 'received:dip0.t-ipconnect.de': 0.16; 'received:io': 0.16; 'received:plane.gmane.org': 0.16; 'received:psf.io': 0.16; 'received:t-ipconnect.de': 0.16; 'webpage,': 0.16; 'xmlhttp': 0.16; 'wrote:': 0.16; 'disable': 0.22; 'explicit': 0.22; 'this:': 0.23; 'header:User-Agent:1': 0.26; 'header:X-Complaints-To:1': 0.26; 'windows,': 0.29; "i'm": 0.30; 'option': 0.31; 'run': 0.33; 'file': 0.34; 'skip:c 30': 0.35; 'next': 0.35; 'but': 0.36; 'to:addr:python-list': 0.36; 'subject:: ': 0.37; 'received:org': 0.37; 'skip:- 60': 0.39; 'to:addr:python.org': 0.40; 'where': 0.40; 'received:de': 0.40; 'save': 0.60; 'believe': 0.66; 'webpage': 0.66; 'subject:write': 0.84
X-Injected-Via-Gmane: http://gmane.org/
X-Gmane-NNTP-Posting-Host: p57bd8162.dip0.t-ipconnect.de
User-Agent: KNode/4.13.3
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <ng73tt$hd2$1@ger.gmane.org>
X-Mailman-Original-References: <ng6jie$1ap$1@dont-email.me> <85vb2xgj2i.fsf@benfinney.id.au> <mailman.298.1462164614.32212.python-list@python.org> <ng6mn7$8nv$1@dont-email.me> <5726ee33$0$1617$c3e8da3$5496439d@news.astraweb.com> <ng6sul$o85$1@dont-email.me>
Xref: csiph.com comp.lang.python:108001

DFS wrote:

>> Is VB using a local web cache, and Python not?
> 
> I'm not specifying a local web cache with either (wouldn't know how or
> where to look).  If you have Windows, you can try it.

I don't have Windows, but if I'm to believe

http://stackoverflow.com/questions/5235464/how-to-make-microsoft-xmlhttprequest-honor-cache-control-directive

the page is indeed cached and you can disable caching with

> Option Explicit
> Dim xmlHTTP, fso, fOut, startTime, endTime, webpage, webfile,i
> webpage = "http://econpy.pythonanywhere.com/ex/001.html"
> webfile  = "D:\econpy001.html"
> startTime = Timer
> For i = 1 to 10
> Set xmlHTTP = CreateObject("MSXML2.serverXMLHTTP")
> xmlHTTP.Open "GET", webpage
  
  xmlHTTP.setRequestHeader "Cache-Control", "max-age=0"

> xmlHTTP.Send
> Set fso = CreateObject("Scripting.FileSystemObject")
> Set fOut = fso.CreateTextFile(webfile, True)
> fOut.WriteLine xmlHTTP.ResponseText
> fOut.Close
> Set fOut    = Nothing
> Set fso     = Nothing
> Set xmlHTTP = Nothing
> Next
> endTime = Timer
> wscript.echo "Finished VBScript in " & FormatNumber(endTime -
> startTime,3) & " seconds"
> -------------------------------------------------------------------
> save it to a .vbs file and run it like this:
> $cscript /nologo filename.vbs
>