Re: Question RE urllib

Path	csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!feeds.phibee-telecom.net!newsfeed.xs4all.nl!newsfeed3.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path	<larry.martell@gmail.com>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.030
X-Spam-Evidence	'H': 0.94; 'S': 0.00; 'jeff': 0.04; 'handler': 0.05; 'subject:Question': 0.07; 'exception,': 0.09; 'try:': 0.09; 'cc:addr:python-list': 0.11; '--\xc2\xa0': 0.16; 'blocked': 0.16; 'internally': 0.16; 'logon': 0.16; 'urllib': 0.16; 'when,': 0.16; '\xc2\xa0i': 0.16; 'component': 0.16; 'all.': 0.16; 'help.': 0.21; 'programming': 0.22; 'import': 0.22; 'cc:addr:python.org': 0.22; 'print': 0.22; 'cc:20': 0.24; 'cc:no real name:20': 0.24; "i've": 0.25; 'script': 0.25; 'suggested': 0.26; 'least': 0.26; 'header:In-Reply-To:1': 0.27; 'am,': 0.29; 'reporting': 0.29; 'dec': 0.30; 'message-id:@mail.gmail.com': 0.30; "i'm": 0.30; 'url:mailman': 0.30; '"",': 0.31; '"': 0.31; 'sites.': 0.31; 'up.': 0.33; 'url:python': 0.33; 'could': 0.34; 'info': 0.35; 'except': 0.35; 'skip:u 20': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'there': 0.35; '8bit%:9': 0.36; 'done': 0.36; 'url:listinfo': 0.36; 'thanks': 0.36; 'url:org': 0.36; 'should': 0.36; 'being': 0.38; 'server': 0.38; 'skip:& 10': 0.38; 'requiring': 0.38; 'url:amazon': 0.38; 'skip:[ 10': 0.38; 'expect': 0.39; 'sure': 0.39; 'url:mail': 0.40; 'lower': 0.61; 'simply': 0.61; 'our': 0.64; 'provide': 0.64; 'due': 0.66; 'here': 0.66; 'determine': 0.67; 'content,': 0.68; 'home': 0.69; 'fact,': 0.69; 'glad': 0.83; 'etc.]': 0.84; 'examples.': 0.84; 'respect.': 0.84; 'to:addr:jeff': 0.84; '2013': 0.98
DKIM-Signature	v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=LYzIcUmSqSbxBJU/nRNB23ctmqSOvIYMU7xpsFtgB4c=; b=p+9D7tjWzDhb5/THc7P6FdWS5VApNhgtd1KWFdxsL+KhVM2KuU53SAZutPHiNg27uB /3JH2HzMA0aq+riuhRPnMvlWcrhPXmBEKXB+llo/N1BURlr+Lz7BDjdnL7MHBNg0A1QD u9urXJjhGMCk4EAR1ZySmt6+Cy56oFEBTQydYhFPSTBe21DKetlUoJxN5gVpL07iaeDC HDfmSNFlNsjYfPIGk0pZuq1ZEsELu+tVLUX2agNu1HowBQkn/2cIrjPSk3LmcywA7h/b PuERAzVFbU6evTt3nigALdP+fCp5O11SP9N7xcx6koqZoaRzidtYXQS2PmS1lH9YWw3g xIuw==
MIME-Version	1.0
X-Received	by 10.195.13.113 with SMTP id ex17mr18320678wjd.0.1387296612695; Tue, 17 Dec 2013 08:10:12 -0800 (PST)
In-Reply-To	<CA+pMb3uR=MCX7x31fRnpmfRqZo2GUMJft2bVgqLaDsJEZGheFw@mail.gmail.com>
References	<CA+pMb3uR=MCX7x31fRnpmfRqZo2GUMJft2bVgqLaDsJEZGheFw@mail.gmail.com>
Date	Tue, 17 Dec 2013 11:10:12 -0500
Subject	Re: Question RE urllib
From	Larry Martell <larry.martell@gmail.com>
To	Jeff James <jeff@jeffljames.com>
Content-Type	multipart/alternative; boundary=047d7bfcf00acc444504edbd29d8
Cc	python-list@python.org
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.15
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list/>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups	comp.lang.python
Message-ID	<mailman.4292.1387296620.18130.python-list@python.org> (permalink)
Lines	121
NNTP-Posting-Host	2001:888:2000:d::a6
X-Trace	1387296620 news.xs4all.nl 2947 [2001:888:2000:d::a6]:42002
X-Complaints-To	abuse@xs4all.nl
Xref	csiph.com comp.lang.python:62209

Show key headers only | View raw

[Multipart message — attachments visible in raw view] - view raw

On Tue, Dec 17, 2013 at 10:26 AM, Jeff James <jeff@jeffljames.com> wrot
>
>  So I'm using the following script to check our sites to make sure they
> are all up and some of them are reporting they are "down" when, in fact,
> they are actually up.   These sites do not require a logon in order for the
> home page to come up.  Could this be due to some port being blocked
> internally ?  Only one of the sites reporting as down is "https" but all
> are internal sites.  Is there some other component I should be including in
> the script ?  There are about 30 or 40 sites that I have listed in all.  I
> just use those in the following script as examples.   Thanks
>
> import urllib
>
> sites = ["http://www.amazon.com/", "https://internalsite.com/intranet.html",
> etc.]
>
> for site in sites:
>     try:
>         urllib.urlopen(site)
>         print site + " "
>     except Exception, e:
>         print site + " is down"
> --
> https://mail.python.org/mailman/listinfo/python-list
>
>
> I've never used urllib, although I've done a fair amount of network
> programming at lower levels.
>
> Are you sure the report of "down" isn't simply a time out due to the
> server being busier than you expect when you hit it?
>
> -Bill
>
> After adding the line suggested by Larry, I was able to determine that the
> URLs reporting as "down" were actually sites requiring authentication in
> order to provide site content, so adding that line to the handler was at
> least enlightening in that  respect.  Thanks Larry.
>
Glad to help. Here is some info on authenticating with urllib:

http://docs.python.org/2.7/howto/urllib2.html#id6

>
>

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Re: Question RE urllib Larry Martell <larry.martell@gmail.com> - 2013-12-17 11:10 -0500
  Re: Question RE urllib Tobiah <toby@tobiah.org> - 2013-12-17 08:25 -0800

csiph-web