Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!feeds.phibee-telecom.net!newsfeed.xs4all.nl!newsfeed3.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.030 X-Spam-Evidence: '*H*': 0.94; '*S*': 0.00; 'jeff': 0.04; 'handler': 0.05; 'subject:Question': 0.07; 'exception,': 0.09; 'try:': 0.09; 'cc:addr:python-list': 0.11; '--\xc2\xa0': 0.16; 'blocked': 0.16; 'internally': 0.16; 'logon': 0.16; 'urllib': 0.16; 'when,': 0.16; '\xc2\xa0i': 0.16; 'component': 0.16; 'all.': 0.16; 'help.': 0.21; 'programming': 0.22; 'import': 0.22; 'cc:addr:python.org': 0.22; 'print': 0.22; 'cc:2**0': 0.24; 'cc:no real name:2**0': 0.24; "i've": 0.25; 'script': 0.25; 'suggested': 0.26; 'least': 0.26; 'header:In-Reply-To:1': 0.27; 'am,': 0.29; 'reporting': 0.29; 'dec': 0.30; 'message-id:@mail.gmail.com': 0.30; "i'm": 0.30; 'url:mailman': 0.30; '"",': 0.31; '"': 0.31; 'sites.': 0.31; 'up.': 0.33; 'url:python': 0.33; 'could': 0.34; 'info': 0.35; 'except': 0.35; 'skip:u 20': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'there': 0.35; '8bit%:9': 0.36; 'done': 0.36; 'url:listinfo': 0.36; 'thanks': 0.36; 'url:org': 0.36; 'should': 0.36; 'being': 0.38; 'server': 0.38; 'skip:& 10': 0.38; 'requiring': 0.38; 'url:amazon': 0.38; 'skip:[ 10': 0.38; 'expect': 0.39; 'sure': 0.39; 'url:mail': 0.40; 'lower': 0.61; 'simply': 0.61; 'our': 0.64; 'provide': 0.64; 'due': 0.66; 'here': 0.66; 'determine': 0.67; 'content,': 0.68; 'home': 0.69; 'fact,': 0.69; 'glad': 0.83; 'etc.]': 0.84; 'examples.': 0.84; 'respect.': 0.84; 'to:addr:jeff': 0.84; '2013': 0.98 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=LYzIcUmSqSbxBJU/nRNB23ctmqSOvIYMU7xpsFtgB4c=; b=p+9D7tjWzDhb5/THc7P6FdWS5VApNhgtd1KWFdxsL+KhVM2KuU53SAZutPHiNg27uB /3JH2HzMA0aq+riuhRPnMvlWcrhPXmBEKXB+llo/N1BURlr+Lz7BDjdnL7MHBNg0A1QD u9urXJjhGMCk4EAR1ZySmt6+Cy56oFEBTQydYhFPSTBe21DKetlUoJxN5gVpL07iaeDC HDfmSNFlNsjYfPIGk0pZuq1ZEsELu+tVLUX2agNu1HowBQkn/2cIrjPSk3LmcywA7h/b PuERAzVFbU6evTt3nigALdP+fCp5O11SP9N7xcx6koqZoaRzidtYXQS2PmS1lH9YWw3g xIuw== MIME-Version: 1.0 X-Received: by 10.195.13.113 with SMTP id ex17mr18320678wjd.0.1387296612695; Tue, 17 Dec 2013 08:10:12 -0800 (PST) In-Reply-To: References: Date: Tue, 17 Dec 2013 11:10:12 -0500 Subject: Re: Question RE urllib From: Larry Martell To: Jeff James Content-Type: multipart/alternative; boundary=047d7bfcf00acc444504edbd29d8 Cc: python-list@python.org X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 121 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1387296620 news.xs4all.nl 2947 [2001:888:2000:d::a6]:42002 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:62209 --047d7bfcf00acc444504edbd29d8 Content-Type: text/plain; charset=UTF-8 On Tue, Dec 17, 2013 at 10:26 AM, Jeff James wrot > > So I'm using the following script to check our sites to make sure they > are all up and some of them are reporting they are "down" when, in fact, > they are actually up. These sites do not require a logon in order for the > home page to come up. Could this be due to some port being blocked > internally ? Only one of the sites reporting as down is "https" but all > are internal sites. Is there some other component I should be including in > the script ? There are about 30 or 40 sites that I have listed in all. I > just use those in the following script as examples. Thanks > > import urllib > > sites = ["http://www.amazon.com/", "https://internalsite.com/intranet.html", > etc.] > > for site in sites: > try: > urllib.urlopen(site) > print site + " " > except Exception, e: > print site + " is down" > -- > https://mail.python.org/mailman/listinfo/python-list > > > I've never used urllib, although I've done a fair amount of network > programming at lower levels. > > Are you sure the report of "down" isn't simply a time out due to the > server being busier than you expect when you hit it? > > -Bill > > After adding the line suggested by Larry, I was able to determine that the > URLs reporting as "down" were actually sites requiring authentication in > order to provide site content, so adding that line to the handler was at > least enlightening in that respect. Thanks Larry. > Glad to help. Here is some info on authenticating with urllib: http://docs.python.org/2.7/howto/urllib2.html#id6 > > --047d7bfcf00acc444504edbd29d8 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
On T= ue, Dec 17, 2013 at 10:26 AM, Jeff James <jeff@jeffljames.com> wrot
So I'm u= sing the following script to check our sites to make sure they are all up a= nd some of them are reporting they are "down" when, in fact, they= are actually up. =C2=A0 These sites do not require a logon in order for th= e home page to come up. =C2=A0Could this be due to some port being blocked = internally ? =C2=A0Only one of the sites reporting as down is "https&q= uot; but all are internal sites. =C2=A0Is there some other component I shou= ld be including in the script ? =C2=A0There are about 30 or 40 sites that I= have listed in all. =C2=A0I just use those in the following script as exam= ples. =C2=A0 Thanks

import urllib


for site in sites:
=C2=A0 =C2=A0 try:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 urllib.urlopen(site)
=C2=A0 =C2=A0= =C2=A0 =C2=A0 print site + " "
=C2=A0 =C2=A0 except Ex= ception, e:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 print site + " is do= wn"
--=C2=A0
https://mail.python.org/mailman/listinfo/py= thon-list

I've never used urllib, al= though I've done a fair amount of network programming at lower levels.<= /div>

Are you sure the report of "down" isn't s= imply a time out due to the server being busier than you expect when you hi= t it?

-Bill

After adding the line suggested by Larry, I was able to determine that the = URLs reporting as "down" were actually sites requiring authentica= tion in order to provide site content, so adding that line to the handler w= as at least enlightening in that=C2=A0=C2=A0respe= ct. =C2=A0Thanks Larry.
Glad to= help. Here is some info on authenticating with urllib:


--047d7bfcf00acc444504edbd29d8--