Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #62209 > unrolled thread

Re: Question RE urllib

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Question RE urllib Larry Martell <larry.martell@gmail.com> - 2013-12-17 11:10 -0500
    Re: Question RE urllib Tobiah <toby@tobiah.org> - 2013-12-17 08:25 -0800

#62209 — Re: Question RE urllib

[Multipart message — attachments visible in raw view] — view raw

On Tue, Dec 17, 2013 at 10:26 AM, Jeff James <jeff@jeffljames.com> wrot
>
>  So I'm using the following script to check our sites to make sure they
> are all up and some of them are reporting they are "down" when, in fact,
> they are actually up.   These sites do not require a logon in order for the
> home page to come up.  Could this be due to some port being blocked
> internally ?  Only one of the sites reporting as down is "https" but all
> are internal sites.  Is there some other component I should be including in
> the script ?  There are about 30 or 40 sites that I have listed in all.  I
> just use those in the following script as examples.   Thanks
>
> import urllib
>
> sites = ["http://www.amazon.com/", "https://internalsite.com/intranet.html",
> etc.]
>
> for site in sites:
>     try:
>         urllib.urlopen(site)
>         print site + " "
>     except Exception, e:
>         print site + " is down"
> --
> https://mail.python.org/mailman/listinfo/python-list
>
>
> I've never used urllib, although I've done a fair amount of network
> programming at lower levels.
>
> Are you sure the report of "down" isn't simply a time out due to the
> server being busier than you expect when you hit it?
>
> -Bill
>
> After adding the line suggested by Larry, I was able to determine that the
> URLs reporting as "down" were actually sites requiring authentication in
> order to provide site content, so adding that line to the handler was at
> least enlightening in that  respect.  Thanks Larry.
>
Glad to help. Here is some info on authenticating with urllib:

http://docs.python.org/2.7/howto/urllib2.html#id6

>
>

[toc] | [next] | [standalone]

#62211

On 12/17/2013 08:10 AM, Larry Martell wrote:
> On Tue, Dec 17, 2013 at 10:26 AM, Jeff James <jeff@jeffljames.com <mailto:jeff@jeffljames.com>> wrot
>
>>     So I'm using the following script to check our sites to make sure they are all up and some of them are reporting they are
>>     "down" when, in fact, they are actually up.   These sites do not require a logon in order for the home page to come up.  Could
>>     this be due to some port being blocked internally ?  Only one of the sites reporting as down is "https" but all are internal
>>     sites.  Is there some other component I should be including in the script ?  There are about 30 or 40 sites that I have listed
>>     in all.  I just use those in the following script as examples.   Thanks
>>
>>     import urllib
>>
>>     sites = ["http://www.amazon.com/", "https://internalsite.com/intranet.html", etc.]
>>
>>     for site in sites:
>>         try:
>>             urllib.urlopen(site)
>>             print site + " "
>>         except Exception, e:
>>             print site + " is down"
>>     --
>>     https://mail.python.org/mailman/listinfo/python-list
>
>     I've never used urllib, although I've done a fair amount of network programming at lower levels.
>
>     Are you sure the report of "down" isn't simply a time out due to the server being busier than you expect when you hit it?
>
>     -Bill
>
>     After adding the line suggested by Larry, I was able to determine that the URLs reporting as "down" were actually sites
>     requiring authentication in order to provide site content, so adding that line to the handler was at least enlightening in that
>     respect.  Thanks Larry.
>
> Glad to help. Here is some info on authenticating with urllib:
>
> http://docs.python.org/2.7/howto/urllib2.html#id6
>
>


It must be a network problem, cuz your code works fine:

:w !python
http://www.amazon.com/
http://google.com
http://tobiah.org
http://notavalidurl.com
http://superreallyforsurenotavalidurlnokidding.com is down

Tobiah

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web