Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #39763 > unrolled thread

"The urlopen() and urlretrieve() functions can cause arbitrarily long delays"

Back to article view | Back to comp.lang.python

Contents

  "The urlopen() and urlretrieve() functions can cause arbitrarily long delays" 7segment <7segment@live.com> - 2013-02-24 11:48 -0600
    Re: "The urlopen() and urlretrieve() functions can cause arbitrarily long delays" Ian Kelly <ian.g.kelly@gmail.com> - 2013-02-24 11:55 -0700
      Re: "The urlopen() and urlretrieve() functions can cause arbitrarily long delays" 7segment <7segment@live.com> - 2013-02-24 13:25 -0600
    Re: "The urlopen() and urlretrieve() functions can cause arbitrarily long delays" MRAB <python@mrabarnett.plus.com> - 2013-02-24 19:04 +0000
      Re: "The urlopen() and urlretrieve() functions can cause arbitrarily long delays" 7segment <7segment@live.com> - 2013-02-24 13:27 -0600
        Re: "The urlopen() and urlretrieve() functions can cause arbitrarily long delays" Thomas Rachel <nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915@spamschutz.glglgl.de> - 2013-02-24 22:30 +0100
    Re: "The urlopen() and urlretrieve() functions can cause arbitrarily long delays" rh <richard_hubbe11@lavabit.com> - 2013-02-24 14:46 -0800

#39763 — "The urlopen() and urlretrieve() functions can cause arbitrarily long delays"

Hi!

The subject is a segment of a sentence which I copied from Python's 
official homepage. In whole, it reads:

"The urlopen() and urlretrieve() functions can cause arbitrarily long 
delays while waiting for a network connection to be set up. This means 
that it is difficult to build an interactive Web client using these 
functions without using threads."

Here is my issue: I am testing my web site (which I have some problems 
with), and comparing its response speed to a couple of other sites, such 
as msn.com and python.org. 

Now, it is very strange that python.org responds always under 0.5 s while 
my site takes about 20 s to respond! And MSN about 60 seconds! So it does 
not seem "arbitrary" to me. And I can figure out the mechanisms behind 
the long delays.

What I am wondering is if there's another way for me to go about. I 
didn't really find any solutions with "threads" as suggested by python's 
document base. Any suggestions?

This is really important because technicians at my web hotel do not use 
proper tools and my site has been nearly dysfunctional since their latest 
upgrade and I want to make a case with proper statistics.

Thank you in advance.

[toc] | [next] | [standalone]

#39772

On Sun, Feb 24, 2013 at 10:48 AM, 7segment <7segment@live.com> wrote:
> Hi!
>
> The subject is a segment of a sentence which I copied from Python's
> official homepage. In whole, it reads:
>
> "The urlopen() and urlretrieve() functions can cause arbitrarily long
> delays while waiting for a network connection to be set up. This means
> that it is difficult to build an interactive Web client using these
> functions without using threads."

I believe what this is warning about is that if a network connection
is not immediately available when the functions are called, they will
block while waiting for one rather than returning control to the
calling function.  If used in an interactive client without threads,
this would cause the application to appear to "hang" while urllib
waits for a response.  I don't think it's meant to suggest that urllib
is responsible for the delays; that sounds like a problem with your
network configuration.

[toc] | [prev] | [next] | [standalone]

#39774

On Sun, 24 Feb 2013 11:55:09 -0700, Ian Kelly wrote:

> On Sun, Feb 24, 2013 at 10:48 AM, 7segment <7segment@live.com> wrote:
>> Hi!
>>
>> The subject is a segment of a sentence which I copied from Python's
>> official homepage. In whole, it reads:
>>
>> "The urlopen() and urlretrieve() functions can cause arbitrarily long
>> delays while waiting for a network connection to be set up. This means
>> that it is difficult to build an interactive Web client using these
>> functions without using threads."
> 
> I believe what this is warning about is that if a network connection is
> not immediately available when the functions are called, they will block
> while waiting for one rather than returning control to the calling
> function.  If used in an interactive client without threads, this would
> cause the application to appear to "hang" while urllib waits for a
> response.  I don't think it's meant to suggest that urllib is
> responsible for the delays; that sounds like a problem with your network
> configuration.

Thanks. I store all the data (response times) in a database. The figures 
I cited are always nearly the same. For example MSN has response times 
between 60-61 seconds consistently, my site 21-35 seconds, but python.org 
only about 0.3 seconds. the calls to the servers happen during the same 
minute.

And there's nothing wrong with my web connection at home.

[toc] | [prev] | [next] | [standalone]

#39773

On 2013-02-24 18:55, Ian Kelly wrote:
> On Sun, Feb 24, 2013 at 10:48 AM, 7segment <7segment@live.com> wrote:
>> Hi!
>>
>> The subject is a segment of a sentence which I copied from Python's
>> official homepage. In whole, it reads:
>>
>> "The urlopen() and urlretrieve() functions can cause arbitrarily long
>> delays while waiting for a network connection to be set up. This means
>> that it is difficult to build an interactive Web client using these
>> functions without using threads."
>
> I believe what this is warning about is that if a network connection
> is not immediately available when the functions are called, they will
> block while waiting for one rather than returning control to the
> calling function.  If used in an interactive client without threads,
> this would cause the application to appear to "hang" while urllib
> waits for a response.  I don't think it's meant to suggest that urllib
> is responsible for the delays; that sounds like a problem with your
> network configuration.
>
When in doubt, check some other way, such as with a browser.

[toc] | [prev] | [next] | [standalone]

#39775

On Sun, 24 Feb 2013 19:04:28 +0000, MRAB wrote:

> On 2013-02-24 18:55, Ian Kelly wrote:
>> On Sun, Feb 24, 2013 at 10:48 AM, 7segment <7segment@live.com> wrote:
>>> Hi!
>>>
>>> The subject is a segment of a sentence which I copied from Python's
>>> official homepage. In whole, it reads:
>>>
>>> "The urlopen() and urlretrieve() functions can cause arbitrarily long
>>> delays while waiting for a network connection to be set up. This means
>>> that it is difficult to build an interactive Web client using these
>>> functions without using threads."
>>
>> I believe what this is warning about is that if a network connection is
>> not immediately available when the functions are called, they will
>> block while waiting for one rather than returning control to the
>> calling function.  If used in an interactive client without threads,
>> this would cause the application to appear to "hang" while urllib waits
>> for a response.  I don't think it's meant to suggest that urllib is
>> responsible for the delays; that sounds like a problem with your
>> network configuration.
>>
> When in doubt, check some other way, such as with a browser.

Thank you Ian. Browser is not a good idea, because I need this tool to 
work automatically. I don't have time to check and compare the response 
times manually and put them into the database.

[toc] | [prev] | [next] | [standalone]

#39799

Am 24.02.2013 20:27 schrieb 7segment:

>> When in doubt, check some other way, such as with a browser.
>
> Thank you Ian. Browser is not a good idea, because I need this tool to
> work automatically. I don't have time to check and compare the response
> times manually and put them into the database.

Of course not for the long term. But you could check e.g. if MSN needs 
60 s as well from the browser. If not, there could be something else wrong.

Thomas

[toc] | [prev] | [next] | [standalone]

#39810

On Sun, 24 Feb 2013 11:48:57 -0600
7segment <7segment@live.com> wrote:

> Hi!
> 
> The subject is a segment of a sentence which I copied from Python's 
> official homepage. In whole, it reads:
> 
> "The urlopen() and urlretrieve() functions can cause arbitrarily long 
> delays while waiting for a network connection to be set up. This
> means that it is difficult to build an interactive Web client using
> these functions without using threads."
> 
> Here is my issue: I am testing my web site (which I have some
> problems with), and comparing its response speed to a couple of other
> sites, such as msn.com and python.org. 

Try some GETs with curl or wget.

> 
> Now, it is very strange that python.org responds always under 0.5 s
> while my site takes about 20 s to respond! And MSN about 60 seconds!
> So it does not seem "arbitrary" to me. And I can figure out the
> mechanisms behind the long delays.

Transparent proxies, edge caches, poorly configured DNS, etc. 
Lots of things can cause delays. Check your server's logs and tell if your
server is causing the delay.

> 
> What I am wondering is if there's another way for me to go about. I 
> didn't really find any solutions with "threads" as suggested by
> python's document base. Any suggestions?
What have you tried? Post your code.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web