Re: urllib2 timeout issue

Path	csiph.com!newsfeed.hal-mli.net!feeder3.hal-mli.net!newsfeed.hal-mli.net!feeder1.hal-mli.net!newsfeed.xs4all.nl!newsfeed1.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path	<jerome@jolimont.fr>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.009
X-Spam-Evidence	'H': 0.98; 'S': 0.00; 'python,': 0.02; '(python': 0.07; 'modifying': 0.07; 'only,': 0.07; 'socket': 0.07; 'http': 0.09; 'ping': 0.09; 'similar,': 0.09; 'skip:/ 10': 0.09; 'url:github': 0.09; 'wrote': 0.14; '"running': 0.16; '(it': 0.16; 'belongs': 0.16; 'context:': 0.16; 'expect,': 0.16; 'header:X-Face:1': 0.16; 'seconds,': 0.16; 'subject:issue': 0.16; 'timeout': 0.16; 'url.': 0.16; 'url:html)': 0.16; 'url:linux': 0.16; 'prevent': 0.16; 'wed,': 0.18; 'seems': 0.21; 'machine': 0.22; 'example': 0.22; 'issue.': 0.22; 'config': 0.24; 'gateway': 0.24; "shouldn't": 0.24; 'server.': 0.24; 'looks': 0.24; 'options': 0.25; '+0200': 0.26; 'second': 0.26; 'least': 0.26; 'skip:" 20': 0.27; 'header:In-Reply-To:1': 0.27; 'resolution': 0.29; 'network.': 0.30; "i'm": 0.30; 'code': 0.31; 'requests': 0.31; 'yes.': 0.31; 'file': 0.32; 'running': 0.33; 'received:fr': 0.33; 'maybe': 0.34; "i'd": 0.34; 'could': 0.34; 'message.': 0.35; 'problem': 0.35; 'skip:- 50': 0.35; 'problem.': 0.35; 'something': 0.35; 'point.': 0.35; 'test': 0.35; 'but': 0.35; 'there': 0.35; 'shorter': 0.36; 'url:org': 0.36; 'should': 0.36; 'behind': 0.37; 'seconds': 0.37; 'so,': 0.37; 'too': 0.37; 'two': 0.37; 'being': 0.38; 'thank': 0.38; 'machines': 0.38; 'to:addr:python-list': 0.38; 'issue': 0.38; 'anything': 0.39; 'does': 0.39; 'skip:8 10': 0.39; 'to:addr:python.org': 0.39; 'address.': 0.39; 'either': 0.39; 'skip:u 10': 0.60; 'radio': 0.60; 'from:charset:utf-8': 0.61; 'first': 0.61; 'back': 0.62; 'received:62': 0.63; 'address': 0.63; 'more': 0.64; 'afraid': 0.65; 'side': 0.67; 'broadcast': 0.68; 'sound': 0.68; 'home': 0.69; 'results': 0.69; 'limit': 0.70; 'topic,': 0.81; 'conclude': 0.84; 'otten': 0.84; 'symptoms': 0.84; 'absolutely': 0.87; 'besides,': 0.93; 'pc.': 0.93; '2013': 0.98
X-Spam-Checker-Version	SpamAssassin 3.3.1 (2010-03-16) on jeftof
X-Spam-Level
X-Spam-Status	No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable version=3.3.1
Date	Thu, 17 Oct 2013 15:34:05 +0200
From	Jérôme <jerome@jolimont.fr>
To	python-list@python.org
Subject	Re: urllib2 timeout issue
In-Reply-To	<acf47c27e418951eae9d050f8d3ccfba@jolimont.fr>
References	<acf47c27e418951eae9d050f8d3ccfba@jolimont.fr>
X-Mailer	Claws Mail 3.9.2 (GTK+ 2.24.21; x86_64-pc-linux-gnu)
X-Face	"kBB1-!wF@,"j_&tJ&7; T,t)PeQkZg5?.:{p,s>/,+?b6pN5!yxZy^nRXA=*?W+\|J9OG!W[rdx^VA^Sx`R"g,; +MzhAq"tZFg27W4qX+ZXvLX=%piZ6c.7@oSDHyQ0Mff#HGx<{
Mime-Version	1.0
Content-Type	text/plain; charset=UTF-8
Content-Transfer-Encoding	quoted-printable
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.15
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list/>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups	comp.lang.python
Message-ID	<mailman.1158.1382016775.18130.python-list@python.org> (permalink)
Lines	118
NNTP-Posting-Host	2001:888:2000:d::a6
X-Trace	1382016775 news.xs4all.nl 16001 [2001:888:2000:d::a6]:41121
X-Complaints-To	abuse@xs4all.nl
Xref	csiph.com comp.lang.python:56958

Show key headers only | View raw

Hi.

Thank you all for your answers.

--------------------------------------------------
Context:

The problem I want to address is the code being stuck too long when the
network is down.

I'm working on a software gateway running on a Raspberry Pi, that forwards
data received through a radio link to the network.

https://github.com/Jerome-github/oem_gateway

This can be sending HTTP requests every 3 seconds, so a 10 secondes timeout
is an issue.
--------------------------------------------------

I was not at home when I wrote my last message. Now back home, I could try on
my own Debian Jessie machine (Python 2.7.5+) and I get the same results as
with the RaspberryPi (long timeout). So there seems to be something going on
with the router/modem.

Wed, 16 Oct 2013 12:04:22 +0200
Vincent Vande Vyvre a écrit:

> The url "dumdgdfgdgmyurl.com/" is just an example ?

Yes.

I wrote dummyurl.com as an example but unfortunately, this actually existed,
so I added gibberish to get a non-existing URL.

Wed, 16 Oct 2013 13:22:34 +0200
Peter Otten a écrit:

> The problem might be ipv6-related. 

I do have a inet6 line in ifconfig on my desktop, but nothing on my Pi:

wlan0     Link encap:Ethernet  HWaddr 80:1f:02:af:36:e4  
          inet addr:192.168.1.7  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2644 errors:0 dropped:42297 overruns:0 frame:0
          TX packets:982 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:362298 (353.8 KiB)  TX bytes:127069 (124.0 KiB)

(I need to connect via Wi-Fi because my ethernet socket is dead thunder
struck...)

And I did this test:
(http://tldp.org/HOWTO/Linux+IPv6-HOWTO/systemcheck-kernel.html):

test -f /proc/net/if_inet6 && echo "Running kernel is IPv6 ready"

IPv6 works on my desktop only, not on the Pi. So it shouldn't be the
culprit, right ?

Wed, 16 Oct 2013 09:06:29 -0700
Tobiah a écrit:

> What happens when you use ping to resolve that address.  Do you get
> the same results?  If so, I'd say you have a DNS problem.  Maybe
> you have two DNS servers listed in /etc/resolv.conf or similar, and
> the first one is unavailable, so it takes 10 seconds to fail over
> to the second working server.

Good point. Same symptoms with ping. Long timeout on machine behind my router,
be it the Pi or the Debian PC.

/etc/resolv.conf:
domain home
search home
nameserver 192.168.1.254

I found this on my server's config:

options timeout:1

I added it to /etc/resolv.conf on my Pi and PC and I get a much shorter
timeout, yet still longer than the value I enter, longer than I expect, and
longer that what I have on other machines (instantaneous).

But at least I see something happening.

--------------------------------------------------

Should I conclude that the issue is in the DNS resolution ?

What can I do then ?

The /etc/resolv.conf trick looks more like a workaround. Besides, this file
is auto-modified. I know I can try to prevent modifications, either by
modifying privileges, or by adding "nohook resolv.conf" to /etc/dhcpcd.conf,
but that does not sound satisfying.

Anyway, it is still not instantaneous.

I don't see anything I can change in my router/modem config (it is closed and
belongs to my ISP, so I can only change what i'm allowed to).

Any other recommandation ?

I'm afraid I'm being out of topic, here.

Back to python, is it normal that urllib2's timeout does not apply to those
DNS delays ?

Is there absolutely nothing I can do from the software side to limit this
delay and give up if no answer is received before a second has passed ?

-- 
Jérôme

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

Re: urllib2 timeout issue Jérôme <jerome@jolimont.fr> - 2013-10-17 15:34 +0200

csiph-web