Re: Real-world use of concurrent.futures

Path	csiph.com!usenet.pasdenom.info!news.albasani.net!rt.uk.eu.org!newsfeed.xs4all.nl!newsfeed2a.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path	<rosuav@gmail.com>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.014
X-Spam-Evidence	'H': 0.97; 'S': 0.00; 'model,': 0.05; 'indicates': 0.09; 'lookup': 0.09; 'separately': 0.09; 'subject:skip:c 10': 0.09; 'cc:addr:python-list': 0.11; 'python': 0.11; 'from:addr:rosuav': 0.16; 'from:name:chris angelico': 0.16; 'here).': 0.16; 'it;': 0.16; 'tcp': 0.16; 'wrote:': 0.18; 'looked': 0.18; 'programming': 0.22; 'cc:addr:python.org': 0.22; 'error': 0.23; 'parse': 0.24; "haven't": 0.24; 'cc:2**0': 0.24; 'handling': 0.26; 'query': 0.26; 'header:In-Reply-To:1': 0.27; 'am,': 0.29; 'possibility': 0.29; 'andrew': 0.30; 'message- id:@mail.gmail.com': 0.30; 'work.': 0.31; 'getting': 0.31; 'easier': 0.31; 'requests': 0.31; 'probably': 0.32; 'option': 0.32; 'another': 0.32; 'addresses': 0.33; 'fri,': 0.33; 'maybe': 0.34; 'could': 0.34; 'something': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'doing': 0.36; 'possible': 0.36; 'too': 0.37; 'starting': 0.37; 'being': 0.38; 'implement': 0.38; 'though,': 0.39; 'how': 0.40; 'easy': 0.60; 'most': 0.60; 'back': 0.62; 'address': 0.63; 'hear': 0.63; 'more': 0.64; 'benefit': 0.68; 'asynchronous': 0.84; 'roads': 0.91; 'to:none': 0.92
DKIM-Signature	v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:cc :content-type; bh=lYMOhO0lnJnkiMfzSAhbXlzC3b6eUONHg4NZaLb94yY=; b=VbNXFibG21+Wljuknqn+bhtEAbQg8tkI+a5BKWYdB1VGAO3pULkHgfsURi3VAWDzJr arl89NsSap4LQaEoHiQPh+bEN8b9tVc03z7wvXLfdjfNV/MtEEkdYJNLzKXl+Qsmafll U0SibSmeRK35yH8K/gaFFEnLlMSfpv3zndn15MG5zI/qWw8d53NWQSFGRFvGqbOM9Dfj 1KAydG00VjyCyju9HA2erYlxhg3+kYljLhmSlBexw53YoYkLd8gG5iOUTGKW/auJbcG7 QCkChLwlEFZVwkoxBkJph3Pf4NWOvHgiL2gIMLfzdZIu+CKtaeyj2NZIIxUoQNHrt0q/ Ij/A==
MIME-Version	1.0
X-Received	by 10.52.173.165 with SMTP id bl5mr3639916vdc.13.1399576015494; Thu, 08 May 2014 12:06:55 -0700 (PDT)
In-Reply-To	<536BD338.4070004@andros.org.uk>
References	<536BD338.4070004@andros.org.uk>
Date	Fri, 9 May 2014 05:06:55 +1000
Subject	Re: Real-world use of concurrent.futures
From	Chris Angelico <rosuav@gmail.com>
Cc	"python-list@python.org" <python-list@python.org>
Content-Type	text/plain; charset=UTF-8
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.15
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list/>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups	comp.lang.python
Message-ID	<mailman.9791.1399576024.18130.python-list@python.org> (permalink)
Lines	21
NNTP-Posting-Host	2001:888:2000:d::a6
X-Trace	1399576024 news.xs4all.nl 2922 [2001:888:2000:d::a6]:60743
X-Complaints-To	abuse@xs4all.nl
Xref	csiph.com comp.lang.python:71119

Show key headers only | View raw

On Fri, May 9, 2014 at 4:55 AM, Andrew McLean <lists@andros.org.uk> wrote:
> Because of latency in the DNS lookup this could
> benefit from multithreading.

Before you go too far down roads that are starting to look
problematic: A DNS lookup is a UDP packet out and a UDP packet in
(ignoring the possibility of TCP queries, which you probably won't be
doing here). Maybe it would be easier to implement it as asynchronous
networking? I don't know that Python makes it easy for you to
construct DNS requests and parse DNS responses; that's something more
in Pike's line of work. But it may be more possible to outright do the
DNS query asynchronously. TBH I haven't looked into it; but it's
another option to consider.

Separately from your programming model, though, how are you handling
timeouts? Any form of DNS error (NXDOMAIN being the most likely), and
the sort-of-error-but-not-error state of getting a response with no
answer, indicates that the address is invalid; but what if you just
don't hear back from the server? Will that mark addresses off as dead?

ChrisA

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Re: Real-world use of concurrent.futures Chris Angelico <rosuav@gmail.com> - 2014-05-09 05:06 +1000
  Re: Real-world use of concurrent.futures Marko Rauhamaa <marko@pacujo.net> - 2014-05-08 22:45 +0300
    Re: Real-world use of concurrent.futures Chris Angelico <rosuav@gmail.com> - 2014-05-09 11:12 +1000

csiph-web