Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #71119 > unrolled thread

Re: Real-world use of concurrent.futures

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Real-world use of concurrent.futures Chris Angelico <rosuav@gmail.com> - 2014-05-09 05:06 +1000
    Re: Real-world use of concurrent.futures Marko Rauhamaa <marko@pacujo.net> - 2014-05-08 22:45 +0300
      Re: Real-world use of concurrent.futures Chris Angelico <rosuav@gmail.com> - 2014-05-09 11:12 +1000

#71119 — Re: Real-world use of concurrent.futures

On Fri, May 9, 2014 at 4:55 AM, Andrew McLean <lists@andros.org.uk> wrote:
> Because of latency in the DNS lookup this could
> benefit from multithreading.

Before you go too far down roads that are starting to look
problematic: A DNS lookup is a UDP packet out and a UDP packet in
(ignoring the possibility of TCP queries, which you probably won't be
doing here). Maybe it would be easier to implement it as asynchronous
networking? I don't know that Python makes it easy for you to
construct DNS requests and parse DNS responses; that's something more
in Pike's line of work. But it may be more possible to outright do the
DNS query asynchronously. TBH I haven't looked into it; but it's
another option to consider.

Separately from your programming model, though, how are you handling
timeouts? Any form of DNS error (NXDOMAIN being the most likely), and
the sort-of-error-but-not-error state of getting a response with no
answer, indicates that the address is invalid; but what if you just
don't hear back from the server? Will that mark addresses off as dead?

ChrisA

[toc] | [next] | [standalone]

#71124

Chris Angelico <rosuav@gmail.com>:

> Before you go too far down roads that are starting to look
> problematic: A DNS lookup is a UDP packet out and a UDP packet in
> (ignoring the possibility of TCP queries, which you probably won't be
> doing here). Maybe it would be easier to implement it as asynchronous
> networking? I don't know that Python makes it easy for you to
> construct DNS requests and parse DNS responses;

That's what I ended up doing when I wrote my home SMTP server.

It's more tedious than difficult. You don't need to have a ready-made
module for everything. An RFC and two hands take you far.


Marko

[toc] | [prev] | [next] | [standalone]

#71143

On Fri, May 9, 2014 at 5:45 AM, Marko Rauhamaa <marko@pacujo.net> wrote:
> Chris Angelico <rosuav@gmail.com>:
>
>> Before you go too far down roads that are starting to look
>> problematic: A DNS lookup is a UDP packet out and a UDP packet in
>> (ignoring the possibility of TCP queries, which you probably won't be
>> doing here). Maybe it would be easier to implement it as asynchronous
>> networking? I don't know that Python makes it easy for you to
>> construct DNS requests and parse DNS responses;
>
> That's what I ended up doing when I wrote my home SMTP server.
>
> It's more tedious than difficult. You don't need to have a ready-made
> module for everything. An RFC and two hands take you far.

Be careful. A naive implementation can open you up to various security
issues. In the case of DNS, source port randomization is important to
the prevention of cache poisoning; if you don't know what I'm talking
about, pick a language/library that does the work for you, rather than
doing everything yourself :)

ChrisA

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web