Efficiency, threading, and concurrent.futures

Path	csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!eternal-september.org!feeder.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From	Rob Gaddi <rgaddi@technologyhighland.invalid>
Newsgroups	comp.lang.python
Subject	Efficiency, threading, and concurrent.futures
Date	Wed, 20 Aug 2014 09:43:35 -0700
Organization	Highland Technology, Inc.
Lines	37
Message-ID	<20140820094335.2345085f@rg.highlandtechnology.com> (permalink)
Mime-Version	1.0
Content-Type	text/plain; charset=US-ASCII
Content-Transfer-Encoding	7bit
Injection-Info	mx05.eternal-september.org; posting-host="903ac420d4384e8fcf51b0ca3b6abd1b"; logging-data="22212"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+W005AVw+3bNropSiiUlnw"
X-Newsreader	Claws Mail 3.9.3 (GTK+ 2.24.23; x86_64-pc-linux-gnu)
Cancel-Lock	sha1:mKuFuY8qkUEGOy616+n4L/e6c/0=
Xref	csiph.com comp.lang.python:76675

Show key headers only | View raw

I've got a situation where I'll be asking an I/O bound process to do
some work (querying an RS-232 device) while my main code is off
running a sleep() bound process.  Everyone always talks about how
expensive thread creation is, so I figured I'd test it out in an
IPython notebook. 

#####

import threading
from concurent.futures import ThreadPoolExecutor as TPE
from time import sleep

def fn():
  sleep(0.001)

%%timeit -r 50 -n 1000
thr = threading.Thread(target=fn)
thr.start()
thr.join()
1000 loops, best of 5: 1.24 ms per loop

%%timeit -r 50 -n 1000 ex=TPE(1)
fut=ex.submit(fn)
fut.result()
1000 loops, best of 5: 1.26 ms per loop

#####

Now, my understanding is that the ThreadPoolExecutor spawns all its
threads at the outset, then stuffs requests into one queue and
fishes results out of another, which should be substantially faster than
having create new threads each time.  And yet those were pretty dead on
even. Any idea what I'm seeing here?

-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Efficiency, threading, and concurrent.futures Rob Gaddi <rgaddi@technologyhighland.invalid> - 2014-08-20 09:43 -0700
  Re: Efficiency, threading, and concurrent.futures Akira Li <4kir4.1i@gmail.com> - 2014-08-20 21:17 +0400

csiph-web