Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #78291

ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function?

Newsgroups comp.lang.python
Date 2014-09-24 07:44 -0700
Message-ID <e2f400e6-359e-4070-a822-d734e85e6a75@googlegroups.com> (permalink)
Subject ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function?
From Praveen Gollakota <pgollakota@gmail.com>

Show all headers | View raw


Hello,

In the ThreadPoolExecutor (TPE), is the callback always guaranteed to run in the same thread as the submitted function?

 For example, I tested this with the following code. I ran it many times and it seemed like `func` and `callback` always ran in the same thread.

    import concurrent.futures
    import random
    import threading
    import time

    executor = concurrent.futures.ThreadPoolExecutor(max_workers=3)

    def func(x):
        time.sleep(random.random())
        return threading.current_thread().name

    def callback(future):
        time.sleep(random.random())
        x = future.result()
        cur_thread = threading.current_thread().name
        if (cur_thread != x):
            print(cur_thread, x)

    print('main thread: %s' % threading.current_thread())
    for i in range(10000):
        future = executor.submit(func, i)
        future.add_done_callback(callback)

However, it seemed to fail when I removed the `time.sleep(random.random())` statements, i.e. at least a few `func` functions and `callbacks` **did not** run in the same thread. 

For a project that I am working on, the callback must always run on the same thread as the submitted function, so I wanted to be sure that this is guaranteed by TPE. (And also the results of the test without the random sleep seemed puzzling).

I looked at the source code for executors (see https://hg.python.org/cpython/file/6d44906344f4/Lib/concurrent/futures/_base.py#l297) and it does not seem like we switch the thread to the main thread before we run the callback. But just wanted to be sure. 

Any insight would be appreciated!

Thanks,
Praveen

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread


Thread

ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Praveen Gollakota <pgollakota@gmail.com> - 2014-09-24 07:44 -0700
  Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? dieter <dieter@handshake.de> - 2014-09-25 08:14 +0200
  Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Ian Kelly <ian.g.kelly@gmail.com> - 2014-09-25 00:39 -0600
    Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Marko Rauhamaa <marko@pacujo.net> - 2014-09-25 12:46 +0300
      Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Ian Kelly <ian.g.kelly@gmail.com> - 2014-09-25 08:16 -0600
        Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Marko Rauhamaa <marko@pacujo.net> - 2014-09-25 17:47 +0300
          Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Ian Kelly <ian.g.kelly@gmail.com> - 2014-09-25 09:40 -0600
          Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Chris Angelico <rosuav@gmail.com> - 2014-09-26 02:04 +1000

csiph-web