Re: threading

From	Ben Finney <ben+python@benfinney.id.au>
Subject	Re: threading
Date	2014-04-07 13:05 +1000
References	<CAChsEZTGWa9Rxhwwg=b8Cnn3tx7FiaWqTJvHRn8hatxmVhOv0A@mail.gmail.com>
Newsgroups	comp.lang.python
Message-ID	<mailman.8969.1396839923.18130.python-list@python.org> (permalink)

Show all headers | View raw

Onder Hazaroglu <oxhazaroglu@ualr.edu> writes:

> I've been using threading library to run some experiments parallel.

Threading is very difficult to get right, much more so than the usual
separation into distinct processes. How did you decide against the
normal means of parallel execution and instead choose threading?

> There is no message passing between my threads but still it messes up
> somehow. The results are different than running it separated.

Yes, this is a common result of threading; problems are much harder to
reproduce and much more entangled with apparently unrelated factors.
This is one of the main reasons to avoid threading if at all feasible.

> Is there a way to make sure that there is no memory sharing between
> threads?

The *whole point* of threading (AFAIK) is to share memory and other
process-distinct resources. If you're looking to avoid that, don't use
threading! Instead, use separate processes for parallel execution.

Parallel processing is achieved much more reliably and deterministically
with separate processes. Python even makes this much easier than most
languages, with the ‘multiprocessing’ module in the standard library
<URL:https://docs.python.org/3/library/multiprocessing.html>.

I would recommend you make easier-to-understand and easier-to-debug code
with that module, and only consider threading if you determine it's
needed.

-- 
 \                “Stop — Drive sideways.” —detour sign, Kyushu, Japan |
  `\                                                                   |
_o__)                                                                  |
Ben Finney

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Re: threading Ben Finney <ben+python@benfinney.id.au> - 2014-04-07 13:05 +1000
  Re: threading Roy Smith <roy@panix.com> - 2014-04-06 23:48 -0400
    Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-07 13:56 +1000
      Re: threading Roy Smith <roy@panix.com> - 2014-04-07 08:26 -0400
        Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-07 22:34 +1000
          Re: threading Roy Smith <roy@panix.com> - 2014-04-07 09:22 -0400
            Re: threading Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-04-07 14:41 +0100
            Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-07 16:49 +0300
              Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-08 00:27 +1000
                Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-07 17:51 +0300
                Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-08 01:12 +1000
            Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-08 00:24 +1000
      Re: threading Rick Johnson <rantingrickjohnson@gmail.com> - 2014-04-08 18:09 -0700
        Re: threading "Neil D. Cerutti" <neilc@norwich.edu> - 2014-04-09 09:50 -0400
          Re: threading Rick Johnson <rantingrickjohnson@gmail.com> - 2014-04-09 08:51 -0700
            Re: threading MRAB <python@mrabarnett.plus.com> - 2014-04-09 18:47 +0100
              Re: threading Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2014-04-10 11:35 +1200
                Re: threading Roy Smith <roy@panix.com> - 2014-04-09 19:53 -0400
                Re: threading Andrew Berg <robotsondrugs@gmail.com> - 2014-04-09 19:02 -0500
                Re: threading Steven D'Aprano <steve@pearwood.info> - 2014-04-10 02:43 +0000
                Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 13:08 +1000
                Re: threading Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-04-10 09:23 +0100
                Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 19:11 +1000
            Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 04:00 +1000
            Re: threading Steven D'Aprano <steve@pearwood.info> - 2014-04-10 03:44 +0000
              Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 13:54 +1000
    Re: threading Ben Finney <ben+python@benfinney.id.au> - 2014-04-07 15:22 +1000
    Re: threading Ethan Furman <ethan@stoneleaf.us> - 2014-04-08 11:09 -0700
    Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 21:41 +0200
      Re: threading Grant Edwards <invalid@invalid.invalid> - 2014-04-08 20:30 +0000
        Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-09 00:32 +0200
          Re: threading Rustom Mody <rustompmody@gmail.com> - 2014-04-08 19:17 -0700
  Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-07 08:10 +0300
    Re: threading Paul Rubin <no.email@nospam.invalid> - 2014-04-06 22:39 -0700
      Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-07 08:46 +0300
      Re: threading Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-04-07 19:47 -0400
        Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-08 08:19 +0300
          Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 10:47 +0000
            Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-08 15:10 +0300
              Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 16:37 +0000
                Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-08 20:17 +0300
            Re: threading Roy Smith <roy@panix.com> - 2014-04-08 09:19 -0400
              Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 15:44 +0000
                Re: threading Paul Rubin <no.email@nospam.invalid> - 2014-04-08 09:38 -0700
                Re: threading Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-04-09 14:42 +0100
          Re: threading "Frank Millman" <frank@chagford.com> - 2014-04-09 15:23 +0200
            Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-09 16:55 +0300
              Re: threading "Frank Millman" <frank@chagford.com> - 2014-04-09 16:46 +0200
                Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-09 20:31 +0300
                Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 03:52 +1000
                Re: threading Mark H Harris <harrismh777@gmail.com> - 2014-04-10 08:29 -0500
                Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-09 19:20 +0000
          Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-09 23:47 +1000
            Re: threading Roy Smith <roy@panix.com> - 2014-04-09 10:44 -0400
          Re: threading "Frank Millman" <frank@chagford.com> - 2014-04-09 16:30 +0200
            Re: threading Roy Smith <roy@panix.com> - 2014-04-09 10:52 -0400
              Re: threading Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2014-04-10 11:19 +1200
            Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-09 19:48 +0300
          Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 00:44 +1000
          Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-09 15:29 +0000
          Re: threading Terry Reedy <tjreedy@udel.edu> - 2014-04-09 12:14 -0400
          Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 02:25 +1000
          Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-09 16:32 +0000
          Re: threading Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-04-09 19:44 -0400
          Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 11:05 +1000
          Re: threading "Frank Millman" <frank@chagford.com> - 2014-04-10 11:17 +0200
          Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 19:40 +1000
          Re: threading "Frank Millman" <frank@chagford.com> - 2014-04-10 13:10 +0200
            Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-10 14:43 +0300
              Re: threading Roy Smith <roy@panix.com> - 2014-04-10 08:56 -0400
              Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-10 15:24 +0000
                Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-10 19:20 +0300
              Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-11 01:32 +1000
                Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-10 19:25 +0300
                Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-11 03:08 +1000
                Re: threading Rustom Mody <rustompmody@gmail.com> - 2014-04-10 11:14 -0700
                Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-10 22:44 +0300
                Re: threading Rustom Mody <rustompmody@gmail.com> - 2014-04-10 13:21 -0700
                Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-10 23:44 +0300
                Re: threading Rustom Mody <rustompmody@gmail.com> - 2014-04-10 22:15 -0700
                Re: threading Rustom Mody <rustompmody@gmail.com> - 2014-04-10 23:50 -0700
                Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-11 18:36 +0300
                Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-12 01:53 +1000
                Re: threading Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-04-11 16:58 +0100
                Re: threading Rustom Mody <rustompmody@gmail.com> - 2014-04-11 11:54 -0700
                Re: threading Marko Rauhamaa <marko@pacujo.net> - 2014-04-11 22:27 +0300
                Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-11 01:51 +0200
                Re: threading Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-04-11 05:35 +0000
                Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-11 09:26 +0000
                Re: threading Roy Smith <roy@panix.com> - 2014-04-11 08:36 -0400
                Re: threading Grant Edwards <invalid@invalid.invalid> - 2014-04-11 16:18 +0000
                Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-11 02:21 +0200
                Re: threading Terry Reedy <tjreedy@udel.edu> - 2014-04-10 20:23 -0400
          Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-10 21:19 +1000
      Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 02:06 +0000
        Re: threading alister <alister.nospam.ware@ntlworld.com> - 2014-04-08 11:07 +0000
          Re: threading Roy Smith <roy@panix.com> - 2014-04-08 09:13 -0400
            Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-08 23:23 +1000
              Re: threading alister <alister.nospam.ware@ntlworld.com> - 2014-04-08 14:15 +0000
                Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 16:06 +0000
            Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 15:40 +0000
              Re: threading Paul Rubin <no.email@nospam.invalid> - 2014-04-08 09:46 -0700
                Re: threading Chris Angelico <rosuav@gmail.com> - 2014-04-09 02:46 +1000
                Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 17:17 +0000
          Re: threading Sturla Molden <sturla.molden@gmail.com> - 2014-04-08 15:19 +0000

csiph-web