Re: Are threads bad? - was: Future of Pypy?

References	(11 earlier) <87h9udb1eq.fsf@jester.gateway.pace.com> <mailman.19058.1424676730.18130.python-list@python.org> <87bnkkb22u.fsf@jester.gateway.pace.com> <mailman.19111.1424738135.18130.python-list@python.org> <87lhjnang1.fsf@jester.gateway.pace.com>
From	Ryan Stuart <ryan.stuart.85@gmail.com>
Date	2015-02-25 00:59 +0000
Subject	Re: Are threads bad? - was: Future of Pypy?
Newsgroups	comp.lang.python
Message-ID	<mailman.19162.1424825985.18130.python-list@python.org> (permalink)

Show all headers | View raw

[Multipart message — attachments visible in raw view] - view raw

On Tue Feb 24 2015 at 3:32:47 PM Paul Rubin <no.email@nospam.invalid> wrote:

> Ryan Stuart <ryan.stuart.85@gmail.com> writes:
> Sure, the shared memory introduces the possibility of some bad errors,
> I'm just saying that I've found that by staying with a certain
> straightforward style, it doesn't seem difficult in practice to avoid
> those errors.
>

And all I'm saying is that it doesn't matter how careful you are, all it
takes is someone up the stack to not be as careful and you will be bitten.
And even if they are careful, it's still very easy to be bitten because
everything runs in shared memory.

> I don't understand what you mean about malloc.
>

My point is malloc, something further up (down?) the stack, is making
modifications to shared state when threads are involved. Modifying shared
state makes it infinitely more difficult to reason about the correctness of
your software.

> That article was interesting in some ways but confused in others.  One
> way it was interesting is it said various non-thread approaches (such as
> coroutines) had about the same problems as threads.  Some ways it
> was confused were:
>

We clearly got completely different things from the article. My
interpretation was that it was making *the exact opposite* point to what
you stated mainly because non-threading approaches don't share state. It
states that quite clearly. For example "it is – literally – exponentially
more difficult to reason about a routine that may be executed from an
arbitrary number of threads concurrently".

>   1) thinking Haskell threads were like processes with separate address
>   spaces.  In fact they are in the same address space and programming
>   with them isn't all that different from Python threads, though the
>   synchronization primitives are a bit different.  There is also an STM
>   library available that is ingenious though apparently somewhat slow.
>

I don't know Haskell, so I can't comment too much, except to say that by
default Haskell looks to use lightweight threads where only 1 thread can be
executing at a time [1] (sounds similar to the GIL to me). That doesn't
seem to be shared state multithreading, which is what the article is
referring to. But again, they will undoubtedly be someone sharing state
higher up the stack.

>   2) it has a weird story about the brass cockroach, that basically
>   signified that they didn't have a robust enough testing system to be
>   able to reproduce the bug.  That is what they should have worked on.
>

The point was that it wasn't feasible to have a robust testing suite
because, you guessed it, shared state and concurrent threads producing n*n
complexity.

>   3) It goes into various hazards of the balance transfer example not
>   mentioning that STM (available in Haskell and Clojure) completely
>   solves it.
>

This is probably correct. Is there any STM implementations out that that
don't significantly compromise performance?

Anyway, I got one thing out of this, which is that the multiprocessing
> module looks pretty nice and I should try it even when I don't need
> multicore parallelism, so thanks for that.
>

It's 1 real advantage is that it side-steps the GIL. So, if you need to
utilise multiple cores for CPU bound tasks, then it might well be the only
option.

Cheers

> In reality though, Python is still my most productive language for
> throwaway, non-concurrent scripting, but for more complicated concurrent
> programs, alternatives like Haskell, Erlang, and Go all have significant
> attractions.
> --
> https://mail.python.org/mailman/listinfo/python-list
>

Thread

Future of Pypy? Dave Farrance <DaveFarrance@OMiTTHiSyahooANDTHiS.co.uk> - 2015-02-22 12:45 +0000
  Re: Future of Pypy? jkn <jkn_gg@nicorp.f9.co.uk> - 2015-02-22 04:58 -0800
    Re: Future of Pypy? Dave Farrance <DaveFarrance@OMiTTHiSyahooANDTHiS.co.uk> - 2015-02-22 15:30 +0000
      [OT] - BASIC is still not a bad choice, was Re: Future of Pypy? Michael Torrie <torriem@gmail.com> - 2015-02-23 17:24 -0700
  Re: Future of Pypy? Laura Creighton <lac@openend.se> - 2015-02-22 14:27 +0100
    Re: Future of Pypy? Dave Farrance <DaveFarrance@OMiTTHiSyahooANDTHiS.co.uk> - 2015-02-22 15:36 +0000
      Re: Future of Pypy? Laura Creighton <lac@openend.se> - 2015-02-22 18:22 +0100
        Re: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-22 11:02 -0800
          Re: Future of Pypy? Laura Creighton <lac@openend.se> - 2015-02-22 20:51 +0100
            Re: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-22 12:14 -0800
              Re: Future of Pypy? Laura Creighton <lac@openend.se> - 2015-02-22 23:13 +0100
                Re: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-22 18:45 -0800
          Re: Future of Pypy? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-02-23 12:18 +1100
            Re: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-22 18:04 -0800
              Re: Future of Pypy? Chris Angelico <rosuav@gmail.com> - 2015-02-23 13:16 +1100
              Re: Future of Pypy? Ryan Stuart <ryan.stuart.85@gmail.com> - 2015-02-23 03:16 +0000
                Re: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-22 19:45 -0800
                Re: Future of Pypy? Ryan Stuart <ryan.stuart.85@gmail.com> - 2015-02-23 04:00 +0000
                Re: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-22 22:13 -0800
                Re: Future of Pypy? Ryan Stuart <ryan.stuart.85@gmail.com> - 2015-02-23 07:32 +0000
                Re: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-23 16:11 -0800
                Re: Future of Pypy? Chris Angelico <rosuav@gmail.com> - 2015-02-24 11:31 +1100
                Re: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-23 17:50 -0800
                Re: Future of Pypy? Chris Angelico <rosuav@gmail.com> - 2015-02-24 13:03 +1100
                Re: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-23 20:40 -0800
                Re: Future of Pypy? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-02-24 17:57 +1100
                Re: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-27 13:40 -0800
                Re: Future of Pypy? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2015-02-27 18:47 -0500
                Are threads bad? - was: Future of Pypy? Ryan Stuart <ryan.stuart.85@gmail.com> - 2015-02-24 00:35 +0000
                Re: Are threads bad? - was: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-23 21:27 -0800
                Re: Are threads bad? - was: Future of Pypy? Chris Angelico <rosuav@gmail.com> - 2015-02-24 16:57 +1100
                Re: Are threads bad? - was: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-23 22:23 -0800
                Re: Are threads bad? - was: Future of Pypy? Marko Rauhamaa <marko@pacujo.net> - 2015-02-24 10:08 +0200
                Re: Are threads bad? - was: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-24 15:53 -0800
                Re: Are threads bad? - was: Future of Pypy? Marko Rauhamaa <marko@pacujo.net> - 2015-02-25 07:25 +0200
                Re: Are threads bad? - was: Future of Pypy? Marcos Almeida Azevedo <marcos.al.azevedo@gmail.com> - 2015-02-25 13:34 +0800
                Re: Are threads bad? - was: Future of Pypy? Marko Rauhamaa <marko@pacujo.net> - 2015-02-25 07:46 +0200
                Re: Are threads bad? - was: Future of Pypy? Chris Angelico <rosuav@gmail.com> - 2015-02-25 16:54 +1100
                Re: Are threads bad? - was: Future of Pypy? Marcos Almeida Azevedo <marcos.al.azevedo@gmail.com> - 2015-02-25 13:58 +0800
                Re: Are threads bad? - was: Future of Pypy? Ian Kelly <ian.g.kelly@gmail.com> - 2015-02-24 23:02 -0700
                Re: Are threads bad? - was: Future of Pypy? Chris Angelico <rosuav@gmail.com> - 2015-02-25 17:07 +1100
                Re: Are threads bad? - was: Future of Pypy? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-02-25 16:37 +0000
                Re: Are threads bad? - was: Future of Pypy? Ian Kelly <ian.g.kelly@gmail.com> - 2015-02-25 10:00 -0700
                Re: Are threads bad? - was: Future of Pypy? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-02-25 17:16 +0000
                Re: Are threads bad? - was: Future of Pypy? Chris Angelico <rosuav@gmail.com> - 2015-02-26 04:22 +1100
                Re: Are threads bad? - was: Future of Pypy? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2015-02-25 19:44 -0500
                Re: Are threads bad? - was: Future of Pypy? Ryan Stuart <ryan.stuart.85@gmail.com> - 2015-02-25 00:59 +0000
                Re: Are threads bad? - was: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-26 21:55 -0800
              Re: Future of Pypy? Chris Angelico <rosuav@gmail.com> - 2015-02-23 14:25 +1100
              Re: Future of Pypy? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-02-23 18:41 +1100
                Re: Future of Pypy? Marko Rauhamaa <marko@pacujo.net> - 2015-02-23 10:16 +0200
                Re: Future of Pypy? Chris Angelico <rosuav@gmail.com> - 2015-02-23 20:19 +1100
                Re: Future of Pypy? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-02-24 17:56 +1100
                Re: Future of Pypy? Chris Angelico <rosuav@gmail.com> - 2015-02-24 18:16 +1100
                Re: Future of Pypy? wxjmfauth@gmail.com - 2015-02-23 23:57 -0800
                Re: Future of Pypy? Ethan Furman <ethan@stoneleaf.us> - 2015-02-23 11:39 -0800
                Re: Future of Pypy? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-02-24 13:15 +1100
                Re: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-23 17:47 -0800
                Re: Future of Pypy? Marko Rauhamaa <marko@pacujo.net> - 2015-02-24 10:12 +0200
      Re: Future of Pypy? Emile van Sebille <emile@fenx.com> - 2015-02-24 09:57 -0800
  Re: Future of Pypy? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-02-23 01:05 +1100
    Re: Future of Pypy? Dave Farrance <DaveFarrance@OMiTTHiSyahooANDTHiS.co.uk> - 2015-02-22 15:44 +0000
      Re: Future of Pypy? Dave Farrance <DaveFarrance@OMiTTHiSyahooANDTHiS.co.uk> - 2015-02-22 19:20 +0000
        Re: Future of Pypy? Laura Creighton <lac@openend.se> - 2015-02-22 22:45 +0100
          Re: Future of Pypy? Dave Farrance <DaveFarrance@OMiTTHiSyahooANDTHiS.co.uk> - 2015-02-23 14:04 +0000
            Re: Future of Pypy? Laura Creighton <lac@openend.se> - 2015-02-23 17:16 +0100
  Re: Future of Pypy? Terry Reedy <tjreedy@udel.edu> - 2015-02-23 01:34 -0500
  Re: Future of Pypy? Dave Cook <davecook@nowhere.net> - 2015-02-23 11:36 +0000
    Re: Future of Pypy? Dave Farrance <DaveFarrance@OMiTTHiSyahooANDTHiS.co.uk> - 2015-02-23 14:13 +0000
      Cython - was: Future of Pypy? Stefan Behnel <stefan_ml@behnel.de> - 2015-02-23 16:43 +0100
      Re: Future of Pypy? Dave Cook <davecook@nowhere.net> - 2015-02-23 23:23 +0000

csiph-web