Are threads bad? - was: Future of Pypy?

References	(9 earlier) <87lhjpb89i.fsf@jester.gateway.pace.com> <mailman.19052.1424664024.18130.python-list@python.org> <87h9udb1eq.fsf@jester.gateway.pace.com> <mailman.19058.1424676730.18130.python-list@python.org> <87bnkkb22u.fsf@jester.gateway.pace.com>
From	Ryan Stuart <ryan.stuart.85@gmail.com>
Date	2015-02-24 00:35 +0000
Subject	Are threads bad? - was: Future of Pypy?
Newsgroups	comp.lang.python
Message-ID	<mailman.19111.1424738135.18130.python-list@python.org> (permalink)

Show all headers | View raw

[Multipart message — attachments visible in raw view] - view raw

On Tue Feb 24 2015 at 10:15:40 AM Paul Rubin <no.email@nospam.invalid>
wrote:
>
> I don't see the error-proneness since nothing there seems to set off
> mutation of shared data.
>

I'm not sure what else to say really. It's just a fact of life that Threads
by definition run in the same memory space and hence always have the
possibility of nasty unforeseen problems. They are unforeseen because it is
extremely difficult (maybe impossible?) to try and map out and understand
all the different possible mutations to state. Sure, your code might not be
making any mutations (that you know of), but malloc definitely is [1], and
that's just the tip of the iceberg. Other things like buffers for stdin and
stdout, DNS resolution etc. all have the same issue.

I have no doubt someone can come up with a scenario where they need to use
threads. I can't come up with one myself, but maybe someone else can. But
in the work I have done, processes have sufficed - even for the example of
dynamic callables you gave.

To borrow from the original article I linked - "Nevertheless I still think
it’s a bad idea to make things harder for ourselves if we can avoid it."

Cheers

[1] Line 70 of glibc malloc -
https://sourceware.org/git/?p=glibc.git;a=blob;f=malloc/arena.c;h=8af51f05eb376ae2ba07e99c8c766a8ae8af425b;hb=bdf1ff052a8e23d637f2c838fa5642d78fcedc33#l70


>
> > Also, the majority can be achieved via the process approach. For
> > example, using fork to take a copy of the current process (including
> > the heap) you want to use will give you access to any callables on the
> > heap.
>
> What if you want to dynamically construct a callable and send it to
> another process?
>
> > Even if you are extra careful to not touch any shared state in your
> > code, you can almost be guaranteed that code higher up the stack, like
> > malloc for example, *will* be using shared state.
>
> This isn't the 1980's any more--any serious malloc implementation these
> days is thread safe.  People write multi-threaded C programs all the
> time and those programs use malloc in more than one thread.
>
> > Even if you aren't sharing state in your code directly, code higher up
> > the stack will be sharing state.  That is the whole point of a thread,
> > that's what they were invented for.  Using threads safely might well
> > be impossible much less verifiable.
>
> You're basically saying it's impossible to write a reliable operating
> system, since OS's by nature have to do that stuff.  Of course there are
> verified OS's, and some of the early pioneers in concurrency were the
> same guys who worked in program verification, e.g. Dijkstra's
> semaphores.  Even Erlang uses data sharing under the hood (ETS tables
> and large binaries) though their API makes it look like the data is
> copied between processes.
>
> What I'd say is that multi-threaded programs tend to have miniature OS's
> inside them, so it helps to have had some exposure to OS implementation
> techniques if you're going to write this kind of code.  But if you've
> had that exposure then it all becomes less scary.
>
> > So when there are other options that are just as viable/functional,
> > result in far less risk and are often much quicker to implement
> > correctly, why wouldn't you use them?
>
> I should give the multiprocessing module a try sometime (haven't used it
> so far because it's relatively new and I'm comfortable with threads).
> It has the disadvantages that I noted, though.
>
> > If it were easy to use threads in a verifiably safe manner, then there
> > probably wouldn't be a GIL.
>
> Nah, the GIL is just a CPython artifact.  As Steven says, IronPython and
> Jython don't have GIL's.  Java has no GIL, OCaml has no GIL, GHC has no
> GIL, etc.  Someone made a CPython version with no GIL some years ago and
> it worked fine and it got a speedup on multiple cores.  The only problem
> was that on a single core, it was significantly slower than regular
> CPython, specifically because of the overhead of having to lock all the
> refcount updates, so it was considered a failure.  Laura Creighton may
> have more to say about this, but I've been under the impression that the
> main obstacle to getting rid of the CPython GIL is the refcount system
> (which is also easy to make mistakes with, by the way).  That's why I
> was surprised to hear that PyPy has a GIL.
> --
> https://mail.python.org/mailman/listinfo/python-list
>

Thread

Future of Pypy? Dave Farrance <DaveFarrance@OMiTTHiSyahooANDTHiS.co.uk> - 2015-02-22 12:45 +0000
  Re: Future of Pypy? jkn <jkn_gg@nicorp.f9.co.uk> - 2015-02-22 04:58 -0800
    Re: Future of Pypy? Dave Farrance <DaveFarrance@OMiTTHiSyahooANDTHiS.co.uk> - 2015-02-22 15:30 +0000
      [OT] - BASIC is still not a bad choice, was Re: Future of Pypy? Michael Torrie <torriem@gmail.com> - 2015-02-23 17:24 -0700
  Re: Future of Pypy? Laura Creighton <lac@openend.se> - 2015-02-22 14:27 +0100
    Re: Future of Pypy? Dave Farrance <DaveFarrance@OMiTTHiSyahooANDTHiS.co.uk> - 2015-02-22 15:36 +0000
      Re: Future of Pypy? Laura Creighton <lac@openend.se> - 2015-02-22 18:22 +0100
        Re: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-22 11:02 -0800
          Re: Future of Pypy? Laura Creighton <lac@openend.se> - 2015-02-22 20:51 +0100
            Re: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-22 12:14 -0800
              Re: Future of Pypy? Laura Creighton <lac@openend.se> - 2015-02-22 23:13 +0100
                Re: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-22 18:45 -0800
          Re: Future of Pypy? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-02-23 12:18 +1100
            Re: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-22 18:04 -0800
              Re: Future of Pypy? Chris Angelico <rosuav@gmail.com> - 2015-02-23 13:16 +1100
              Re: Future of Pypy? Ryan Stuart <ryan.stuart.85@gmail.com> - 2015-02-23 03:16 +0000
                Re: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-22 19:45 -0800
                Re: Future of Pypy? Ryan Stuart <ryan.stuart.85@gmail.com> - 2015-02-23 04:00 +0000
                Re: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-22 22:13 -0800
                Re: Future of Pypy? Ryan Stuart <ryan.stuart.85@gmail.com> - 2015-02-23 07:32 +0000
                Re: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-23 16:11 -0800
                Re: Future of Pypy? Chris Angelico <rosuav@gmail.com> - 2015-02-24 11:31 +1100
                Re: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-23 17:50 -0800
                Re: Future of Pypy? Chris Angelico <rosuav@gmail.com> - 2015-02-24 13:03 +1100
                Re: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-23 20:40 -0800
                Re: Future of Pypy? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-02-24 17:57 +1100
                Re: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-27 13:40 -0800
                Re: Future of Pypy? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2015-02-27 18:47 -0500
                Are threads bad? - was: Future of Pypy? Ryan Stuart <ryan.stuart.85@gmail.com> - 2015-02-24 00:35 +0000
                Re: Are threads bad? - was: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-23 21:27 -0800
                Re: Are threads bad? - was: Future of Pypy? Chris Angelico <rosuav@gmail.com> - 2015-02-24 16:57 +1100
                Re: Are threads bad? - was: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-23 22:23 -0800
                Re: Are threads bad? - was: Future of Pypy? Marko Rauhamaa <marko@pacujo.net> - 2015-02-24 10:08 +0200
                Re: Are threads bad? - was: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-24 15:53 -0800
                Re: Are threads bad? - was: Future of Pypy? Marko Rauhamaa <marko@pacujo.net> - 2015-02-25 07:25 +0200
                Re: Are threads bad? - was: Future of Pypy? Marcos Almeida Azevedo <marcos.al.azevedo@gmail.com> - 2015-02-25 13:34 +0800
                Re: Are threads bad? - was: Future of Pypy? Marko Rauhamaa <marko@pacujo.net> - 2015-02-25 07:46 +0200
                Re: Are threads bad? - was: Future of Pypy? Chris Angelico <rosuav@gmail.com> - 2015-02-25 16:54 +1100
                Re: Are threads bad? - was: Future of Pypy? Marcos Almeida Azevedo <marcos.al.azevedo@gmail.com> - 2015-02-25 13:58 +0800
                Re: Are threads bad? - was: Future of Pypy? Ian Kelly <ian.g.kelly@gmail.com> - 2015-02-24 23:02 -0700
                Re: Are threads bad? - was: Future of Pypy? Chris Angelico <rosuav@gmail.com> - 2015-02-25 17:07 +1100
                Re: Are threads bad? - was: Future of Pypy? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-02-25 16:37 +0000
                Re: Are threads bad? - was: Future of Pypy? Ian Kelly <ian.g.kelly@gmail.com> - 2015-02-25 10:00 -0700
                Re: Are threads bad? - was: Future of Pypy? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-02-25 17:16 +0000
                Re: Are threads bad? - was: Future of Pypy? Chris Angelico <rosuav@gmail.com> - 2015-02-26 04:22 +1100
                Re: Are threads bad? - was: Future of Pypy? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2015-02-25 19:44 -0500
                Re: Are threads bad? - was: Future of Pypy? Ryan Stuart <ryan.stuart.85@gmail.com> - 2015-02-25 00:59 +0000
                Re: Are threads bad? - was: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-26 21:55 -0800
              Re: Future of Pypy? Chris Angelico <rosuav@gmail.com> - 2015-02-23 14:25 +1100
              Re: Future of Pypy? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-02-23 18:41 +1100
                Re: Future of Pypy? Marko Rauhamaa <marko@pacujo.net> - 2015-02-23 10:16 +0200
                Re: Future of Pypy? Chris Angelico <rosuav@gmail.com> - 2015-02-23 20:19 +1100
                Re: Future of Pypy? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-02-24 17:56 +1100
                Re: Future of Pypy? Chris Angelico <rosuav@gmail.com> - 2015-02-24 18:16 +1100
                Re: Future of Pypy? wxjmfauth@gmail.com - 2015-02-23 23:57 -0800
                Re: Future of Pypy? Ethan Furman <ethan@stoneleaf.us> - 2015-02-23 11:39 -0800
                Re: Future of Pypy? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-02-24 13:15 +1100
                Re: Future of Pypy? Paul Rubin <no.email@nospam.invalid> - 2015-02-23 17:47 -0800
                Re: Future of Pypy? Marko Rauhamaa <marko@pacujo.net> - 2015-02-24 10:12 +0200
      Re: Future of Pypy? Emile van Sebille <emile@fenx.com> - 2015-02-24 09:57 -0800
  Re: Future of Pypy? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-02-23 01:05 +1100
    Re: Future of Pypy? Dave Farrance <DaveFarrance@OMiTTHiSyahooANDTHiS.co.uk> - 2015-02-22 15:44 +0000
      Re: Future of Pypy? Dave Farrance <DaveFarrance@OMiTTHiSyahooANDTHiS.co.uk> - 2015-02-22 19:20 +0000
        Re: Future of Pypy? Laura Creighton <lac@openend.se> - 2015-02-22 22:45 +0100
          Re: Future of Pypy? Dave Farrance <DaveFarrance@OMiTTHiSyahooANDTHiS.co.uk> - 2015-02-23 14:04 +0000
            Re: Future of Pypy? Laura Creighton <lac@openend.se> - 2015-02-23 17:16 +0100
  Re: Future of Pypy? Terry Reedy <tjreedy@udel.edu> - 2015-02-23 01:34 -0500
  Re: Future of Pypy? Dave Cook <davecook@nowhere.net> - 2015-02-23 11:36 +0000
    Re: Future of Pypy? Dave Farrance <DaveFarrance@OMiTTHiSyahooANDTHiS.co.uk> - 2015-02-23 14:13 +0000
      Cython - was: Future of Pypy? Stefan Behnel <stefan_ml@behnel.de> - 2015-02-23 16:43 +0100
      Re: Future of Pypy? Dave Cook <davecook@nowhere.net> - 2015-02-23 23:23 +0000

csiph-web