Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #45581

Re: Please help with Threading

Show all headers | View raw

On Sun, 19 May 2013 21:04:42 -0400, Dave Angel <davea@davea.name>
declaimed the following in gmane.comp.python.general:

> So what's the mapping between real (OS) threads, and the fake ones 
> Python uses?  The OS keeps track of a separate stack and context for 
> each thread it knows about;  are they one-to-one with the ones you're 
> describing here?  If so, then any OS thread that gets scheduled will 
> almost always find it can't get the GIL, and spend time thrashing.   But 
> the change that CPython does intentionally would be equivalent to a 
> sleep(0).
>
	No. The first time that thread attempts to gain the GIL it will be
blocked. It will not be made ready again until the current owner of the
GIL frees it (at which point it competes with all other threads that
were blocked). 

	No thrashing -- but a lot of threads blocked waiting for the GIL,
and why multicore processors won't see a speed up in number crunching
applications using threads. Multiprocessing creates copies of the
interpreter, and each copy has its own GIL which won't conflict with the
others -- so intense number crunchers can benefit even with the overhead
of creating a new/independent process. I/O bound tasks don't gain as
much from multiprocessing as you have the overhead of creating a system
process, only to spend most of the time waiting for an I/O operation to
complete. Threads work well for that situation (and Twisted even gets by
without threading -- though my mind just can't work with the Twisted
architecture <G>; even one number cruncher in Twisted has to be
cooperative, working in chunks and returning so the dispatcher can
handle events).

 
> On the other hand, if these threads are distinct from the OS threads, is 
> it done with some sort of thread pool, where CPython has its own stack, 
> and doesn't really use the one managed by the OS?
>

	Even the common GNAT Ada releases rely upon the OS for tasking.

	One pretty much has to use the OS task scheduler, otherwise one
thread inside the interpreter/runtime that blocks on an OS system call
will block the entire interpreter/runtime and hence any other thread
would be blocked too.

> Understand the only OS threading I really understand is the one in 
> Windows (which I no longer use).  So assuming Linux has some form of 
> lightweight threading, the distinction above may not map very well.

	And I'm most familiar with the Amiga, even though I've not used it
in 20 years. In it, the OS scheduled "tasks" -- but above tasks were
"processes". A process contained structures for stdin/stdout/stderr,
current directory, environment variables. These structures were just
extensions of the task control block holding signal bits, register
contents (when not running) etc.

	Windows "lightweight" threads would probably be "fibers" -- which
require the Windows application itself to schedule them, rather than the
OS. IOW, they are closer to co-routines (and run within a thread that is
controlling them).


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
        wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Please help with Threading Jurgens de Bruin <debruinjj@gmail.com> - 2013-05-18 01:58 -0700
  Re: Please help with Threading Peter Otten <__peter__@web.de> - 2013-05-18 11:09 +0200
    Re: Please help with Threading Jurgens de Bruin <debruinjj@gmail.com> - 2013-05-18 04:23 -0700
      Re: Please help with Threading Peter Otten <__peter__@web.de> - 2013-05-18 14:01 +0200
  Re: Please help with Threading Dave Angel <davea@davea.name> - 2013-05-18 09:55 -0400
  Re: Please help with Threading Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2013-05-18 15:28 -0400
  RE: Please help with Threading Carlos Nepomuceno <carlosnepomuceno@outlook.com> - 2013-05-19 03:02 +0300
  Re: Please help with Threading Chris Angelico <rosuav@gmail.com> - 2013-05-19 10:38 +1000
  Re: Please help with Threading Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2013-05-19 17:46 -0400
  Re: Please help with Threading Chris Angelico <rosuav@gmail.com> - 2013-05-20 07:52 +1000
  Re: Please help with Threading Dave Angel <davea@davea.name> - 2013-05-19 21:04 -0400
  Re: Please help with Threading Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2013-05-19 22:58 -0400
  Re: Please help with Threading Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2013-05-19 23:25 -0400
  Re: Please help with Threading Fábio Santos <fabiosantosart@gmail.com> - 2013-05-20 07:25 +0100
  Re: Please help with Threading Jurgens de Bruin <debruinjj@gmail.com> - 2013-06-02 18:47 -0700

csiph-web