Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #24746 > unrolled thread

when "normal" parallel computations in CPython will be implemented at last?

Back to article view | Back to comp.lang.python

Contents

  when "normal" parallel computations in CPython will be implemented at last? dmitrey <dmitrey15@gmail.com> - 2012-07-01 10:51 -0700
    Re: when "normal" parallel computations in CPython will be implemented at last? Thomas Jollans <t@jollybox.de> - 2012-07-01 20:53 +0200
      Re: when "normal" parallel computations in CPython will be implemented at last? Ross Ridge <rridge@csclub.uwaterloo.ca> - 2012-07-01 16:03 -0400
    Re: when "normal" parallel computations in CPython will be implemented at last? Thomas Jollans <t@jollybox.de> - 2012-07-01 20:58 +0200
    Re: when "normal" parallel computations in CPython will be implemented at last? Andrew Berg <bahamutzero8825@gmail.com> - 2012-07-01 14:09 -0500
    Re: when "normal" parallel computations in CPython will be implemented at last? Thomas Jollans <t@jollybox.de> - 2012-07-01 21:36 +0200
    Re: when "normal" parallel computations in CPython will be implemented at last? John Nagle <nagle@animats.com> - 2012-07-02 10:51 -0700
      Re: when "normal" parallel computations in CPython will be implemented at last? Tim Roberts <timr@probo.com> - 2012-07-03 20:16 -0700
    Re: when "normal" parallel computations in CPython will be implemented at last? Stefan Behnel <stefan_ml@behnel.de> - 2012-07-02 20:37 +0200

#24746 — when "normal" parallel computations in CPython will be implemented at last?

hi all,
are there any information about upcoming availability of parallel
computations in CPython without modules like  multiprocessing? I mean
something like parallel "for" loops, or, at least, something without
forking with copying huge amounts of RAM each time and possibility to
involve unpiclable data (vfork would be ok, but AFAIK it doesn't work
with CPython due to GIL).

AFAIK in PyPy some progress have been done (
http://morepypy.blogspot.com/2012/06/stm-with-threads.html )

Thank you in advance, D.

[toc] | [next] | [standalone]

#24747

On 07/01/2012 07:51 PM, dmitrey wrote:
> hi all,
> are there any information about upcoming availability of parallel
> computations in CPython without modules like  multiprocessing? I mean
> something like parallel "for" loops, or, at least, something without
> forking with copying huge amounts of RAM each time and possibility to
> involve unpiclable data (vfork would be ok, but AFAIK it doesn't work
> with CPython due to GIL).
> 
> AFAIK in PyPy some progress have been done (
> http://morepypy.blogspot.com/2012/06/stm-with-threads.html )

As far as I can tell, there are no concrete plans to integrate
concurrency better, or get rid of the GIL, at the moment. To quote
http://wiki.python.org/moin/GlobalInterpreterLock

  """Getting rid of the GIL is an occasional topic on the python-dev
mailing list. No one has managed it yet."""

There are currently no open or accepted PEPs on the subject of concurrency.
http://www.python.org/dev/peps/

There is, of course, Stackless Python.
http://stackless.com/

[toc] | [prev] | [next] | [standalone]

#24751

Thomas Jollans  <t@jollybox.de> wrote:
>There is, of course, Stackless Python.
>http://stackless.com/

Stackless Python doesn't really address the original poster's problem
as the GIL still effectively limits Python code running in one thread
at a time.

					Ross Ridge

-- 
 l/  //	  Ross Ridge -- The Great HTMU
[oo][oo]  rridge@csclub.uwaterloo.ca
-()-/()/  http://www.csclub.uwaterloo.ca/~rridge/ 
 db  //	  

[toc] | [prev] | [next] | [standalone]

#24748

On 07/01/2012 08:44 PM, Dan Stromberg wrote:
> IronPython, sadly, lacks a python standard library.


Beg pardon?

https://github.com/IronLanguages/main/tree/master/External.LCA_RESTRICTED/Languages/IronPython/27/Lib

[toc] | [prev] | [next] | [standalone]

#24749

On 7/1/2012 1:53 PM, Thomas Jollans wrote:
> As far as I can tell, there are no concrete plans to integrate
> concurrency better, or get rid of the GIL, at the moment. To quote
> http://wiki.python.org/moin/GlobalInterpreterLock
> 
>   """Getting rid of the GIL is an occasional topic on the python-dev
> mailing list. No one has managed it yet."""
There's also this, recently written by one of CPython's core devs:
http://python-notes.boredomandlaziness.org/en/latest/python3/questions_and_answers.html#but-but-surely-fixing-the-gil-is-more-important-than-fixing-unicode
-- 
CPython 3.3.0a4 | Windows NT 6.1.7601.17803

[toc] | [prev] | [next] | [standalone]

#24750

On 07/01/2012 09:28 PM, Dan Stromberg wrote:
> 
> 
> On Sun, Jul 1, 2012 at 11:58 AM, Thomas Jollans <t@jollybox.de
> <mailto:t@jollybox.de>> wrote:
> 
>     On 07/01/2012 08:44 PM, Dan Stromberg wrote:
>     > IronPython, sadly, lacks a python standard library.
> 
> 
>     Beg pardon?
> 
>     https://github.com/IronLanguages/main/tree/master/External.LCA_RESTRICTED/Languages/IronPython/27/Lib
> 
> Perhaps things have changed.
> 
> When I last checked the situation, IronPython came with no standard
> library, but you could bolt one on that hadn't been tested well - IIRC,
> just a simple "import os" gave a traceback.  FePy was IronPython with a
> standard library and some degree of testing, but their emphasis was
> windows-only.
> 
> I'd be open to using FePy instead, and I might even call it IronPython,
> but I'm not in the habit of happily using software that is Windows only.
> 

That must have been quite a while ago?

I haven't tested it recently, but I'm fairly sure that IronPython
includes a Python standard library which works reasonably well, and it's
not Windows-only. (it works with Mono)

[toc] | [prev] | [next] | [standalone]

#24782

On 7/1/2012 10:51 AM, dmitrey wrote:
> hi all,
> are there any information about upcoming availability of parallel
> computations in CPython without modules like  multiprocessing? I mean
> something like parallel "for" loops, or, at least, something without
> forking with copying huge amounts of RAM each time and possibility to
> involve unpiclable data (vfork would be ok, but AFAIK it doesn't work
> with CPython due to GIL).
>
> AFAIK in PyPy some progress have been done (
> http://morepypy.blogspot.com/2012/06/stm-with-threads.html )
>
> Thank you in advance, D.
>

    It would be "un-Pythonic" to have real concurrency in Python.
You wouldn't be able to patch code running in one thread from
another thread.  Some of the dynamic features of Python
would break.   If you want fine-grained concurrency, you need
controlled isolation between concurrent tasks, so they interact
only at well-defined points.  That's un-Pythonic.

				John Nagle

[toc] | [prev] | [next] | [standalone]

#24845

John Nagle <nagle@animats.com> wrote:
>
>    It would be "un-Pythonic" to have real concurrency in Python.
>You wouldn't be able to patch code running in one thread from
>another thread.  Some of the dynamic features of Python
>would break.   If you want fine-grained concurrency, you need
>controlled isolation between concurrent tasks, so they interact
>only at well-defined points.  That's un-Pythonic.

I disagree.  The situation in Python is no different than the situation in
other programming languages.  If you have shared state, you protect it with
some kind of lock.  After all, you don't patch code on a byte-by-byte basis
-- you just change function bindings.  That can be done atomically.
-- 
Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.

[toc] | [prev] | [next] | [standalone]

#24784

Dan Stromberg, 01.07.2012 21:28:
> On Sun, Jul 1, 2012 at 11:58 AM, Thomas Jollans wrote:
>> On 07/01/2012 08:44 PM, Dan Stromberg wrote:
>>> IronPython, sadly, lacks a python standard library.
>>
>> Beg pardon?
>>
>> https://github.com/IronLanguages/main/tree/master/External.LCA_RESTRICTED/Languages/IronPython/27/Lib
>
> Perhaps things have changed.

Yes, they have. The original restriction came from Microsoft enforcing a
clean-room implementation for their code, which obviously excluded the
standard library - which is well tested and licence cleared and all that,
but nevertheless non-MS. When they dropped the IronPython project, it
became free to integrate with other software.

Clearly a cultural thing.

Stefan

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web