Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #6446 > unrolled thread

Re: GIL in alternative implementations

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: GIL in alternative implementations Daniel Kluev <dan.kluev@gmail.com> - 2011-05-28 13:06 +1100
    Re: GIL in alternative implementations Marc Christiansen <usenet@solar-empire.de> - 2011-05-28 11:55 +0200
      Re: GIL in alternative implementations Wolfgang Rohdewald <wolfgang@rohdewald.de> - 2011-05-28 13:16 +0200
    Re: GIL in alternative implementations John Nagle <nagle@animats.com> - 2011-05-28 09:39 -0700
      Re: GIL in alternative implementations Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-05-28 17:05 +0000
        Re: GIL in alternative implementations "Gabriel Genellina" <gagsl-py2@yahoo.com.ar> - 2011-06-07 01:03 -0300
          Re: GIL in alternative implementations Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-06-07 07:09 +0000
          Re: GIL in alternative implementations Jean-Paul Calderone <calderone.jeanpaul@gmail.com> - 2011-06-07 05:07 -0700

#6446 — Re: GIL in alternative implementations

> So I'd like to know: how do these other implementations handle concurrency
> matters for their primitive types, and prevent them from getting corrupted
> in multithreaded programs (if they do) ? I'm not only thinking about python
> types, but also primitive containers and types used in .Net and Java VMs,
> which aren't atomic elements either at an assembly-level point of view.

Well, they definitely have some shortcomings:

test.py:

from threading import Thread
class X(object):
    pass
obj = X()
obj.x = 0

def f(*args):
   for i in range(10000):
       obj.x += 1

threads = []
for i in range(100):
    t = Thread(target=f)
    threads.append(t)
    t.start()

for t in threads:
    while t.isAlive():
        t.join(1)

print(obj.x)

> python test.py
1000000
> pypy test.py
1000000
> jython-2.5 test.py
19217
> ipy test.py
59040

Not that this thing is reasonable to do in real code, but cpython and
other implementations with GIL at least give you some safety margin.

-- 
With best regards,
Daniel Kluev

[toc] | [next] | [standalone]

#6461

Daniel Kluev <dan.kluev@gmail.com> wrote:
> test.py:
> 
> from threading import Thread
> class X(object):
>    pass
> obj = X()
> obj.x = 0
> 
> def f(*args):
>   for i in range(10000):
>       obj.x += 1
> 
> threads = []
> for i in range(100):
>    t = Thread(target=f)
>    threads.append(t)
>    t.start()
> 
> for t in threads:
>    while t.isAlive():
>        t.join(1)
> 
> print(obj.x)
> 
>> python test.py
> 1000000
>> pypy test.py
> 1000000
>> jython-2.5 test.py
> 19217
>> ipy test.py
> 59040
> 
> Not that this thing is reasonable to do in real code, but cpython and
> other implementations with GIL at least give you some safety margin.
> 
Sure? ;)
 > for p in python2.4 python2.5 python2.6 python2.7 python3.1 python3.2; do echo $p; $p /tmp/test.py; $p /tmp/test.py; done
 python2.4
 525369
 736202
 python2.5
 449496
 551023
 python2.6
 903405
 937335
 python2.7
 885834
 910144
 python3.1
 866557
 766842
 python3.2
 1000000
 1000000

So even CPython (at least < 3.2) isn't safe. And I wouldn't rely on 3.2
not to break.

Marc

[toc] | [prev] | [next] | [standalone]

#6463

On Samstag 28 Mai 2011, Marc Christiansen wrote:
> And I wouldn't rely on 3.2
> not to break.

it breaks too like it should, but only rarely
like one out of 10 times

i5:/pub/src/gitgames/kajongg/src$ python3.2 test.py 
1000000
i5:/pub/src/gitgames/kajongg/src$ python3.2 test.py 
1000000
i5:/pub/src/gitgames/kajongg/src$ python3.2 test.py 
980000
i5:/pub/src/gitgames/kajongg/src$ python3.2 test.py 
1000000


-- 
Wolfgang

[toc] | [prev] | [next] | [standalone]

#6482

On 5/27/2011 7:06 PM, Daniel Kluev wrote:
>> So I'd like to know: how do these other implementations handle concurrency
>> matters for their primitive types, and prevent them from getting corrupted
>> in multithreaded programs (if they do) ? I'm not only thinking about python
>> types, but also primitive containers and types used in .Net and Java VMs,
>> which aren't atomic elements either at an assembly-level point of view.

   "+=" is not guaranteed to be atomic in most languages.  Some
C and C++ implementations have "atomic_inc", etc., which is guaranteed
to execute as an atomic operation.

   How do most safe languages handle concurrency? For the unboxed 
primitive types, like numbers, the hardware handles it. For
memory allocation, there's a lock.  Most don't allow patching code
on the fly.  Concurrent garbage collection prevents deleting
something if there's a pointer to it anywhere.  This was all worked
out for LISP and SELF decades ago.

   Python allows patching code while the code is executing.  This
implies a sizable performance penalty, and causes incredible
complexity in PyPy.

					John Nagle

[toc] | [prev] | [next] | [standalone]

#6483

On Sat, 28 May 2011 09:39:08 -0700, John Nagle wrote:

> Python allows patching code while the code is executing.

Can you give an example of what you mean by this?

If I have a function:


def f(a, b):
    c = a + b
    d = c*3
    return "hello world"*d


how would I patch this function while it is executing?


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#7133

En Sat, 28 May 2011 14:05:16 -0300, Steven D'Aprano  
<steve+comp.lang.python@pearwood.info> escribió:

> On Sat, 28 May 2011 09:39:08 -0700, John Nagle wrote:
>
>> Python allows patching code while the code is executing.
>
> Can you give an example of what you mean by this?
>
> If I have a function:
>
>
> def f(a, b):
>     c = a + b
>     d = c*3
>     return "hello world"*d
>
>
> how would I patch this function while it is executing?

I think John Nagle was thinking about rebinding names:


def f(self, a, b):
   while b>0:
     b = g(b)
     c = a + b
     d = self.h(c*3)
   return "hello world"*d

both g and self.h may change its meaning from one iteration to the next,  
so a complete name lookup is required at each iteration. This is very  
useful sometimes, but affects performance a lot.

-- 
Gabriel Genellina

[toc] | [prev] | [next] | [standalone]

#7141

On Tue, 07 Jun 2011 01:03:55 -0300, Gabriel Genellina wrote:

> En Sat, 28 May 2011 14:05:16 -0300, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> escribió:
> 
>> On Sat, 28 May 2011 09:39:08 -0700, John Nagle wrote:
>>
>>> Python allows patching code while the code is executing.
>>
>> Can you give an example of what you mean by this?
[...]
> I think John Nagle was thinking about rebinding names:
> 
> 
> def f(self, a, b):
>    while b>0:
>      b = g(b)
>      c = a + b
>      d = self.h(c*3)
>    return "hello world"*d
> 
> both g and self.h may change its meaning from one iteration to the next,
> so a complete name lookup is required at each iteration. This is very
> useful sometimes, but affects performance a lot.

Ah, that was what I was missing.

Thanks Gabriel.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#7149

On Jun 7, 12:03 am, "Gabriel Genellina" <gagsl-...@yahoo.com.ar>
wrote:
> En Sat, 28 May 2011 14:05:16 -0300, Steven D'Aprano  
> <steve+comp.lang.pyt...@pearwood.info> escribi :
>
>
>
>
>
>
>
>
>
> > On Sat, 28 May 2011 09:39:08 -0700, John Nagle wrote:
>
> >> Python allows patching code while the code is executing.
>
> > Can you give an example of what you mean by this?
>
> > If I have a function:
>
> > def f(a, b):
> >     c = a + b
> >     d = c*3
> >     return "hello world"*d
>
> > how would I patch this function while it is executing?
>
> I think John Nagle was thinking about rebinding names:
>
> def f(self, a, b):
>    while b>0:
>      b = g(b)
>      c = a + b
>      d = self.h(c*3)
>    return "hello world"*d
>
> both g and self.h may change its meaning from one iteration to the next,  
> so a complete name lookup is required at each iteration. This is very  
> useful sometimes, but affects performance a lot.
>

And even the original example, with only + and * can have side-
effects.  Who knows how a defines __add__?

Jean-Paul

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web