Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!selfless.tophat.at!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
To: python-list@python.org
From: Peter Otten <__peter__@web.de>
Subject: Re: GIL in alternative implementations
Date: Sat, 28 May 2011 13:09:44 +0200
Organization: None
References: <4DE015EA.4040600@gmail.com> <BANLkTikbXA1S2Yw_DkNL-xPeSTVTXAKMFw@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 7Bit
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.2188.1306581000.9059.python-list@python.org>
Lines: 78
NNTP-Posting-Host: 82.94.164.166
Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:6462

Daniel Kluev wrote:

>> So I'd like to know: how do these other implementations handle
>> concurrency matters for their primitive types, and prevent them from
>> getting corrupted in multithreaded programs (if they do) ? I'm not only
>> thinking about python types, but also primitive containers and types used
>> in .Net and Java VMs, which aren't atomic elements either at an
>> assembly-level point of view.
> 
> Well, they definitely have some shortcomings:
> 
> test.py:
> 
> from threading import Thread
> class X(object):
>     pass
> obj = X()
> obj.x = 0
> 
> def f(*args):
>    for i in range(10000):
>        obj.x += 1
> 
> threads = []
> for i in range(100):
>     t = Thread(target=f)
>     threads.append(t)
>     t.start()
> 
> for t in threads:
>     while t.isAlive():
>         t.join(1)
> 
> print(obj.x)
> 
>> python test.py
> 1000000
>> pypy test.py
> 1000000
>> jython-2.5 test.py
> 19217
>> ipy test.py
> 59040
> 
> Not that this thing is reasonable to do in real code, but cpython and
> other implementations with GIL at least give you some safety margin.

The problem with your code is that it gives the wrong result when a thread 
reads obj.x, then suspends, then another thread reads obj.x and finally both 
threads store the same value+1 back. That problem has nothing to do with the 
GIL which just prohibits that two threads can run at the same time, on 
different cores. 

It occurs because obj.x += 1 is not an atomic operation. That lack of 
atomicity should be obvious if you take a look at the byte-code:

>>> def f():
...     obj.x += 1
...
>>> dis.dis(f)
  2           0 LOAD_GLOBAL              0 (obj)
              3 DUP_TOP
              4 LOAD_ATTR                1 (x)
              7 LOAD_CONST               1 (1)
             10 INPLACE_ADD
             11 ROT_TWO
             12 STORE_ATTR               1 (x)
             15 LOAD_CONST               0 (None)
             18 RETURN_VALUE

[Marc Christiansen]
> So even CPython (at least < 3.2) isn't safe. And I wouldn't rely on 3.2
> not to break.

I don't know why it /seems/ to work in 3.2 more often than in 2.x, but in 
general bugs that occur only sporadically are typical for multithreaded 
code...