Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #17353 > unrolled thread

Make a small function thread safe

Back to article view | Back to comp.lang.python

Contents

  Make a small function thread safe Brad Tilley <kj4eit@gmail.com> - 2011-12-16 05:21 -0800
    Re: Make a small function thread safe Tim Wintle <tim.wintle@teamrubber.com> - 2011-12-16 13:33 +0000
    Re: Make a small function thread safe Tim Wintle <tim.wintle@teamrubber.com> - 2011-12-16 14:36 +0000
      Re: Make a small function thread safe Brad Tilley <kj4eit@gmail.com> - 2011-12-16 07:05 -0800
    Re: Make a small function thread safe Lie Ryan <lie.1296@gmail.com> - 2011-12-17 09:08 +1100
      Re: Make a small function thread safe John Nagle <nagle@animats.com> - 2011-12-16 18:18 -0800
        Re: Make a small function thread safe RangerElf <gustavo.cordova@gmail.com> - 2011-12-18 00:52 -0800
          Re: Make a small function thread safe Ian Kelly <ian.g.kelly@gmail.com> - 2011-12-18 23:57 -0700
    Re: Make a small function thread safe ting@thsu.org - 2011-12-19 15:56 -0800

#17353 — Make a small function thread safe

Hey guys,

I have a C++ function that I'd like to replicate (as closely as
possible) in Python. Here's an example:

107         void increment_counter( unsigned int& counter )
108         {
109                 boost::mutex::scoped_lock lock( counter_lock );
110                 ++counter;
111         }

A thread locks the function on entrance and then releases it on exit.
What is the equivalent way to do this in Python?

Many thanks!

Brad

[toc] | [next] | [standalone]

#17354

On Fri, 2011-12-16 at 05:21 -0800, Brad Tilley wrote:
> 107         void increment_counter( unsigned int& counter )
> 108         {
> 109                 boost::mutex::scoped_lock lock( counter_lock );
> 110                 ++counter;
> 111         }


with counter_lock:
    counter += 1


... where counter_lock is a threading.Lock instance.

(see docs for the threading module)

Tim

[toc] | [prev] | [next] | [standalone]

#17357

On Fri, 2011-12-16 at 09:24 -0500, Brad Tilley wrote:
> So something like this then:
> 
> import threading
> 
> shared_container = []
> lock = threading.Lock()
> 
> class thread_example( threading.Thread ):
> 
>     def __init__( self ):
>         threading.Thread.__init__ (self)
> 
>     def run(t):
>         lock
>         shared_container.append(t.name)

should be:
      def run(t):
          with lock:
              shared_container.append(t.name)

(or lock.acquire() and lock.release() as you mentioned)

> # main
> 
> threads = []
> for i in xrange(10):
>     thread = thread_example()
>     threads.append(thread)
>     
> for thread in threads:
>     thread.start()

you'll either need to lock again here, or join each thread:

for thread in threads:
      thread.join()

> for item in shared_container:
>     print item 

Tim

[toc] | [prev] | [next] | [standalone]

#17359

On Dec 16, 9:36 am, Tim Wintle <tim.win...@teamrubber.com> wrote:

> should be:
>       def run(t):
>           with lock:
>               shared_container.append(t.name)
>
> (or lock.acquire() and lock.release() as you mentioned)


Thanks Tim. The with statement is closer to the C++ code (IMO) more so
than the explicit acquire() and release() so I'll use that approach. I
appreciate your advice.

Brad

[toc] | [prev] | [next] | [standalone]

#17389

On 12/17/2011 01:30 AM, Brad Tilley wrote:
> Or perhaps run should look like this instead:
>
>      def run(t):
>          lock.acquire()
>          shared_container.append(t.name <http://t.name>)
>          lock.release()
>
> That seems a bit barbaric to me, not sure.

change that to:

def run(t):
     with lock:
         shared_container.append(t.name <http://t.name>)


the `with-statement` will call lock.acquire() and lock.release().

[toc] | [prev] | [next] | [standalone]

#17400

On 12/16/2011 2:08 PM, Lie Ryan wrote:
> On 12/17/2011 01:30 AM, Brad Tilley wrote:
>> Or perhaps run should look like this instead:
>>
>> def run(t):
>> lock.acquire()
>> shared_container.append(t.name <http://t.name>)
>> lock.release()
>>
>> That seems a bit barbaric to me, not sure.
>
> change that to:
>
> def run(t):
> with lock:
> shared_container.append(t.name <http://t.name>)
>
>
> the `with-statement` will call lock.acquire() and lock.release().
>

    And, importantly, if you leave the "with" via an exception, the
lock will be unlocked.

					John Nagle

[toc] | [prev] | [next] | [standalone]

#17444

Which is why the original .acquire() ... .release() idiom was wrong, this would better express the intent:

try:
  lock.acquire()
  shared_container.append(...)
finally:
  lock.release()

But like everyone mentions, the with statement takes care of all that in a more readable and compact fashion.

[toc] | [prev] | [next] | [standalone]

#17501

On Sun, Dec 18, 2011 at 6:27 PM, Tim Delaney
<timothy.c.delaney@gmail.com> wrote:
> On 18 December 2011 19:52, RangerElf <gustavo.cordova@gmail.com> wrote:
>>
>> Which is why the original .acquire() ... .release() idiom was wrong, this
>> would better express the intent:
>>
>> try:
>>  lock.acquire()
>>  shared_container.append(...)
>> finally:
>>  lock.release()
>
>
> No - this is very bad. The lock must be acquired outside the try: -
> otherwise if an exception is thrown while acquiring, you will try to release
> a lock that you have not acquired.
>
> Which again is why using with is a much better option - you can't make this
> kind of mistake.

Well, not unless you make the same mistake in the context manager itself.

from contextlib import contextmanager

@contextmanager
def bad_context_manager(lock):
    try:
        lock.acquire()
        yield
    finally:
        lock.release()

class Lock(object):
    def __init__(self):
        self.is_locked = False
    def acquire(self):
        assert False
    def release(self):
        assert self.is_locked, "Tried to release lock without acquiring it"

with bad_context_manager(Lock()):
    pass

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "<stdin>", line 7, in bad_context_manager
  File "<stdin>", line 7, in release
AssertionError: Tried to release lock without acquiring it


Perhaps a (small) reason to avoid the contextmanager decorator and
implement __enter__ and __exit__ instead.

[toc] | [prev] | [next] | [standalone]

#17530

On Dec 16, 8:21 am, Brad Tilley <kj4...@gmail.com> wrote:
> A thread locks the function on entrance and then releases it on exit.
> What is the equivalent way to do this in Python?

I'm not sure if this applies in your case, but much of the time, you
can use thread local storage, rather thread locking, in order to make
your code thread safe. You tend to run into far less bottleneck and
race condition issues by using thread local storage rather than thread
locking, whenever possible.
--
// T.Hsu

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web