Path: csiph.com!usenet.pasdenom.info!news.albasani.net!news2.arglkargh.de!feeder.erje.net!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.001 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'intermediate': 0.05; 'iterate': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'itself.': 0.11; 'thread': 0.11; 'sections': 0.13; '(the': 0.15; 'appends': 0.16; 'deque': 0.16; 'iterating': 0.16; 'iteration': 0.16; 'iteration.': 0.16; 'operation:': 0.16; 'received:80.91.229.3': 0.16; 'received:dip0.t-ipconnect.de': 0.16; 'received:plane.gmane.org': 0.16; 'received:t-ipconnect.de': 0.16; 'striving': 0.16; 'memory': 0.18; '(by': 0.22; 'seems': 0.23; 'non': 0.24; 'header:User-Agent:1': 0.26; '(which': 0.26; 'extend': 0.26; 'right.': 0.27; 'header:X-Complaints-To:1': 0.28; 'appending': 0.29; 'exposed': 0.29; 'locks': 0.29; 'thus,': 0.29; 'writes:': 0.29; 'case,': 0.29; 'point': 0.31; 'to:addr:python- list': 0.33; 'operations': 0.33; 'involving': 0.35; 'something': 0.35; 'there': 0.35; 'received:org': 0.36; 'but': 0.36; 'beyond': 0.37; 'subject:: ': 0.38; 'performance': 0.39; 'to:addr:python.org': 0.39; 'short': 0.39; 'subject:-': 0.40; 'header:Received:5': 0.40; 'think': 0.40; 'side': 0.61; '8bit%:20': 0.62; 'duration': 0.62; 'safety': 0.65; 'skip:\xe2 10': 0.66; 'received:217': 0.68; 'protect': 0.69 X-Injected-Via-Gmane: http://gmane.org/ To: python-list@python.org From: Dieter Maurer Subject: Re: deque and thread-safety Date: Sat, 13 Oct 2012 08:35:19 +0200 References: Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Gmane-NNTP-Posting-Host: pd9e0878a.dip0.t-ipconnect.de User-Agent: Gnus/5.1008 (Gnus v5.10.8) XEmacs/21.4.22 (linux) Cancel-Lock: sha1:KREeww5g2xD/oVOnNQh0WaHi65g= X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 28 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1350110132 news.xs4all.nl 6983 [2001:888:2000:d::a6]:35676 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:31186 Christophe Vandeplas writes: > ... > From the documentation I understand that deques are thread-safe: >> Deques are a generalization of stacks and queues (the name is pronounced “deck” >> and is short for “double-ended queue”). Deques support thread-safe, memory >> efficient appends and pops from either side of the deque with approximately the >> same O(1) performance in either direction. > > It seems that appending to deques is indeed thread-safe, but not > iterating over them. You are right. And when you think about it, then there is not much point in striving for thread safety for iteration (alone). Iteration is (by nature) a non atomic operation: you iterate because you want to do something with the intermediate results; this "doing" is not part of the iteration itself. Thus, you are looking for thread safety not for only the iteration but for the iteration combined with additional operations (which may well extend beyond the duration of the iteration). Almost surely, the "deque" implementation is using locks to ensure thread safety for its "append" and "pop". Check whether this lock is exposed to the application. In this case, use it to protect you atomic sections involving iteration.