Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!selfless.tophat.at!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.012 X-Spam-Evidence: '*H*': 0.98; '*S*': 0.00; 'raises': 0.07; 'adjusting': 0.09; 'am,': 0.14; 'received:209.85.214.174': 0.14; 'received:mail-iw0-f174.google.com': 0.14; 'wrote:': 0.14; 'arbitrary.': 0.16; 'disk.': 0.16; 'from:addr:rosuav': 0.16; 'from:name:chris angelico': 0.16; 'set,': 0.16; 'stop.': 0.16; 'subject:stop': 0.16; 'mon,': 0.17; 'header:In-Reply-To:1': 0.21; 'figure': 0.21; '(b)': 0.23; 'subject: : ': 0.26; 'fixed': 0.27; 'message-id:@mail.gmail.com': 0.28; '(the': 0.28; 'problem': 0.28; 'received:209.85.214': 0.28; 'disk': 0.29; 'fact': 0.30; 'terminate': 0.30; 'threads': 0.30; 'worker': 0.30; 'does': 0.33; 'to:addr:python-list': 0.33; 'copying': 0.33; 'starting': 0.33; 'error': 0.33; 'file': 0.34; 'fail': 0.34; '(a)': 0.35; 'flag': 0.35; 'probably': 0.36; 'too.': 0.37; 'received:google.com': 0.37; 'received:209.85': 0.37; '20,': 0.37; 'thread': 0.37; 'subject:: ': 0.38; "i'd": 0.39; 'received:209': 0.39; 'empty': 0.39; 'to:addr:python.org': 0.39; 'current': 0.40; 'stop': 0.62; 'huge': 0.62; 'fighting': 0.67; 'subject:make': 0.73; 'optimum': 0.91 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=4j+HU5sDmYK3isrM9Bp9arWNMEhsEKNjffGqgthmC+o=; b=gu4iWrBHHHbysGMsSSLSAazzN9kWpLHeYbBc1gxX8+LuhYWtrbb9/8OMfJU8wxIDFD C2Q51Bn6GhOmb3XpOkDjCWf41MIIaEOlsCi0+d4oqRhShKvq/UjoW2vFomasHXrhEVoD +OFl5PQZQ7TdOYHy8/biv/rUJBBwVe0OqYmTY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=so1I9uzaKZswtG9NfWd7F7cWBVJDqLXIFasc7S21CUYcUKaSmcofs4zzZ381QrSoXw aePTWmYm2BWcdt2NxSu8k/AYONVcqW3BpJo0l8ooyCSK9a3ftr/sqaqQ6EHtYxTl0BRj Ry1N/ZY4S+0rhyr6DUywQIhQmA9m4LG2HoA/U= MIME-Version: 1.0 In-Reply-To: References: Date: Mon, 20 Jun 2011 02:03:11 +1000 Subject: Re: threading : make stop the caller From: Chris Angelico To: python-list@python.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 19 NNTP-Posting-Host: 82.94.164.166 X-Trace: 1308499395 news.xs4all.nl 49178 [::ffff:82.94.164.166]:49526 X-Complaints-To: abuse@xs4all.nl Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:7977 On Mon, Jun 20, 2011 at 1:39 AM, Laurent Claessens wrote: > My problem is that when FileToCopyTask raises an error, the program does not > stop. > In fact when the error is Disk Full, I want to stop the whole program > because I know that the next task will fail too. If you're starting a thread for every file you're copying, you're starting a huge number of threads that probably will just end up fighting over the disk. To get a reasonably efficient early-abort, I'd recommend having a fixed pool of worker threads (say, ten of them), and have each thread (a) check if the early-abort flag is set, and then (b) start copying the next file in queue. Once the queue's empty or the early-abort flag is set, all ten threads will terminate when they finish their current transfers. (The ten threads figure is arbitrary. Optimum value for performance will come by adjusting this.) ChrisA