Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.xs4all.nl!newsfeed3.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.000 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'api.': 0.05; 'debugging': 0.07; 'only,': 0.07; 'stops': 0.07; 'arguments': 0.09; 'assigning': 0.09; 'except:': 0.09; 'msg': 0.09; 'propagate': 0.09; 'try:': 0.09; 'python': 0.11; 'def': 0.12; 'thread': 0.14; '"with"': 0.16; '(either': 0.16; 'anyways,': 0.16; 'disconnect': 0.16; 'doing,': 0.16; 'exit.': 0.16; 'logger': 0.16; 'main():': 0.16; 'merely': 0.16; 'msg)': 0.16; 'msg):': 0.16; 'mutated': 0.16; 'raised,': 0.16; 'range(0,': 0.16; 'reraise': 0.16; 'statement.': 0.16; 'stdout': 0.16; 'subject:Thread': 0.16; 'tables,': 0.16; 'thread,': 0.16; 'threads,': 0.16; 'exception': 0.16; 'wrote:': 0.18; 'variable': 0.18; 'wed,': 0.18; 'module': 0.19; 'commit': 0.19; 'aug': 0.22; 'email addr:gmail.com>': 0.22; 'to:name:python-list@python.org': 0.22; 'creating': 0.23; 'error': 0.23; '(such': 0.24; 'server.': 0.24; '(see': 0.26; 'handling': 0.26; 'possibly': 0.26; 'tables': 0.26; 'least': 0.26; 'header:In-Reply-To:1': 0.27; 'function': 0.29; 'correct': 0.29; 'am,': 0.29; 'generally': 0.29; 'raise': 0.29; 'thus': 0.29; "doesn't": 0.30; 'operations,': 0.30; 'statement': 0.30; 'especially': 0.30; 'message-id:@mail.gmail.com': 0.30; 'skip:( 20': 0.30; 'code': 0.31; '3.2': 0.31; 'block,': 0.31; 'exceptions': 0.31; 'overhead': 0.31; 'raised': 0.31; 'terminate': 0.31; 'allows': 0.31; 'class': 0.32; 'probably': 0.32; 'run': 0.32; 'another': 0.32; 'open': 0.33; 'running': 0.33; 'skip:_ 10': 0.34; 'received:74.125.82': 0.34; 'except': 0.35; 'skip:s 30': 0.35; 'something': 0.35; 'case,': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'really': 0.36; 'complete.': 0.36; 'object,': 0.36; 'sequence': 0.36; 'doing': 0.36; 'possible': 0.36; 'too': 0.37; 'performance': 0.37; 'starting': 0.37; 'being': 0.38; 'skip:& 10': 0.38; 'problems': 0.38; 'handle': 0.38; 'to:addr:python-list': 0.38; 'fact': 0.38; 'skip:& 20': 0.39; 'received:74.125': 0.39; 'does': 0.39; 'sure': 0.39; 'to:addr:python.org': 0.39; 'either': 0.39; 'even': 0.60; 'skip:u 10': 0.60; 'skip:t 30': 0.61; 'entire': 0.61; 'first': 0.61; 'connecting': 0.64; 'provide': 0.64; 'due': 0.66; 'sample': 0.67; 'close': 0.67; 'connection.': 0.74; '\xc2\xa0\xc2\xa0': 0.74; 'bare': 0.84; 'itself?': 0.84; 'off,': 0.84; 'received:74.125.82.41': 0.84; 'capture': 0.91; 'connection,': 0.95; 'directly.': 0.95 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type; bh=0vIh3C+AGj3DiJWaAwSDlgxYuGtl9qbkb6e1KA6P4UI=; b=j9U56OVoRLJy1S6ifx/ZSiF0hWkkMqPWowLQuJGtKpvsTpIEW/wF0i0kX6xMnmseSw mXkJLZv6bcgRFSOzJ0/29K+z0vu5X57wSWwrP32aj7o026rkTIPt276/DbrYqeRhJM/i m25yq8f+8wjiWll9nj04hrE11oPB7f7uInIjzzIElEU2x4c6mDgsSeYIA4l3ViQ+7bso M3guwrGkziGxHzobD4Jzpn0Qoeu2hvXOPJ773OgNgv/UCmm3OOBgm/FhZM5yfmNIDbXA SwmHTO0sDvdVw7IQnNoya41tBDKxWUDmO0L5LMik9zUED0DyChWUsktpQu60zWYVVRxd 14gg== X-Gm-Message-State: ALoCoQkmxHisWvqfPTU9A0u6j2451ASTdHZOuVTYVmE/WBQHytgokj9g6bCzPxukhH4ghLzvt8/S X-Received: by 10.194.191.165 with SMTP id gz5mr39492031wjc.16.1409167628343; Wed, 27 Aug 2014 12:27:08 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <20140827185514.GA18660@arxnet.hu> References: <20140827185514.GA18660@arxnet.hu> From: Chris Kaynor Date: Wed, 27 Aug 2014 12:26:48 -0700 Subject: Re: Thread terminate To: "python-list@python.org" Content-Type: multipart/alternative; boundary=047d7b873878eab4f70501a16721 X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 293 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1409169517 news.xs4all.nl 2895 [2001:888:2000:d::a6]:60393 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:77159 --047d7b873878eab4f70501a16721 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Wed, Aug 27, 2014 at 11:55 AM, Ervin Heged=C3=BCs wr= ote: > what's the correct way to terminate a thread by itself? > To terminate the thread, the run function must exit. This can be either from an exception or a return statement. > > I mean: > > class MyThread(threading.Thread): > def __init__(self, queueitem): > threading.Thread.__init__(self) > ... > > def run(self): > """pseudo code below""" > > try: > self.connect_to_database() > except: > First off, especially if you are going to raise an exception anyways, I would recommend not silencing the orignal error - it will make debugging harder. This is especially true with a bare "except:" statement - at least capture only some exceptions. Using a logger to handle the exception or use your except, log the error, than reraise with a bare "raise" statement would generally be preferable. self.terminate("Error when connect to db") > > try: > self.create_db_table() > except: > self.terminate("Error when create table") > You probably want to make sure to disconnect from the database in here. Possibly via a "with" statement or a "try...finally" over the entire block, depending on your database API. .... > def terminate(self, msg): > self.error =3D True > self.errormsg =3D msg > syslog.syslog(syslog.LOG_DEBUG, msg) > raise Exception(msg) > > > in main(): > > for q in range(0, 10): > mt =3D MyThread() > try: > mt.run() > This is not actually starting a thread, but is running it in-line, just like a normal function. You probably want "mt.start()" instead, which will cause "mt.run" to execute in another thread. If you use mt.start() to actually start a thread, the only time it will fail is if the thread fails to start (such as too many running threads, or other out-of-resource errors). The exceptions raised inside the thread will not propagate outside the start call, and thus the mt.join() call inside the except statement will have no effect - no thread is running. except: > mt.join() > but it doesn't works - the exception showed on stdout only, and > the thread stops of course, but I would like to handle the > exceptions. If I call the join() inside from thread object, then > an exception raised, cause the thread object of Threading module > doesn't allows that... > Not being able to call join from the thread you are joining with is probably a good thing - doing so would result in a dead lock: the thread is waiting for the thread to exit. In your case, you may want to just handle the exceptions inside the thread's run() function directly. If that is not possible and you really need to handle them inside the main thread, you would need to store off the error data in a variable (either passed into the thread as the "args" or "kwargs" arguments to the MyThread() call, or global or class variables) then use mt.join() to wait for the thread(s) to exit. In the case of handling the problems in the main thread, your main thread code would look something like: # Start all of the threads up-front. threads =3D [] for q in range(0, 10): mt =3D MyThread() mt.start() # NOT mt.run() - that does not do any threading! threads.append(mt) # Now wait for them to complete. for thread in threads: thread.join() # Check for the status of thread here. Possibly on the thread object, which can be mutated in MyThread.run by assigning to members of self. In the sample code you are doing, I am not even sure if I would use threads due to the added complexities associated with them. In the case of merely connecting to a database and creating a few tables, you can probably get by with doing them in a single thread: open a single connection, creating the tables as a sequence of operations, commit operations, and close connection= . If you do in fact need threads, you may also be better off using a thread pool (see concurrent.futures, available in Python 3.2 and higher only). This can reduce the overhead associated with threads and provide better performance - both locally and, with database connections, on the server. --047d7b873878eab4f70501a16721 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
On W= ed, Aug 27, 2014 at 11:55 AM, Ervin Heged=C3=BCs <airween@gmail.com>= ; wrote:
what's the correct wa= y to terminate a thread by itself?

To terminate the thread, the run function = must exit. This can be either from an exception or a return statement.
=C2=A0

I mean:

class MyThread(threading.Thread):
=C2=A0 =C2=A0 def __init__(self, queueitem):
=C2=A0 =C2=A0 =C2=A0 =C2=A0 threading.Thread.__init__(self)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 ...

=C2=A0 =C2=A0 def run(self):
=C2=A0 =C2=A0 =C2=A0 =C2=A0 """pseudo code below""= "

=C2=A0 =C2=A0 =C2=A0 =C2=A0 try:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.connect_to_database()
=C2=A0 =C2=A0 =C2=A0 =C2=A0 except:
=C2=A0
F= irst off, especially if you are going to raise an exception anyways, I woul= d recommend not silencing the orignal error - it will make debugging harder= . This is especially true with a bare "except:" statement - at le= ast capture only some exceptions. Using a logger to handle the exception or= use your except, log the error, than reraise with a bare "raise"= statement would generally be preferable.

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.terminate("Error when co= nnect to db")

=C2=A0 =C2=A0 =C2=A0 =C2=A0 try:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.create_db_table()
=C2=A0 =C2=A0 =C2=A0 =C2=A0 except:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.terminate("Error when cr= eate table")

You probably want to = make sure to disconnect from the database in here. Possibly via a "wit= h" statement or a "try...finally" over the entire block, dep= ending on your database API.

=C2=A0 =C2=A0 ....
=C2=A0 =C2=A0 def terminate(self, msg):
=C2=A0 =C2=A0 =C2=A0 =C2=A0 self.error =3D True
=C2=A0 =C2=A0 =C2=A0 =C2=A0 self.errormsg =3D msg
=C2=A0 =C2=A0 =C2=A0 =C2=A0 syslog.syslog(syslog.LOG_DEBUG, msg)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 raise Exception(msg)


in main():

for q in range(0, 10):
=C2=A0 =C2=A0 mt =3D MyThread()
=C2=A0 =C2=A0 try:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 mt.run()
=C2=A0
= This is not actually starting a thread, but is running it in-line, just lik= e a normal function. You probably want "mt.start()" instead, whic= h will cause "mt.run" to execute in another thread.

If you use mt.start() to actually start a thread, the o= nly time it will fail is if the thread fails to start (such as too many run= ning threads, or other out-of-resource errors). The exceptions raised insid= e the thread will not propagate outside the start call, and thus the mt.joi= n() call inside the except statement will have no effect - no thread is run= ning.

=C2=A0 =C2=A0 except:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 mt.join() =C2=A0=C2=A0

but it doesn't works - the exception showed on stdout only, and
the thread stops of course, but I would like to handle the
exceptions. If I call the join() inside from thread object, then
an exception raised, cause the thread object of Threading module
doesn't allows that...

Not being ab= le to call join from the thread you are joining with is probably a good thi= ng - doing so would result in a dead lock: the thread is waiting for the th= read to exit.


In your case, you may want to just handl= e the exceptions inside the thread's run() function directly. If that i= s not possible and you really need to handle them inside the main thread, y= ou would need to store off the error data in a variable (either passed into= the thread as the "args" or "kwargs" arguments to the = MyThread() call, or global or class variables) then use mt.join() to wait f= or the thread(s) to exit.

In the case of handling the problems in the main thread= , your main thread code would look something like:
# Start all of the threads up-front.
<= /div>
thr= eads =3D []
for = q in range(0, 10):
=C2=A0 =C2=A0 mt =3D MyThread()
=C2=A0 =C2=A0 mt.start() # NOT mt.run() - t= hat does not do any threading!
=
=C2=A0 =C2=A0 threads.append(mt)

# Now wait for them to complete.
for thre= ad in threads:
=C2=A0 =C2=A0 thread.join()
=C2=A0 =C2=A0 # Check for the status of t= hread here. Possibly on the thread object, which can be mutated in MyThread= .run by assigning to members of self.

In the sample code you are doing, I am not even sur= e if I would use threads due to the added complexities associated with them= . In the case of merely connecting to a database and creating a few tables,= you can probably get by with doing them in a single thread: open a single = connection, creating the tables as a sequence of operations, commit operati= ons, and close connection.

If you do in fact need threads, you may also be better = off using a thread pool (see=C2=A0concurrent.futures, available in Python 3= .2 and higher only). This can reduce the overhead associated with threads a= nd provide better performance - both locally and, with database connections= , on the server.
--047d7b873878eab4f70501a16721--