Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #19624 > unrolled thread

Killing threads, and os.system()

Back to article view | Back to comp.lang.python

Contents

  Killing threads, and os.system() Laurent Claessens <moky.math@gmail.com> - 2012-01-31 08:45 +0100
    Re: Killing threads, and os.system() Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2012-01-31 11:04 -0500
      Re: Killing threads, and os.system() Laurent Claessens <moky.math@gmail.com> - 2012-01-31 18:18 +0100
        Re: Killing threads, and os.system() Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2012-01-31 14:34 -0500
      Re: Killing threads, and os.system() Laurent Claessens <moky.math@gmail.com> - 2012-01-31 18:18 +0100
      Re: Killing threads, and os.system() John Nagle <nagle@animats.com> - 2012-02-03 00:14 -0800
        Re: Killing threads, and os.system() Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-02-03 09:25 +0000
        Re: Killing threads, and os.system() Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2012-02-03 11:20 -0500
        Re: Killing threads, and os.system() Paul Rubin <no.email@nospam.invalid> - 2012-02-03 15:42 -0800

#19624 — Killing threads, and os.system()

     Hello all


   I've a program that launches a lot of  threads and each of them 
launches a os.system("my_command").

My program also keeps a list of the launched threads, so I can make 
"for" loops on the threads.


My aim is to kill everything with ctrl-C (KeyboardInterrupt).

Of course I tried to do

try:
    [...]
except KeyboardInterrupt :
    for task in task_list :
       task.stop()
       #task_list is the list of threads to be killed


It does not work.

How can I produce an "emergency" stop of all processes, including the 
externals programs that were called by os.system() ?

My aim is of course to write an ultimate log file containing the status 
of the program when KeyboardInterupt was raised. (if not, the unix 
command "kill" does the job ;) )


Thanks for any help
have a good day
Laurent

[toc] | [next] | [standalone]

#19644

On Tue, 31 Jan 2012 08:45:53 +0100, Laurent Claessens
<moky.math@gmail.com> wrote:

>
>My aim is to kill everything with ctrl-C (KeyboardInterrupt).
>
>Of course I tried to do
>
>try:
>    [...]
>except KeyboardInterrupt :
>    for task in task_list :
>       task.stop()
>       #task_list is the list of threads to be killed
>
	When did .stop() become a method of threads? Or is it a method
created by you which is supposed to set some local data in the thread to
signal is should exit?

>
>It does not work.
>
	For years, the recommended coding style requires the thread to wait
on an event/condition or poll a local/global flag value; the thread is
then responsible for cleanly exiting /itself/.

	Of course, if that thread is stuck waiting for a call to os.system()
to complete, then it can not do anything...

	os.system()	is a rather limited, restrictive, call -- best used for
quick one-of operations. If running Python 2.6+, I'd recommend
converting from os.system() to subprocess.Popen(). .Popen() objects now
have .terminate() and .kill() methods.

	Heck, if the only reason you have been using threads is to permit
multiple parallel os.system() calls, you can probably drop the threads
entirely, and use a collection of subprocess.Popen() objects directly.
You will/should code something to handle stdin/stdout/stderr of those
objects -- unfortunately lowest-common-denominator IPC system does not
make it easy to do interactive operations ({muse: who do we have to kill
to persuade OS designers to incorporate something like the Amiga ARexx
"rexxport" system <G>}).

>How can I produce an "emergency" stop of all processes, including the 
>externals programs that were called by os.system() ?

	As hinted above: Python 2.6 or newer, subprocess.Popen(), using
.terminate(), .kill(), or maybe .send_signal()
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
        wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]

#19649

Le 31/01/2012 17:04, Dennis Lee Bieber a écrit :

> 	Of course, if that thread is stuck waiting for a call to os.system()
> to complete, then it can not do anything...
>
> 	os.system()	is a rather limited, restrictive, call -- best used for
> quick one-of operations. If running Python 2.6+, I'd recommend
> converting from os.system() to subprocess.Popen(). .Popen() objects now
> have .terminate() and .kill() methods.

Ok, I'll try that Popen. Indeed I think that my threads are stuck 
waiting os.system to complete.

Thanks
Laurent

[toc] | [prev] | [next] | [standalone]

#19655

On Tue, 31 Jan 2012 18:18:28 +0100, Laurent Claessens
<moky.math@gmail.com> wrote:

>
>Ok, I'll try that Popen. Indeed I think that my threads are stuck 
>waiting os.system to complete.
>
	Since the return value of os.system() is the /completion status/ of
the spawned command, that would seem to be a given.
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
        wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]

#19650

Le 31/01/2012 17:04, Dennis Lee Bieber a écrit :

> 	Of course, if that thread is stuck waiting for a call to os.system()
> to complete, then it can not do anything...
>
> 	os.system()	is a rather limited, restrictive, call -- best used for
> quick one-of operations. If running Python 2.6+, I'd recommend
> converting from os.system() to subprocess.Popen(). .Popen() objects now
> have .terminate() and .kill() methods.

Ok, I'll try that Popen. Indeed I think that my threads are stuck 
waiting os.system to complete.

Thanks
Laurent

[toc] | [prev] | [next] | [standalone]

#19828

On 1/31/2012 8:04 AM, Dennis Lee Bieber wrote:
> ({muse: who do we have to kill
> to persuade OS designers to incorporate something like the Amiga ARexx
> "rexxport" system<G>}).

    QNX, which is a real-time microkernel which looks like POSIX to
applications.  actually got interprocess communication right.  It
has to; everything in QNX is done by interprocess communication,
including all I/O.  File systems and drivers are ordinary programs.
The kernel just handles message passing, CPU dispatching, and timers.
QNX's message passing looks more like a subroutine call than an
I/O operation, and this has important implications for efficient CPU 
dispatching.

    Any QNX system call that can block is really a message pass.  Message
passes can be given a timeout, and they can be canceled from another
thread.  The "system call" then returns with an error status.  This
provides a way to keep threads from getting "stuck" in a system call.

(Unfortunately, QNX, which survived as a separate company for decades,
sold out to Harmon (car audio) a few years ago. They had no clue
what to do with an OS.  They sold it to Research In Motion, the
Blackberry company, which is in the process of tanking.)

     Python's thread model is unusually dumb.  You can't send signals
to other threads, you can't force an exception in another thread, and
I won't even get into the appalling mess around the Global Interpreter
Lock.  This has forced the use of subprocesses where, in other 
languages, you'd use threads.  Of course, you load a new copy of the
interpreter in each thread, so this bloats memory usage.

				John Nagle

[toc] | [prev] | [next] | [standalone]

#19830

On Fri, 03 Feb 2012 00:14:33 -0800, John Nagle wrote:

> I won't even get into the appalling mess around the Global Interpreter
> Lock.

You know full well that IronPython and Jython don't have a GIL. If the 
GIL was as harmful as you repeatedly tell us, why haven't you, and 
everyone else, migrated to IronPython and Jython?

Oh yeah, maybe it's because CPython, even with the GIL, is significantly 
faster than the two implementations without a GIL. Go figure.

But never mind the facts, spreading the FUD about Python is much more 
fun. Hey, I hear that Python 3 is tanking too, and millions of Python 
developers are rushing, rushing I say, to port their code to Go. Or was 
it Ruby? Possibly Lua. Maybe VBScript.


> This has forced the use of subprocesses where, in other 
> languages, you'd use threads.  

Only if by "forced" you mean "not forced at all". 

http://docs.python.org/library/multiprocessing.html
http://www.stackless.com/
http://pypi.python.org/pypi/greenlet/
http://twistedmatrix.com/trac/
http://www.gevent.org/
http://code.google.com/p/cogen/
http://www.kamaelia.org/Home
http://code.google.com/p/syncless/
http://opensource.hyves.org/concurrence/
http://www.tornadoweb.org/
http://docs.python.org/library/asyncore.html
http://pyro.sourceforge.net/
http://wiki.python.org/moin/ParallelProcessing


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#19836

On Fri, 03 Feb 2012 00:14:33 -0800, John Nagle <nagle@animats.com>
wrote:

>On 1/31/2012 8:04 AM, Dennis Lee Bieber wrote:
>> ({muse: who do we have to kill
>> to persuade OS designers to incorporate something like the Amiga ARexx
>> "rexxport" system<G>}).
>
>    QNX, which is a real-time microkernel which looks like POSIX to
>applications.  actually got interprocess communication right.  It
>has to; everything in QNX is done by interprocess communication,
>including all I/O.  File systems and drivers are ordinary programs.
>The kernel just handles message passing, CPU dispatching, and timers.
>QNX's message passing looks more like a subroutine call than an
>I/O operation, and this has important implications for efficient CPU 
>dispatching.
>
	Sounds a lot like the Amiga kernal. Though, since message passing
was just a matter of adding/removing entries from a linked list, it
prevented easy porting to protected virtual memory spaces.

	I/O was user code passing an I/O request to a file-handler, which
then determined which device driver was to receive the next lower level
operation... Eventual these messages passed back in the other direction
with the result data.

	Rexxports were just regular Amiga message ports with some ARexx
overhead... From an ARexx script, sending to another program that
implemented a Rexxport was as simple as

	address <targetport> command_and_data_parameters

	The library included statements for ARexx scripts to create their
own ports, and to receive packets via them.

	Unfortunately, the limitations of most OS's have led to
implementations like the Regina Rexx -- where the ADDRESS command is
basically a synonym for Python's subprocess.Popen() and the "targetport"
controls whether the command is executed directly or if a shell
processor is invoked to handle it.
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
        wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]

#19848

John Nagle <nagle@animats.com> writes:
> QNX's message passing looks more like a subroutine call than an
> I/O operation, 

How do they enforce process isolation, or do they decide they don't need to?

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web