Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #98644 > unrolled thread

cross platform alternative for signal.SIGALRM?

Back to article view | Back to comp.lang.python

Contents

  cross platform alternative for signal.SIGALRM? Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2015-11-11 16:16 +0000
    Re: cross platform alternative for signal.SIGALRM? Marko Rauhamaa <marko@pacujo.net> - 2015-11-11 18:30 +0200
      Re: cross platform alternative for signal.SIGALRM? Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2015-11-11 17:06 +0000
        Re: cross platform alternative for signal.SIGALRM? Marko Rauhamaa <marko@pacujo.net> - 2015-11-11 20:03 +0200
          Re: cross platform alternative for signal.SIGALRM? Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2015-11-11 22:42 +0000
        Re: cross platform alternative for signal.SIGALRM? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2015-11-11 19:42 -0500
    Re: cross platform alternative for signal.SIGALRM? Terry Reedy <tjreedy@udel.edu> - 2015-11-11 20:37 -0500
      Re: cross platform alternative for signal.SIGALRM? Marko Rauhamaa <marko@pacujo.net> - 2015-11-12 08:14 +0200
        Re: cross platform alternative for signal.SIGALRM? Christian Gollwitzer <auriocus@gmx.de> - 2015-11-12 07:43 +0100
          Re: cross platform alternative for signal.SIGALRM? Chris Angelico <rosuav@gmail.com> - 2015-11-12 18:37 +1100
          Re: cross platform alternative for signal.SIGALRM? Terry Reedy <tjreedy@udel.edu> - 2015-11-12 05:15 -0500
          Re: cross platform alternative for signal.SIGALRM? Chris Angelico <rosuav@gmail.com> - 2015-11-12 22:38 +1100
          Re: cross platform alternative for signal.SIGALRM? Terry Reedy <tjreedy@udel.edu> - 2015-11-12 09:01 -0500
        Re: cross platform alternative for signal.SIGALRM? Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2015-11-12 07:22 +0000
          Re: cross platform alternative for signal.SIGALRM? Marko Rauhamaa <marko@pacujo.net> - 2015-11-12 10:15 +0200
      Re: cross platform alternative for signal.SIGALRM? Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2015-11-12 06:58 +0000
    Re: cross platform alternative for signal.SIGALRM? Cameron Simpson <cs@zip.com.au> - 2015-11-12 16:20 +1100

#98644 — cross platform alternative for signal.SIGALRM?

I am rewriting a Perl program into Python (2.7).
It must run on Linux and Windows.
With Linux I have no problems, but Windows... :-(

The current show stopper is signal.SIGALRM which is not available on
Windows:

  File "fexit.py", line 674, in formdata_post
    signal.signal(signal.SIGALRM,timeout_handler)
  AttributeError: 'module' object has no attribute 'SIGALRM'


  https://docs.python.org/2/library/signal.html

  signal.alarm(time) (...) Availability: Unix.

Perl for Windows has had SIGALRM support (or some kind of emulation).

Ok, I have to redesign this part of my code:

  def timeout_handler(sig,frame): 
    raise ValueError("timeout!")
    
  signal.signal(signal.SIGALRM,timeout_handler)

  while True:
    chunk = fileo.read(bs)
    sock.sendall(chunk)
    (...)


What is the best practise for a cross platform timeout handler?




-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum IZUS/TIK         E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart         Tel:    ++49-711-68565868
Allmandring 30a                Fax:    ++49-711-682357
70550 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/

[toc] | [next] | [standalone]

#98646

Ulli Horlacher <framstag@rus.uni-stuttgart.de>:

> What is the best practise for a cross platform timeout handler?

Here's the simplest answer:

   https://docs.python.org/3/library/threading.html#threading.Timer

(Also available in Python 2.)


Marko

[toc] | [prev] | [next] | [standalone]

#98650

Marko Rauhamaa <marko@pacujo.net> wrote:
> Ulli Horlacher <framstag@rus.uni-stuttgart.de>:
> 
> > What is the best practise for a cross platform timeout handler?
> 
> Here's the simplest answer:
> 
>    https://docs.python.org/3/library/threading.html#threading.Timer
> 
> (Also available in Python 2.)

Hmmm... not so simple for me. My test code:

from time import *
import threading
import sys

def hello(): 
raise ValueError("hello!!!")

t = threading.Timer(3.0,hello)
t.start()
try:
  print "start"
  sleep(5)
  print "end"
except ValueError as e:
  print e.args[0]
  sys.exit(1)


gives:


start
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 759, in run
    self.function(*self.args, **self.kwargs)
  File "x.py", line 7, in hello
    def hello(): raise ValueError("hello!!!")
ValueError: hello!!!

end



-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum IZUS/TIK         E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart         Tel:    ++49-711-68565868
Allmandring 30a                Fax:    ++49-711-682357
70550 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/

[toc] | [prev] | [next] | [standalone]

#98657

Ulli Horlacher <framstag@rus.uni-stuttgart.de>:

> Hmmm... not so simple for me. My test code:
>
> from time import *
> import threading
> import sys
>
> def hello(): 
> raise ValueError("hello!!!")
>
> t = threading.Timer(3.0,hello)
> t.start()
> try:
>   print "start"
>   sleep(5)
>   print "end"
> except ValueError as e:
>   print e.args[0]
>   sys.exit(1)

Correct. The timer callback function (hello) would be called in a
separate thread. An exception raised in one thread cannot be caught in
the main thread. In general, there is no way for a thread to interrupt a
sibling thread that is in a blocking function call.


Marko

[toc] | [prev] | [next] | [standalone]

#98660

Marko Rauhamaa <marko@pacujo.net> wrote:

> Correct. The timer callback function (hello) would be called in a
> separate thread. An exception raised in one thread cannot be caught in
> the main thread. In general, there is no way for a thread to interrupt a
> sibling thread that is in a blocking function call.

Then threading.Timer is not a solution for my problem.

-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum IZUS/TIK         E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart         Tel:    ++49-711-68565868
Allmandring 30a                Fax:    ++49-711-682357
70550 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/

[toc] | [prev] | [next] | [standalone]

#98661

On Wed, 11 Nov 2015 17:06:46 +0000 (UTC), Ulli Horlacher
<framstag@rus.uni-stuttgart.de> declaimed the following:

>Marko Rauhamaa <marko@pacujo.net> wrote:
>> Ulli Horlacher <framstag@rus.uni-stuttgart.de>:
>> 
>> > What is the best practise for a cross platform timeout handler?
>> 
>> Here's the simplest answer:
>> 
>>    https://docs.python.org/3/library/threading.html#threading.Timer
>> 
>> (Also available in Python 2.)
>
>Hmmm... not so simple for me. My test code:
>
>from time import *
>import threading
>import sys
>
>def hello(): 
>raise ValueError("hello!!!")
>
	Indentation!

	If that is a cut&paste, you have defined a "hello" function that does
nothing, followed by immediately raising ValueError (so nothing below will
be executed)

>t = threading.Timer(3.0,hello)
>t.start()
>try:
>  print "start"
>  sleep(5)
>  print "end"
>except ValueError as e:
>  print e.args[0]
>  sys.exit(1)
>
>
>gives:
>
>
>start
>Exception in thread Thread-1:
>Traceback (most recent call last):
>  File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
>    self.run()
>  File "/usr/lib/python2.7/threading.py", line 759, in run
>    self.function(*self.args, **self.kwargs)
>  File "x.py", line 7, in hello
>    def hello(): raise ValueError("hello!!!")
>ValueError: hello!!!
>

	But this indicates the thread stuff did run, so your copying into the
post is incorrect.

{Please take my post, then, as an advisory to take care when posting Python
code that the indentation is correct in the posted version -- not taking
such care has caused many wild-goose chases for other posters}
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]

#98662

On 11/11/2015 11:16 AM, Ulli Horlacher wrote:
> I am rewriting a Perl program into Python (2.7).

I recommend using 3.4+ if you possibly can.

> It must run on Linux and Windows.
> With Linux I have no problems, but Windows... :-(
>
> The current show stopper is signal.SIGALRM which is not available on
> Windows:

> Perl for Windows has had SIGALRM support (or some kind of emulation).
>
> Ok, I have to redesign this part of my code:
>
>    def timeout_handler(sig,frame):
>      raise ValueError("timeout!")
>
>    signal.signal(signal.SIGALRM,timeout_handler)
>
>    while True:
>      chunk = fileo.read(bs)
>      sock.sendall(chunk)
>      (...)
>
> What is the best practise for a cross platform timeout handler?

The cross-platform 3.4 asyncio module has some functions with timeouts.
(3.5 has new 'async' syntac which supposedly makes it easier to use.  I 
have not looked at this yet.)

For instance: coroutine asyncio.wait(futures, *, loop=None, 
timeout=None, return_when=ALL_COMPLETED)
     Wait for the Futures and coroutine objects given by the sequence 
futures to complete. Coroutines will be wrapped in Tasks. Returns two 
sets of Future: (done, pending).
...
Usage:
   done, pending = yield from asyncio.wait(fs)

I believe the backport on pypi.python.org, called tulip, works on 2.7.

In the example above, the read/send would be a task.  Wait on the task, 
and when it returns, cancel the task if in pending.


-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#98673

Terry Reedy <tjreedy@udel.edu>:

> The cross-platform 3.4 asyncio module has some functions with
> timeouts.

Even that doesn't forcefully interrupt an obnoxious blocking function
call like

   time.sleep(10000)

The original question claimed signal.alarm() would do the trick in
Linux. However, even that cannot be relied on as "man alarm" states:

   sleep(3) may be implemented using SIGALRM; mixing calls to alarm()
   and sleep(3) is a bad idea.

I'm thinking the only portable way is to run a watchdog process with
subprocess or multiprocessing.


Marko

[toc] | [prev] | [next] | [standalone]

#98675

Am 12.11.15 um 07:14 schrieb Marko Rauhamaa:
> Terry Reedy <tjreedy@udel.edu>:
>
>> The cross-platform 3.4 asyncio module has some functions with
>> timeouts.
>
> Even that doesn't forcefully interrupt an obnoxious blocking function
> call like
>
>     time.sleep(10000)

A blocking call - granted. But what happens in a blocking loop, i.e.

for i in range(10000000000000000000000000):
	pass

?

My understanding of async is that it creates an event loop. In which 
case the loop has no chance to run within a block of code that computes 
anything, is that correct? Or does it hook into the interpreter and is 
able to interrupt the program between bytecodes?

> I'm thinking the only portable way is to run a watchdog process with
> subprocess or multiprocessing.

What about a thread which calls exit() after the timeout? Does that 
forcefully kill the whole process?

	Christian

[toc] | [prev] | [next] | [standalone]

#98678

On Thu, Nov 12, 2015 at 5:43 PM, Christian Gollwitzer <auriocus@gmx.de> wrote:
> My understanding of async is that it creates an event loop. In which case
> the loop has no chance to run within a block of code that computes anything,
> is that correct?

This is correct. At its simplest, asynchronous code is an abstraction
over the select() call, which basically says "Hey system, tell me when
(a) I can read from here, (b) I can write to here, or (c) I've been
waiting this long". The most common use is sockets; a web server has
its main listening socket (it becomes readable when someone connects),
any clients that haven't finished sending their requests yet (they
become readable when more data arrives), any clients that you're still
sending to (they become writeable when there's room in their output
buffers), and maybe some sort of periodic checks ("every hour, do
maintenance"). Whenever you finish a bit of processing (reading from a
client, sending to a client, whatever), you return to the "event
loop", which in this case would be select().

An async library makes all this look a lot cleaner in your code, but
ultimately, it's not preemptive. You still have to make sure the
processing doesn't take too long.

ChrisA

[toc] | [prev] | [next] | [standalone]

#98690

On 11/12/2015 2:37 AM, Chris Angelico wrote:
> On Thu, Nov 12, 2015 at 5:43 PM, Christian Gollwitzer <auriocus@gmx.de> wrote:
>> My understanding of async is that it creates an event loop. In which case
>> the loop has no chance to run within a block of code that computes anything,
>> is that correct?
>
> This is correct. At its simplest, asynchronous code is an abstraction
> over the select() call,

True on Unix-derived systems, where 'select' includes the various 
derivatives.  It is also an abstraction over the Windows completion 
calls, which are quite different.  The latter is why one must generally 
use a different event loop on Windows.  The point is that asyncio 
provides an *abstraction* such that after choosing the event loop, the 
rest of one's code is os-agnostic.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#98692

On Thu, Nov 12, 2015 at 9:15 PM, Terry Reedy <tjreedy@udel.edu> wrote:
> On 11/12/2015 2:37 AM, Chris Angelico wrote:
>>
>> On Thu, Nov 12, 2015 at 5:43 PM, Christian Gollwitzer <auriocus@gmx.de>
>> wrote:
>>>
>>> My understanding of async is that it creates an event loop. In which case
>>> the loop has no chance to run within a block of code that computes
>>> anything,
>>> is that correct?
>>
>>
>> This is correct. At its simplest, asynchronous code is an abstraction
>> over the select() call,
>
>
> True on Unix-derived systems, where 'select' includes the various
> derivatives.  It is also an abstraction over the Windows completion calls,
> which are quite different.  The latter is why one must generally use a
> different event loop on Windows.  The point is that asyncio provides an
> *abstraction* such that after choosing the event loop, the rest of one's
> code is os-agnostic.

I've never done that kind of thing on Windows, so I'm not sure how it
works; it's still broadly based on I/O availability, right? And
ultimately, it comes down to "go back to the event loop so others can
run".

ChrisA

[toc] | [prev] | [next] | [standalone]

#98694

On 11/12/2015 6:38 AM, Chris Angelico wrote:
> On Thu, Nov 12, 2015 at 9:15 PM, Terry Reedy <tjreedy@udel.edu> wrote:
>> On 11/12/2015 2:37 AM, Chris Angelico wrote:
>>>
>>> On Thu, Nov 12, 2015 at 5:43 PM, Christian Gollwitzer <auriocus@gmx.de>
>>> wrote:
>>>>
>>>> My understanding of async is that it creates an event loop. In which case
>>>> the loop has no chance to run within a block of code that computes
>>>> anything,
>>>> is that correct?
>>>
>>>
>>> This is correct. At its simplest, asynchronous code is an abstraction
>>> over the select() call,
>>
>>
>> True on Unix-derived systems, where 'select' includes the various
>> derivatives.  It is also an abstraction over the Windows completion calls,
>> which are quite different.  The latter is why one must generally use a
>> different event loop on Windows.  The point is that asyncio provides an
>> *abstraction* such that after choosing the event loop, the rest of one's
>> code is os-agnostic.
>
> I've never done that kind of thing on Windows, so I'm not sure how it
> works; it's still broadly based on I/O availability, right?

That is the general abstraction.  But there was something in the design 
discussion about a difference between 'edge' versus 'level' triggering 
that made it a challenge to get something more detailed that covers both 
implementations.

> ultimately, it comes down to "go back to the event loop so others can
> run".


-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#98677

Marko Rauhamaa <marko@pacujo.net> wrote:

> I'm thinking the only portable way is to run a watchdog process with
> subprocess or multiprocessing.

How can a subprocess interrupt a function in another process?

For example: waiting for user input with a timeout.

raw_input("Hit ENTER to continue or wait 10 s")



-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum IZUS/TIK         E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart         Tel:    ++49-711-68565868
Allmandring 30a                Fax:    ++49-711-682357
70550 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/

[toc] | [prev] | [next] | [standalone]

#98679

Ulli Horlacher <framstag@rus.uni-stuttgart.de>:

> Marko Rauhamaa <marko@pacujo.net> wrote:
>
>> I'm thinking the only portable way is to run a watchdog process with
>> subprocess or multiprocessing.
>
> How can a subprocess interrupt a function in another process?
>
> For example: waiting for user input with a timeout.
>
> raw_input("Hit ENTER to continue or wait 10 s")

By sending the parent a signal with os.kill().

Now, signal handling in Python is brittle so you must be careful:

  There is no way to “block” signals temporarily from critical sections
  <URL: https://docs.python.org/2/library/signal.html>


Marko

[toc] | [prev] | [next] | [standalone]

#98676

Terry Reedy <tjreedy@udel.edu> wrote:
> On 11/11/2015 11:16 AM, Ulli Horlacher wrote:
> > I am rewriting a Perl program into Python (2.7).
> 
> I recommend using 3.4+ if you possibly can.

It is not possible.
The main target platform offers only python 2.7

-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum IZUS/TIK         E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart         Tel:    ++49-711-68565868
Allmandring 30a                Fax:    ++49-711-682357
70550 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/

[toc] | [prev] | [next] | [standalone]

#98672

On 11Nov2015 16:16, Ulli Horlacher <framstag@rus.uni-stuttgart.de> wrote:
>I am rewriting a Perl program into Python (2.7).
>It must run on Linux and Windows.
>With Linux I have no problems, but Windows... :-(
>
>The current show stopper is signal.SIGALRM which is not available on
>Windows:
>
>  File "fexit.py", line 674, in formdata_post
>    signal.signal(signal.SIGALRM,timeout_handler)
>  AttributeError: 'module' object has no attribute 'SIGALRM'
>
>  https://docs.python.org/2/library/signal.html
>
>  signal.alarm(time) (...) Availability: Unix.
>
>Perl for Windows has had SIGALRM support (or some kind of emulation).
>
>Ok, I have to redesign this part of my code:
>
>  def timeout_handler(sig,frame):
>    raise ValueError("timeout!")
>
>  signal.signal(signal.SIGALRM,timeout_handler)
>
>  while True:
>    chunk = fileo.read(bs)
>    sock.sendall(chunk)
>    (...)
>
>What is the best practise for a cross platform timeout handler?

I suggest you look at the socket.settimeout function. Avoid SIGALRM altogether.  
Then (untested):

  import socket
  ...
  socket.settimeout(timeout_in_seconds)
  ...
  while True:
    ...
    chunk = fileo.read(bs)
    try:
      sock.sendall(chunk)
    except socket.timeout as e:
      ... complain about timeout, reciting "e" in the message ...

Cheers,
Cameron Simpson <cs@zip.com.au>

I think you're confusing "recognizing" and "understanding" with "caring".
The net is cruel, sometimes, but always fair.
        - Rick Gordon <rickg@crl.com>

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web