Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #19710 > unrolled thread

python reliability with EINTR handling in general modules

Back to article view | Back to comp.lang.python

Contents

  python reliability with EINTR handling in general modules oleg korenevich <void.of.time@gmail.com> - 2012-02-01 06:15 -0800
    Re: python reliability with EINTR handling in general modules Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2012-02-01 11:07 -0500
      Re: python reliability with EINTR handling in general modules oleg korenevich <void.of.time@gmail.com> - 2012-02-01 23:25 -0800
        Re: python reliability with EINTR handling in general modules Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2012-02-02 11:34 -0500
          Re: python reliability with EINTR handling in general modules Mel Wilson <mwilson@the-wire.com> - 2012-02-02 14:42 -0500

#19710 — python reliability with EINTR handling in general modules

I have linux board on samsung SoC s3c6410 (ARM11). I build rootfs with
buildroot: Python 2.7.1, uClibc-0.9.31. Linux kernel: Linux buildroot
2.6.28.6 #177 Mon Oct 3 12:50:57 EEST 2011 armv6l GNU/Linux

My app, written on python, in some mysterios conditons raise this
exceptions:

1) exception:

 File "./dfbUtils.py", line 3209, in setItemData
ValueError: (4, 'Interrupted system call')
code:

currentPage=int(math.floor(float(rowId)/
self.pageSize))==self.selectedPage
2) exception:

File "./terminalGlobals.py", line 943, in getFirmawareName
OSError: [Errno 4] Interrupted system call: 'firmware'
code:

for fileName in os.listdir('firmware'):
Some info about app: it have 3-7 threads, listen serial ports via
'serial' module, use gui implemented via c extension that wrap
directfb, i can't reproduce this exceptions, they are not predictable.

I googled for EINTR exceptions in python, but only found that EINTR
can occur only on slow system calls and python's modules socket,
subprocess and another one is already process EINTR. So what happens
in my app? Why simple call of math function can interrupt program at
any time, it's not reliable at all. I have only suggestions: ulibc
bug, kernel/hw handling bug. But this suggestions don't show me
solution.

Now i created wrap functions (that restart opertion in case of EINTR)
around some functions from os module, but wrapping math module will
increase execution time in 2 times. There another question: if math
can be interrutped than other module also can and how to get
reliability?

[toc] | [next] | [standalone]

#19715

On Wed, 1 Feb 2012 06:15:22 -0800 (PST), oleg korenevich
<void.of.time@gmail.com> wrote:

>I have linux board on samsung SoC s3c6410 (ARM11). I build rootfs with
>buildroot: Python 2.7.1, uClibc-0.9.31. Linux kernel: Linux buildroot
>2.6.28.6 #177 Mon Oct 3 12:50:57 EEST 2011 armv6l GNU/Linux
>
>My app, written on python, in some mysterios conditons raise this
>exceptions:
>
>1) exception:
>
> File "./dfbUtils.py", line 3209, in setItemData
>ValueError: (4, 'Interrupted system call')
>code:
>
>currentPage=int(math.floor(float(rowId)/
>self.pageSize))==self.selectedPage
>2) exception:
>
>File "./terminalGlobals.py", line 943, in getFirmawareName
>OSError: [Errno 4] Interrupted system call: 'firmware'
>code:
>
>for fileName in os.listdir('firmware'):
>Some info about app: it have 3-7 threads, listen serial ports via
>'serial' module, use gui implemented via c extension that wrap
>directfb, i can't reproduce this exceptions, they are not predictable.
>
>I googled for EINTR exceptions in python, but only found that EINTR
>can occur only on slow system calls and python's modules socket,
>subprocess and another one is already process EINTR. So what happens
>in my app? Why simple call of math function can interrupt program at
>any time, it's not reliable at all. I have only suggestions: ulibc
>bug, kernel/hw handling bug. But this suggestions don't show me
>solution.

	I see nothing in your traceback that indicates that the interrupt
occurred in the math library call -- unless you deleted that line. In
the first one, I'd be more likely to suspect your C extension/wrapper...
(are the fields .pageSize and .selectedPage coming from an object
implemented in C?)

	As for the math stuff... I presume both rowID and .pageSize are
constrained to be 0 or positive integers. If that is the case, invoking
math.floor() is just redundant overhead as the documented behavior of
int() is to truncate towards 0, which for a positive value, is the same
as floor()

>>> neg = -3.141592654
>>> pos = 3.141592654
>>> int(neg)
-3
>>> math.floor(neg)
-4.0
>>> int(pos)
3
>>> math.floor(pos)
3.0
>>> 

	In the second case... Well, os.listdir() is most likely translated
into some operating system call.

http://www.gnu.org/software/libc/manual/html_node/Interrupted-Primitives.html

And, while that call is waiting for I/O to complete, some sort of signal
is being received.
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
        wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]

#19775

On Feb 1, 6:07 pm, Dennis Lee Bieber <wlfr...@ix.netcom.com> wrote:
> On Wed, 1 Feb 2012 06:15:22 -0800 (PST), oleg korenevich
>
>
>
>
>
>
>
>
>
> <void.of.t...@gmail.com> wrote:
> >I have linux board on samsung SoC s3c6410 (ARM11). I build rootfs with
> >buildroot: Python 2.7.1, uClibc-0.9.31. Linux kernel: Linux buildroot
> >2.6.28.6 #177 Mon Oct 3 12:50:57 EEST 2011 armv6l GNU/Linux
>
> >My app, written on python, in some mysterios conditons raise this
> >exceptions:
>
> >1) exception:
>
> > File "./dfbUtils.py", line 3209, in setItemData
> >ValueError: (4, 'Interrupted system call')
> >code:
>
> >currentPage=int(math.floor(float(rowId)/
> >self.pageSize))==self.selectedPage
> >2) exception:
>
> >File "./terminalGlobals.py", line 943, in getFirmawareName
> >OSError: [Errno 4] Interrupted system call: 'firmware'
> >code:
>
> >for fileName in os.listdir('firmware'):
> >Some info about app: it have 3-7 threads, listen serial ports via
> >'serial' module, use gui implemented via c extension that wrap
> >directfb, i can't reproduce this exceptions, they are not predictable.
>
> >I googled for EINTR exceptions in python, but only found that EINTR
> >can occur only on slow system calls and python's modules socket,
> >subprocess and another one is already process EINTR. So what happens
> >in my app? Why simple call of math function can interrupt program at
> >any time, it's not reliable at all. I have only suggestions: ulibc
> >bug, kernel/hw handling bug. But this suggestions don't show me
> >solution.
>
>         I see nothing in your traceback that indicates that the interrupt
> occurred in the math library call -- unless you deleted that line. In
> the first one, I'd be more likely to suspect your C extension/wrapper...
> (are the fields .pageSize and .selectedPage coming from an object
> implemented in C?)
>
>         As for the math stuff... I presume both rowID and .pageSize are
> constrained to be 0 or positive integers. If that is the case, invoking
> math.floor() is just redundant overhead as the documented behavior of
> int() is to truncate towards 0, which for a positive value, is the same
> as floor()
>
>
>
> >>> neg = -3.141592654
> >>> pos = 3.141592654
> >>> int(neg)
> -3
> >>> math.floor(neg)
> -4.0
> >>> int(pos)
> 3
> >>> math.floor(pos)
> 3.0
>
>         In the second case... Well, os.listdir() is most likely translated
> into some operating system call.
>
> http://www.gnu.org/software/libc/manual/html_node/Interrupted-Primiti...
>
> And, while that call is waiting for I/O to complete, some sort of signal
> is being received.
> --
>         Wulfraed                 Dennis Lee Bieber         AF6VN
>         wlfr...@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

Thanks for help. In first case all vars is python integers, maybe
math.floor is redundant, but i'm afraid that same error with math
module call will occur in other places of app, where math is needed.
Strange thing here is that math library call is not a system call, and
strange exception ValueError (all values have right values) and why in
braces i have (4, Interruted system call).

For second case: if python really does some slow system call from
module os, why it doesn't handle EINTR and not restart call. Is
SA_RESTART flag in signal can be solution? But how i can set this
flag? By placing flag for signal handler in c extension (or ctypes
manipulation)?

[toc] | [prev] | [next] | [standalone]

#19801

On Wed, 1 Feb 2012 23:25:36 -0800 (PST), oleg korenevich
<void.of.time@gmail.com> wrote:


>Thanks for help. In first case all vars is python integers, maybe
>math.floor is redundant, but i'm afraid that same error with math
>module call will occur in other places of app, where math is needed.
>Strange thing here is that math library call is not a system call, and
>strange exception ValueError (all values have right values) and why in
>braces i have (4, Interruted system call).
>
	math.floor() may still be a system call of some sort if access to
the math processor requires synchronization between processes (that is,
the math processor/registers are maintained as a separate structure
apart from the task status during process switches). {Yes -- that is a
wild hypothesis}

	Or perhaps you haven't enabled the full math support in the runtime
-- from http://uclibc.org/FAQ.html

       >In other cases, uClibc leaves certain features (such as full C99
Math library support, wordexp, IPV6, and RPC support) disabled by
default. 

FYI: uClibc 9.33 has been released -- might be worth checking the change
logs to see if something might apply to the problems.

>For second case: if python really does some slow system call from
>module os, why it doesn't handle EINTR and not restart call. Is
>SA_RESTART flag in signal can be solution? But how i can set this
>flag? By placing flag for signal handler in c extension (or ctypes
>manipulation)?

	Note that it is identifying "firmware" as the system call (If I
understand the nature of the traceback). Is "firmware" a real file
system directory or some virtual entry that translates into scanning the
hardware/ROM for information. (If it isn't obvious, I'm not an OS
internals expert -- though I was able to get around in DEC (Open)VMS
back in the 90s).

	What has me curious is: WHAT signal is being seen that is
interrupting the system call.
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
        wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]

#19808

Dennis Lee Bieber wrote:

> On Wed, 1 Feb 2012 23:25:36 -0800 (PST), oleg korenevich
> <void.of.time@gmail.com> wrote:
> 
> 
>>Thanks for help. In first case all vars is python integers, maybe
>>math.floor is redundant, but i'm afraid that same error with math
>>module call will occur in other places of app, where math is needed.
>>Strange thing here is that math library call is not a system call, and
>>strange exception ValueError (all values have right values) and why in
>>braces i have (4, Interruted system call).
>>
> math.floor() may still be a system call of some sort if access to
> the math processor requires synchronization between processes (that is,
> the math processor/registers are maintained as a separate structure
> apart from the task status during process switches). {Yes -- that is a
> wild hypothesis}

One thing to remember about errno is that C library code will set it to a 
non-zero value when an error is encountered, but (I believe) there's no 
requirement to clear it in the absence of an error.  EINTR might just be 
left over from some long-gone I/O call, then reported "just in case" in 
handling an exception that didn't involve the C library at all.

As a C coder there are times when it's wise to clear errno yourself to make 
sure your code doesn't get fooled.

	Mel.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web