Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.programming.threads > #1089 > unrolled thread

Safe accesses of global arrays in signal handlers?

Back to article view | Back to comp.programming.threads

Contents

  Safe accesses of global arrays in signal handlers? Markus Elfring <Markus.Elfring@web.de> - 2012-09-25 08:01 +0200
    Re: Safe accesses of global arrays in signal handlers? Nobody <nobody@nowhere.com> - 2012-09-25 12:15 +0100
      Re: Safe accesses of global arrays in signal handlers? Markus Elfring <Markus.Elfring@web.de> - 2012-09-27 11:20 +0200
        Re: Safe accesses of global arrays in signal handlers? Rainer Weikusat <rweikusat@mssgmbh.com> - 2012-09-27 13:23 +0100
          Re: Safe accesses of global arrays in signal handlers? Markus Elfring <Markus.Elfring@web.de> - 2012-09-28 13:25 +0200
        Re: Safe accesses of global arrays in signal handlers? Nobody <nobody@nowhere.com> - 2012-09-27 13:35 +0100
          Re: Safe accesses of global arrays in signal handlers? Markus Elfring <Markus.Elfring@web.de> - 2012-09-28 13:48 +0200
            Re: Safe accesses of global arrays in signal handlers? Johann Klammer <klammerj@NOSPAM.a1.net> - 2012-09-28 18:34 +0200
              Re: Safe accesses of global arrays in signal handlers? scott@slp53.sl.home (Scott Lurndal) - 2012-09-28 17:54 +0000
            Re: Safe accesses of global arrays in signal handlers? Nobody <nobody@nowhere.com> - 2012-09-28 18:26 +0100
              Re: Safe accesses of global arrays in signal handlers? Markus Elfring <Markus.Elfring@web.de> - 2012-09-29 13:44 +0200
                Re: Safe accesses of global arrays in signal handlers? Nobody <nobody@nowhere.com> - 2012-09-29 20:37 +0100
                  Re: Safe accesses of global arrays in signal handlers? William Ahern <william@wilbur.25thandClement.com> - 2012-09-29 17:31 -0700
                    Signal handlers writing into pipes Markus Elfring <Markus.Elfring@web.de> - 2012-09-30 13:21 +0200
                    Re: Safe accesses of global arrays in signal handlers? Rainer Weikusat <rweikusat@mssgmbh.com> - 2012-10-01 18:04 +0100
                      Signal handlers writing into pipes Markus Elfring <Markus.Elfring@web.de> - 2012-10-03 16:36 +0200
                        Re: Signal handlers writing into pipes Rainer Weikusat <rweikusat@mssgmbh.com> - 2012-10-03 17:07 +0100
                  Re: Safe accesses of global arrays in signal handlers? Markus Elfring <Markus.Elfring@web.de> - 2012-09-30 13:12 +0200
                    Re: Safe accesses of global arrays in signal handlers? Nobody <nobody@nowhere.com> - 2012-09-30 20:46 +0100
          Re: Safe accesses of global arrays in signal handlers? Geoff Clare <geoff@clare.See-My-Signature.invalid> - 2012-09-28 13:39 +0100
          Re: Safe accesses of global arrays in signal handlers? Markus Elfring <Markus.Elfring@web.de> - 2012-09-29 13:15 +0200
            Re: Safe accesses of global arrays in signal handlers? William Ahern <william@wilbur.25thandClement.com> - 2012-09-29 17:42 -0700
              Re: Safe accesses of global arrays in signal handlers? Markus Elfring <Markus.Elfring@web.de> - 2012-09-30 14:04 +0200
    Re: Safe accesses of global arrays in signal handlers? Rainer Weikusat <rweikusat@mssgmbh.com> - 2012-09-25 13:27 +0100

Page 1 of 2  [1] 2  Next page →

#1089 — Safe accesses of global arrays in signal handlers?

Hello,

A secure programming guideline contains the following information.

https://www.securecoding.cert.org/confluence/display/seccode/SIG31-C.+Do+not+access+or+modify+shared+objects+in+signal+handlers
:
'Accessing or modifying shared objects in signal handlers can result in race
conditions that can leave data in an inconsistent state. The exception to this
rule is the ability to read and write to variables of volatile sig_atomic_t.
...
It is important to note that the behavior of a program that accesses an object
of any other type from a signal handler is undefined.'


I interpret this description in the way that there are risks for the portable
use of a shared variable which has got a data type like "array" within a signal
handler.
http://en.wikipedia.org/wiki/Array_data_type


Source code example:
  pthread_mutex_t my_lock = PTHREAD_MUTEX_INITIALIZER;
  struct my_element my_list[MY_ARRAY_SIZE];

  void my_status_log(int number)
  {
   int result = pthread_mutex_lock(&my_lock);

   if (result) exit(result);

   if (  my_list[number - 1].catched
      && printf("Signal %d was received.", number) < 22)
      perror("printf");

   result = pthread_mutex_unlock(&my_lock);

   if (result) exit(result);
  }


  static void my_signal_handler(int number)
  {
   struct my_data * pointer = my_list[number - 1].context;

   if (!pointer) return;

   my_list[number - 1].catched = 1;

   /* Call more functions eventually ... */
  }


I would appreciate your advices for the clarification of the involved
implementation details. Would you like to suggest any solutions for such an use
case?

Regards,
Markus

[toc] | [next] | [standalone]

#1090

On Tue, 25 Sep 2012 08:01:47 +0200, Markus Elfring wrote:

> I would appreciate your advices for the clarification of the involved
> implementation details. Would you like to suggest any solutions for such
> an use case?

The recommended way to handle signals is to block all relevant async
signals in all threads, then have a single thread which does nothing but
poll signals using sigwaitinfo() then handle them (e.g. by notifying
condition variables, appending information to a queue, etc).

[toc] | [prev] | [next] | [standalone]

#1095

> The recommended way to handle signals is to block all relevant async
> signals in all threads, then have a single thread which does nothing but
> poll signals using sigwaitinfo() then handle them (e.g. by notifying
> condition variables, appending information to a queue, etc).

Do you know any other solutions which can work without such an initial signal
blockage like it is needed in the suggested approach for the handling of
real-time systems?

Regards,
Markus

[toc] | [prev] | [next] | [standalone]

#1097

Markus Elfring <Markus.Elfring@web.de> writes:
>> The recommended way to handle signals is to block all relevant async
>> signals in all threads, then have a single thread which does nothing but
>> poll signals using sigwaitinfo() then handle them (e.g. by notifying
>> condition variables, appending information to a queue, etc).
>
> Do you know any other solutions which can work without such an initial signal
> blockage like it is needed in the suggested approach for the handling of
> real-time systems?

Judgeing from you initial post, you're targetting a hypothetical
computer without any known properties and the example code you
provided is also incomplete (it lacks information about any actual
data types). Consequently, no one can answer this question.

[toc] | [prev] | [next] | [standalone]

#1106

> Judgeing from you initial post, you're targetting a hypothetical
> computer without any known properties and the example code you
> provided is also incomplete (it lacks information about any actual
> data types). Consequently, no one can answer this question.

Would you like to look at a more concrete source code example again?
http://cvs.schmorp.de/libev/ev.c?revision=1.449&view=markup#l2029

How do you think about the feedback by Marc Lehmann for my request "Fix signal
handler"?
http://lists.schmorp.de/pipermail/libev/2012q3/002007.html

Regards,
Markus

[toc] | [prev] | [next] | [standalone]

#1098

On Thu, 27 Sep 2012 11:20:14 +0200, Markus Elfring wrote:

>> The recommended way to handle signals is to block all relevant async
>> signals in all threads, then have a single thread which does nothing but
>> poll signals using sigwaitinfo() then handle them (e.g. by notifying
>> condition variables, appending information to a queue, etc).
> 
> Do you know any other solutions which can work without such an initial signal
> blockage like it is needed in the suggested approach for the handling of
> real-time systems?

Doing anything substantial in a signal handler is problematic. So
typically you just want to notify a thread which blocks until
notification. There are a few async-signal-safe functions which can be
used for notification, including sem_post(), sigqueue(), raise() and
write(). A complete list is at:

http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_04_03

Note that none of the pthread functions are in that list.

[toc] | [prev] | [next] | [standalone]

#1107

> Doing anything substantial in a signal handler is problematic.

Are the following approaches valid software design options for the discussed use
case?

1. "Self-pipe trick" that was described by Daniel J. Bernstein.
   http://cr.yp.to/docs/selfpipe.html

2. Are lock-free algorithms applicable like an implementation is described in
the article "Signal-Safe Locks"?
   http://locklessinc.com/articles/signalsafe_locks/

   http://en.wikipedia.org/wiki/Non-blocking_algorithm

Regards,
Markus

[toc] | [prev] | [next] | [standalone]

#1110

Markus Elfring wrote:
>> Doing anything substantial in a signal handler is problematic.
>
> Are the following approaches valid software design options for the discussed use
> case?
>
> 1. "Self-pipe trick" that was described by Daniel J. Bernstein.
>     http://cr.yp.to/docs/selfpipe.html
>
This should work nicely, as long as you need some kind of FIFO. The 
write() is usable in signal handler.
> 2. Are lock-free algorithms applicable like an implementation is described in
> the article "Signal-Safe Locks"?
>     http://locklessinc.com/articles/signalsafe_locks/
>
>     http://en.wikipedia.org/wiki/Non-blocking_algorithm
>
It depends if your architecture allows atomic increments. Better avoid that.

With both approaches you'll have to consider that under some 
circumstances, multiple signals may be merged to one even before your 
handler will see them.

> Regards,
> Markus

[toc] | [prev] | [next] | [standalone]

#1112

Johann Klammer <klammerj@NOSPAM.a1.net> writes:
>Markus Elfring wrote:
>>> Doing anything substantial in a signal handler is problematic.
>>
>> Are the following approaches valid software design options for the discussed use
>> case?
>>
>> 1. "Self-pipe trick" that was described by Daniel J. Bernstein.
>>     http://cr.yp.to/docs/selfpipe.html

A common method used to awaken a poll loop, when for example, one needs
to add "POLLOUT" to the events mask for a network file descriptor.  I usually
pass a single byte through the pipe, and a POLLIN event on the read-end of
the pipe will fire.   Consume the byte, then 'continue' to the top of the
poll loop.    To terminate the loop, set the boolean condition on the loop
to false[*] prior to writing to the pipe.   Otherwise, the loop will examine
the outbound queue, and if not empty, set POLLOUT on the appropriate
network connection file descriptor which, when tripped, will result in
a send/write/sendto/writev of the outbound packet.

>>
>This should work nicely, as long as you need some kind of FIFO. The 
>write() is usable in signal handler.
>> 2. Are lock-free algorithms applicable like an implementation is described in
>> the article "Signal-Safe Locks"?
>>     http://locklessinc.com/articles/signalsafe_locks/
>>
>>     http://en.wikipedia.org/wiki/Non-blocking_algorithm
>>
>It depends if your architecture allows atomic increments. Better avoid that.

gcc provides atomic primitives that are portable across all architectures
(and C11/C++11 have now specified atomic primitives as well).  On architectures
that don't support atomics in hardware, the compiler will generate calls to
library functions that will emulate them with LL/SC, compare and swap, et. al.

>
>With both approaches you'll have to consider that under some 
>circumstances, multiple signals may be merged to one even before your 
>handler will see them.

Unless one is using queued aka real-time signals.

scott

[*] Use the following macro to test the condition at the top of the loop
    to avoid compiler optimization related issues when using threads or signals.

#define ACCESS_ONCE(x) (*(volatile typeof(x) *)&(x))

[toc] | [prev] | [next] | [standalone]

#1111

On Fri, 28 Sep 2012 13:48:40 +0200, Markus Elfring wrote:

>> Doing anything substantial in a signal handler is problematic.
> 
> Are the following approaches valid software design options for the
> discussed use case?
> 
> 1. "Self-pipe trick" that was described by Daniel J. Bernstein.
>    http://cr.yp.to/docs/selfpipe.html

Yes. write() is async-signal-safe, so one thread can do a blocking read()
and the signal handler can use write() to wake it.

> 2. Are lock-free algorithms applicable like an implementation is described
> in the article "Signal-Safe Locks"?

Lock-free algorithms require memory semantics beyond those specified in
the C and POSIX standards. E.g. they typically require the use of a memory
barrier (fence).

[toc] | [prev] | [next] | [standalone]

#1119

> Yes. write() is async-signal-safe, so one thread can do a blocking read()
> and the signal handler can use write() to wake it.

Thanks for your feedback.

Do you see any difficulties or software design challenges for proper handling of
the situation that more data is written into the pipe than the receiving thread
can read and process in a timely manner?


> Lock-free algorithms require memory semantics beyond those specified in
> the C and POSIX standards. E.g. they typically require the use of a memory
> barrier (fence).

I hope that the support for this use case will be improved by the current (and
following) standard version for the programming languages C and C++.
http://en.wikipedia.org/wiki/C++11#Threading_facilities

Regards,
Markus

[toc] | [prev] | [next] | [standalone]

#1121

On Sat, 29 Sep 2012 13:44:50 +0200, Markus Elfring wrote:

>> Yes. write() is async-signal-safe, so one thread can do a blocking read()
>> and the signal handler can use write() to wake it.
> 
> Thanks for your feedback.
> 
> Do you see any difficulties or software design challenges for proper
> handling of the situation that more data is written into the pipe than
> the receiving thread can read and process in a timely manner?

The signal handler only needs to write one byte at a time; the receiving
thread can read a pipe-full at a time. If the receiving thread does
nothing but notify worker threads or enqueue messages, there shouldn't be
a problem.

But you have a similar issue with the signals themselves. If non-realtime
signals are generated faster than they are handled, pending signals
aren't guaranteed to be queued, i.e. the handler may only be invoked once
for multiple signals.

[toc] | [prev] | [next] | [standalone]

#1122

Nobody <nobody@nowhere.com> wrote:
> On Sat, 29 Sep 2012 13:44:50 +0200, Markus Elfring wrote:

> >> Yes. write() is async-signal-safe, so one thread can do a blocking
> >> read() and the signal handler can use write() to wake it.
> > 
> > Thanks for your feedback.
> > 
> > Do you see any difficulties or software design challenges for proper
> > handling of the situation that more data is written into the pipe than
> > the receiving thread can read and process in a timely manner?

> The signal handler only needs to write one byte at a time; the receiving
> thread can read a pipe-full at a time. If the receiving thread does
> nothing but notify worker threads or enqueue messages, there shouldn't be
> a problem.

> But you have a similar issue with the signals themselves. If non-realtime
> signals are generated faster than they are handled, pending signals aren't
> guaranteed to be queued, i.e. the handler may only be invoked once for
> multiple signals.

The problematic issue is when SIGFOO fills the pipe, causing SIGBAR to get
dropped on the floor. That seems exceedingly unlikely, but I suppose you
could use one pipe per signal.

Or better yet, just use a pipe for signalling and an atomic operation to
clear a separate data structure, basically mimicking what the kernel does
with it's pending queue and syscall interruption.

FWIW, Linux has signalfd, and the BSDs have the EVFILT_SIGNAL kevent. The
proper solution on Solaris is to block all signals, and have a dedicated
thread block on sigwait, sigwaitinfo, or sigtimedwait. That also works on
Linux and most of the BSDs--OpenBSD threads and signals are currently
broken, however.

EVFILT_SIGNAL is the cleanest interface, because it requires no global data
structures and its use has no side effects--multiple pollers can catch
notifications, and it works regardless of whether a signal is blocked or
ignored.

If only every platform implemented kqueue...

[toc] | [prev] | [next] | [standalone]

#1125 — Signal handlers writing into pipes

> The problematic issue is when SIGFOO fills the pipe, causing SIGBAR to get
> dropped on the floor. That seems exceedingly unlikely, but I suppose you
> could use one pipe per signal.

Can the probability for a full pipe be calculated for the discussed use case?
How big is the pipe capacity on the various software platforms?

Regards,
Markus

[toc] | [prev] | [next] | [standalone]

#1147

William Ahern <william@wilbur.25thandClement.com> writes:
> Nobody <nobody@nowhere.com> wrote:
>> On Sat, 29 Sep 2012 13:44:50 +0200, Markus Elfring wrote:
>
>> >> Yes. write() is async-signal-safe, so one thread can do a blocking
>> >> read() and the signal handler can use write() to wake it.
>> > 
>> > Thanks for your feedback.
>> > 
>> > Do you see any difficulties or software design challenges for proper
>> > handling of the situation that more data is written into the pipe than
>> > the receiving thread can read and process in a timely manner?
>
>> The signal handler only needs to write one byte at a time; the receiving
>> thread can read a pipe-full at a time. If the receiving thread does
>> nothing but notify worker threads or enqueue messages, there shouldn't be
>> a problem.
>
>> But you have a similar issue with the signals themselves. If non-realtime
>> signals are generated faster than they are handled, pending signals aren't
>> guaranteed to be queued, i.e. the handler may only be invoked once for
>> multiple signals.
>
> The problematic issue is when SIGFOO fills the pipe, causing SIGBAR to get
> dropped on the floor. That seems exceedingly unlikely, but I suppose you
> could use one pipe per signal.

Technically, when the program is receiving signals so fast that it
doesn't get around to processing them anymore, that's a livelock and
in this case, something has to be dropped on the floor in order to
restore the ability to make forward progress. Consequently, just
setting the pipe descriptor to non-blocking and ignoring EAGAIN in the
signal handler might be a sensible approach, especially considering
that this is unlikely to happen for typical uses of signals.

[toc] | [prev] | [next] | [standalone]

#1165 — Signal handlers writing into pipes

> Technically, when the program is receiving signals so fast that it
> doesn't get around to processing them anymore, that's a livelock and
> in this case, something has to be dropped on the floor in order to
> restore the ability to make forward progress. Consequently, just
> setting the pipe descriptor to non-blocking and ignoring EAGAIN in the
> signal handler might be a sensible approach, especially considering
> that this is unlikely to happen for typical uses of signals.

Can the probability for a live-lock be reduced if a named pipe would be used
instead of an anonymous one?

Regards,
Markus

[toc] | [prev] | [next] | [standalone]

#1167 — Re: Signal handlers writing into pipes

Markus Elfring <Markus.Elfring@web.de> writes:
>> Technically, when the program is receiving signals so fast that it
>> doesn't get around to processing them anymore, that's a livelock and
>> in this case, something has to be dropped on the floor in order to
>> restore the ability to make forward progress. Consequently, just
>> setting the pipe descriptor to non-blocking and ignoring EAGAIN in the
>> signal handler might be a sensible approach, especially considering
>> that this is unlikely to happen for typical uses of signals.
>
> Can the probability for a live-lock be reduced if a named pipe would be used
> instead of an anonymous one?

No because this is not a buffering problem. It just means that the
pipe buffer fills up because signals are persistently arriving faster
than the appplication code outside of the signal handler can process
them.

[toc] | [prev] | [next] | [standalone]

#1124

> The signal handler only needs to write one byte at a time;
> the receiving thread can read a pipe-full at a time. If the receiving
> thread does nothing but notify worker threads or enqueue messages,
> there shouldn't be a problem.

Is it a software design challenge to forward the number that was passed to this
callback function into the dedicated pipe?
Does the conversion of the signal number from an integer to a single character
mean an interesting portability issue if you would like to use it as an index
for an array?
(What would be the maximum value in this use case?)

Regards,
Markus

[toc] | [prev] | [next] | [standalone]

#1127

On Sun, 30 Sep 2012 13:12:27 +0200, Markus Elfring wrote:

>> The signal handler only needs to write one byte at a time;
>> the receiving thread can read a pipe-full at a time. If the receiving
>> thread does nothing but notify worker threads or enqueue messages,
>> there shouldn't be a problem.
> 
> Is it a software design challenge to forward the number that was passed
> to this callback function into the dedicated pipe?

Using a pipe is fine if you just want to wake a thread when something
interesting has happened, but if you're trying to turn signals into
events, the sigwaitinfo() approach is the way to go.

[toc] | [prev] | [next] | [standalone]

#1109

Nobody wrote:

> Doing anything substantial in a signal handler is problematic. So
> typically you just want to notify a thread which blocks until
> notification. There are a few async-signal-safe functions which can be
> used for notification, including sem_post(), sigqueue(), raise() and
> write(). A complete list is at:
>
> http://pubs.opengroup.org/[...]/functions/V2_chap02.html#tag_15_04_03
>
> Note that none of the pthread functions are in that list.

That's going to change in SUSv4 TC1 (which is finished but awaiting
final approval by IEEE and The Open Group). It adds pthread_equal(),
pthread_kill() and pthread_sigmask() to the list of async-signal-safe
functions.

Note that the additions are not because the standard developers' expect
them to be used for anything useful in signal handlers. They are being
done to correct some inconsistencies. Namely that raise(sig) is required
to be equivalent to pthread_kill(pthread_self(), sig) and raise()
is async-signal-safe, and that pthread_sigmask() is required to be
equivalent to sigprocmask(), without the single-thread restriction, and
sigprocmask() is async-signal-safe.

-- 
Geoff Clare <netnews@gclare.org.uk>

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.programming.threads

csiph-web