Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.arch.embedded > #24632

Re: Safety-Critical Software Design

Show all headers | View raw

Randy Yates wrote:

> Hi Everyone,
> 
> Are there any formal requirements, guidelines, or recommendations for
> software which will run in a safety-critical environment in the United
> States or world-wide?
> 
> By a "safety-critical" environment I mean an environment in which a
> failure can lead to loss of, or serious injury to, human life. For
> example, automobile navigation systems, medical devices, lasers, etc.
> 
> I know there is the MISRA association and MISRA C. I am wondering if
> there are others.
> 
> My gut and experience tells me there should NEVER be software DIRECTLY
> controlling signals of devices that might lead to human injury. Rather,
> such devices should be controlled by discrete hardware, perhaps as
> complex as an FPGA. There is always going to be a chance that a real
> processor that, e.g., controls the enable signal to a laser is going to
> crash with the signal enabled.
> 
> I realize that hardware-only control is subject to failures as well,
> but they wouldn't seem to be nearly as likely as a coding failure.
> 
> Let me get even more specific: would it be acceptable to use a processor
> running linux in such an application? My gut reaction is "Not only no,
> but HELL no," but I'm not sure if I'm being overly cautious.
> 
> Any guidance, suggestions, comments, etc., would be appreciated.

There are several regulations:
IEC 61508 for industrial functional safety
ISO 26262 for automotive
DO-178 for avionics
??? for medical

I am not at home with the first two, but develop avionics systems to DO-178. 
At its highest safety level (DAL-A) you have to trace back every line of 
code to system/high-level/low-level requirements and develop test procedures 
to verify that the software (and that is the whole system, from boot loader 
to OS, to libraries, to the application) fulfills these requirements and 
also that every decision ("if (a && b)...) and every combination of inputs 
("a" and "b" in this example) is correctly processed.
Also every low level requirement must be derived from high-level reqs, every 
HLR must be derived from Sys-Reqs and it must be shown that the step from 
Sys-Reqs to HLR covers all Sys-Req. The same for HLR to LLR.
You even have to certify or verify all tools that can contribute to errors 
or are used for verification and can lead to not detecting an error during 
verification.

If you think you can do it instead in hardware like an FPGA something 
similar applies: There is DO-254 for "complex hardware".

For any software that needs certification to DO-178 e.g. Level A the 
hardware has to be certified to the corresponding DO-254 level. Often you 
will also have not just one CPU but at least two doing the same by software 
written by different teams and some kind of interlock between these which 
brings the system into fail safe state in case of discrepancies. In such a 
case you also have to asses what fail has to be. Shutting down the engines 
in mid flight probably is not very fail safe.

Thing like MISRA C does by itself not guaranty any safety. It is just a set 
of guidelines. Mostly of what not to do because a less competent programmer 
might misuse a feature. 
Something in the line of "Somebody has cut himself with a knife, so we 
forbid to use of knifes."

The most critical part is the development of the Sys-Reqs, since everything 
derives from these. This is something that is in principle outside of the 
realm of the software designers. From what I know about some catastrophic 
failures, the problem was often rooted in incorrect Sys-Reqs.

One example was the fatal crash during landing with heavy side wind of an 
Airbus in Warsaw several years ago. It was caused by an interlock that 
prevented the thrust reversal to be activated when not both wheels signal 
"weight on wheel". This interlock was introduced because a Lauda-Air bird 
dropped from the sky in Malaysia (?) because the pilot had erroneously 
activated thrust reversal during normal flight. The people who had changed 
the Sys-Reqs for the flight software had not thought everything to its end. 
The software people were not to be blamed for correct implementation of this 
"feature" according to the Sys-Reqs.

My guideline is:
- first do safety assessments, identify any potential thread
- develop Sys-Reqs, take into account every dangerous situation and how to 
handle it (like: have a upper time limit for activation of the laser). 
Decide what to to in HW what in SW, the upper limit could better be a HW 
monoflop.
- review the Sys-Reqs and fix/freeze them, signed by the customer. Can every 
Sys-Req be verified? How do different Sys-Reqs interact to create an 
additional thread (see above fatal plane crash)
- develop HLRs. This will also define the overall design. Do nothing fancy, 
keep everything simple.
- review and freeze, Make sure each HLR can be verified during integration 
and verification.
- and so on...

In parallel develop the requirements for the test cases and develop the 
tests. These will help you during coding.

Never ever change HLRs or even Sys-Reqs when coding. When such changes need 
to be done, re-start the process including the reviews and the impact 
analysis for the changes.

-- 
Reinhardt

Back to comp.arch.embedded | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Safety-Critical Software Design Randy Yates <yates@digitalsignallabs.com> - 2016-07-17 00:57 -0400
  Re: Safety-Critical Software Design Paul Rubin <no.email@nospam.invalid> - 2016-07-16 23:10 -0700
  Re: Safety-Critical Software Design Niklas Holsti <niklas.holsti@tidorum.invalid> - 2016-07-17 09:16 +0300
  Re: Safety-Critical Software Design Reinhardt Behm <rbehm@hushmail.com> - 2016-07-17 14:38 +0800
  Re: Safety-Critical Software Design upsidedown@downunder.com - 2016-07-17 10:37 +0300
  Re: Safety-Critical Software Design Don Y <blockedofcourse@foo.invalid> - 2016-07-17 03:09 -0700
    Re: Safety-Critical Software Design Don Y <blockedofcourse@foo.invalid> - 2016-07-17 03:32 -0700
      Re: Safety-Critical Software Design Niklas Holsti <niklas.holsti@tidorum.invalid> - 2016-07-17 22:49 +0300
      Re: Safety-Critical Software Design Les Cargill <lcargill99@comcast.com> - 2016-07-26 11:20 -0500
  Re: Safety-Critical Software Design Tim Wescott <tim@seemywebsite.com> - 2016-07-17 20:51 -0500
  Re: Safety-Critical Software Design FreeRTOS info <noemail@given.com> - 2016-07-21 10:33 +0100
  Re: Safety-Critical Software Design Les Cargill <lcargill99@comcast.com> - 2016-07-26 11:17 -0500

csiph-web