Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.arch.embedded > #24637

Re: Safety-Critical Software Design

Show all headers | View raw

On 7/16/2016 9:57 PM, Randy Yates wrote:
> Hi Everyone,
>
> Are there any formal requirements, guidelines, or recommendations for
> software which will run in a safety-critical environment in the United
> States or world-wide?
>
> By a "safety-critical" environment I mean an environment in which a
> failure can lead to loss of, or serious injury to, human life. For
> example, automobile navigation systems, medical devices, lasers, etc.

Like, say, a self-driving car?  XRay?  CAT?  Fly-by-wire aircraft?
Countless process control devices?

> I know there is the MISRA association and MISRA C. I am wondering if
> there are others.

IME (in a few different "regulated" industries), *process* seems to
be what gets stressed.  *How* did you come to this design?  Can you
SHOW your "work"?  If there's any hand-waving, EXPECT to have problems!

MISRA is really inconsequential -- despite being adopted by some
industries.  It's like telling folks not to run while carrying scissors.
They're guidelines just like HR has guidelines for whose CV's they'll
consider (the analogy is deliberate:  guidelines can be unnecessarily
limiting!  Sort of like trying to avoid putting a goto in your code!)

Do you put an explicit test to verify the divisor in each division
instance is NOT zero?  That the argument to each sqrt() is non-negative?
(Why not??)

> My gut and experience tells me there should NEVER be software DIRECTLY
> controlling signals of devices that might lead to human injury. Rather,
> such devices should be controlled by discrete hardware, perhaps as
> complex as an FPGA. There is always going to be a chance that a real
> processor that, e.g., controls the enable signal to a laser is going to
> crash with the signal enabled.

That's the purpose of failsafes and watchdogs.

Note that something (i.e., software) still has to TELL that hardware to
allow the device to turn on.  So, you're still "trusting" the software...

I use hardware protections for things like keeping personnel and body
parts out of areas where they can be harmed by mechanisms in motion.
I.e., "guards" with interlocks that make it hard for a user to
casually/carelessly put himself in harms way

> I realize that hardware-only control is subject to failures as well,
> but they wouldn't seem to be nearly as likely as a coding failure.
>
> Let me get even more specific: would it be acceptable to use a processor
> running linux in such an application? My gut reaction is "Not only no,
> but HELL no," but I'm not sure if I'm being overly cautious.

The rationale for "hell no" is that you can't vouch for the code
you're "leveraging"!  And, probably couldn't find a small army of
people who, in concert, *could*!

> Any guidance, suggestions, comments, etc., would be appreciated.

As with any design:  make everything as simple as it can be -- but
no simpler!

You want to be able to convince yourself that you know EVERYTHING
that can affect a safety-related issue -- without having to wonder
how much stuff is obfuscated in the countless "black boxes" in
most designs.

Remember, those black boxes can evolve -- in ways that make your
safety-related ASSUMPTIONS about them invalid!

When I design a solution (hardware and/or software), I am ALWAYS
asking myself:  how can I *break* this?  And, the goal is NOT to
convince myself that the vulnerability CAN'T HAPPEN but, rather,
to figure out how I will protect against it happening (as unlikely
as it may seem!).

And, *record* all of this so its obvious (to yourself and others)
why you did something that *seems* "impossible".

Back to comp.arch.embedded | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Safety-Critical Software Design Randy Yates <yates@digitalsignallabs.com> - 2016-07-17 00:57 -0400
  Re: Safety-Critical Software Design Paul Rubin <no.email@nospam.invalid> - 2016-07-16 23:10 -0700
  Re: Safety-Critical Software Design Niklas Holsti <niklas.holsti@tidorum.invalid> - 2016-07-17 09:16 +0300
  Re: Safety-Critical Software Design Reinhardt Behm <rbehm@hushmail.com> - 2016-07-17 14:38 +0800
  Re: Safety-Critical Software Design upsidedown@downunder.com - 2016-07-17 10:37 +0300
  Re: Safety-Critical Software Design Don Y <blockedofcourse@foo.invalid> - 2016-07-17 03:09 -0700
    Re: Safety-Critical Software Design Don Y <blockedofcourse@foo.invalid> - 2016-07-17 03:32 -0700
      Re: Safety-Critical Software Design Niklas Holsti <niklas.holsti@tidorum.invalid> - 2016-07-17 22:49 +0300
      Re: Safety-Critical Software Design Les Cargill <lcargill99@comcast.com> - 2016-07-26 11:20 -0500
  Re: Safety-Critical Software Design Tim Wescott <tim@seemywebsite.com> - 2016-07-17 20:51 -0500
  Re: Safety-Critical Software Design FreeRTOS info <noemail@given.com> - 2016-07-21 10:33 +0100
  Re: Safety-Critical Software Design Les Cargill <lcargill99@comcast.com> - 2016-07-26 11:17 -0500

csiph-web