Groups | Search | Server Info | Login | Register

Groups > comp.arch.embedded > #32266

Re: Validation in non-regulated industries/markets

Cross-posted to 2 groups.

Show all headers | View raw

On 08/11/2024 05:10, Don Y wrote:
> In *regulated* industries (FDA, aviation, etc.), products are
> validate (hardware and software) in their "as sold" configurations.
> This adds constraints to what can be tested, and how.  E.g.,
> invariants in code need to remain in the production configuration
> if relied upon during validation.
> 
> But, *testing* (as distinct from validation) is usually more
> thorough and benefits from test-specific changes to the
> hardware and software.  These to allow for fault injection
> and observation.
> 
> In *unregulated* industries (common in the US but not so abroad),
> how much of a stickler is the validation process for this level
> of "purity"?
> 
> E.g., I have "test" hardware that I use to exercise the algorithms
> in my code to verify they operate as intended and detect the
> faults against which they are designed to protect.  So, I can inject
> EDAC errors in my memory interface, SEUs, multiple row/column
> faults, read/write disturb errors, pin/pad driver faults, etc.
> 
> These are useful (essential?) to proving the software can
> detect these faults -- without having to wait for a "naturally
> occurrence".  But, because they are verified/validated on non
> production hardware, they wouldn't "fly" in regulated
> markets.
> 
> Do you "assume" your production hardware/software mimics
> the "test" configuration, just by a thought exercise
> governing the differences between the two situations?
> 
> Without specialty devices (e.g., bond-outs), how can you
> address these issues, realistically?
> 

I think perhaps this is confusing systems testing with product testing. 
You need to make a clear distinction between the two.

Systems testing is about checking that a /design/ is correct.  Much of 
that is usually about software testing, but it applies to hardware too. 
This will often be done using modified hardware so that you can, for 
example, inject /realistic/ faults and check that the hardware and 
software function as expected.  Depending on the application, you might 
also run test boards at high temperatures or otherwise abuse them to 
confirm the design.

Production testing is about ensuring that the products made are correct 
according to the design.  You don't check that the memory works, or the 
ECC handler works - you check that you have correctly mounted and 
soldered the memory chip and that the memory chip supplier has checked 
for production faults.


There are some products where the likelihood of developing partial 
faults in the field are high and the consequences of that are serious 
but it is useful to be able to keep a partially failed system in action. 
  There are also products with user-serviceable parts.  Then it is often 
helpful to have some kind of self-test to identify failing subsystems.


Unfortunately, in some regulated markets, or for some types of "safety 
certification", the rule-makers don't understand how this works.  The 
result is that they insist on extra fault-checking hardware and/or 
software that actually decreases the total reliability of the system, 
and introduces new parts that in themselves cannot be checked 
(systematically, in production, and/or in the field).


How do you deal with it?  You follow the rules, even though some of them 
were written by muppets.

Back to comp.arch.embedded | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

Validation in non-regulated industries/markets Don Y <blockedofcourse@foo.invalid> - 2024-11-07 21:10 -0700
  Re: Validation in non-regulated industries/markets David Brown <david.brown@hesbynett.no> - 2024-11-08 10:53 +0100
    Re: Validation in non-regulated industries/markets "Niocláiſín Cóilín de Ġloſtéir" <Master_Fontaine_is_dishonest@Strand_in_London.Gov.UK> - 2024-11-13 00:27 +0100
      Re: Validation in non-regulated industries/markets Don Y <blockedofcourse@foo.invalid> - 2024-11-12 17:21 -0700
        Re: Validation in non-regulated industries/markets Don Y <blockedofcourse@foo.invalid> - 2024-11-12 17:28 -0700
        Re: Validation in non-regulated industries/markets David Brown <david.brown@hesbynett.no> - 2024-11-13 09:58 +0100
  Re: Validation in non-regulated industries/markets bitrex <user@example.net> - 2024-11-13 14:18 -0500
    Re: Validation in non-regulated industries/markets Don Y <blockedofcourse@foo.invalid> - 2024-11-13 12:59 -0700
      Re: Validation in non-regulated industries/markets Don Y <blockedofcourse@foo.invalid> - 2024-11-13 14:42 -0700

csiph-web