Testing (Was: arm-gcc: pointer to constant string)

From	Jacob Sparre Andersen <jacob@jacob-sparre.dk>
Newsgroups	comp.arch.embedded
Subject	Testing (Was: arm-gcc: pointer to constant string)
Date	2018-09-22 12:16 +0200
Organization	JSA Research & Innovation
Message-ID	<87efdlu9af.fsf_-_@jacob-sparre.dk> (permalink)
References	(14 earlier) <pnmd4r$3h8$1@dont-email.me> <pnpn9t$i6j$1@dont-email.me> <pnq76n$kos$1@dont-email.me> <po4bpk$5nm$1@dont-email.me> <po53eb$poj$1@dont-email.me>

Show all headers | View raw

David Brown wrote:

> But you are simply /wrong/ to think that non-exhaustive testing can
> somehow "prove" that code works.

I hope we all agree on this.

> And there are some classes of problem for which testing is extremely
> unlikely to identify the problem.  In particular, for anything
> involving race conditions or timing, you cannot /test/ your way to
> working code.

If you know the possible origin of race conditions, you can actually
detect them using faked versions of selected functions in an API.  It
still doesn't prove anything, and I'm not sure how often it is cost
effective, but it is worth being aware of the possibility.

We've done it on one legacy project I've worked on.  The system is
database-heavy, and has a high risk of problems due to race-conditions
on database modifications.  We created a modified version of the
database API implementation, where we easily can inject what will appear
as competing modifications of the database from other processes.  It
doesn't find all the problems, but it easily reveals to worst blunders.

I am pretty sure you can do similar things in other cases - like file
system interactions - but I'm not sure how often it is worth the effort.
Maybe such a modified test implementation should be distributed with all
libraries, where there is a serious risk of race-conditions in using the
API.  On the other hand, if you know that much, when you create a
library, you might as well try to write the API so it (to the extent
possible) lets the user avoid race-conditions completely.

Greetings,

Jacob
-- 
"I remember being impressed with Ada because you could write
 an infinite loop without a faked up condition. The idea
 being that in Ada the typical infinite loop would normally
 be terminated by detonation."                 -- Larry Wall

Thread

arm-gcc: pointer to constant string pozz <pozzugno@gmail.com> - 2018-09-14 09:50 +0200
  Re: arm-gcc: pointer to constant string David Brown <david.brown@hesbynett.no> - 2018-09-14 11:18 +0200
    Re: arm-gcc: pointer to constant string pozz <pozzugno@gmail.com> - 2018-09-14 13:05 +0200
      Re: arm-gcc: pointer to constant string David Brown <david.brown@hesbynett.no> - 2018-09-14 13:25 +0200
  Re: arm-gcc: pointer to constant string Nils M Holm <nmh@ananda.local> - 2018-09-14 09:47 +0000
    Re: arm-gcc: pointer to constant string David Brown <david.brown@hesbynett.no> - 2018-09-14 12:26 +0200
      Re: arm-gcc: pointer to constant string Stef <stef33d@yahooI-N-V-A-L-I-D.com.invalid> - 2018-09-14 14:52 +0200
        Re: arm-gcc: pointer to constant string David Brown <david.brown@hesbynett.no> - 2018-09-14 16:04 +0200
          Re: arm-gcc: pointer to constant string Tom Gardner <spamjunk@blueyonder.co.uk> - 2018-09-14 15:48 +0100
            Re: arm-gcc: pointer to constant string David Brown <david.brown@hesbynett.no> - 2018-09-14 17:02 +0200
              Re: arm-gcc: pointer to constant string Tom Gardner <spamjunk@blueyonder.co.uk> - 2018-09-14 18:45 +0100
                Re: arm-gcc: pointer to constant string David Brown <david.brown@hesbynett.no> - 2018-09-14 20:37 +0200
                Re: arm-gcc: pointer to constant string Tom Gardner <spamjunk@blueyonder.co.uk> - 2018-09-14 20:51 +0100
                Re: arm-gcc: pointer to constant string David Brown <david.brown@hesbynett.no> - 2018-09-15 16:24 +0200
                Re: arm-gcc: pointer to constant string Tom Gardner <spamjunk@blueyonder.co.uk> - 2018-09-15 17:28 +0100
                Re: arm-gcc: pointer to constant string Clifford Heath <no.spam@please.net> - 2018-09-16 13:58 +1000
                Re: arm-gcc: pointer to constant string Tom Gardner <spamjunk@blueyonder.co.uk> - 2018-09-16 08:42 +0100
                Re: arm-gcc: pointer to constant string Tom Gardner <spamjunk@blueyonder.co.uk> - 2018-09-16 08:47 +0100
                Re: arm-gcc: pointer to constant string Clifford Heath <no.spam@please.net> - 2018-09-16 20:18 +1000
                Re: arm-gcc: pointer to constant string Tom Gardner <spamjunk@blueyonder.co.uk> - 2018-09-16 12:25 +0100
                Re: arm-gcc: pointer to constant string Clifford Heath <no.spam@please.net> - 2018-09-16 22:22 +1000
                Re: arm-gcc: pointer to constant string David Brown <david.brown@hesbynett.no> - 2018-09-16 15:00 +0200
                Re: arm-gcc: pointer to constant string Tom Gardner <spamjunk@blueyonder.co.uk> - 2018-09-16 14:33 +0100
                Re: arm-gcc: pointer to constant string David Brown <david.brown@hesbynett.no> - 2018-09-16 22:06 +0200
                Re: arm-gcc: pointer to constant string Tom Gardner <spamjunk@blueyonder.co.uk> - 2018-09-16 21:42 +0100
                Re: arm-gcc: pointer to constant string Les Cargill <lcargill99@comcast.com> - 2018-09-17 21:24 -0500
                Re: arm-gcc: pointer to constant string David Brown <david.brown@hesbynett.no> - 2018-09-18 08:49 +0200
                Re: arm-gcc: pointer to constant string Les Cargill <lcargill99@comcast.com> - 2018-09-21 22:15 -0500
                Re: arm-gcc: pointer to constant string David Brown <david.brown@hesbynett.no> - 2018-09-22 11:53 +0200
                Testing (Was: arm-gcc: pointer to constant string) Jacob Sparre Andersen <jacob@jacob-sparre.dk> - 2018-09-22 12:16 +0200
                Re: Testing (Was: arm-gcc: pointer to constant string) David Brown <david.brown@hesbynett.no> - 2018-09-22 13:31 +0200
                Re: Testing (Was: arm-gcc: pointer to constant string) Clifford Heath <no.spam@please.net> - 2018-09-23 09:26 +1000
                Re: arm-gcc: pointer to constant string Niklas Holsti <niklas.holsti@tidorum.invalid> - 2018-09-22 20:06 +0300
                Re: arm-gcc: pointer to constant string David Brown <david.brown@hesbynett.no> - 2018-09-23 14:39 +0200
                Re: arm-gcc: pointer to constant string Les Cargill <lcargill99@comcast.com> - 2018-09-22 13:50 -0500
                Re: arm-gcc: pointer to constant string David Brown <david.brown@hesbynett.no> - 2018-09-23 15:21 +0200
                Re: arm-gcc: pointer to constant string Tom Gardner <spamjunk@blueyonder.co.uk> - 2018-09-23 16:06 +0100
                Re: arm-gcc: pointer to constant string julio@diegidio.name - 2018-09-23 08:41 -0700
                Re: arm-gcc: pointer to constant string Tom Gardner <spamjunk@blueyonder.co.uk> - 2018-09-23 16:49 +0100
                Re: arm-gcc: pointer to constant string julio@diegidio.name - 2018-09-23 08:56 -0700
                Re: arm-gcc: pointer to constant string Tom Gardner <spamjunk@blueyonder.co.uk> - 2018-09-23 18:20 +0100
                Re: arm-gcc: pointer to constant string julio@diegidio.name - 2018-09-23 12:05 -0700
                Re: arm-gcc: pointer to constant string Tom Gardner <spamjunk@blueyonder.co.uk> - 2018-09-23 20:50 +0100
                Re: arm-gcc: pointer to constant string Les Cargill <lcargill99@comcast.com> - 2018-09-23 12:04 -0500
                Re: arm-gcc: pointer to constant string Tom Gardner <spamjunk@blueyonder.co.uk> - 2018-09-23 18:27 +0100
                Re: arm-gcc: pointer to constant string David Brown <david.brown@hesbynett.no> - 2018-09-23 19:37 +0200
                Re: arm-gcc: pointer to constant string Jacob Sparre Andersen <jacob@jacob-sparre.dk> - 2018-09-24 12:53 +0200
                Re: arm-gcc: pointer to constant string David Brown <david.brown@hesbynett.no> - 2018-09-24 13:49 +0200
                Re: arm-gcc: pointer to constant string Les Cargill <lcargill99@comcast.com> - 2018-09-28 20:17 -0500
                Re: arm-gcc: pointer to constant string Tom Gardner <spamjunk@blueyonder.co.uk> - 2018-09-18 08:32 +0100
                Re: arm-gcc: pointer to constant string David Brown <david.brown@hesbynett.no> - 2018-09-18 10:02 +0200
                Re: arm-gcc: pointer to constant string Les Cargill <lcargill99@comcast.com> - 2018-09-21 22:17 -0500
          Re: arm-gcc: pointer to constant string Stef <stef33d@yahooI-N-V-A-L-I-D.com.invalid> - 2018-09-14 18:30 +0200
            Re: arm-gcc: pointer to constant string David Brown <david.brown@hesbynett.no> - 2018-09-14 20:46 +0200
      Re: arm-gcc: pointer to constant string Stefan Reuther <stefan.news@arcor.de> - 2018-09-14 19:57 +0200
        Re: arm-gcc: pointer to constant string David Brown <david.brown@hesbynett.no> - 2018-09-14 20:56 +0200
          Re: arm-gcc: pointer to constant string Stefan Reuther <stefan.news@arcor.de> - 2018-09-15 18:32 +0200
            Re: arm-gcc: pointer to constant string David Brown <david.brown@hesbynett.no> - 2018-09-16 15:06 +0200
      Re: arm-gcc: pointer to constant string Richard Damon <Richard@Damon-Family.org> - 2018-09-14 19:59 -0400
        Re: arm-gcc: pointer to constant string David Brown <david.brown@hesbynett.no> - 2018-09-15 16:25 +0200
  Re: arm-gcc: pointer to constant string Tauno Voipio <tauno.voipio@notused.fi.invalid> - 2018-09-14 13:49 +0300

csiph-web