Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #35143

Re: C unit testing and regression testing

Show all headers | View raw

On 08/08/2013 02:49 PM, Malcolm McLean wrote:
> On Thursday, August 8, 2013 6:29:02 PM UTC+1, James Kuyper wrote:
>> On 08/08/2013 12:46 PM, James Harris wrote:
>>
>>> I am looking for an approach in preference to a package or a framework. 
>>> Simple and adequate is better than comprehensive and complex.
>>
>> For me, "simple" and "adequate" would be inherently conflicting goals,
>> while "comprehensive" would be implied by "adequate" - but it all
>> depends upon what you mean by those terms.
>>
> Adequate means that a bug might slip through, but the software is likely still
> to be usable.

With very few exceptions, for my software, most of the things that could
possibly go wrong with it render its products unusable. It is possible
for it to contain errors too small to matter, but most of the plausible
ways a defect could occur cause unacceptable errors.

> ... Comprehensive would mean that, as far as humanly impossible, 
> all bugs are caught, regardless of expense. ...

The amount of expense that could be expended by humans to catch all bugs
in a piece of software has no upper limit that I'm aware of, just an
unbounded increase in the marginal cost per bug caught as the expense
gets higher.

> ... Comprehensive only equates to
> adequate if any bug at all renders the software unacceptable, which might
> be the case in a life support system (but only if the chance of a software
> failure is of the same or greater order of magnitude as the chance of a 
> hardware component failing).

For my group, "adequate" testing has been officially defined as testing
that each branch of the code gets exercised by at least one of the test
cases, and that it has been confirmed that the expected test results
have been achieved for each of those cases, and that the test plan has
been designed to make sure that test cases corresponding to different
branches have distinguishable expected results (I'm amazed at how often
people writing test plans forget that last issue).

I also consider that fairly comprehensive testing, even though it is in
fact humanly possible to do more to catch bugs, if we could afford to
spend the time needed to do so. We can't quite afford the time needed to
do "adequate" testing, as defined above.

Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

C unit testing and regression testing "James Harris" <james.harris.1@gmail.com> - 2013-08-08 17:46 +0100
  Re: C unit testing and regression testing James Kuyper <jameskuyper@verizon.net> - 2013-08-08 13:29 -0400
    Re: C unit testing and regression testing Malcolm McLean <malcolm.mclean5@btinternet.com> - 2013-08-08 11:49 -0700
      Re: C unit testing and regression testing James Kuyper <jameskuyper@verizon.net> - 2013-08-08 16:33 -0400
        Re: C unit testing and regression testing Malcolm McLean <malcolm.mclean5@btinternet.com> - 2013-08-08 14:55 -0700
          Re: C unit testing and regression testing James Kuyper <jameskuyper@verizon.net> - 2013-08-08 18:35 -0400
            Re: C unit testing and regression testing "James Harris" <james.harris.1@gmail.com> - 2013-08-09 07:34 +0100
          Re: C unit testing and regression testing Ian Collins <ian-news@hotmail.com> - 2013-08-09 18:08 +1200
            Re: C unit testing and regression testing Malcolm McLean <malcolm.mclean5@btinternet.com> - 2013-08-09 03:16 -0700
      Re: C unit testing and regression testing Ian Collins <ian-news@hotmail.com> - 2013-08-09 18:04 +1200
    Re: C unit testing and regression testing "James Harris" <james.harris.1@gmail.com> - 2013-08-09 07:30 +0100
  Re: C unit testing and regression testing Les Cargill <lcargill99@comcast.com> - 2013-08-08 12:54 -0500
    Re: C unit testing and regression testing "James Harris" <james.harris.1@gmail.com> - 2013-08-09 07:37 +0100
      Re: C unit testing and regression testing Les Cargill <lcargill99@comcast.com> - 2013-08-09 07:46 -0500
  Re: C unit testing and regression testing Jorgen Grahn <grahn+nntp@snipabacken.se> - 2013-08-08 18:55 +0000
    Re: C unit testing and regression testing "James Harris" <james.harris.1@gmail.com> - 2013-08-09 07:58 +0100
      Re: C unit testing and regression testing Ian Collins <ian-news@hotmail.com> - 2013-08-09 19:54 +1200
        Re: C unit testing and regression testing "James Harris" <james.harris.1@gmail.com> - 2013-08-13 12:35 +0100
          Expect (was Re: C unit testing and regression testing) Jorgen Grahn <grahn+nntp@snipabacken.se> - 2013-08-13 19:10 +0000
      Re: C unit testing and regression testing Jorgen Grahn <grahn+nntp@snipabacken.se> - 2013-08-09 11:54 +0000
        Re: C unit testing and regression testing "James Harris" <james.harris.1@gmail.com> - 2013-08-13 12:45 +0100
          Re: C unit testing and regression testing Jorgen Grahn <grahn+nntp@snipabacken.se> - 2013-08-13 19:50 +0000
            Re: C unit testing and regression testing "James Harris" <james.harris.1@gmail.com> - 2013-08-14 10:14 +0100
              Re: C unit testing and regression testing Jorgen Grahn <grahn+nntp@snipabacken.se> - 2013-08-14 19:44 +0000
                Re: C unit testing and regression testing "James Harris" <james.harris.1@gmail.com> - 2013-08-15 13:10 +0100
                Re: C unit testing and regression testing Jorgen Grahn <grahn+nntp@snipabacken.se> - 2013-08-17 07:32 +0000
          Re: C unit testing and regression testing Ian Collins <ian-news@hotmail.com> - 2013-08-14 10:58 +1200
            Re: C unit testing and regression testing "James Harris" <james.harris.1@gmail.com> - 2013-08-14 10:25 +0100
              Re: C unit testing and regression testing Ian Collins <ian-news@hotmail.com> - 2013-08-14 21:53 +1200
  Re: C unit testing and regression testing Ian Collins <ian-news@hotmail.com> - 2013-08-09 07:17 +1200
    Re: C unit testing and regression testing Bart van Ingen Schenau <bart@ingen.ddns.info.invalid> - 2013-08-09 10:30 +0000
      Re: C unit testing and regression testing Ian Collins <ian-news@hotmail.com> - 2013-08-10 08:21 +1200
  Re: C unit testing and regression testing <william@wilbur.25thandClement.com> - 2013-08-08 13:37 -0700
    Re: C unit testing and regression testing Jorgen Grahn <grahn+nntp@snipabacken.se> - 2013-08-08 22:52 +0000
      Re: C unit testing and regression testing "James Harris" <james.harris.1@gmail.com> - 2013-08-09 08:17 +0100
        Re: C unit testing and regression testing Jorgen Grahn <grahn+nntp@snipabacken.se> - 2013-08-09 09:34 +0000
        Re: C unit testing and regression testing Les Cargill <lcargill99@comcast.com> - 2013-08-09 07:52 -0500
  Re: C unit testing and regression testing Siri Cruise <chine.bleu@yahoo.com> - 2013-08-09 05:29 -0700
  Re: C unit testing and regression testing Roberto Waltman <usenet@rwaltman.com> - 2013-08-11 20:19 -0400
    Test-driven development (was Re: C unit testing and regression testing) Jorgen Grahn <grahn+nntp@snipabacken.se> - 2013-08-17 07:17 +0000
      Re: Test-driven development (was Re: C unit testing and regression   testing) Ian Collins <ian-news@hotmail.com> - 2013-08-17 19:40 +1200
      Re: Test-driven development (was Re: C unit testing and regression testing) Les Cargill <lcargill99@comcast.com> - 2013-08-17 11:40 -0500

csiph-web