Re: C unit testing and regression testing

From	Ian Collins <ian-news@hotmail.com>
Newsgroups	comp.lang.c
Subject	Re: C unit testing and regression testing
Date	2013-08-09 18:08 +1200
Message-ID	<b6jfb2FgltmU2@mid.individual.net> (permalink)
References	<ku0i0g$e0v$1@dont-email.me> <5203D55E.9060202@verizon.net> <d9fb0feb-1d3d-446a-81af-bf662462f50f@googlegroups.com> <5204008B.6090107@verizon.net> <52c3a48c-f03c-42b7-b7a6-105c4adf526f@googlegroups.com>

Show all headers | View raw

Malcolm McLean wrote:
> On Thursday, August 8, 2013 9:33:15 PM UTC+1, James Kuyper wrote:
>> On 08/08/2013 02:49 PM, Malcolm McLean wrote:
>>
>> For my group, "adequate" testing has been officially defined as testing
>> that each branch of the code gets exercised by at least one of the test
>> cases, and that it has been confirmed that the expected test results
>> have been achieved for each of those cases, and that the test plan has
>> been designed to make sure that test cases corresponding to different
>> branches have distinguishable expected results (I'm amazed at how often
>> people writing test plans forget that last issue).
>>
> Your group might have officially defined the term "adequate testing". But
> most of us don't work for your group, and are unlikely to adopt the
> definition.
>
> Coverage is a reasonable criterion, however. It doesn't prove a program
> is correct, because you've got a combinatorial problem with branch
> points. But it will catch most bugs.
> However a lot of code has branch points for memory allocation failures
> which are extremely unlikely to happen. You could argue that if it's worth
> handle the allocation failure, it's also worth testing it. But it does
> increase the difficulty of testing considerably - either you've got to
> alter the source code, or you need a special malloc package, or you need
> to fiddle with a debugger.

Or you use a testing framework that can mock malloc and friends.

> You can make a strong case that not going to
> the trouble is "adequate".

Not really, if the code isn't covered by a test, it shouldn't be there. 
  One of my teams used to pay the testers who did the product acceptance 
testing in beer if they found bugs in our code.  Most "bugs" turned out 
to be ambiguities in the specification.

-- 
Ian Collins

Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

C unit testing and regression testing "James Harris" <james.harris.1@gmail.com> - 2013-08-08 17:46 +0100
  Re: C unit testing and regression testing James Kuyper <jameskuyper@verizon.net> - 2013-08-08 13:29 -0400
    Re: C unit testing and regression testing Malcolm McLean <malcolm.mclean5@btinternet.com> - 2013-08-08 11:49 -0700
      Re: C unit testing and regression testing James Kuyper <jameskuyper@verizon.net> - 2013-08-08 16:33 -0400
        Re: C unit testing and regression testing Malcolm McLean <malcolm.mclean5@btinternet.com> - 2013-08-08 14:55 -0700
          Re: C unit testing and regression testing James Kuyper <jameskuyper@verizon.net> - 2013-08-08 18:35 -0400
            Re: C unit testing and regression testing "James Harris" <james.harris.1@gmail.com> - 2013-08-09 07:34 +0100
          Re: C unit testing and regression testing Ian Collins <ian-news@hotmail.com> - 2013-08-09 18:08 +1200
            Re: C unit testing and regression testing Malcolm McLean <malcolm.mclean5@btinternet.com> - 2013-08-09 03:16 -0700
      Re: C unit testing and regression testing Ian Collins <ian-news@hotmail.com> - 2013-08-09 18:04 +1200
    Re: C unit testing and regression testing "James Harris" <james.harris.1@gmail.com> - 2013-08-09 07:30 +0100
  Re: C unit testing and regression testing Les Cargill <lcargill99@comcast.com> - 2013-08-08 12:54 -0500
    Re: C unit testing and regression testing "James Harris" <james.harris.1@gmail.com> - 2013-08-09 07:37 +0100
      Re: C unit testing and regression testing Les Cargill <lcargill99@comcast.com> - 2013-08-09 07:46 -0500
  Re: C unit testing and regression testing Jorgen Grahn <grahn+nntp@snipabacken.se> - 2013-08-08 18:55 +0000
    Re: C unit testing and regression testing "James Harris" <james.harris.1@gmail.com> - 2013-08-09 07:58 +0100
      Re: C unit testing and regression testing Ian Collins <ian-news@hotmail.com> - 2013-08-09 19:54 +1200
        Re: C unit testing and regression testing "James Harris" <james.harris.1@gmail.com> - 2013-08-13 12:35 +0100
          Expect (was Re: C unit testing and regression testing) Jorgen Grahn <grahn+nntp@snipabacken.se> - 2013-08-13 19:10 +0000
      Re: C unit testing and regression testing Jorgen Grahn <grahn+nntp@snipabacken.se> - 2013-08-09 11:54 +0000
        Re: C unit testing and regression testing "James Harris" <james.harris.1@gmail.com> - 2013-08-13 12:45 +0100
          Re: C unit testing and regression testing Jorgen Grahn <grahn+nntp@snipabacken.se> - 2013-08-13 19:50 +0000
            Re: C unit testing and regression testing "James Harris" <james.harris.1@gmail.com> - 2013-08-14 10:14 +0100
              Re: C unit testing and regression testing Jorgen Grahn <grahn+nntp@snipabacken.se> - 2013-08-14 19:44 +0000
                Re: C unit testing and regression testing "James Harris" <james.harris.1@gmail.com> - 2013-08-15 13:10 +0100
                Re: C unit testing and regression testing Jorgen Grahn <grahn+nntp@snipabacken.se> - 2013-08-17 07:32 +0000
          Re: C unit testing and regression testing Ian Collins <ian-news@hotmail.com> - 2013-08-14 10:58 +1200
            Re: C unit testing and regression testing "James Harris" <james.harris.1@gmail.com> - 2013-08-14 10:25 +0100
              Re: C unit testing and regression testing Ian Collins <ian-news@hotmail.com> - 2013-08-14 21:53 +1200
  Re: C unit testing and regression testing Ian Collins <ian-news@hotmail.com> - 2013-08-09 07:17 +1200
    Re: C unit testing and regression testing Bart van Ingen Schenau <bart@ingen.ddns.info.invalid> - 2013-08-09 10:30 +0000
      Re: C unit testing and regression testing Ian Collins <ian-news@hotmail.com> - 2013-08-10 08:21 +1200
  Re: C unit testing and regression testing <william@wilbur.25thandClement.com> - 2013-08-08 13:37 -0700
    Re: C unit testing and regression testing Jorgen Grahn <grahn+nntp@snipabacken.se> - 2013-08-08 22:52 +0000
      Re: C unit testing and regression testing "James Harris" <james.harris.1@gmail.com> - 2013-08-09 08:17 +0100
        Re: C unit testing and regression testing Jorgen Grahn <grahn+nntp@snipabacken.se> - 2013-08-09 09:34 +0000
        Re: C unit testing and regression testing Les Cargill <lcargill99@comcast.com> - 2013-08-09 07:52 -0500
  Re: C unit testing and regression testing Siri Cruise <chine.bleu@yahoo.com> - 2013-08-09 05:29 -0700
  Re: C unit testing and regression testing Roberto Waltman <usenet@rwaltman.com> - 2013-08-11 20:19 -0400
    Test-driven development (was Re: C unit testing and regression testing) Jorgen Grahn <grahn+nntp@snipabacken.se> - 2013-08-17 07:17 +0000
      Re: Test-driven development (was Re: C unit testing and regression   testing) Ian Collins <ian-news@hotmail.com> - 2013-08-17 19:40 +1200
      Re: Test-driven development (was Re: C unit testing and regression testing) Les Cargill <lcargill99@comcast.com> - 2013-08-17 11:40 -0500

csiph-web