Re: Regression testing for pointers

From	Ian Collins <ian-news@hotmail.com>
Newsgroups	comp.lang.c
Subject	Re: Regression testing for pointers
Date	2012-03-13 20:34 +1300
Message-ID	<9s8bjmFhvvU5@mid.individual.net> (permalink)
References	(8 earlier) <jjl7n8$mt4$1@speranza.aioe.org> <9s6t8nFa8vU6@mid.individual.net> <jjm0hj$vtl$1@speranza.aioe.org> <9s80hcFhvvU1@mid.individual.net> <jjmser$jnv$1@speranza.aioe.org>

Show all headers | View raw

On 03/13/12 08:19 PM, Don Y wrote:
> On 3/12/2012 9:25 PM, Ian Collins wrote:
>> On 03/13/12 12:23 PM, Don Y wrote:
>>> Hi Ian,
>>>
>>> On 3/12/2012 11:23 AM, Ian Collins wrote:
>>>
>>>> So add a test.
>>>
>>>> So add a test.
>>>
>>>> So add a test.
>>>
>>>> So add a test.
>>>
>>>> So add a test.
>>>
>>> So I -- and anyone modifying the test suite -- should be diligent
>>> enough to verify all of these issues INSTEAD OF something like:
>>> ASSERT(result == 0x80032440)
>>> which performs *all* of these "additional tests". If result is
>>> *not* 0x80032440 then one of the other enumerated issues was
>>> not satisfied!
>>
>> If you have a condition you want to test, test it.
>>
>>> If you were testing strchr(3), would you do:
>>> template = "ABCDEABCFF
>>> sought = 'C'
>>> result = strchr(template, sought)
>>> ASSERT(result>=&template[0])
>>> ASSERT(result<  &template[strlen(template)])
>>> ASSERT(*result == sought)
>>> ASSERT(result[1] == sought+1)
>>>
>>> Or, would you do:
>>> ASSERT(result ==&template[2])
>>>
>>>>> This embodies *all* of the above tests and gives the
>>>>> function far less "wiggle room".
>>
>> That doesn't make any sense and bears no resemblance to the questions
>> you asked and subsequently snipped.
>
> If your comment applies to the strchr example, that is intended
> to show that you can present lots of tests/criteria to *infer*
> that the result is correct.   *OR*, you could just CHECK THE
> RESULT!  You *know* what the conditions are when the test
> is executed (i.e., you know what the resolution of your math
> library is, you know the value(s) of the inputs to the routines
> under test) so all you have to do is verify that the *result*
> is EXACTLY what you expect it to be.  No need to look at
> indirect evidence when you have a concrete result that you
> can test!

Your typical memory allocator is a little more complex and less 
deterministic than strchr().

>> You added additional requirements, I
>> suggested you add tests for them.
>
> ASSERT(result == 0x80032440)
>
> exactly and unequivocally proves ALL CONDITIONS are met.
> Specifically, it states (knowing the conditions in which
> the test suite executed) that:
> - the request was satisfied (i.e., the heap's condition
>     at the time was such that the criteria associated with
>     the request could be met)
> - the *correct* fragment in the free list was selected
>     (because I *knew* what the free list should have contained)
> - the correct *portion* of the selected fragment was chosen
>     to be returned to the caller
> - the correct *alignment* was enforced
> - the correct size chunk was returned to the caller
> - the internal state of the free list remains consistent
>     (as will be evidenced in subsequent requests)
>
> All of those facts are expressed in that single assertion.
> Change any digit in the constant shown and one or more of
> these facts are no longer true.

Nowhere in your earlier descriptions was that requirement mentioned.  I 
thought you were test a general purpose allocator.

If your requirements are that specific, what's wrong with

ASSERT(result == heapBase+0x32440)?

The base address can still be flexible.

> I don't have to, separately, test:
> - did the function return non-NULL?  OK, then the request
>     was satisfied
> - did the function select the LASTFIT fragment as directed
>     (how do you know without peeking inside the free list?)
> - did the function pick the right *part* of that fragment
>     to use in satisfying the request (how do you know if
>     you can't see the actual bounds of the fragment BEFORE
>     it was selected/manipulated?)
> - did the request get properly aligned per the requirements
>     of the processor and compiler (again, without seeing the
>     pre-selected fragment, how do you know that any "work"
>     was done on that selection?)
> - did the right amount of "excess" get trimmed off the
>     fragment prior to being returned
> - what are the chances that the new state of the free list
>     is as it should be?  how many more of these questions do
>     I need to answer in the next N example invocations to
>     assure myself of that?
>
> *And*, any other criteria embedded in the function now
> or in the future are also proven by this.

I would probably test the competent functions individually (those 
working with slabs) and mock them in the allocator tests.  If you were 
to build your allocator test first, the design would end up with 
wouldn't require tests that poke around in the internal structures.

-- 
Ian Collins

Thread

Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-09 16:43 -0700
  Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 12:56 +1300
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-09 19:40 -0700
      Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 17:53 +1300
        Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 18:12 +1300
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 02:05 -0700
          Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 23:17 +1300
            Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-10 12:00 +0000
              Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 11:00 +1300
                Re: Regression testing for pointers Ike Naar <ike@iceland.freeshell.org> - 2012-03-10 22:23 +0000
                Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-10 22:40 +0000
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 12:12 +1300
                Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-11 00:42 +0000
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 14:26 +1300
                Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-11 03:09 +0000
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 18:16 +1300
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 12:01 -0700
              Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 10:35 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 09:19 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 07:23 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 16:23 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 17:25 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 00:19 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 20:34 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 01:00 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 21:22 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 10:19 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 07:25 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 02:00 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 22:18 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 11:32 -0700
                Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-15 05:58 -0700
                Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-15 05:55 -0700
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-15 11:32 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-16 18:14 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-20 01:43 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-20 21:59 +1300
                Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-16 03:05 -0700
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-20 02:33 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-24 11:49 +1300
          Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-11 05:01 -0700
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 09:44 -0700
              Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-12 09:56 +1300
    Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-10 02:28 -0800
      Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 11:09 -0700
      Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-10 18:29 +0000
        Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-11 05:22 -0700
          Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 09:50 -0700
            Re: Regression testing for pointers "io_x" <a@b.c.invalid> - 2012-03-13 13:09 +0100
  Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-10 03:14 +0000
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 12:42 -0700
  Re: Regression testing for pointers Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-10 08:42 +0000
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 02:19 -0700
      Re: Regression testing for pointers "BartC" <bc@freeuk.com> - 2012-03-10 15:54 +0000
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 11:49 -0700
          Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-11 06:09 -0700
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 12:33 -0700
              Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-12 03:29 -0700
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 10:01 -0700
                Re: Regression testing for pointers "BartC" <bc@freeuk.com> - 2012-03-13 18:56 +0000
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 16:31 -0700
      Re: Regression testing for pointers Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-12 00:30 +0000
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 16:41 -0700
      Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-12 13:39 +1300
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 09:36 -0700
          Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 07:25 +1300
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 16:28 -0700
              Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-13 04:20 +0000
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 17:34 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 00:29 -0700
              Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 17:31 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 01:05 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 21:30 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 10:43 -0700
  Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-10 15:41 -0800
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 11:23 -0700
      Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-13 13:43 -0700
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 01:45 -0700
          Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-14 15:15 -0700
            Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-14 22:57 +0000
              Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-15 13:10 -0700
                Re: Regression testing for pointers jgk@panix.com (Joe keane) - 2012-03-16 22:20 +0000
                Re: Regression testing for pointers "io_x" <a@b.c.invalid> - 2012-03-18 07:15 +0100
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 23:13 -0700
  Re: Regression testing for pointers gordonb.b29ri@burditt.org (Gordon Burditt) - 2012-03-12 19:37 -0500
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 19:23 -0700
  Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 13:12 +1300
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 01:54 -0700
      Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 22:05 +1300
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 11:54 -0700

csiph-web