Re: Regression testing for pointers

From	Don Y <this@isnotme.com>
Newsgroups	comp.lang.c
Subject	Re: Regression testing for pointers
Date	2012-03-13 01:00 -0700
Organization	Aioe.org NNTP Server
Message-ID	<jjmurk$ohs$1@speranza.aioe.org> (permalink)
References	(9 earlier) <9s6t8nFa8vU6@mid.individual.net> <jjm0hj$vtl$1@speranza.aioe.org> <9s80hcFhvvU1@mid.individual.net> <jjmser$jnv$1@speranza.aioe.org> <9s8bjmFhvvU5@mid.individual.net>

Show all headers | View raw

On 3/13/2012 12:34 AM, Ian Collins wrote:

>>>> So I -- and anyone modifying the test suite -- should be diligent
>>>> enough to verify all of these issues INSTEAD OF something like:
>>>> ASSERT(result == 0x80032440)
>>>> which performs *all* of these "additional tests". If result is
>>>> *not* 0x80032440 then one of the other enumerated issues was
>>>> not satisfied!
>>>
>>> If you have a condition you want to test, test it.
>>>
>>>> If you were testing strchr(3), would you do:
>>>> template = "ABCDEABCFF
>>>> sought = 'C'
>>>> result = strchr(template, sought)
>>>> ASSERT(result>=&template[0])
>>>> ASSERT(result< &template[strlen(template)])
>>>> ASSERT(*result == sought)
>>>> ASSERT(result[1] == sought+1)
>>>>
>>>> Or, would you do:
>>>> ASSERT(result ==&template[2])
>>>>
>>>>>> This embodies *all* of the above tests and gives the
>>>>>> function far less "wiggle room".
>>>
>>> That doesn't make any sense and bears no resemblance to the questions
>>> you asked and subsequently snipped.
>>
>> If your comment applies to the strchr example, that is intended
>> to show that you can present lots of tests/criteria to *infer*
>> that the result is correct. *OR*, you could just CHECK THE
>> RESULT! You *know* what the conditions are when the test
>> is executed (i.e., you know what the resolution of your math
>> library is, you know the value(s) of the inputs to the routines
>> under test) so all you have to do is verify that the *result*
>> is EXACTLY what you expect it to be. No need to look at
>> indirect evidence when you have a concrete result that you
>> can test!
>
> Your typical memory allocator is a little more complex and less
> deterministic than strchr().

But it still produces *a* result.  It is a deterministic
algorithm (even if the specification -- like malloc -- doesn't
make that determinism explicit).  With a given set of operating
conditions and inputs, you **KNOW** what the result will be.
TEST FOR THAT RESULT!

>>> You added additional requirements, I
>>> suggested you add tests for them.
>>
>> ASSERT(result == 0x80032440)
>>
>> exactly and unequivocally proves ALL CONDITIONS are met.
>> Specifically, it states (knowing the conditions in which
>> the test suite executed) that:
>> - the request was satisfied (i.e., the heap's condition
>> at the time was such that the criteria associated with
>> the request could be met)
>> - the *correct* fragment in the free list was selected
>> (because I *knew* what the free list should have contained)
>> - the correct *portion* of the selected fragment was chosen
>> to be returned to the caller
>> - the correct *alignment* was enforced
>> - the correct size chunk was returned to the caller
>> - the internal state of the free list remains consistent
>> (as will be evidenced in subsequent requests)
>>
>> All of those facts are expressed in that single assertion.
>> Change any digit in the constant shown and one or more of
>> these facts are no longer true.
>
> Nowhere in your earlier descriptions was that requirement mentioned.

Because I didn't consider it germane to the concept of
"testing pointer values".  The specification is 15 pages.

> I thought you were test a general purpose allocator.
>
> If your requirements are that specific, what's wrong with
>
> ASSERT(result == heapBase+0x32440)?

What happens if heapBase is effectively (char *) -32400?
What happens if the processor only has a 16b address space?
What happens if that +32440 was a consequence of pointer
alignment requirements on THIS PLATFORM?
What happens if there isn't physical memory present at
that location?

> The base address can still be flexible.
>
>> I don't have to, separately, test:
>> - did the function return non-NULL? OK, then the request
>> was satisfied
>> - did the function select the LASTFIT fragment as directed
>> (how do you know without peeking inside the free list?)
>> - did the function pick the right *part* of that fragment
>> to use in satisfying the request (how do you know if
>> you can't see the actual bounds of the fragment BEFORE
>> it was selected/manipulated?)
>> - did the request get properly aligned per the requirements
>> of the processor and compiler (again, without seeing the
>> pre-selected fragment, how do you know that any "work"
>> was done on that selection?)
>> - did the right amount of "excess" get trimmed off the
>> fragment prior to being returned
>> - what are the chances that the new state of the free list
>> is as it should be? how many more of these questions do
>> I need to answer in the next N example invocations to
>> assure myself of that?
>>
>> *And*, any other criteria embedded in the function now
>> or in the future are also proven by this.
>
> I would probably test the competent functions individually (those
> working with slabs) and mock them in the allocator tests. If you were to
> build your allocator test first, the design would end up with wouldn't
> require tests that poke around in the internal structures.

The approach I have taken (to testing on *one particular* platform)
eliminates the need for "peeking inside".

If I know where (on this particular platform) the heap is located
and how large it is and alignment constraints, etc.  AND have just
initialized the heap... THEN:
- I know what the internal free list looks like, "conceptually"
   (I might not know specific bit patterns but I know what the
   free list *reflects* -- a heap that is completely FULL!
- given that free list, I know how any particular allocation
   request WILL be handled.
- given how that request is handled, I now know what the NEW
   state of the free list will be.
- lather, rinse, repeat

And, all I need to do to convince myself that my idea of what's
happening "under the covers" agrees with what is *really* under
the covers is to repeat this process a sufficient number of times
and in an appropriately clever order to remove any doubt.

Thread

Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-09 16:43 -0700
  Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 12:56 +1300
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-09 19:40 -0700
      Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 17:53 +1300
        Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 18:12 +1300
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 02:05 -0700
          Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 23:17 +1300
            Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-10 12:00 +0000
              Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 11:00 +1300
                Re: Regression testing for pointers Ike Naar <ike@iceland.freeshell.org> - 2012-03-10 22:23 +0000
                Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-10 22:40 +0000
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 12:12 +1300
                Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-11 00:42 +0000
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 14:26 +1300
                Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-11 03:09 +0000
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 18:16 +1300
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 12:01 -0700
              Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 10:35 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 09:19 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 07:23 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 16:23 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 17:25 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 00:19 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 20:34 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 01:00 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 21:22 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 10:19 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 07:25 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 02:00 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 22:18 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 11:32 -0700
                Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-15 05:58 -0700
                Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-15 05:55 -0700
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-15 11:32 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-16 18:14 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-20 01:43 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-20 21:59 +1300
                Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-16 03:05 -0700
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-20 02:33 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-24 11:49 +1300
          Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-11 05:01 -0700
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 09:44 -0700
              Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-12 09:56 +1300
    Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-10 02:28 -0800
      Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 11:09 -0700
      Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-10 18:29 +0000
        Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-11 05:22 -0700
          Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 09:50 -0700
            Re: Regression testing for pointers "io_x" <a@b.c.invalid> - 2012-03-13 13:09 +0100
  Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-10 03:14 +0000
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 12:42 -0700
  Re: Regression testing for pointers Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-10 08:42 +0000
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 02:19 -0700
      Re: Regression testing for pointers "BartC" <bc@freeuk.com> - 2012-03-10 15:54 +0000
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 11:49 -0700
          Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-11 06:09 -0700
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 12:33 -0700
              Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-12 03:29 -0700
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 10:01 -0700
                Re: Regression testing for pointers "BartC" <bc@freeuk.com> - 2012-03-13 18:56 +0000
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 16:31 -0700
      Re: Regression testing for pointers Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-12 00:30 +0000
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 16:41 -0700
      Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-12 13:39 +1300
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 09:36 -0700
          Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 07:25 +1300
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 16:28 -0700
              Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-13 04:20 +0000
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 17:34 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 00:29 -0700
              Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 17:31 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 01:05 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 21:30 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 10:43 -0700
  Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-10 15:41 -0800
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 11:23 -0700
      Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-13 13:43 -0700
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 01:45 -0700
          Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-14 15:15 -0700
            Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-14 22:57 +0000
              Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-15 13:10 -0700
                Re: Regression testing for pointers jgk@panix.com (Joe keane) - 2012-03-16 22:20 +0000
                Re: Regression testing for pointers "io_x" <a@b.c.invalid> - 2012-03-18 07:15 +0100
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 23:13 -0700
  Re: Regression testing for pointers gordonb.b29ri@burditt.org (Gordon Burditt) - 2012-03-12 19:37 -0500
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 19:23 -0700
  Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 13:12 +1300
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 01:54 -0700
      Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 22:05 +1300
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 11:54 -0700

csiph-web