Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #18651

Re: Regression testing for pointers

Show key headers only | View raw

On 03/13/12 09:00 PM, Don Y wrote:
> On 3/13/2012 12:34 AM, Ian Collins wrote:
>
>>>>> So I -- and anyone modifying the test suite -- should be diligent
>>>>> enough to verify all of these issues INSTEAD OF something like:
>>>>> ASSERT(result == 0x80032440)
>>>>> which performs *all* of these "additional tests". If result is
>>>>> *not* 0x80032440 then one of the other enumerated issues was
>>>>> not satisfied!
>>>>
>>>> If you have a condition you want to test, test it.
>>>>
>>>>> If you were testing strchr(3), would you do:
>>>>> template = "ABCDEABCFF
>>>>> sought = 'C'
>>>>> result = strchr(template, sought)
>>>>> ASSERT(result>=&template[0])
>>>>> ASSERT(result<  &template[strlen(template)])
>>>>> ASSERT(*result == sought)
>>>>> ASSERT(result[1] == sought+1)
>>>>>
>>>>> Or, would you do:
>>>>> ASSERT(result ==&template[2])
>>>>>
>>>>>>> This embodies *all* of the above tests and gives the
>>>>>>> function far less "wiggle room".
>>>>
>>>> That doesn't make any sense and bears no resemblance to the questions
>>>> you asked and subsequently snipped.
>>>
>>> If your comment applies to the strchr example, that is intended
>>> to show that you can present lots of tests/criteria to *infer*
>>> that the result is correct. *OR*, you could just CHECK THE
>>> RESULT! You *know* what the conditions are when the test
>>> is executed (i.e., you know what the resolution of your math
>>> library is, you know the value(s) of the inputs to the routines
>>> under test) so all you have to do is verify that the *result*
>>> is EXACTLY what you expect it to be. No need to look at
>>> indirect evidence when you have a concrete result that you
>>> can test!
>>
>> Your typical memory allocator is a little more complex and less
>> deterministic than strchr().
>
> But it still produces *a* result.  It is a deterministic
> algorithm (even if the specification -- like malloc -- doesn't
> make that determinism explicit).  With a given set of operating
> conditions and inputs, you **KNOW** what the result will be.
> TEST FOR THAT RESULT!

It may produce a result, but more often than not, the exact value is of 
little on no consequence.  If you start to test for specific values 
based on knowledge of the internals, you end up with a very fragile test 
harness.  I think this may have been what you were saying elsewhere.  If 
your tests become bonded to single implementation, you are stuck with 
it.  If your tests don't assume specific values, you are free to change 
the internals of the function as long as you don't break its published 
behaviour.  Obviously if your internal implementation is specified you 
are stuck.

Memory allocators are definitely one of those cases where one size does 
not fit all, just look at the range of malloc libraries available for a 
typical non-embedded environment.  They would all pass the same test 
suite assuming the tests didn't make assumptions about the implementation.

>> I thought you were test a general purpose allocator.
>>
>> If your requirements are that specific, what's wrong with
>>
>> ASSERT(result == heapBase+0x32440)?
>
> What happens if heapBase is effectively (char *) -32400?
> What happens if the processor only has a 16b address space?
> What happens if that +32440 was a consequence of pointer
> alignment requirements on THIS PLATFORM?
> What happens if there isn't physical memory present at
> that location?

You break 0x32440 down into platform independent variables.

>> The base address can still be flexible.
>>
>>> I don't have to, separately, test:
>>> - did the function return non-NULL? OK, then the request
>>> was satisfied
>>> - did the function select the LASTFIT fragment as directed
>>> (how do you know without peeking inside the free list?)
>>> - did the function pick the right *part* of that fragment
>>> to use in satisfying the request (how do you know if
>>> you can't see the actual bounds of the fragment BEFORE
>>> it was selected/manipulated?)
>>> - did the request get properly aligned per the requirements
>>> of the processor and compiler (again, without seeing the
>>> pre-selected fragment, how do you know that any "work"
>>> was done on that selection?)
>>> - did the right amount of "excess" get trimmed off the
>>> fragment prior to being returned
>>> - what are the chances that the new state of the free list
>>> is as it should be? how many more of these questions do
>>> I need to answer in the next N example invocations to
>>> assure myself of that?
>>>
>>> *And*, any other criteria embedded in the function now
>>> or in the future are also proven by this.
>>
>> I would probably test the competent functions individually (those
>> working with slabs) and mock them in the allocator tests. If you were to
>> build your allocator test first, the design would end up with wouldn't
>> require tests that poke around in the internal structures.
>
> The approach I have taken (to testing on *one particular* platform)
> eliminates the need for "peeking inside".
>
> If I know where (on this particular platform) the heap is located
> and how large it is and alignment constraints, etc.  AND have just
> initialized the heap... THEN:
> - I know what the internal free list looks like, "conceptually"
>     (I might not know specific bit patterns but I know what the
>     free list *reflects* -- a heap that is completely FULL!
> - given that free list, I know how any particular allocation
>     request WILL be handled.
> - given how that request is handled, I now know what the NEW
>     state of the free list will be.
> - lather, rinse, repeat

Until someone introduces a critical optimisation that changes the layout...

I still think you are testing at too high a level, test the functions 
that manipulate your internal independently.

-- 
Ian Collins

Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-09 16:43 -0700
  Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 12:56 +1300
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-09 19:40 -0700
      Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 17:53 +1300
        Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 18:12 +1300
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 02:05 -0700
          Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 23:17 +1300
            Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-10 12:00 +0000
              Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 11:00 +1300
                Re: Regression testing for pointers Ike Naar <ike@iceland.freeshell.org> - 2012-03-10 22:23 +0000
                Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-10 22:40 +0000
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 12:12 +1300
                Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-11 00:42 +0000
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 14:26 +1300
                Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-11 03:09 +0000
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 18:16 +1300
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 12:01 -0700
              Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 10:35 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 09:19 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 07:23 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 16:23 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 17:25 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 00:19 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 20:34 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 01:00 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 21:22 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 10:19 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 07:25 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 02:00 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 22:18 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 11:32 -0700
                Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-15 05:58 -0700
                Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-15 05:55 -0700
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-15 11:32 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-16 18:14 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-20 01:43 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-20 21:59 +1300
                Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-16 03:05 -0700
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-20 02:33 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-24 11:49 +1300
          Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-11 05:01 -0700
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 09:44 -0700
              Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-12 09:56 +1300
    Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-10 02:28 -0800
      Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 11:09 -0700
      Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-10 18:29 +0000
        Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-11 05:22 -0700
          Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 09:50 -0700
            Re: Regression testing for pointers "io_x" <a@b.c.invalid> - 2012-03-13 13:09 +0100
  Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-10 03:14 +0000
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 12:42 -0700
  Re: Regression testing for pointers Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-10 08:42 +0000
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 02:19 -0700
      Re: Regression testing for pointers "BartC" <bc@freeuk.com> - 2012-03-10 15:54 +0000
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 11:49 -0700
          Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-11 06:09 -0700
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 12:33 -0700
              Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-12 03:29 -0700
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 10:01 -0700
                Re: Regression testing for pointers "BartC" <bc@freeuk.com> - 2012-03-13 18:56 +0000
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 16:31 -0700
      Re: Regression testing for pointers Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-12 00:30 +0000
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 16:41 -0700
      Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-12 13:39 +1300
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 09:36 -0700
          Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 07:25 +1300
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 16:28 -0700
              Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-13 04:20 +0000
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 17:34 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 00:29 -0700
              Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 17:31 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 01:05 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 21:30 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 10:43 -0700
  Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-10 15:41 -0800
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 11:23 -0700
      Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-13 13:43 -0700
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 01:45 -0700
          Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-14 15:15 -0700
            Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-14 22:57 +0000
              Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-15 13:10 -0700
                Re: Regression testing for pointers jgk@panix.com (Joe keane) - 2012-03-16 22:20 +0000
                Re: Regression testing for pointers "io_x" <a@b.c.invalid> - 2012-03-18 07:15 +0100
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 23:13 -0700
  Re: Regression testing for pointers gordonb.b29ri@burditt.org (Gordon Burditt) - 2012-03-12 19:37 -0500
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 19:23 -0700
  Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 13:12 +1300
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 01:54 -0700
      Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 22:05 +1300
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 11:54 -0700

csiph-web