Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #18420 > unrolled thread

Regression testing for pointers

Back to article view | Back to comp.lang.c

Contents

  Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-09 16:43 -0700
    Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 12:56 +1300
      Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-09 19:40 -0700
        Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 17:53 +1300
          Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 18:12 +1300
          Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 02:05 -0700
            Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 23:17 +1300
              Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-10 12:00 +0000
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 11:00 +1300
                  Re: Regression testing for pointers Ike Naar <ike@iceland.freeshell.org> - 2012-03-10 22:23 +0000
                  Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-10 22:40 +0000
                    Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 12:12 +1300
                      Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-11 00:42 +0000
                        Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 14:26 +1300
                          Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-11 03:09 +0000
                            Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 18:16 +1300
              Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 12:01 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 10:35 +1300
                  Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 09:19 -0700
                    Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 07:23 +1300
                      Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 16:23 -0700
                        Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 17:25 +1300
                          Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 00:19 -0700
                            Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 20:34 +1300
                              Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 01:00 -0700
                                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 21:22 +1300
                                  Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 10:19 -0700
                                    Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 07:25 +1300
                                      Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 02:00 -0700
                                        Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 22:18 +1300
                                          Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 11:32 -0700
                                          Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-15 05:58 -0700
                                        Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-15 05:55 -0700
                                          Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-15 11:32 -0700
                                            Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-16 18:14 +1300
                                              Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-20 01:43 -0700
                                                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-20 21:59 +1300
                                            Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-16 03:05 -0700
                                              Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-20 02:33 -0700
                                                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-24 11:49 +1300
            Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-11 05:01 -0700
              Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 09:44 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-12 09:56 +1300
      Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-10 02:28 -0800
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 11:09 -0700
        Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-10 18:29 +0000
          Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-11 05:22 -0700
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 09:50 -0700
              Re: Regression testing for pointers "io_x" <a@b.c.invalid> - 2012-03-13 13:09 +0100
    Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-10 03:14 +0000
      Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 12:42 -0700
    Re: Regression testing for pointers Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-10 08:42 +0000
      Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 02:19 -0700
        Re: Regression testing for pointers "BartC" <bc@freeuk.com> - 2012-03-10 15:54 +0000
          Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 11:49 -0700
            Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-11 06:09 -0700
              Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 12:33 -0700
                Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-12 03:29 -0700
                  Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 10:01 -0700
                    Re: Regression testing for pointers "BartC" <bc@freeuk.com> - 2012-03-13 18:56 +0000
                      Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 16:31 -0700
        Re: Regression testing for pointers Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-12 00:30 +0000
          Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 16:41 -0700
        Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-12 13:39 +1300
          Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 09:36 -0700
            Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 07:25 +1300
              Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 16:28 -0700
                Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-13 04:20 +0000
                  Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 17:34 +1300
                  Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 00:29 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 17:31 +1300
                  Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 01:05 -0700
                    Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 21:30 +1300
                      Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 10:43 -0700
    Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-10 15:41 -0800
      Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 11:23 -0700
        Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-13 13:43 -0700
          Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 01:45 -0700
            Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-14 15:15 -0700
              Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-14 22:57 +0000
                Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-15 13:10 -0700
                  Re: Regression testing for pointers jgk@panix.com (Joe keane) - 2012-03-16 22:20 +0000
                  Re: Regression testing for pointers "io_x" <a@b.c.invalid> - 2012-03-18 07:15 +0100
              Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 23:13 -0700
    Re: Regression testing for pointers gordonb.b29ri@burditt.org (Gordon Burditt) - 2012-03-12 19:37 -0500
      Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 19:23 -0700
    Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 13:12 +1300
      Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 01:54 -0700
        Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 22:05 +1300
          Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 11:54 -0700

Page 1 of 5  [1] 2 3 4 5  Next page →

#18420 — Regression testing for pointers

Hi,

There's rarely a problem generating regression tests for
functions/procedures that use concrete *values*:

sqrt.test

3 --> 1.732
4 --> 2
5 --> 2.236

(apologies if I have misremembered any of these values  :> )

And, it is easy to erect the necessary scaffolding for those
sorts of tests without introducing any noticeable uncertainty
in the veracity of the results obtained.

But, I can't see a way to handle this sort of testing on
functions that manipulate *pointers* to objects -- without
adding a fair bit of code to the scaffolding that, itself,
represents a testing issue!  (i.e., does the scaffolding
have any bugs that result in false positives or negatives??)

In particular, I am trying to develop a test suite for my
dynamic memory management functions (e.g., malloc and free
types of routines).

I.e., I can't come up with "constant" results against
which to compare test-time results.  Sticking with the
traditional malloc/free for example (my routines are
more heavily parameterized), I can create a malloc.test
that pushes size_t's to malloc.  But, aside from verifying
the result is not NULL (or, *is* NULL, in some cases!),
there's nothing I can do to check that the result is
actually correct!

[Recall, the location and size of the free store might change
from environment to environment; alignment constraints might
vary, etc.]

E.g., if a test invocation of malloc(100) returns 0x12345678
in one environment, it could just as easily return 0x1234
in *another* (smaller address space).  A second, identical test
invocation returning 0x87654321 could likewise be "correct".

About the only thing I can check for is if that second
invocation returned 0x12345699 (which overlaps the previous
allocation at 0x12345678!).

Furthermore, just examining the results from malloc don't
tell me that the freelist is being maintained, properly.

Of course, I can write code to inspect these objects in
greater detail.  But, then the test scaffolding starts to
rival the complexity of the functions being tested!  (bugs)

I guess the "ideal" I see would be to be able to build a
"memory image" and verify that the functions cause expected
and predicted changes in that "image".

E.g., as the allocation policy is varied, to see the
effects of that reflected in the changes to the free list
before vs. after the invocation.

But, I can't see any way to make this NOT require some sort
of active probing of memory by the test scaffolding (which
means the internals of the functions have to be exposed to
the test suite).

So:  how do other folks test for pointer manipulation?  etc.

[toc] | [next] | [standalone]

#18422

On 03/10/12 12:43 PM, Don Y wrote:
> Hi,
>
> There's rarely a problem generating regression tests for
> functions/procedures that use concrete *values*:
>
> sqrt.test
>
> 3 -->  1.732
> 4 -->  2
> 5 -->  2.236
>
> (apologies if I have misremembered any of these values  :>  )
>
> And, it is easy to erect the necessary scaffolding for those
> sorts of tests without introducing any noticeable uncertainty
> in the veracity of the results obtained.
>
> But, I can't see a way to handle this sort of testing on
> functions that manipulate *pointers* to objects -- without
> adding a fair bit of code to the scaffolding that, itself,
> represents a testing issue!  (i.e., does the scaffolding
> have any bugs that result in false positives or negatives??)

As a general point, it a lot easier and more satisfying for the 
programmer to write the tests before the code under test.

> In particular, I am trying to develop a test suite for my
> dynamic memory management functions (e.g., malloc and free
> types of routines).
>
> I.e., I can't come up with "constant" results against
> which to compare test-time results.  Sticking with the
> traditional malloc/free for example (my routines are
> more heavily parameterized), I can create a malloc.test
> that pushes size_t's to malloc.  But, aside from verifying
> the result is not NULL (or, *is* NULL, in some cases!),
> there's nothing I can do to check that the result is
> actually correct!

Mock the functions your code calls.

I have a mocking library that can be used to mock any C function (given 
its return and parameter types) for use in unit tests which enables me 
to specify expected parameters and returned results.  I find this 
invaluable for comprehensive uni testing.

-- 
Ian Collins

[toc] | [prev] | [next] | [standalone]

#18429

Hi Ian,

On 3/9/2012 4:56 PM, Ian Collins wrote:
>> There's rarely a problem generating regression tests for
>> functions/procedures that use concrete *values*:
>>
>> sqrt.test
>>
>> 3 --> 1.732
>> 4 --> 2
>> 5 --> 2.236
>>
>> (apologies if I have misremembered any of these values :> )
>>
>> And, it is easy to erect the necessary scaffolding for those
>> sorts of tests without introducing any noticeable uncertainty
>> in the veracity of the results obtained.
>>
>> But, I can't see a way to handle this sort of testing on
>> functions that manipulate *pointers* to objects -- without
>> adding a fair bit of code to the scaffolding that, itself,
>> represents a testing issue! (i.e., does the scaffolding
>> have any bugs that result in false positives or negatives??)
>
> As a general point, it a lot easier and more satisfying for the
> programmer to write the tests before the code under test.

But that's the whole point -- you can't create test cases
independant of the execution environment!

What should "malloc(500)" return?  0x1234?  0x12345678?
NULL?

>> In particular, I am trying to develop a test suite for my
>> dynamic memory management functions (e.g., malloc and free
>> types of routines).
>>
>> I.e., I can't come up with "constant" results against
>> which to compare test-time results. Sticking with the
>> traditional malloc/free for example (my routines are
>> more heavily parameterized), I can create a malloc.test
>> that pushes size_t's to malloc. But, aside from verifying
>> the result is not NULL (or, *is* NULL, in some cases!),
>> there's nothing I can do to check that the result is
>> actually correct!
>
> Mock the functions your code calls.
>
> I have a mocking library that can be used to mock any C function (given
> its return and parameter types) for use in unit tests which enables me
> to specify expected parameters and returned results. I find this
> invaluable for comprehensive uni testing.

I don't see how that will help me.  Malloc doesn't "call* anything.
It massages blocks of memory in ways that vary depending on the
platform involved, etc.

E.g.,

    foo = malloc(500);
    result = myfree(foo, 451);

can work on some platforms and not on others; and, among
those where it does work, can give different results from
platform to platform.

The problem becomes even more difficult when you start
varying allocation strategies, fragmentation criteria, etc.

[toc] | [prev] | [next] | [standalone]

#18431

On 03/10/12 03:40 PM, Don Y wrote:
> Hi Ian,
>
> On 3/9/2012 4:56 PM, Ian Collins wrote:
>>> There's rarely a problem generating regression tests for
>>> functions/procedures that use concrete *values*:
>>>
>>> sqrt.test
>>>
>>> 3 -->  1.732
>>> 4 -->  2
>>> 5 -->  2.236
>>>
>>> (apologies if I have misremembered any of these values :>  )
>>>
>>> And, it is easy to erect the necessary scaffolding for those
>>> sorts of tests without introducing any noticeable uncertainty
>>> in the veracity of the results obtained.
>>>
>>> But, I can't see a way to handle this sort of testing on
>>> functions that manipulate *pointers* to objects -- without
>>> adding a fair bit of code to the scaffolding that, itself,
>>> represents a testing issue! (i.e., does the scaffolding
>>> have any bugs that result in false positives or negatives??)
>>
>> As a general point, it a lot easier and more satisfying for the
>> programmer to write the tests before the code under test.
>
> But that's the whole point -- you can't create test cases
> independant of the execution environment!

Of course you can.  How else would you test embedded code for a device 
without external communications?

> What should "malloc(500)" return?  0x1234?  0x12345678?
> NULL?

What ever *your test harness'* malloc is told to return.

>>> In particular, I am trying to develop a test suite for my
>>> dynamic memory management functions (e.g., malloc and free
>>> types of routines).
>>>
>>> I.e., I can't come up with "constant" results against
>>> which to compare test-time results. Sticking with the
>>> traditional malloc/free for example (my routines are
>>> more heavily parameterized), I can create a malloc.test
>>> that pushes size_t's to malloc. But, aside from verifying
>>> the result is not NULL (or, *is* NULL, in some cases!),
>>> there's nothing I can do to check that the result is
>>> actually correct!
>>
>> Mock the functions your code calls.
>>
>> I have a mocking library that can be used to mock any C function (given
>> its return and parameter types) for use in unit tests which enables me
>> to specify expected parameters and returned results. I find this
>> invaluable for comprehensive uni testing.
>
> I don't see how that will help me.  Malloc doesn't "call* anything.
> It massages blocks of memory in ways that vary depending on the
> platform involved, etc.

That's why you want to use a mock - to make your tests platform neutral.

-- 
Ian Collins

[toc] | [prev] | [next] | [standalone]

#18433

On 03/10/12 05:53 PM, Ian Collins wrote:
> On 03/10/12 03:40 PM, Don Y wrote:
>
>> What should "malloc(500)" return?  0x1234?  0x12345678?
>> NULL?
>
> What ever *your test harness'* malloc is told to return.

Maybe an example is called for:

Given

typedef struct { int n; } X;

X* makeAnX( int nx; )
{
   X* p = malloc( sizeof *p );
   p->n = n;
   return p;
}

A Googletest test (my library is C++, C simply can't do what it does. 
But that doesn't matter as for testing C functions) function using my 
mocks may look something like:

void testSomething()
{
   X testX;

   malloc::willReturn &testX;
   malloc::expect( sizeof(X) );

   X* p = makeAnX( 42 );

   ASSERT_EQ( p, &testX );
   ASSERT_EQ( 42, n-> );
}

The above tests that makeAnX passes the expected value to malloc and the 
correct value is returned from makeAnX.

-- 
Ian Collins

[toc] | [prev] | [next] | [standalone]

#18439

Hi Ian,

On 3/9/2012 9:53 PM, Ian Collins wrote:

>>>> There's rarely a problem generating regression tests for
>>>> functions/procedures that use concrete *values*:
>>>>
>>>> And, it is easy to erect the necessary scaffolding for those
>>>> sorts of tests without introducing any noticeable uncertainty
>>>> in the veracity of the results obtained.
>>>>
>>>> But, I can't see a way to handle this sort of testing on
>>>> functions that manipulate *pointers* to objects -- without
>>>> adding a fair bit of code to the scaffolding that, itself,
>>>> represents a testing issue! (i.e., does the scaffolding
>>>> have any bugs that result in false positives or negatives??)
>>>
>>> As a general point, it a lot easier and more satisfying for the
>>> programmer to write the tests before the code under test.
>>
>> But that's the whole point -- you can't create test cases
>> independant of the execution environment!
>
> Of course you can. How else would you test embedded code for a device
> without external communications?

For an embedded device, you *know* the environment you
are operating in.  You *know* where the heap resides
(actual addresses).  You know what the alignment
restrictions of the processor are.  You know that a
request for 0x61 bytes with the heap starting at 0x40000000
having a length of 0x100 in a byte alignable environment
allocating from the *end* of the free list (as opposed to
the start) trimming the allocation unit to the requested
size *should* give you a result of 0x61 bytes located
at 0x4000009F.  You *then* know that an attempt to
allocate 0x9E bytes will FAIL under the same calling
conditions (because a 1 byte crumb will be left in the
free list -- which "/* CAN'T HAPPEN */".

I write MyMalloc() and hand it to you WITH A TEST SUITE.
You run it on <whatever> processor and rely on the test
suite to prove to you that the code works as advertised.

Repeat the above example with a heap at 0x2000 (!!) of size
0x1000 and you get different results.

>> What should "malloc(500)" return? 0x1234? 0x12345678?
>> NULL?
>
> What ever *your test harness'* malloc is told to return.

What "test harness malloc"?  There *is* no "test harness
malloc!  Recall, I said "malloc and free *types* of functions"
not "malloc and free" -- in particular, note the semantics of
myfree() differ from free(3).

All I can compare against is myself -- and if I have a bug
then the other instance will faithfully repeat the same
bug.

>>>> In particular, I am trying to develop a test suite for my
>>>> dynamic memory management functions (e.g., malloc and free
>>>> types of routines).
>>>>
>>>> I.e., I can't come up with "constant" results against
>>>> which to compare test-time results. Sticking with the
>>>> traditional malloc/free for example (my routines are
>>>> more heavily parameterized), I can create a malloc.test
>>>> that pushes size_t's to malloc. But, aside from verifying
>>>> the result is not NULL (or, *is* NULL, in some cases!),
>>>> there's nothing I can do to check that the result is
>>>> actually correct!
>>>
>>> Mock the functions your code calls.
>>>
>>> I have a mocking library that can be used to mock any C function (given
>>> its return and parameter types) for use in unit tests which enables me
>>> to specify expected parameters and returned results. I find this
>>> invaluable for comprehensive uni testing.
>>
>> I don't see how that will help me. Malloc doesn't "call* anything.
>> It massages blocks of memory in ways that vary depending on the
>> platform involved, etc.
>
> That's why you want to use a mock - to make your tests platform neutral.

Sorry, I just don't see this.

I'm going to give you some code to implement a MyMalloc
(-like) function.  I'm not going to tell you where
the heap resides in memory.  I'm not going to tell you
if memory accesses must be byte/word/cache-line aligned.
Heck, I'm not even going to tell you how *big* the heap
is!

What are your test cases?  How do you know if they have
succeeded or not?  I.e., just because MyMalloc *appears*
to return a value, are you sure that it is the *correct*
value?

Are you sure that the allocation strategy actually enacted
was "smallest fit" and not "last fit" or "largest fit"?
(my memory management system is highly parameterized)
How do you know that the allocated block was *intentionally*
aligned with whatever the processor wants -- instead of
*coincidentally* happening to be aligned?  Are you sure
that any "crumbs" left behind in the free list as a
result of that alignment are recoverable?  (i.e., if
you left a single byte behind, then you probably can never
recover it since its not big enough to contain the list
overhead necessary to tie it into the free list)

[toc] | [prev] | [next] | [standalone]

#18441

On 03/10/12 10:05 PM, Don Y wrote:
> Hi Ian,
>
> On 3/9/2012 9:53 PM, Ian Collins wrote:
>>>
>>> But that's the whole point -- you can't create test cases
>>> independant of the execution environment!
>>
>> Of course you can. How else would you test embedded code for a device
>> without external communications?
>
> For an embedded device, you *know* the environment you
> are operating in.  You *know* where the heap resides
> (actual addresses).  You know what the alignment
> restrictions of the processor are.  You know that a
> request for 0x61 bytes with the heap starting at 0x40000000
> having a length of 0x100 in a byte alignable environment
> allocating from the *end* of the free list (as opposed to
> the start) trimming the allocation unit to the requested
> size *should* give you a result of 0x61 bytes located
> at 0x4000009F.  You *then* know that an attempt to
> allocate 0x9E bytes will FAIL under the same calling
> conditions (because a 1 byte crumb will be left in the
> free list -- which "/* CAN'T HAPPEN */".
>
> I write MyMalloc() and hand it to you WITH A TEST SUITE.
> You run it on<whatever>  processor and rely on the test
> suite to prove to you that the code works as advertised.
>
> Repeat the above example with a heap at 0x2000 (!!) of size
> 0x1000 and you get different results.

Ah, I see.  I though you were testing something that used malloc, rather 
than an implementation of malloc.

In the case above, just use conditionally compiled constants for heap 
base and size.

I don't see how a function like malloc is any different (form a unit 
testing perspective) than any other.

-- 
Ian Collins

[toc] | [prev] | [next] | [standalone]

#18447

Ian Collins <ian-news@hotmail.com> writes:
<snip>
> I don't see how a function like malloc is any different (form a unit
> testing perspective) than any other.

To take a simple well-know case (so I don't have to find a way to write
the specification!) what would unit tests for strdup look like?  In
particular, it would be a bug if strdup returned a pointer to a region
of memory that overlapped with one returned by any other allocating
function or, indeed, by strdup itself.  Now you may rule out testing for
the former because this is, after all a *unit* test, but not,
presumably, the latter.

I've been "out of the loop" as far as testing goes for some time so I
don't know how modern unit test systems deal with cases like this.  No
doubt they have sophisticated memory region checking tools, but surely
strdup unit tests will look far more complex than those for, say,
strcpy?

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#18483

On 03/11/12 01:00 AM, Ben Bacarisse wrote:
> Ian Collins<ian-news@hotmail.com>  writes:
> <snip>
>> I don't see how a function like malloc is any different (form a unit
>> testing perspective) than any other.
>
> To take a simple well-know case (so I don't have to find a way to write
> the specification!) what would unit tests for strdup look like?  In
> particular, it would be a bug if strdup returned a pointer to a region
> of memory that overlapped with one returned by any other allocating
> function or, indeed, by strdup itself.  Now you may rule out testing for
> the former because this is, after all a *unit* test, but not,
> presumably, the latter.

Well I guess you could rule out both by asserting strdup calls malloc.

> I've been "out of the loop" as far as testing goes for some time so I
> don't know how modern unit test systems deal with cases like this.  No
> doubt they have sophisticated memory region checking tools, but surely
> strdup unit tests will look far more complex than those for, say,
> strcpy?

I don't think so, assuming the behaviour of malloc is well defined.  All 
strdup is doing is something along the lines of:

char* pNew = malloc( strlen(in) );

if( pNew )
   strcpy( pNew, p );

return pNew;

-- 
Ian Collins

[toc] | [prev] | [next] | [standalone]

#18491

On 2012-03-10, Ian Collins <ian-news@hotmail.com> wrote:
> I don't think so, assuming the behaviour of malloc is well defined.  All 
> strdup is doing is something along the lines of:
>
> char* pNew = malloc( strlen(in) );

s/strlen(in)/strlen(in)+1/ (one extra byte for the null terminator)

> if( pNew )
>    strcpy( pNew, p );
>
> return pNew;

[toc] | [prev] | [next] | [standalone]

#18493

Ian Collins <ian-news@hotmail.com> writes:

> On 03/11/12 01:00 AM, Ben Bacarisse wrote:
>> Ian Collins<ian-news@hotmail.com>  writes:
>> <snip>
>>> I don't see how a function like malloc is any different (form a unit
>>> testing perspective) than any other.
>>
>> To take a simple well-know case (so I don't have to find a way to write
>> the specification!) what would unit tests for strdup look like?  In
>> particular, it would be a bug if strdup returned a pointer to a region
>> of memory that overlapped with one returned by any other allocating
>> function or, indeed, by strdup itself.  Now you may rule out testing for
>> the former because this is, after all a *unit* test, but not,
>> presumably, the latter.
>
> Well I guess you could rule out both by asserting strdup calls malloc.

That would be to miss the point!  Assume that strdup does not use
malloc -- this is an environment where dynamic allocation is not
permitted but we want to be able to copy some limited number of
strings.

>> I've been "out of the loop" as far as testing goes for some time so I
>> don't know how modern unit test systems deal with cases like this.  No
>> doubt they have sophisticated memory region checking tools, but surely
>> strdup unit tests will look far more complex than those for, say,
>> strcpy?
>
> I don't think so, assuming the behaviour of malloc is well defined.
> All strdup is doing is something along the lines of:
>
> char* pNew = malloc( strlen(in) );
>
> if( pNew )
>   strcpy( pNew, p );
>
> return pNew;

See above.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#18496

On 03/11/12 11:40 AM, Ben Bacarisse wrote:
> Ian Collins<ian-news@hotmail.com>  writes:
>
>> On 03/11/12 01:00 AM, Ben Bacarisse wrote:
>>> Ian Collins<ian-news@hotmail.com>   writes:
>>> <snip>
>>>> I don't see how a function like malloc is any different (form a unit
>>>> testing perspective) than any other.
>>>
>>> To take a simple well-know case (so I don't have to find a way to write
>>> the specification!) what would unit tests for strdup look like?  In
>>> particular, it would be a bug if strdup returned a pointer to a region
>>> of memory that overlapped with one returned by any other allocating
>>> function or, indeed, by strdup itself.  Now you may rule out testing for
>>> the former because this is, after all a *unit* test, but not,
>>> presumably, the latter.
>>
>> Well I guess you could rule out both by asserting strdup calls malloc.
>
> That would be to miss the point!  Assume that strdup does not use
> malloc -- this is an environment where dynamic allocation is not
> permitted but we want to be able to copy some limited number of
> strings.

Um, OK.  I was using the description in the Solaris man page:

      The strdup() function returns a pointer to a new string that
      is  a duplicate of the string pointed to by  s. The returned
      pointer can be passed to  free().  The  space  for  the  new
      string  is obtained using malloc(3C).

Given your caveat, I would first test the function that retuned the 
memory and then carry on as before by asserting strdup calls that 
function.  I would then add tests to ensure the string was correctly 
copied.  This separates testing the memory management functionality from 
the string copying.

See my earlier reply to "Don Y" for an example of how those tests would 
be built.

-- 
Ian Collins

[toc] | [prev] | [next] | [standalone]

#18502

Ian Collins <ian-news@hotmail.com> writes:

> On 03/11/12 11:40 AM, Ben Bacarisse wrote:
>> Ian Collins<ian-news@hotmail.com>  writes:
>>
>>> On 03/11/12 01:00 AM, Ben Bacarisse wrote:
>>>> Ian Collins<ian-news@hotmail.com>   writes:
>>>> <snip>
>>>>> I don't see how a function like malloc is any different (form a unit
>>>>> testing perspective) than any other.
>>>>
>>>> To take a simple well-know case (so I don't have to find a way to write
>>>> the specification!) what would unit tests for strdup look like?  In
>>>> particular, it would be a bug if strdup returned a pointer to a region
>>>> of memory that overlapped with one returned by any other allocating
>>>> function or, indeed, by strdup itself.  Now you may rule out testing for
>>>> the former because this is, after all a *unit* test, but not,
>>>> presumably, the latter.
>>>
>>> Well I guess you could rule out both by asserting strdup calls malloc.
>>
>> That would be to miss the point!  Assume that strdup does not use
>> malloc -- this is an environment where dynamic allocation is not
>> permitted but we want to be able to copy some limited number of
>> strings.
>
> Um, OK.  I was using the description in the Solaris man page:
>
>      The strdup() function returns a pointer to a new string that
>      is  a duplicate of the string pointed to by  s. The returned
>      pointer can be passed to  free().  The  space  for  the  new
>      string  is obtained using malloc(3C).
>
> Given your caveat, I would first test the function that retuned the
> memory and then carry on as before by asserting strdup calls that
> function.  I would then add tests to ensure the string was correctly
> copied.  This separates testing the memory management functionality
> from the string copying.
>
> See my earlier reply to "Don Y" for an example of how those tests
> would be built.

Hmm... I'm not seeing it.  Can we take a specific case?  Your previous
post happened to have a strdup with a bug (storage returned was one byte
too short).  What sort of unit test would pick that bug up?  I may be
misreading the allocation tests you are referring to but they don't seem
to cover that.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#18509

On 03/11/12 01:42 PM, Ben Bacarisse wrote:
> Ian Collins<ian-news@hotmail.com>  writes:
>
>> On 03/11/12 11:40 AM, Ben Bacarisse wrote:
>>> Ian Collins<ian-news@hotmail.com>   writes:
>>>
>>>> On 03/11/12 01:00 AM, Ben Bacarisse wrote:
>>>>> Ian Collins<ian-news@hotmail.com>    writes:
>>>>> <snip>
>>>>>> I don't see how a function like malloc is any different (form a unit
>>>>>> testing perspective) than any other.
>>>>>
>>>>> To take a simple well-know case (so I don't have to find a way to write
>>>>> the specification!) what would unit tests for strdup look like?  In
>>>>> particular, it would be a bug if strdup returned a pointer to a region
>>>>> of memory that overlapped with one returned by any other allocating
>>>>> function or, indeed, by strdup itself.  Now you may rule out testing for
>>>>> the former because this is, after all a *unit* test, but not,
>>>>> presumably, the latter.
>>>>
>>>> Well I guess you could rule out both by asserting strdup calls malloc.
>>>
>>> That would be to miss the point!  Assume that strdup does not use
>>> malloc -- this is an environment where dynamic allocation is not
>>> permitted but we want to be able to copy some limited number of
>>> strings.
>>
>> Um, OK.  I was using the description in the Solaris man page:
>>
>>       The strdup() function returns a pointer to a new string that
>>       is  a duplicate of the string pointed to by  s. The returned
>>       pointer can be passed to  free().  The  space  for  the  new
>>       string  is obtained using malloc(3C).
>>
>> Given your caveat, I would first test the function that retuned the
>> memory and then carry on as before by asserting strdup calls that
>> function.  I would then add tests to ensure the string was correctly
>> copied.  This separates testing the memory management functionality
>> from the string copying.
>>
>> See my earlier reply to "Don Y" for an example of how those tests
>> would be built.
>
> Hmm... I'm not seeing it.  Can we take a specific case?  Your previous
> post happened to have a strdup with a bug (storage returned was one byte
> too short).  What sort of unit test would pick that bug up?  I may be
> misreading the allocation tests you are referring to but they don't seem
> to cover that.

I would probably be testing this a two separate libraries.  The memory 
allocation functions would have their own set of tests and I would then 
mock those functions (let's call them staticMalloc and staticFree) in 
the string library tests.

So in my string library tests I would have a test like:

TEST_F( MyStingTest,
         requestedBufferSizeOneMoreThanStringLength )
{
   const char* string = "hello";

   test::staticMalloc::expect( sizeof(string)+1 );

   strdup( string );

   ASSERT_TRUE( test::staticMalloc::called );
}


-- 
Ian Collins

[toc] | [prev] | [next] | [standalone]

#18512

Ian Collins <ian-news@hotmail.com> writes:

> On 03/11/12 01:42 PM, Ben Bacarisse wrote:
>> Ian Collins<ian-news@hotmail.com>  writes:
>>
>>> On 03/11/12 11:40 AM, Ben Bacarisse wrote:
>>>> Ian Collins<ian-news@hotmail.com>   writes:
>>>>
>>>>> On 03/11/12 01:00 AM, Ben Bacarisse wrote:
>>>>>> Ian Collins<ian-news@hotmail.com>    writes:
>>>>>> <snip>
>>>>>>> I don't see how a function like malloc is any different (form a unit
>>>>>>> testing perspective) than any other.
>>>>>>
>>>>>> To take a simple well-know case (so I don't have to find a way to write
>>>>>> the specification!) what would unit tests for strdup look like?  In
>>>>>> particular, it would be a bug if strdup returned a pointer to a region
>>>>>> of memory that overlapped with one returned by any other allocating
>>>>>> function or, indeed, by strdup itself.  Now you may rule out testing for
>>>>>> the former because this is, after all a *unit* test, but not,
>>>>>> presumably, the latter.
>>>>>
>>>>> Well I guess you could rule out both by asserting strdup calls malloc.
>>>>
>>>> That would be to miss the point!  Assume that strdup does not use
>>>> malloc -- this is an environment where dynamic allocation is not
>>>> permitted but we want to be able to copy some limited number of
>>>> strings.
>>>
>>> Um, OK.  I was using the description in the Solaris man page:
>>>
>>>       The strdup() function returns a pointer to a new string that
>>>       is  a duplicate of the string pointed to by  s. The returned
>>>       pointer can be passed to  free().  The  space  for  the  new
>>>       string  is obtained using malloc(3C).
>>>
>>> Given your caveat, I would first test the function that retuned the
>>> memory and then carry on as before by asserting strdup calls that
>>> function.  I would then add tests to ensure the string was correctly
>>> copied.  This separates testing the memory management functionality
>>> from the string copying.
>>>
>>> See my earlier reply to "Don Y" for an example of how those tests
>>> would be built.
>>
>> Hmm... I'm not seeing it.  Can we take a specific case?  Your previous
>> post happened to have a strdup with a bug (storage returned was one byte
>> too short).  What sort of unit test would pick that bug up?  I may be
>> misreading the allocation tests you are referring to but they don't seem
>> to cover that.
>
> I would probably be testing this a two separate libraries.  The memory
> allocation functions would have their own set of tests and I would
> then mock those functions (let's call them staticMalloc and
> staticFree) in the string library tests.
>
> So in my string library tests I would have a test like:
>
> TEST_F( MyStingTest,
>         requestedBufferSizeOneMoreThanStringLength )
> {
>   const char* string = "hello";
>
>   test::staticMalloc::expect( sizeof(string)+1 );
>
>   strdup( string );
>
>   ASSERT_TRUE( test::staticMalloc::called );
> }

I see, thank you.

You have a bug in the test that is interesting -- the size is also wrong
here.  That was a similar bug to the one in strdup.  It seems a shame
that one of the things that needs to be tested (the size expression)
must be written out here as well -- I suppose that's why the code's
author should not write the tests.  BTW, I might change the expect
call was to expect_at_least(...) so as to uncouple the test from
the implementation a little more.

If I can prevail on you some more...  What about the requirement that
the malloced store be unique and not overlapping any others?  Is that
done statistically by setting the memory to some known data pattern?

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#18517

On 03/11/12 04:09 PM, Ben Bacarisse wrote:
>>
>> I would probably be testing this a two separate libraries.  The memory
>> allocation functions would have their own set of tests and I would
>> then mock those functions (let's call them staticMalloc and
>> staticFree) in the string library tests.
>>
>> So in my string library tests I would have a test like:
>>
>> TEST_F( MyStingTest,
>>          requestedBufferSizeOneMoreThanStringLength )
>> {
>>    const char* string = "hello";
>>
>>    test::staticMalloc::expect( sizeof(string)+1 );
>>
>>    strdup( string );
>>
>>    ASSERT_TRUE( test::staticMalloc::called );
>> }
>
> I see, thank you.
>
> You have a bug in the test that is interesting -- the size is also wrong
> here.  That was a similar bug to the one in strdup.

I would test with more than one string, so the bug would soon come to 
light.  I also try to use a different  expression in the tests (such as 
size of rather than strlen) to minimise common errors.

> It seems a shame
> that one of the things that needs to be tested (the size expression)
> must be written out here as well -- I suppose that's why the code's
> author should not write the tests.

The odds on the same bug appearing in two different expression are less 
than a single occurrence.  If I am working with a pair, we often swap 
roles between tests.  If I am working on my own, I have to take more care!

 > BTW, I might change the expect
> call was to expect_at_least(...) so as to uncouple the test from
> the implementation a little more.

That would be fine for numeric values, but not much use for other types. 
  My harness' expect function accepts function objects as well as values 
(one handy feature of C++), so I could use an at_least functor here.

> If I can prevail on you some more...  What about the requirement that
> the malloced store be unique and not overlapping any others?  Is that
> done statistically by setting the memory to some known data pattern?

That would be a memory allocator test.  I'll ruminate some more on how 
to write it!

-- 
Ian Collins

[toc] | [prev] | [next] | [standalone]

#18460

Hi Ian,

On 3/10/2012 3:17 AM, Ian Collins wrote:
>> On 3/9/2012 9:53 PM, Ian Collins wrote:
>>>>
>>>> But that's the whole point -- you can't create test cases
>>>> independant of the execution environment!
>>>
>>> Of course you can. How else would you test embedded code for a device
>>> without external communications?
>>
>> For an embedded device, you *know* the environment you
>> are operating in. You *know* where the heap resides
>> (actual addresses). You know what the alignment
>> restrictions of the processor are. You know that a
>> request for 0x61 bytes with the heap starting at 0x40000000
>> having a length of 0x100 in a byte alignable environment
>> allocating from the *end* of the free list (as opposed to
>> the start) trimming the allocation unit to the requested
>> size *should* give you a result of 0x61 bytes located
>> at 0x4000009F. You *then* know that an attempt to
>> allocate 0x9E bytes will FAIL under the same calling
>> conditions (because a 1 byte crumb will be left in the
>> free list -- which "/* CAN'T HAPPEN */".
>>
>> I write MyMalloc() and hand it to you WITH A TEST SUITE.
>> You run it on<whatever> processor and rely on the test
>> suite to prove to you that the code works as advertised.
>>
>> Repeat the above example with a heap at 0x2000 (!!) of size
>> 0x1000 and you get different results.
>
> Ah, I see. I though you were testing something that used malloc, rather
> than an implementation of malloc.

Exactly.  And, since so much relies on this subsystem, I
want to "beat on it" pretty severely to assure myself (and
others porting to other platforms) that it *does* handle
every potential "situation".

> In the case above, just use conditionally compiled constants for heap
> base and size.

That assumes you can put the heap where your test suite wants
it.  E.g., on a tiny processor, the heap might be at 0xA000;
on a desktop, at 0x40020000; on a 64b machine...

Similarly, the amount of memory *in* the heap might vary.
You could allocate 1MB on a desktop platform but 10KB on
a tiny machine might be "oodles".  If you've written with
the tiny machine in mind, perhaps you've a latent bug where
you expected pointers to be 16b instead of 32b (sure, you
can code to deal with this; but, you can also FORGET TO
CODE to deal with this!).

The alignment constraints of the processor might change from
what your test cases "expect".

Etc.  You could *code* a "test routine" that examines these
things and *intelligently* invokes the UUT.  But, that means
you start migrating complexity into the "tester" that can be
just as tricky (if not trickier!) as the code you are hoping
to "test".

> I don't see how a function like malloc is any different (form a unit
> testing perspective) than any other.

Because addresses aren't as constrained (in the language) as
*values*.  "3" is "3" is "3".  But, for a given object, you
can have environments where &object can NEVER be 0x0...3 -- or
0x0...2 (!), etc.

At the same time, that same code can execute in environments
where this assertion doesn't hold.

Yet, BOTH are "correct".

[toc] | [prev] | [next] | [standalone]

#18481

On 03/11/12 08:01 AM, Don Y wrote:
> Hi Ian,
>
> On 3/10/2012 3:17 AM, Ian Collins wrote:
>
>> In the case above, just use conditionally compiled constants for heap
>> base and size.
>
> That assumes you can put the heap where your test suite wants
> it.  E.g., on a tiny processor, the heap might be at 0xA000;
> on a desktop, at 0x40020000; on a 64b machine...
>
> Similarly, the amount of memory *in* the heap might vary.
> You could allocate 1MB on a desktop platform but 10KB on
> a tiny machine might be "oodles".  If you've written with
> the tiny machine in mind, perhaps you've a latent bug where
> you expected pointers to be 16b instead of 32b (sure, you
> can code to deal with this; but, you can also FORGET TO
> CODE to deal with this!).
>
> The alignment constraints of the processor might change from
> what your test cases "expect".
>
> Etc.  You could *code* a "test routine" that examines these
> things and *intelligently* invokes the UUT.  But, that means
> you start migrating complexity into the "tester" that can be
> just as tricky (if not trickier!) as the code you are hoping
> to "test".

OK, it's a dull Sunday morning, so I cooked up a quick working example 
using Google Test.

The simple function myMalloc() will:

Return NULL for a zero size request.
Return a correctly aligned pointer within the heap for a non-zero request.
Use a fixed heap base of 0xA000 and size of 0x1000 on TARGET_X.

#if defined TARGET_X
const uint8_t* heapBase = 0xA000;
const size_t   heapSize = 0x1000;
const size_t   minAlign = 8;
#else
uint8_t* heapBase = NULL;
size_t heapSize = 0x1000;
size_t minAlign = 8;
#endif

void* myMalloc( size_t size )
{
   size_t offset = 0;

   if( size )
   {
     while( (size_t)(heapBase+offset)%minAlign ) ++offset;

     return heapBase+offset;
   }
   return NULL;
}

TEST_F( MyMallocTest,
         returnIsNullForZeroByteRequest )
{
   ASSERT_FALSE( myMalloc(0) );
}

TEST_F( MyMallocTest,
         returnIsInHeapForOneByteRequest )
{
   ASSERT_TRUE( myMalloc(1) >= heapBase );
   ASSERT_TRUE( myMalloc(1) < heapBase+heapSize );
}

TEST_F( MyMallocTest,
         returnForOneByteRequestCorrectlyAlligned )
{
   ASSERT_EQ( 0, (size_t)myMalloc(1)%minAlign );
}

TEST_F( MyMallocTest,
         returnForOneByteRequestCorrectlyAllignedFor128Bytes )
{
   minAlign = 128;
   ASSERT_EQ( 0, (size_t)myMalloc(1)%minAlign );
}

The Google test code is at the end to avoid clutter.

>> I don't see how a function like malloc is any different (form a unit
>> testing perspective) than any other.
>
> Because addresses aren't as constrained (in the language) as
> *values*.  "3" is "3" is "3".  But, for a given object, you
> can have environments where&object can NEVER be 0x0...3 -- or
> 0x0...2 (!), etc.
>
> At the same time, that same code can execute in environments
> where this assertion doesn't hold.
>
> Yet, BOTH are "correct".

Does the exaple above help?

-- 
Google test bits for the test:

#include <gtest/gtest.h>

struct MyMallocTest : testing::Test
{
   uint8_t* heap;

   MyMallocTest()
   {
     heap = new uint8_t[heapSize];
     heapBase = heap;
   }

   ~MyMallocTest()
   {
     heapSize = 0x1000;
     delete heap;
   }

   void SetUp() { memset( heap, '@', heapSize ); }
};

// tests go here

int main( int argc, char **argv )
{
   testing::InitGoogleTest(&argc, argv);

   return RUN_ALL_TESTS();
}

-- 
Ian Collins

[toc] | [prev] | [next] | [standalone]

#18591

Hi Ian,

On 3/10/2012 2:35 PM, Ian Collins wrote:

>>> In the case above, just use conditionally compiled constants for heap
>>> base and size.
>>
>> That assumes you can put the heap where your test suite wants
>> it. E.g., on a tiny processor, the heap might be at 0xA000;
>> on a desktop, at 0x40020000; on a 64b machine...
>>
>> Similarly, the amount of memory *in* the heap might vary.
>> You could allocate 1MB on a desktop platform but 10KB on
>> a tiny machine might be "oodles". If you've written with
>> the tiny machine in mind, perhaps you've a latent bug where
>> you expected pointers to be 16b instead of 32b (sure, you
>> can code to deal with this; but, you can also FORGET TO
>> CODE to deal with this!).
>>
>> The alignment constraints of the processor might change from
>> what your test cases "expect".
>>
>> Etc. You could *code* a "test routine" that examines these
>> things and *intelligently* invokes the UUT. But, that means
>> you start migrating complexity into the "tester" that can be
>> just as tricky (if not trickier!) as the code you are hoping
>> to "test".
>
> OK, it's a dull Sunday morning, so I cooked up a quick working example
> using Google Test.
>
> The simple function myMalloc() will:
>
> Return NULL for a zero size request.
> Return a correctly aligned pointer within the heap for a non-zero request.
> Use a fixed heap base of 0xA000 and size of 0x1000 on TARGET_X.

But those criteria might not test the full capabilities of
myMalloc.  E.g., does it behave when the heap size is increased
to 0x6000?  Does it behave if I move the heap to 0x0000?  Does
it work for "big" requests -- e.g., a megabyte?  (it *should*
on TARGET_Y but that wouldn't make sense on TARGET_X... two
different test suites?)

> #if defined TARGET_X
> const uint8_t* heapBase = 0xA000;
> const size_t heapSize = 0x1000;
> const size_t minAlign = 8;
> #else
> uint8_t* heapBase = NULL;
> size_t heapSize = 0x1000;
> size_t minAlign = 8;
> #endif

You are relying on being able to enumerate all of the test
conditions for each potential platform.  But you can't write
that test suite "today" without knowledge of what they will be
(e.g., 1MB request).

The problem is inherent in the fact that this is an area
(pointers, size_t) where the language gives the implementation
lots of flexibility.  OTOH, you *know* that a double will
be good for ~10+ digits...

> TEST_F( MyMallocTest,
> returnIsNullForZeroByteRequest )
> {
> ASSERT_FALSE( myMalloc(0) );
> }

Should be null for "too big" request.

> TEST_F( MyMallocTest,
> returnIsInHeapForOneByteRequest )
> {
> ASSERT_TRUE( myMalloc(1) >= heapBase );
> ASSERT_TRUE( myMalloc(1) < heapBase+heapSize );
> }

Should be "in heap" for *all* requests.

> TEST_F( MyMallocTest,
> returnForOneByteRequestCorrectlyAlligned )
> {
> ASSERT_EQ( 0, (size_t)myMalloc(1)%minAlign );
> }

Should be correctly aligned for *all* requests.

> TEST_F( MyMallocTest,
> returnForOneByteRequestCorrectlyAllignedFor128Bytes )
> {
> minAlign = 128;
> ASSERT_EQ( 0, (size_t)myMalloc(1)%minAlign );
> }
>
> The Google test code is at the end to avoid clutter.

The point of my "should be's" is that you can pin the
function's return values down much tighter than that.  And,
you *need* to be able to do so in order to assure yourself
that it truly is operating properly.

If I run the returnIsInHeapForOneByteRequest 0x1000 times,
will it always pass (without an intervening free())?
Shouldn't it fail at the 0x1000/minAlign point, instead?
Am I sure that no two of the returned pointers EVER coincide?

E.g., you wouldn't write:

ASSERT(sqrt(3) > 1)
ASSERT(sqrt(3) < 2)
ASSERT(sqrt(3) > sqrt(2))
ASSERT(sqrt(3) < sqrt(4))

*All* of these are appropriate.  But, what you really want
to know is

ASSERT(sqrt(3) =~ 1.732...)

This embodies *all* of the above tests and gives the
function far less "wiggle room".

(malloc -- and your myMalloc -- are too loosely defined for
robust formal testing.  That's one of the reasons why I
write my own memory management tools.  I need to be able
to *predict* the return values for each invocation -- since
I know how I've called it in each *prior* invocation, this
should be straightforward for a deterministic function)

>>> I don't see how a function like malloc is any different (form a unit
>>> testing perspective) than any other.
>>
>> Because addresses aren't as constrained (in the language) as
>> *values*. "3" is "3" is "3". But, for a given object, you
>> can have environments where&object can NEVER be 0x0...3 -- or
>> 0x0...2 (!), etc.
>>
>> At the same time, that same code can execute in environments
>> where this assertion doesn't hold.
>>
>> Yet, BOTH are "correct".
>
> Does the exaple above help?

[toc] | [prev] | [next] | [standalone]

#18600

On 03/13/12 05:19 AM, Don Y wrote:
> Hi Ian,
>
> On 3/10/2012 2:35 PM, Ian Collins wrote:
>
>>>> In the case above, just use conditionally compiled constants for heap
>>>> base and size.
>>>
>>> That assumes you can put the heap where your test suite wants
>>> it. E.g., on a tiny processor, the heap might be at 0xA000;
>>> on a desktop, at 0x40020000; on a 64b machine...
>>>
>>> Similarly, the amount of memory *in* the heap might vary.
>>> You could allocate 1MB on a desktop platform but 10KB on
>>> a tiny machine might be "oodles". If you've written with
>>> the tiny machine in mind, perhaps you've a latent bug where
>>> you expected pointers to be 16b instead of 32b (sure, you
>>> can code to deal with this; but, you can also FORGET TO
>>> CODE to deal with this!).
>>>
>>> The alignment constraints of the processor might change from
>>> what your test cases "expect".
>>>
>>> Etc. You could *code* a "test routine" that examines these
>>> things and *intelligently* invokes the UUT. But, that means
>>> you start migrating complexity into the "tester" that can be
>>> just as tricky (if not trickier!) as the code you are hoping
>>> to "test".
>>
>> OK, it's a dull Sunday morning, so I cooked up a quick working example
>> using Google Test.
>>
>> The simple function myMalloc() will:
>>
>> Return NULL for a zero size request.
>> Return a correctly aligned pointer within the heap for a non-zero request.
>> Use a fixed heap base of 0xA000 and size of 0x1000 on TARGET_X.
>
> But those criteria might not test the full capabilities of
> myMalloc.  E.g., does it behave when the heap size is increased
> to 0x6000?  Does it behave if I move the heap to 0x0000?  Does
> it work for "big" requests -- e.g., a megabyte?  (it *should*
> on TARGET_Y but that wouldn't make sense on TARGET_X... two
> different test suites?)

This is a demonstration, not a full test suite.  I've been designing 
allocators for 20 odd year and non of them have worked differently is 
different locations.

>> #if defined TARGET_X
>> const uint8_t* heapBase = 0xA000;
>> const size_t heapSize = 0x1000;
>> const size_t minAlign = 8;
>> #else
>> uint8_t* heapBase = NULL;
>> size_t heapSize = 0x1000;
>> size_t minAlign = 8;
>> #endif
>
> You are relying on being able to enumerate all of the test
> conditions for each potential platform.

No I'm not, that's just an example of allowing either constant or 
variable parameters. The allocator code should be position independent.

> But you can't write
> that test suite "today" without knowledge of what they will be
> (e.g., 1MB request).

Why not?  Just add maxRequest as a constant/variable to the mix.

> The problem is inherent in the fact that this is an area
> (pointers, size_t) where the language gives the implementation
> lots of flexibility.  OTOH, you *know* that a double will
> be good for ~10+ digits...

Flexibility that should not change the relative behaviour of the code.

>> TEST_F( MyMallocTest,
>> returnIsNullForZeroByteRequest )
>> {
>> ASSERT_FALSE( myMalloc(0) );
>> }
>
> Should be null for "too big" request.

So add a test.

>> TEST_F( MyMallocTest,
>> returnIsInHeapForOneByteRequest )
>> {
>> ASSERT_TRUE( myMalloc(1)>= heapBase );
>> ASSERT_TRUE( myMalloc(1)<  heapBase+heapSize );
>> }
>
> Should be "in heap" for *all* requests.

So add a test.

>> TEST_F( MyMallocTest,
>> returnForOneByteRequestCorrectlyAlligned )
>> {
>> ASSERT_EQ( 0, (size_t)myMalloc(1)%minAlign );
>> }
>
> Should be correctly aligned for *all* requests.

So add a test.

>> TEST_F( MyMallocTest,
>> returnForOneByteRequestCorrectlyAllignedFor128Bytes )
>> {
>> minAlign = 128;
>> ASSERT_EQ( 0, (size_t)myMalloc(1)%minAlign );
>> }
>>
>> The Google test code is at the end to avoid clutter.
>
> The point of my "should be's" is that you can pin the
> function's return values down much tighter than that.  And,
> you *need* to be able to do so in order to assure yourself
> that it truly is operating properly.
>
> If I run the returnIsInHeapForOneByteRequest 0x1000 times,
> will it always pass (without an intervening free())?

So add a test.

> Shouldn't it fail at the 0x1000/minAlign point, instead?
> Am I sure that no two of the returned pointers EVER coincide?

So add a test.

> E.g., you wouldn't write:
>
> ASSERT(sqrt(3)>  1)
> ASSERT(sqrt(3)<  2)
> ASSERT(sqrt(3)>  sqrt(2))
> ASSERT(sqrt(3)<  sqrt(4))
>
> *All* of these are appropriate.  But, what you really want
> to know is
>
> ASSERT(sqrt(3) =~ 1.732...)
>
> This embodies *all* of the above tests and gives the
> function far less "wiggle room".
>
> (malloc -- and your myMalloc -- are too loosely defined for
> robust formal testing.

It's only a demonstration to address some of your questions of a work in 
progress.

-- 
Ian Collins

[toc] | [prev] | [next] | [standalone]

Page 1 of 5  [1] 2 3 4 5  Next page →

Back to top | Article view | comp.lang.c

csiph-web