Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
| From | Don Y <this@isnotme.com> |
|---|---|
| Newsgroups | comp.lang.c |
| Subject | Re: Regression testing for pointers |
| Date | 2012-03-10 12:42 -0700 |
| Organization | Aioe.org NNTP Server |
| Message-ID | <jjgaru$phb$1@speranza.aioe.org> (permalink) |
| References | <jje4jg$l8k$1@speranza.aioe.org> <20120309184942.431@kylheku.com> |
Hi Kaz,
On 3/9/2012 8:14 PM, Kaz Kylheku wrote:
> On 2012-03-09, Don Y<this@isnotme.com> wrote:
>> In particular, I am trying to develop a test suite for my
>> dynamic memory management functions (e.g., malloc and free
>> types of routines).
>>
>> I.e., I can't come up with "constant" results against
>> which to compare test-time results. Sticking with the
>> traditional malloc/free for example (my routines are
>> more heavily parameterized), I can create a malloc.test
>> that pushes size_t's to malloc. But, aside from verifying
>> the result is not NULL (or, *is* NULL, in some cases!),
>> there's nothing I can do to check that the result is
>> actually correct!
>
> A possible regression test case for malloc routines is a program which beats up
> the routines, performs certain validation on the inputs and terminates
> successfully.
But that doesn't "prove" that the functions are operating as
*designed*/expected.
E.g., my allocator allows the caller to tell it *how* to select
a chunk of memory from the free list, *where* in the free list
that selection should be made, how the selection should be modified
to fit the specifications of the request, etc.
The allocator can "satisfy" all of the requests that I make
of it (i.e., can return non NULL) yet could have done so
in entirely the wrong "way".
>> About the only thing I can check for is if that second
>> invocation returned 0x12345699 (which overlaps the previous
>> allocation at 0x12345678!).
>
> You could allocate some random blocks and keep them in an array. All such
> blocks should be non-overlapping.
But that still doesn't tell you that it is returning the *correct*
results. You're testing an *aspect* of the allocator which doesn;t
guarantee that the allocator itself is functioning properly.
E.g., imagine testing sqrt() with (pseudocode):
lastvalue = 0.
lastresult = 0
do {
value = lastvalue + 1.
result = sqrt(value)
if ! (result > lastresult)
break;
lastvalue = value
lastresult = result
} while (FOREVER)
I.e., a functioning sqrt() will comply with this test. But,
a MALfunctioning one will, as well!
> Another thing to test for is that the allocator is not getting confused and
> overwriting the data areas of the blocks.
>
> Put data in the blocks and maintain checksums. Check that the allocator
> operations do not change anything.
Again, that's just poking an *aspect* of the allocation
not testing the allocator's functionality. Before the blocks
are allocated, their contents are indeterminate. You don't
*care* if the allocator screws with them prior to allocation.
All you care is that it doesn't alter the contents of ALLOCATED
blocks.
> Also, the regression test does not have to use only the public routines. Give
> your allocator routines some debugging functions.
>
> The test could check, for instance, that the allocator's data structures have
> returned to some proper state after the blocks have been freed.
The code already is littered with invariants. I am *sure* it
"works". :> The problem is making sure that it continues to
work wen ported to other platforms, when "tweeked" by well
meaning developers, etc.
> This could be repeated for different orders of freeing the blocks: in order, in
> reverse order, in scrambled order.
In my case, its considerably harder than that. You tell the allocator
(and reclamation function) how you want the free list managed. So,
you can allocate using a set of criteria (call them "A1") and release
with some criteria R1. The *next* allocation would yield M1.
OTOH, allocating with A1 and releasing with R2 could cause the next
allocation to yield M2 (!) -- and be operating exactly as designed!
> You could have some self-check routines in the allocator. For instance if it
> keeps counters about how much has been allocated, etc, a function could be
> provided which performs a check by walking the data structure to validate that
> the meta-data matches the structure.
See above (invariants)
>> Furthermore, just examining the results from malloc don't
>> tell me that the freelist is being maintained, properly.
>
> See above: expose an internal routine that validates the sanity of the
> free list, and use it in the regression test case.
>
>> Of course, I can write code to inspect these objects in
>> greater detail. But, then the test scaffolding starts to
>> rival the complexity of the functions being tested! (bugs)
>
> But that is fine because that scaffolding isn't what is being deployed.
>
> A wind tunnel is going to be larger than the aircraft part being tested.
> That's a given.
But the effort to VALIDATE the wind tunnel is amortized over
ALL of the aircraft parts being tested in that tunnel. Imagine
building a special wind tunnel for the wing. Of *this* ONE
airframe. And, yet another for the tail section. Another for
the fuselage, etc.
Each needs to be validated in its own right before being relied
upon to vouch for the "correctness" of the items tested within.
>> I guess the "ideal" I see would be to be able to build a
>> "memory image" and verify that the functions cause expected
>> and predicted changes in that "image".
>
> That's not a bad idea.
>
> The image will likely contain pointers. Those can be normalized by changing
> them to offsets from the base address.
>
> As a substitute for a binary image dump, you could have a
> symbolic dump which traverses everything and produces
> a printed representation of it.
The problem is building the "template" image. You can't just
put pointers anywhere. E.g., how do you create an "unaligned
pointer" in the language?
>> But, I can't see any way to make this NOT require some sort
>> of active probing of memory by the test scaffolding (which
>> means the internals of the functions have to be exposed to
>> the test suite).
>
> Exposing internals to a regression test: fine idea.
>
> Would you not test electronic hardware with some probes
> to check voltage levels and signals?
Sure. But I wouldn't grab a length of #12AWG and use it to
probe the front end of a satellite receiver! I'd get a
probe that was VALIDATED AS APPROPRIATE for that sort of
testing.
Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-09 16:43 -0700
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 12:56 +1300
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-09 19:40 -0700
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 17:53 +1300
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 18:12 +1300
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 02:05 -0700
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 23:17 +1300
Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-10 12:00 +0000
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 11:00 +1300
Re: Regression testing for pointers Ike Naar <ike@iceland.freeshell.org> - 2012-03-10 22:23 +0000
Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-10 22:40 +0000
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 12:12 +1300
Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-11 00:42 +0000
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 14:26 +1300
Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-11 03:09 +0000
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 18:16 +1300
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 12:01 -0700
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 10:35 +1300
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 09:19 -0700
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 07:23 +1300
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 16:23 -0700
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 17:25 +1300
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 00:19 -0700
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 20:34 +1300
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 01:00 -0700
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 21:22 +1300
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 10:19 -0700
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 07:25 +1300
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 02:00 -0700
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 22:18 +1300
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 11:32 -0700
Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-15 05:58 -0700
Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-15 05:55 -0700
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-15 11:32 -0700
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-16 18:14 +1300
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-20 01:43 -0700
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-20 21:59 +1300
Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-16 03:05 -0700
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-20 02:33 -0700
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-24 11:49 +1300
Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-11 05:01 -0700
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 09:44 -0700
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-12 09:56 +1300
Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-10 02:28 -0800
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 11:09 -0700
Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-10 18:29 +0000
Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-11 05:22 -0700
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 09:50 -0700
Re: Regression testing for pointers "io_x" <a@b.c.invalid> - 2012-03-13 13:09 +0100
Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-10 03:14 +0000
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 12:42 -0700
Re: Regression testing for pointers Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-10 08:42 +0000
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 02:19 -0700
Re: Regression testing for pointers "BartC" <bc@freeuk.com> - 2012-03-10 15:54 +0000
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 11:49 -0700
Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-11 06:09 -0700
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 12:33 -0700
Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-12 03:29 -0700
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 10:01 -0700
Re: Regression testing for pointers "BartC" <bc@freeuk.com> - 2012-03-13 18:56 +0000
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 16:31 -0700
Re: Regression testing for pointers Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-12 00:30 +0000
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 16:41 -0700
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-12 13:39 +1300
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 09:36 -0700
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 07:25 +1300
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 16:28 -0700
Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-13 04:20 +0000
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 17:34 +1300
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 00:29 -0700
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 17:31 +1300
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 01:05 -0700
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 21:30 +1300
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 10:43 -0700
Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-10 15:41 -0800
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 11:23 -0700
Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-13 13:43 -0700
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 01:45 -0700
Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-14 15:15 -0700
Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-14 22:57 +0000
Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-15 13:10 -0700
Re: Regression testing for pointers jgk@panix.com (Joe keane) - 2012-03-16 22:20 +0000
Re: Regression testing for pointers "io_x" <a@b.c.invalid> - 2012-03-18 07:15 +0100
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 23:13 -0700
Re: Regression testing for pointers gordonb.b29ri@burditt.org (Gordon Burditt) - 2012-03-12 19:37 -0500
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 19:23 -0700
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 13:12 +1300
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 01:54 -0700
Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 22:05 +1300
Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 11:54 -0700
csiph-web