Re: Regression testing for pointers

From	Ian Collins <ian-news@hotmail.com>
Newsgroups	comp.lang.c
Subject	Re: Regression testing for pointers
Date	2012-03-20 21:59 +1300
Message-ID	<9sqv87Fc6rU2@mid.individual.net> (permalink)
References	(18 earlier) <jjpmn4$9j0$1@speranza.aioe.org> <20559117.329.1331816158025.JavaMail.geo-discussion-forums@vbhy1> <jjtcje$t89$1@speranza.aioe.org> <9sg0iaFgvgU1@mid.individual.net> <jk9fuo$puo$1@speranza.aioe.org>

Show all headers | View raw

On 03/20/12 09:43 PM, Don Y wrote:
> Hi Ian,
>
> On 3/15/2012 10:14 PM, Ian Collins wrote:
>
>>>>>>>> I still think you are testing at too high a level, test the
>>>>>>>> functions
>>>>>>>> that manipulate your internal independently.
>>>>>>>
>>>>>>> There *is* nothing else. the allocator and the deallocator
>>>>>>> are the *only* things that massage the internal state of the
>>>>>>> heap!
>>>>>>
>>>>>> The you should extract some of the internal functionality and test it.
>>>>>
>>>>> I can essentially do this by examining the results of
>>>>> successive invocations designed to expose aspects of that
>>>>> state. (see other post to Malcolm, this date)
>>>>
>>>> seems like doing things the hard way. *And* it's error prone. You
>>>> need white-box knowledge to test your function thoughourly- so why
>>>> not make it explicit in your test harness?
>>>
>>> Doing so means you have now determined the *implementation*
>>> as well as the functionality.
>>>
>>> If you want to split out "sub-functions", do so. Then,
>>> formally specify how they must work. And, formally
>>> specify a test suite for them. Thereafter, the implementation
>>> of the allocator must preserve all of these interfaces.
>>
>> No, you do not. The interface of the allocator is formally specified,
>> not its internals.
>
> Great!   What is your internal test SUPPOSED TO DO when
> given ___________?  How do you *know* what it is supposed to
> do in that case?

As I said a while back, most platforms have a number of interchangeable 
allocators optimised for various applications.  Their internals may be 
completely different, but they still conform to the the standard 
allocator requirements.

> What happens when your boss wants to
> throw some manpower at your task to improve your chances of
> meeting a deadline and he decides splitting of the development
> of the test suite for<whatever>  should be a nice, easily
> compartmentalized task.  The guy who is writing the test suite
> for you wants to know what he should *expect* your code to do
> given ___________.

We don't work that way.  In the TDD world, the bloke (or pair) writing 
the tests is the bloke (or pair) writing the code.

> Once you *have* specified this *internal* interface, any
> changes you make to *that* interface require the rewriting
> and revalidation of the layers *above* and *below*.  I.e.,
> the parts of your<top_level_function>  that rely on that
> <lower_level_implementation>  now have to change.  And, that
> <lower_level_implementation>  also has to change to agree
> with the new interface definition.

That argument could go round in circles for ever.

>>> The whole point of regression testing is to formalize the
>>> test cases and ensure changes to a function preserve its
>>> interface semantics and functionality.
>>
>> Quite. If you make a change to the internals that breaks the published
>> interface, the published interface test will fail. You could have
>> several stets of internals for different environments and be free to
>> substitute one for another without breaking the published interface.
>
> While you are away at lunch, I replace the code for function()
> with a new implementation.  It adheres to the PUBLISHED interface
> for function().  But, fails ALL of your "internal tests" because
> it is implemented in an entirely different manner (that is NOT
> mandated by the published specification).

It couldn't. You didn't change the internal code, you replaced it with 
some new code, presumably with its own tests.  In other words, you 
produced another alternative implementation.

> I haven't done anything wrong.  But, my code fails your
> tests -- because your tests are looking at undocumented
> aspects of *an* implementation.

Nope.

> I rewrite sqrt():  the new RUN-TIME version causes a
> packet to be sent across the internet to an accountant
> sitting in a smoke-filled office.  The value for which
> you are seeking the sqrt appears on his display.  He
> takes out his pencil and a clean ream of paper and
> gets to work on the problem.  When he is done, some
> time later, he types the answer in on his keyboard and
> a packet is returned to the WAITING application.  The
> sqrt() code then converts this ASCII decimal string
> into a suitable "double" and returns the value to the
> caller.
>
> There is *nothing* about this implementation that
> violates the letter of the sqrt() interface specification.
> Granted, it's dog slow and only works on machines with
> internet connectivity.  But, it still *works*.  It
> passes the regression tests for sqrt().
>
> But, *doesn't* pass any of your undocumented internal
> tests -- because you probably wouldn't have conceived of
> such an approach.

While sightly exaggerated, that isn't an uncommon situation.  For 
example an application may happily be working away processing data from 
a local store.  At some point that data is moved to another system and 
is accessed through a web service.  So the persistence layer gets 
replaced.  The application still passes all its regression tests because 
they don't care how the application outputs 2 when 4 is input, They just 
care that it does.

-- 
Ian Collins

Thread

Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-09 16:43 -0700
  Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 12:56 +1300
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-09 19:40 -0700
      Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 17:53 +1300
        Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 18:12 +1300
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 02:05 -0700
          Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 23:17 +1300
            Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-10 12:00 +0000
              Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 11:00 +1300
                Re: Regression testing for pointers Ike Naar <ike@iceland.freeshell.org> - 2012-03-10 22:23 +0000
                Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-10 22:40 +0000
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 12:12 +1300
                Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-11 00:42 +0000
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 14:26 +1300
                Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-11 03:09 +0000
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 18:16 +1300
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 12:01 -0700
              Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 10:35 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 09:19 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 07:23 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 16:23 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 17:25 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 00:19 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 20:34 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 01:00 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 21:22 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 10:19 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 07:25 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 02:00 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 22:18 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 11:32 -0700
                Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-15 05:58 -0700
                Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-15 05:55 -0700
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-15 11:32 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-16 18:14 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-20 01:43 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-20 21:59 +1300
                Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-16 03:05 -0700
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-20 02:33 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-24 11:49 +1300
          Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-11 05:01 -0700
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 09:44 -0700
              Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-12 09:56 +1300
    Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-10 02:28 -0800
      Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 11:09 -0700
      Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-10 18:29 +0000
        Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-11 05:22 -0700
          Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 09:50 -0700
            Re: Regression testing for pointers "io_x" <a@b.c.invalid> - 2012-03-13 13:09 +0100
  Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-10 03:14 +0000
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 12:42 -0700
  Re: Regression testing for pointers Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-10 08:42 +0000
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 02:19 -0700
      Re: Regression testing for pointers "BartC" <bc@freeuk.com> - 2012-03-10 15:54 +0000
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 11:49 -0700
          Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-11 06:09 -0700
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 12:33 -0700
              Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-12 03:29 -0700
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 10:01 -0700
                Re: Regression testing for pointers "BartC" <bc@freeuk.com> - 2012-03-13 18:56 +0000
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 16:31 -0700
      Re: Regression testing for pointers Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-12 00:30 +0000
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 16:41 -0700
      Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-12 13:39 +1300
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 09:36 -0700
          Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 07:25 +1300
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 16:28 -0700
              Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-13 04:20 +0000
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 17:34 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 00:29 -0700
              Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 17:31 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 01:05 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 21:30 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 10:43 -0700
  Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-10 15:41 -0800
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 11:23 -0700
      Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-13 13:43 -0700
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 01:45 -0700
          Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-14 15:15 -0700
            Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-14 22:57 +0000
              Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-15 13:10 -0700
                Re: Regression testing for pointers jgk@panix.com (Joe keane) - 2012-03-16 22:20 +0000
                Re: Regression testing for pointers "io_x" <a@b.c.invalid> - 2012-03-18 07:15 +0100
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 23:13 -0700
  Re: Regression testing for pointers gordonb.b29ri@burditt.org (Gordon Burditt) - 2012-03-12 19:37 -0500
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 19:23 -0700
  Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 13:12 +1300
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 01:54 -0700
      Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 22:05 +1300
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 11:54 -0700

csiph-web