Re: Regression testing for pointers

From	Don Y <this@isnotme.com>
Newsgroups	comp.lang.c
Subject	Re: Regression testing for pointers
Date	2012-03-11 12:33 -0700
Organization	Aioe.org NNTP Server
Message-ID	<jjium2$4lp$1@speranza.aioe.org> (permalink)
References	(1 earlier) <slrnjlm4v4.1ls.grahn+nntp@frailea.sa.invalid> <jjf6ap$ot3$1@speranza.aioe.org> <jjftfp$ksf$1@dont-email.me> <jjg7o0$hh7$1@speranza.aioe.org> <19145559.3836.1331471399326.JavaMail.geo-discussion-forums@vbas10>

Show all headers | View raw

Hi Nick,

On 3/11/2012 6:09 AM, nick_keighley_nospam@hotmail.com wrote:

<snip>

>>> Maybe you're just trying too hard. Are you trying to produce some software,
>>> which, without benefit of alpha and beta versions, can be released as a
>>> version 1.000 with *no* bugs?
>>
>> First of all, I consider it sinful to release code that hasn't
>> been tested.  "Well, I ran it a couple of times and it *looked*
>> like it was working.  Play with it and tell me what you think..."
>> is not a valid attitude, IMO.
>
> that's not quite the same as "no bugs"

Exactly!  Yet that seems to be the most common approach to testing
nowadays (how many folks have formal specifications, test suites,
validation procedures, etc.).

You have to consciously *attack*/challenge a piece of code to
identify deficiencies.  Where are the boundary conditions?
What did the developer *not* expect?  etc.

I tend to be pretty good at coming up with "unexpected"
operating conditions.  When the developer invariably
*whines* saying "but you're not supposed to *do* that!"
I grin and reply, "Then don't LET me!"

>>> And then there's performance to be considered; it might work properly, but
>>> too slowly. Or it might only slow down after a particular pattern of
>>> inputs.
>>
>> That, in itself (in this particular case) is actually one of the
>> things you want to test for!  *If* the code is functioning
>> properly, then I can expect performance to be X in this situation
>> (based on the deliberate actions I have taken to fragment the
>> free list).
>
> I'll be impressed if you succeed

This is really not hard to do.  Since most of my work is for
real-time applications, *how* the algorithm performs (timeliness)
is as important as what it *does*.

This was the motivation for developing this version of the
memory management library -- since an application must be
able to accurately predict how a particular request will
be satisfied INCLUDING HOW LONG IT WILL TAKE.

>>> Functions such as allocators are very difficult to test. And it might be
>>> difficult to just think up suitable tests, without feedback from actual
>>> trials.
>>
>> How could a "trial" possibly give me more information than I
>> have available when examining the source?
>
> because the real world is a constant surprise

Engineering is the art of anticipating those surprises!
In the "ethereal" world of software, this is considerably easier
than in the physical world  ("What happens if a guy whacks it
with a 20lb sledge hammer?")

>> If I know my art,
>> then I should be able to anticipate damn near every possible
>> case that the code will encounter -- moreso than an application
>> is likely to stumble upon in the course of pursuing *its* goal.
>
> "should" being the operative word!

Of course!  That's what makes me good at what I do!  :>

> "
> GLENDOWER
> I can call spirits from the vasty deep.
>
> HOTSPUR
>    Why, so can I, or so can any man,
>    But will they come when you do call for them?
> "
>
> This is one of the reasons the formal methods approach has been sluggish.
> In theory you prove a program is correct without ever executing it. But
> people rarely do this on any large scale.

Actually, this is common on very large scale projects (which,
otherwise, would never be able to run a single invocation to
completion, otherwise).

Where it *doesn't* happen is smaller projects, "lone wolf"
developers, "immature" organizations, etc.  "Pay me now or
pay me later" is a great mantra when it comes to software
quality and testing, in general.

> The writers of the Space Shuttle software might have acheived  to what
> you're trying to do. At enourmous cost.

There are many industries where formal testing and validation
are the norm and "required" practices.  Surprisingly, the
cost for this isn't as great as one would think.  Once a
module has been validated, that cost is behind you -- any
future use of the module comes with an *assurance* of proper
operation (within the constraints of the contract/spec).

In these environments, you really benefit from code AND
design reuse.

OTOH, if you are developing "ad hoc" and testing is left
to the whims of the developer, chances are you can't
afford a proper validation environment and you'll just keep
paying for it in OTHER ways (bugs, feature creep, training,
etc.) without ever being able to wrap your hands around the
entire problem.

[Do you get the impression that I am a big advocate of
formalized methods?  :> ]

Thread

Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-09 16:43 -0700
  Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 12:56 +1300
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-09 19:40 -0700
      Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 17:53 +1300
        Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 18:12 +1300
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 02:05 -0700
          Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-10 23:17 +1300
            Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-10 12:00 +0000
              Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 11:00 +1300
                Re: Regression testing for pointers Ike Naar <ike@iceland.freeshell.org> - 2012-03-10 22:23 +0000
                Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-10 22:40 +0000
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 12:12 +1300
                Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-11 00:42 +0000
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 14:26 +1300
                Re: Regression testing for pointers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-03-11 03:09 +0000
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 18:16 +1300
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 12:01 -0700
              Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-11 10:35 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 09:19 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 07:23 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 16:23 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 17:25 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 00:19 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 20:34 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 01:00 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 21:22 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 10:19 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 07:25 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 02:00 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 22:18 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 11:32 -0700
                Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-15 05:58 -0700
                Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-15 05:55 -0700
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-15 11:32 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-16 18:14 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-20 01:43 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-20 21:59 +1300
                Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-16 03:05 -0700
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-20 02:33 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-24 11:49 +1300
          Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-11 05:01 -0700
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 09:44 -0700
              Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-12 09:56 +1300
    Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-10 02:28 -0800
      Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 11:09 -0700
      Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-10 18:29 +0000
        Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-11 05:22 -0700
          Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 09:50 -0700
            Re: Regression testing for pointers "io_x" <a@b.c.invalid> - 2012-03-13 13:09 +0100
  Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-10 03:14 +0000
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 12:42 -0700
  Re: Regression testing for pointers Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-10 08:42 +0000
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 02:19 -0700
      Re: Regression testing for pointers "BartC" <bc@freeuk.com> - 2012-03-10 15:54 +0000
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-10 11:49 -0700
          Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-11 06:09 -0700
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 12:33 -0700
              Re: Regression testing for pointers nick_keighley_nospam@hotmail.com - 2012-03-12 03:29 -0700
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 10:01 -0700
                Re: Regression testing for pointers "BartC" <bc@freeuk.com> - 2012-03-13 18:56 +0000
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 16:31 -0700
      Re: Regression testing for pointers Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-12 00:30 +0000
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 16:41 -0700
      Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-12 13:39 +1300
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 09:36 -0700
          Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 07:25 +1300
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 16:28 -0700
              Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-13 04:20 +0000
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 17:34 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 00:29 -0700
              Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 17:31 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 01:05 -0700
                Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-13 21:30 +1300
                Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-13 10:43 -0700
  Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-10 15:41 -0800
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-11 11:23 -0700
      Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-13 13:43 -0700
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 01:45 -0700
          Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-14 15:15 -0700
            Re: Regression testing for pointers Kaz Kylheku <kaz@kylheku.com> - 2012-03-14 22:57 +0000
              Re: Regression testing for pointers Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-03-15 13:10 -0700
                Re: Regression testing for pointers jgk@panix.com (Joe keane) - 2012-03-16 22:20 +0000
                Re: Regression testing for pointers "io_x" <a@b.c.invalid> - 2012-03-18 07:15 +0100
            Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 23:13 -0700
  Re: Regression testing for pointers gordonb.b29ri@burditt.org (Gordon Burditt) - 2012-03-12 19:37 -0500
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-12 19:23 -0700
  Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 13:12 +1300
    Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 01:54 -0700
      Re: Regression testing for pointers Ian Collins <ian-news@hotmail.com> - 2012-03-14 22:05 +1300
        Re: Regression testing for pointers Don Y <this@isnotme.com> - 2012-03-14 11:54 -0700

csiph-web