Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #18816

Re: Method to test a program against different C implementations

Show all headers | View raw

On 03/15/2012 06:09 PM, Jorgen Grahn wrote:
> On Sun, 2012-03-11, pozz wrote:
>> We perfectly know that C implementations can be very different in terms of:
>> - number of bits per byte
>> - number of bytes per int
>> - number of bytes per long
>> - signed or unsigned char
>> - negative number implementation (two's or one's complement)
>> - big or little endian or others
>>
>> I'm asking if there's a way to test a small C source (a function or a 
>> small set of functions) against all possible C implementations.  Of 
>> course I'm thinking of an automatic test, because the number of C 
>> implementations is high.
>>
>> I think this could be done with an ad-hoc C compiler that can be 
>> configured (I imagine thorugh command-line parameteres) in several ways. 
>>   For example, I can say to the compiler to use signed or unsigned char 
>> (when the signedness is not specified), 16-bit ints and 32-bit longs, 
>> big-ending representation, two's complement for negative numbers.
>>
>> So I can write a small script that runs the compiler several times with 
>> different configurations, producing different executables.  Now I can 
>> run these exe and compare the results (their outputs) against "the 
>> correct output" I prepared before.
>>
>> I think this test could be very useful to verify the correctness of our 
>> sources not only against the implementation where we are developing, but 
>> also for other possible implementations that could be used in the future 
>> by the users of our program.
> 
> Most/all of the responses have been surprisingly negative, so here's
> one in favor.
> 
> Yes please, I'd like to have something like what you describe. I see
> practical problems, and I don't think I'll ever see such a system, but
> it's not a stupid idea.

It's a stupid idea as long as it's supposed to test "against all
possible C implementations", apparently under the misconception that the
number of such implementation is not only finite, but indeed small
enough to make it possible to imagine that such testing would be
feasible using "a small script"!

As soon as you select a subset of all possible C implementations that is
small enough to make such testing feasible, you are talking about an
automatic test procedure whose details are highly dependent upon which
subset you've chosen. Such a procedure could be of considerable interest
to those who would have selected a sufficiently similar subset, while
being of little or no interest in those who'd prefer a significantly
different subset.

To be useful, the test should also, ideally, cover all possible sets of
input data - a space that usually has a very high dimensionality, and a
huge volume. For most non-trivial programs it's absolutely, completely,
infeasible to cover that entire space. The best that can be done is a
finite sample of the possible sets. Unless the automated system has
intelligence comparable to that of the professional tester, it's stuck
with randomly searching the space of possible input sets, which is
unlikely to find all the special cases that a professional tester could
identify as worthy of special attention.

Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Method to test a program against different C implementations pozz <pozzugno@gmail.com> - 2012-03-11 12:27 +0100
  Re: Method to test a program against different C implementations "Joachim Schmitz" <nospam.jojo@schmitz-digital.de> - 2012-03-11 13:48 +0100
    Re: Method to test a program against different C implementations pete <pfiland@mindspring.com> - 2012-03-11 09:03 -0500
      Re: Method to test a program against different C implementations James Kuyper <jameskuyper@verizon.net> - 2012-03-11 10:43 -0400
  Re: Method to test a program against different C implementations Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-03-11 09:08 -0400
    Re: Method to test a program against different C implementations pozz <pozzugno@gmail.com> - 2012-03-11 22:12 +0100
      Re: Method to test a program against different C implementations Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-03-11 17:44 -0400
  Re: Method to test a program against different C implementations Don Y <this@isnotme.com> - 2012-03-11 09:39 -0700
  Re: Method to test a program against different C implementations Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-15 22:09 +0000
    Re: Method to test a program against different C implementations James Kuyper <jameskuyper@verizon.net> - 2012-03-15 18:32 -0400
      Re: Method to test a program against different C implementations pozz <pozzugno@gmail.com> - 2012-03-17 16:26 +0100
        Re: Method to test a program against different C implementations James Kuyper <jameskuyper@verizon.net> - 2012-03-17 11:52 -0400
    Re: Method to test a program against different C implementations Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-03-15 20:31 -0400
      Re: Method to test a program against different C implementations Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-20 22:18 +0000
        Re: Method to test a program against different C implementations Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-03-20 22:38 -0400
          Re: Method to test a program against different C implementations Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-22 21:48 +0000
            Re: Method to test a program against different C implementations James Kuyper <jameskuyper@verizon.net> - 2012-03-22 18:39 -0400
            Re: Method to test a program against different C implementations Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-03-22 21:07 -0400
  Re: Method to test a program against different C implementations Robert Miles <robertmilesxyz@gmail.com> - 2012-03-15 17:41 -0700

csiph-web