Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #18532

Re: Method to test a program against different C implementations

Show all headers | View raw

On 3/11/2012 7:27 AM, pozz wrote:
> We perfectly know that C implementations can be very different in terms of:
> - number of bits per byte
> - number of bytes per int
> - number of bytes per long
> - signed or unsigned char
> - negative number implementation (two's or one's complement)
> - big or little endian or others
>
> I'm asking if there's a way to test a small C source (a function or a
> small set of functions) against all possible C implementations. Of
> course I'm thinking of an automatic test, because the number of C
> implementations is high.

     Infinite. (Proof: sizeof(int) can be 1,2,3,... without bound,
and each of these implies a C implementation that differs in at
least one respect from all those with more or fewer bytes per int.
I was going to say "countably infinite," but then I bethought me
of undefined behavior and decided to delete "countably.")

     So, let's give up on "all possible" and consider just "many"
C implementations.  An important part of the question is then the
choice of which actual or possible implementations to test.

> I think this could be done with an ad-hoc C compiler that can be
> configured (I imagine thorugh command-line parameteres) in several ways.
> For example, I can say to the compiler to use signed or unsigned char
> (when the signedness is not specified), 16-bit ints and 32-bit longs,
> big-ending representation, two's complement for negative numbers.
>
> So I can write a small script that runs the compiler several times with
> different configurations, producing different executables. Now I can run
> these exe and compare the results (their outputs) against "the correct
> output" I prepared before.

     Do the programs take any inputs?  File contents, environment
variables, command-line parameters, ...?  Will you test with "all"
combinations of inputs?

     Further: Suppose your "correct" program outputs 2.718281828459045
and another prints 2.718281828459046 -- is the second "wrong?"  How
did you establish that the first was "correct" to begin with?

> I think this test could be very useful to verify the correctness of our
> sources not only against the implementation where we are developing, but
> also for other possible implementations that could be used in the future
> by the users of our program.

     While I agree that testing for portability is a Good Thing, I
think you've underestimated both the difficulty and the effectiveness.
Testing on one system is already difficult and expensive; testing on
N systems is more so, testing on M^N would be worse still.  If your
target platforms are Windows and MacOS, it seems unlikely that the
folks responsible for the development budget will choose to expend
a lot of resources trying to future-proof the code for hypothetical
platforms with expected total sales of zero ...

     Besides, remember Dijkstra: "Testing shows the presence, not the
absence of bugs."

-- 
Eric Sosman
esosman@ieee-dot-org.invalid

Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Method to test a program against different C implementations pozz <pozzugno@gmail.com> - 2012-03-11 12:27 +0100
  Re: Method to test a program against different C implementations "Joachim Schmitz" <nospam.jojo@schmitz-digital.de> - 2012-03-11 13:48 +0100
    Re: Method to test a program against different C implementations pete <pfiland@mindspring.com> - 2012-03-11 09:03 -0500
      Re: Method to test a program against different C implementations James Kuyper <jameskuyper@verizon.net> - 2012-03-11 10:43 -0400
  Re: Method to test a program against different C implementations Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-03-11 09:08 -0400
    Re: Method to test a program against different C implementations pozz <pozzugno@gmail.com> - 2012-03-11 22:12 +0100
      Re: Method to test a program against different C implementations Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-03-11 17:44 -0400
  Re: Method to test a program against different C implementations Don Y <this@isnotme.com> - 2012-03-11 09:39 -0700
  Re: Method to test a program against different C implementations Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-15 22:09 +0000
    Re: Method to test a program against different C implementations James Kuyper <jameskuyper@verizon.net> - 2012-03-15 18:32 -0400
      Re: Method to test a program against different C implementations pozz <pozzugno@gmail.com> - 2012-03-17 16:26 +0100
        Re: Method to test a program against different C implementations James Kuyper <jameskuyper@verizon.net> - 2012-03-17 11:52 -0400
    Re: Method to test a program against different C implementations Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-03-15 20:31 -0400
      Re: Method to test a program against different C implementations Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-20 22:18 +0000
        Re: Method to test a program against different C implementations Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-03-20 22:38 -0400
          Re: Method to test a program against different C implementations Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-03-22 21:48 +0000
            Re: Method to test a program against different C implementations James Kuyper <jameskuyper@verizon.net> - 2012-03-22 18:39 -0400
            Re: Method to test a program against different C implementations Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-03-22 21:07 -0400
  Re: Method to test a program against different C implementations Robert Miles <robertmilesxyz@gmail.com> - 2012-03-15 17:41 -0700

csiph-web