Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #140541 > unrolled thread

Compiler stress test

Back to article view | Back to comp.lang.c

Contents

  Compiler stress test jacobnavia <jacob@jacob.remcomp.fr> - 2019-02-05 10:30 +0100
    Re: Compiler stress test Ian Collins <ian-news@hotmail.com> - 2019-02-05 22:50 +1300
      Re: Compiler stress test Ian Collins <ian-news@hotmail.com> - 2019-02-05 22:56 +1300
        strict aliasing rule on buffers slacked.misha@gmail.com - 2019-02-05 13:33 +0100
          Re: strict aliasing rule on buffers Richard Damon <Richard@Damon-Family.org> - 2019-02-05 08:13 -0500
          Re: strict aliasing rule on buffers Barry Schwarz <schwarzb@dqel.com> - 2019-02-05 06:00 -0800
          Re: strict aliasing rule on buffers David Brown <david.brown@hesbynett.no> - 2019-02-05 15:33 +0100
          Re: strict aliasing rule on buffers Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-02-05 14:36 +0000
          Re: strict aliasing rule on buffers James Kuyper <jameskuyper@alumni.caltech.edu> - 2019-02-05 07:27 -0800
      Re: Compiler stress test Ian Collins <ian-news@hotmail.com> - 2019-02-07 08:28 +1300
        Re: Compiler stress test Bart <bc@freeuk.com> - 2019-02-06 19:48 +0000
        Re: Compiler stress test jacobnavia <jacob@jacob.remcomp.fr> - 2019-02-06 22:11 +0100
          Re: Compiler stress test David Brown <david.brown@hesbynett.no> - 2019-02-07 00:34 +0100
            Re: Compiler stress test jacobnavia <jacob@jacob.remcomp.fr> - 2019-02-07 07:13 +0100
              Re: Compiler stress test David Brown <david.brown@hesbynett.no> - 2019-02-07 08:32 +0100
                Re: Compiler stress test Bart <bc@freeuk.com> - 2019-02-07 12:06 +0000
                  Re: Compiler stress test David Brown <david.brown@hesbynett.no> - 2019-02-07 14:08 +0100
                    Re: Compiler stress test Bart <bc@freeuk.com> - 2019-02-07 13:49 +0000
                      Re: Compiler stress test David Brown <david.brown@hesbynett.no> - 2019-02-07 17:01 +0100
                    Re: Compiler stress test jacobnavia <jacob@jacob.remcomp.fr> - 2019-02-07 22:26 +0100
                      Re: Compiler stress test David Brown <david.brown@hesbynett.no> - 2019-02-07 23:24 +0100
                        Re: Compiler stress test Bart <bc@freeuk.com> - 2019-02-07 23:20 +0000
                          Re: Compiler stress test David Brown <david.brown@hesbynett.no> - 2019-02-08 10:42 +0100
                        Re: Compiler stress test Ian Collins <ian-news@hotmail.com> - 2019-02-08 16:37 +1300
                          Re: Compiler stress test David Brown <david.brown@hesbynett.no> - 2019-02-08 12:30 +0100
                            Re: Compiler stress test Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-02-08 08:18 -0800
                          Re: Compiler stress test scott@slp53.sl.home (Scott Lurndal) - 2019-02-08 16:04 +0000
                            Re: Compiler stress test Jorgen Grahn <grahn+nntp@snipabacken.se> - 2019-02-11 20:34 +0000
                      Re: Compiler stress test Ian Collins <ian-news@hotmail.com> - 2019-02-08 16:30 +1300
    Re: Compiler stress test Bart <bc@freeuk.com> - 2019-02-05 12:20 +0000
      Re: Compiler stress test Ian Collins <ian-news@hotmail.com> - 2019-02-06 09:59 +1300
        Re: Compiler stress test Bart <bc@freeuk.com> - 2019-02-05 21:57 +0000
          Re: Compiler stress test David Brown <david.brown@hesbynett.no> - 2019-02-06 00:29 +0100
            Re: Compiler stress test Bart <bc@freeuk.com> - 2019-02-05 23:58 +0000
              Re: Compiler stress test Ian Collins <ian-news@hotmail.com> - 2019-02-06 14:10 +1300
                Re: Compiler stress test Bart <bc@freeuk.com> - 2019-02-06 01:16 +0000
                  Re: Compiler stress test Ian Collins <ian-news@hotmail.com> - 2019-02-06 14:21 +1300
                    Re: Compiler stress test Paul <nospam@needed.invalid> - 2019-02-06 01:28 -0500
                      Re: Compiler stress test Paul <nospam@needed.invalid> - 2019-02-08 03:51 -0500
              Re: Compiler stress test David Brown <david.brown@hesbynett.no> - 2019-02-06 08:13 +0100
        Re: Compiler stress test BGB <cr88192@hotmail.com> - 2019-02-06 13:13 -0600

Page 1 of 3  [1] 2 3  Next page →

#140541 — Compiler stress test

Prompted by Bart's discussion, I wrote a small C program that generates 
C source files.
The file is just N functions like:

int fn1(void)
{
         int a=1,b=2,c=3,d=4;
         return a=b+c*d;

}
int fn2(void)
{
         int a=1,b=2,c=3,d=4;
         return a=b+c*d;

}


Etc.

You can pass it as a parameter the number of functions you want to 
generate.

I passed it 1 000 000 and it generated a file of 68 888 896 bytes with 1 
million small functions.

Tests done with an i7 CPU, windows 7 machine with 32GB RAM.

LCC (64 bits) took 5 hours 47 minutes and 50 seconds. The 64 bit object 
file was 98 000 358 bytes.

TCC took 10 seconds, no object file produced, no diagnostics. It just quits.

Microsoft was the winner with just 52 minutes and 13 seconds for its 32 
bit version. The 64 bit version was slower with 1 hour, 27 minutes and 
15 seconds. The 64 bit object file was 234 778 394 bytes.

gcc runs out of memory in 11 seconds (1 second more than tcc). When the 
out of memory condition appears, the machine has still 18 GB of memory free.

d:\lcc>gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=E:/TDM-GCC-64/bin/../libexec/gcc/x86_64-w64-mingw32/5.1.0/lto-wrapper.exe
Target: x86_64-w64-mingw32
Configured with: ../../../src/gcc-5.1.0/configure 
--build=x86_64-w64-mingw32 --enable-targets=all 
--enable-languages=ada,c,c++,fortran,lto,o
bjc,obj-c++ --enable-libgomp --enable-lto --enable-graphite 
--enable-cxx-flags=-DWINPTHREAD_STATIC --disable-build-with-cxx 
--disable-build-
poststage1-with-cxx --enable-libstdcxx-debug --enable-threads=posix 
--enable-version-specific-runtime-libs --enable-fully-dynamic-string --e
nable-libstdcxx-threads --enable-libstdcxx-time --with-gnu-ld 
--disable-werror --disable-nls --disable-win32-registry 
--prefix=/mingw64tdm -
-with-local-prefix=/mingw64tdm --with-pkgversion=tdm64-1 
--with-bugurl=http://tdm-gcc.tdragon.net/bugs
Thread model: posix
gcc version 5.1.0 (tdm64-1)
bash-3.2$

[toc] | [next] | [standalone]

#140543

On 05/02/2019 22:30, jacobnavia wrote:
> Prompted by Bart's discussion, I wrote a small C program that generates
> C source files.
> The file is just N functions like:
> 
> int fn1(void)
> {
>           int a=1,b=2,c=3,d=4;
>           return a=b+c*d;
> 
> }
> int fn2(void)
> {
>           int a=1,b=2,c=3,d=4;
>           return a=b+c*d;
> 
> }
> 
> 
> Etc.
> 
> You can pass it as a parameter the number of functions you want to
> generate.
> 
> I passed it 1 000 000 and it generated a file of 68 888 896 bytes with 1
> million small functions.
> 
> Tests done with an i7 CPU, windows 7 machine with 32GB RAM.
> 
> LCC (64 bits) took 5 hours 47 minutes and 50 seconds. The 64 bit object
> file was 98 000 358 bytes.
> 
> TCC took 10 seconds, no object file produced, no diagnostics. It just quits.
> 
> Microsoft was the winner with just 52 minutes and 13 seconds for its 32
> bit version. The 64 bit version was slower with 1 hour, 27 minutes and
> 15 seconds. The 64 bit object file was 234 778 394 bytes.
> 
> gcc runs out of memory in 11 seconds (1 second more than tcc). When the
> out of memory condition appears, the machine has still 18 GB of memory free.

Is gcc a 32 bit executable on your machine?

On mine it's happily to chug away with 13.6G of RAM...

$ wc -l zz.c
5000000 zz.c

$ time gcc zz.c
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/Scrt1.o: In 
function `_start':
(.text+0x20): undefined reference to `main'
collect2: error: ld returned 1 exit status

real	5m46.880s
user	5m38.221s
sys	0m8.236s

I forgot -c...
$ gcc --version
gcc (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0

-- 
Ian.

[toc] | [prev] | [next] | [standalone]

#140544

On 05/02/2019 22:50, Ian Collins wrote:
> On 05/02/2019 22:30, jacobnavia wrote:
>> Prompted by Bart's discussion, I wrote a small C program that generates
>> C source files.
>> The file is just N functions like:
>>
>> int fn1(void)
>> {
>>            int a=1,b=2,c=3,d=4;
>>            return a=b+c*d;
>>
>> }
>> int fn2(void)
>> {
>>            int a=1,b=2,c=3,d=4;
>>            return a=b+c*d;
>>
>> }
>>
>>
>> Etc.
>>
>> You can pass it as a parameter the number of functions you want to
>> generate.
>>
>> I passed it 1 000 000 and it generated a file of 68 888 896 bytes with 1
>> million small functions.
>>
>> Tests done with an i7 CPU, windows 7 machine with 32GB RAM.
>>
>> LCC (64 bits) took 5 hours 47 minutes and 50 seconds. The 64 bit object
>> file was 98 000 358 bytes.
>>
>> TCC took 10 seconds, no object file produced, no diagnostics. It just quits.
>>
>> Microsoft was the winner with just 52 minutes and 13 seconds for its 32
>> bit version. The 64 bit version was slower with 1 hour, 27 minutes and
>> 15 seconds. The 64 bit object file was 234 778 394 bytes.
>>
>> gcc runs out of memory in 11 seconds (1 second more than tcc). When the
>> out of memory condition appears, the machine has still 18 GB of memory free.
> 
> Is gcc a 32 bit executable on your machine?
> 
> On mine it's happily to chug away with 13.6G of RAM...
> 
> $ wc -l zz.c
> 5000000 zz.c
> 
> $ time gcc zz.c
> /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/Scrt1.o: In
> function `_start':
> (.text+0x20): undefined reference to `main'
> collect2: error: ld returned 1 exit status
> 
> real	5m46.880s
> user	5m38.221s
> sys	0m8.236s
> 
> I forgot -c...
> $ gcc --version
> gcc (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0
> 
With -c: 


$ time gcc -c zz.c

real	5m41.981s
user	5m34.321s

$ ls -lh zz.o
-rw-r--r-- 1 ian ian 137M Feb  5 22:54 zz.o

-- 
Ian.

[toc] | [prev] | [next] | [standalone]

#140549 — strict aliasing rule on buffers

Let's consider following example


    #include <stdio.h>

    struct msg
    {
        int a;
        int b;
    };

    /* simulate receiving data from another pc (serial, network etc)
     */
    void read_data(void *buf)
    {
        int i;
        char *b = buf;

        for (i = 0; i != sizeof(struct msg); ++i)
        {
            b[i] = i;
        }
    }

    void process_data(void *buf)
    {
        /* now does it break strict aliasing rule? (void *) can be casted
         * to any type, but here (char *) was casted to (void *) and then
         * to (struct msg *)
         */
        struct msg *m = buf;
        printf("%d, %d\n", m->a, m->b);
    }

    int main(void)
    {
        char b[128] = {0};
        read_data(b);
        process_data(b);
        return 0;
    }


The question is: does this code break strict aliasing rule? According to C
standard, it is:

 - legal to cast anything to (void *) and then back to same anything. 
 - legal to cast anything to (char *) but you cannot cast
 - not legal to cast anything to (char *)

But in this example (char *) is casted to (void *) and then, that (void *)
is casted to (struct msg *). Compiler doesn't complain about anything - thanks
to those (void *) casts, but that doesn't mean it is ok.

So, does this code really break strict aliasing and is undefined behaviour?
If so, are there any other options to case received data to struct waithout
doing copies (like simple casting)? This doesn't seem to be the problem on
big machines where Linux is running, but it may be a huge memory overheat on
small microcontrollers.

And please, let's not talk about other problems this code has, like endianess,
padding, int size etc. Please, only consider aliasing problem.

[toc] | [prev] | [next] | [standalone]

#140552 — Re: strict aliasing rule on buffers

On 2/5/19 7:33 AM, slacked.misha@gmail.com wrote:
> Let's consider following example
> 
> 
>     #include <stdio.h>
> 
>     struct msg
>     {
>         int a;
>         int b;
>     };
> 
>     /* simulate receiving data from another pc (serial, network etc)
>      */
>     void read_data(void *buf)
>     {
>         int i;
>         char *b = buf;
> 
>         for (i = 0; i != sizeof(struct msg); ++i)
>         {
>             b[i] = i;
>         }
>     }
> 
>     void process_data(void *buf)
>     {
>         /* now does it break strict aliasing rule? (void *) can be casted
>          * to any type, but here (char *) was casted to (void *) and then
>          * to (struct msg *)
>          */
>         struct msg *m = buf;
>         printf("%d, %d\n", m->a, m->b);
>     }
> 
>     int main(void)
>     {
>         char b[128] = {0};
>         read_data(b);
>         process_data(b);
>         return 0;
>     }
> 
> 
> The question is: does this code break strict aliasing rule? According to C
> standard, it is:
> 
>  - legal to cast anything to (void *) and then back to same anything. 
>  - legal to cast anything to (char *) but you cannot cast
>  - not legal to cast anything to (char *)
> 
> But in this example (char *) is casted to (void *) and then, that (void *)
> is casted to (struct msg *). Compiler doesn't complain about anything - thanks
> to those (void *) casts, but that doesn't mean it is ok.
> 
> So, does this code really break strict aliasing and is undefined behaviour?
> If so, are there any other options to case received data to struct waithout
> doing copies (like simple casting)? This doesn't seem to be the problem on
> big machines where Linux is running, but it may be a huge memory overheat on
> small microcontrollers.
> 
> And please, let's not talk about other problems this code has, like endianess,
> padding, int size etc. Please, only consider aliasing problem.
> 

One issue with talking about aliasing in this code is that accessing via
a char* is one of the exceptions to the aliasing rule, you are allowed
to access anything via a char* pointer.

Now, since the real object is a char array, accessing that through a
pointer to struct, causes aliasing issues (as well as alignment issues).
If main had declared the buffer as a struct msg, then the read call,
which access via a char point would be ok because of that exception.

Not sure where you got the rule that is wasn't legal to cast anything to
char*, it is guaranteed that any pointer will round trip through a char*
as well as a void* (it won't implicitly convert, but you can explicitly
cast), and you can also access the representation of any object via a
char* (or unsigned char* or signed char*).

[toc] | [prev] | [next] | [standalone]

#140554 — Re: strict aliasing rule on buffers

On Tue, 5 Feb 2019 13:33:21 +0100, slacked.misha@gmail.com wrote:

>Let's consider following example
>
>
>    #include <stdio.h>
>
>    struct msg
>    {
>        int a;
>        int b;
>    };
>
>    /* simulate receiving data from another pc (serial, network etc)
>     */
>    void read_data(void *buf)
>    {
>        int i;
>        char *b = buf;
>
>        for (i = 0; i != sizeof(struct msg); ++i)
>        {
>            b[i] = i;
>        }
>    }
>
>    void process_data(void *buf)
>    {
>        /* now does it break strict aliasing rule? (void *) can be casted

There are no casts in your code.  All your pointer assignments perform
implicit conversions specifically allowed by section 6.3.2.3-1 so
there is no reason for the compiler to complain.

Section 6.3.2.3-7 identifies the conditions under which performing
such a conversion results in undefined behavior during execution.  It
is your job as a programmer to insure this does not happen.

>         * to any type, but here (char *) was casted to (void *) and then
>         * to (struct msg *)

Since char* and void* are required to have the same representation and
alignment requirements (6.2.5-28), your use of void* does change
anything.  You could achieve the same affect by using a cast operator
directly on a char*, as in
     void process_data(char *buf){
          struct msg *m = (struct msg*)buf;

>         */
>        struct msg *m = buf;
>        printf("%d, %d\n", m->a, m->b);
>    }
>
>    int main(void)
>    {
>        char b[128] = {0};
>        read_data(b);
>        process_data(b);
>        return 0;
>    }
>
>
>The question is: does this code break strict aliasing rule? According to C
>standard, it is:

The aliasing rule is in section 6.5-7.  YES, you do violate this rule.
The following are stated incorrectly and even when corrected deal with
aliasing only tangentially

> - legal to cast anything to (void *) and then back to same anything. 
> - legal to cast anything to (char *) but you cannot cast
> - not legal to cast anything to (char *)

It is legal to convert in either direction between void* and any
pointer to object type.  Conversion between different types of object
pointer cannot be performed implicitly.  They require a cast operator.

>But in this example (char *) is casted to (void *) and then, that (void *)
>is casted to (struct msg *). Compiler doesn't complain about anything - thanks
>to those (void *) casts, but that doesn't mean it is ok.

There are no casts and while it may appear to work it is definitely
not OK.

>So, does this code really break strict aliasing and is undefined behaviour?

Yes, it breaks aliasing.  Whether it results in undefined behavior
depends on the things you ask not to talk about below.

>If so, are there any other options to case received data to struct waithout
>doing copies (like simple casting)? This doesn't seem to be the problem on
>big machines where Linux is running, but it may be a huge memory overheat on
>small microcontrollers.

Look at unions.  See if section 6.7.2.1-16 and footnote 95 in section
6.5.2.3-3 solves your problem.

>And please, let's not talk about other problems this code has, like endianess,
>padding, int size etc. Please, only consider aliasing problem.

-- 
Remove del for email

[toc] | [prev] | [next] | [standalone]

#140555 — Re: strict aliasing rule on buffers

On 05/02/2019 13:33, slacked.misha@gmail.com wrote:
<snip>
> The question is: does this code break strict aliasing rule? According to C
> standard, it is:
> 
>  - legal to cast anything to (void *) and then back to same anything. 
>  - legal to cast anything to (char *) but you cannot cast
>  - not legal to cast anything to (char *)
> 
> But in this example (char *) is casted to (void *) and then, that (void *)
> is casted to (struct msg *). Compiler doesn't complain about anything - thanks
> to those (void *) casts, but that doesn't mean it is ok.
> 
> So, does this code really break strict aliasing and is undefined behaviour?
> If so, are there any other options to case received data to struct waithout
> doing copies (like simple casting)? This doesn't seem to be the problem on
> big machines where Linux is running, but it may be a huge memory overheat on
> small microcontrollers.
> 
> And please, let's not talk about other problems this code has, like endianess,
> padding, int size etc. Please, only consider aliasing problem.
> 

But first, let us talk about a problem with the post.  You have made the
post as a reply to an existing post, thus starting a new thread in the
middle of another thread.  This is confusing and unhelpful - for a new
topic and a new question (which we greatly welcome in this group),
please start a new thread.

(You'll find it all much easier if you drop the horrendous "google
groups" interface and get a proper newsreader and newsserver.  I
recommend Thunderbird as a popular newsreader that works for any system,
and eternal-september.org as a free newserver.  Other people have
different choices, but those are fine to start with.)


On to the code.  As others have said, you don't have casts - you have
conversions.  A "cast" is just an explicit way to make a conversion,
while "conversion" is the general term and also applies when you assign
to and from "void*".

I believe a main point of your confusion is mixing the concepts of
converting pointers, and accessing data via pointers.  It is fine in C
to do the kinds of conversions of the pointers that you have in your
code.  But that does not mean that it is fine to use those pointers to
access data.  In particular, you can convert your pointer-to-char into a
pointer-to-void, and then into a pointer-to-struct, but that does not
mean that you can use it to access data via the pointer-to-struct.
About the only thing you can do (guaranteed by the C standards) is to
convert it back to a pointer-to-void and then back to a pointer-to-char.
 Even that is not guaranteed to give you the original pointer, though it
will do on almost all processors.

Separate your thoughts about what /accesses/ are allowed, and what
/conversions/ are allowed.  See if that helps you.

[toc] | [prev] | [next] | [standalone]

#140557 — Re: strict aliasing rule on buffers

slacked.misha@gmail.com writes:

>     #include <stdio.h>
>
>     struct msg
>     {
>         int a;
>         int b;
>     };
>
>     /* simulate receiving data from another pc (serial, network etc)
>      */
>     void read_data(void *buf)
>     {
>         int i;
>         char *b = buf;
>
>         for (i = 0; i != sizeof(struct msg); ++i)
>         {
>             b[i] = i;
>         }
>     }
>
>     void process_data(void *buf)
>     {
>         /* now does it break strict aliasing rule? (void *) can be casted
>          * to any type, but here (char *) was casted to (void *) and then
>          * to (struct msg *)
>          */
>         struct msg *m = buf;
>         printf("%d, %d\n", m->a, m->b);
>     }
>
>     int main(void)
>     {
>         char b[128] = {0};
>         read_data(b);
>         process_data(b);
>         return 0;
>     }

There are no aliasing issues that I can see.  But you do have an
alignment issue.: you can't be sure that b is suitable aligned for a
struct msg object.  In the actual code, it's possible that b will be
allocated by malloc, in which case all is fine, but if you need to
declare b (and you don't have C11's new alignment control), put b into a
union:

  union { char b[128]; struct msg _dummy } u = {0};
  read_data(u.b);
  process_data(u.b);

> And please, let's not talk about other problems this code has, like endianess,
> padding, int size etc. Please, only consider aliasing problem.

You don't exclude alignment, so I talked about it!

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#140560 — Re: strict aliasing rule on buffers

On Tuesday, February 5, 2019 at 7:33:36 AM UTC-5, slacke...@gmail.com wrote:
> Let's consider following example
> 
> 
>     #include <stdio.h>
> 
>     struct msg
>     {
>         int a;
>         int b;
>     };
> 
>     /* simulate receiving data from another pc (serial, network etc)
>      */
>     void read_data(void *buf)
>     {
>         int i;
>         char *b = buf;
> 
>         for (i = 0; i != sizeof(struct msg); ++i)
>         {
>             b[i] = i;
>         }
>     }
> 
>     void process_data(void *buf)
>     {
>         /* now does it break strict aliasing rule? (void *) can be casted
>          * to any type, but here (char *) was casted to (void *) and then
>          * to (struct msg *)
>          */
>         struct msg *m = buf;
>         printf("%d, %d\n", m->a, m->b);
>     }
> 
>     int main(void)
>     {
>         char b[128] = {0};

Since b has automatic storage duration, it's effective type is the same as it's declared type, char[128].

>         read_data(b);
>         process_data(b);
>         return 0;
>     }
> 
> 
> The question is: does this code break strict aliasing rule? According to C
> standard, it is:
> 
>  - legal to cast anything to (void *) and then back to same anything. 
>  - legal to cast anything to (char *) but you cannot cast
>  - not legal to cast anything to (char *)
> 
> But in this example (char *) is casted to (void *) and then, that (void *)
> is casted to (struct msg *). Compiler doesn't complain about anything - thanks
> to those (void *) casts, but that doesn't mean it is ok.
> 
> So, does this code really break strict aliasing and is undefined behaviour?

m->a and m->b are lvalues of type 'int'. The effective type of the memory those expressions are accessing is an array of char; they're each actually accessing sizeof(int) elements of that array. It's not entirely clear to me whether I should apply the aliasing rules to "char" or "char[sizeof(int)]", but the results are equally bad either way, so I'll go with char: "An object shall have its stored value accessed only by an lvalue expression that has one of the following types:" (6.5p7). None of the cases listed below that sentence covers this case. Going over the permitted options one by one:

- "int" is not compatible with "char"
- "int" is not a qualified version of "char" (for example, const char).
- "int" is a signed type, but is not the signed type corresponding to "char" (which is "signed char").
- "int" is not a signed type corresponding to any qualified version of "char"
- "int" is not a struct or union type.
- "int" is not a character type.

"If a ‘‘shall’’ or ‘‘shall not’’ requirement that appears outside of a constraint or runtimeconstraint is violated, the behavior is undefined." (4p2).

> If so, are there any other options to case received data to struct waithout
> doing copies (like simple casting)? This doesn't seem to be the problem on
> big machines where Linux is running, but it may be a huge memory overheat on
> small microcontrollers.

Sure. Change "char b[128]" to "struct msg b". The rest of your code could remain unchanged (though I would not recommend it - I'd change process_data() to take a struct msg* argument, since that would simplify the code).

> And please, let's not talk about other problems this code has, like endianess,
> padding, int size etc. Please, only consider aliasing problem.

OK, as you request, I'll ignore the other important reasons why your code may have undefined behavior. But my recommended fix will cover most of those issues, too.

[toc] | [prev] | [next] | [standalone]

#140660

On 05/02/2019 22:50, Ian Collins wrote:
> On 05/02/2019 22:30, jacobnavia wrote:
>>
>> gcc runs out of memory in 11 seconds (1 second more than tcc). When the
>> out of memory condition appears, the machine has still 18 GB of memory free.
> 
> Is gcc a 32 bit executable on your machine?

Ghosting?

-- 
Ian.

[toc] | [prev] | [next] | [standalone]

#140662

On 06/02/2019 19:28, Ian Collins wrote:
> On 05/02/2019 22:50, Ian Collins wrote:
>> On 05/02/2019 22:30, jacobnavia wrote:
>>>
>>> gcc runs out of memory in 11 seconds (1 second more than tcc). When the
>>> out of memory condition appears, the machine has still 18 GB of 
>>> memory free.
>>
>> Is gcc a 32 bit executable on your machine?
> 
> Ghosting?

Yes, it could have been bait, to hopefully let me waste 5 hours waiting 
for a compiler to complete (did he really hang about for 6 hours on one 
test?).

However, I think I spent somewhat less than an hour on my first reply. 
Not having the patience to wait more than 1-2 minutes for any test helped.

[toc] | [prev] | [next] | [standalone]

#140666

Le 06/02/2019 à 20:28, Ian Collins a écrit :
> On 05/02/2019 22:50, Ian Collins wrote:
>> On 05/02/2019 22:30, jacobnavia wrote:
>>>
>>> gcc runs out of memory in 11 seconds (1 second more than tcc). When the
>>> out of memory condition appears, the machine has still 18 GB of 
>>> memory free.
>>
>> Is gcc a 32 bit executable on your machine?
> 
> Ghosting?
> 
not really. I didn't have any access to my windows machine

I am using TDM-GCC, and I suppose that a 64 bit compiler would be a 64 
bit executable

Wrong!

It is a 32 bit executable... GREAT!

It just fails then.

[toc] | [prev] | [next] | [standalone]

#140670

On 06/02/2019 22:11, jacobnavia wrote:
> Le 06/02/2019 à 20:28, Ian Collins a écrit :
>> On 05/02/2019 22:50, Ian Collins wrote:
>>> On 05/02/2019 22:30, jacobnavia wrote:
>>>>
>>>> gcc runs out of memory in 11 seconds (1 second more than tcc). When the
>>>> out of memory condition appears, the machine has still 18 GB of 
>>>> memory free.
>>>
>>> Is gcc a 32 bit executable on your machine?
>>
>> Ghosting?
>>
> not really. I didn't have any access to my windows machine
> 
> I am using TDM-GCC, and I suppose that a 64 bit compiler would be a 64 
> bit executable
> 
> Wrong!
> 
> It is a 32 bit executable... GREAT!
> 
> It just fails then.

There is no particular reason why a compiler for a 64-bit target should 
itself be a 64-bit program.  It is only in very odd cases (like this 
stress test) where the memory limitations of a 32-bit program would be 
relevant in compilation.  And on Windows, 32-bit programs are typically 
faster than 64-bit versions.

[toc] | [prev] | [next] | [standalone]

#140699

Le 07/02/2019 à 00:34, David Brown a écrit :

> There is no particular reason why a compiler for a 64-bit target should 
> itself be a 64-bit program.  

Rule 1:

GCC is always right even if it fails.

> It is only in very odd cases (like this 
> stress test) where the memory limitations of a 32-bit program would be 
> relevant in compilation.  

RULE 2:

If GCC fails, RULE 1 applies.


> And on Windows, 32-bit programs are typically 
> faster than 64-bit versions.

Sure, but all 32 bit processes under windows can use up to 4GB of RAM if 
they turn on the LARGEADDRESSAWARE flag in the executable header.

gcc does NOT turn ON that flag, and doesn't even use the 2GB available 
to it but...

RULE 3
Technical resons do not count, because RULE 1 always applies.

[toc] | [prev] | [next] | [standalone]

#140702

On 07/02/2019 07:13, jacobnavia wrote:
> Le 07/02/2019 à 00:34, David Brown a écrit :
> 
>> There is no particular reason why a compiler for a 64-bit target 
>> should itself be a 64-bit program. 
> 
> Rule 1:
> 
> GCC is always right even if it fails.

If you say so.  No one will agree with you, but whatever makes you happy.

And it is, of course, irrelevant to my point - the host system and bit 
size of a compiler does not need any correlation with the target 
processor and bit size.

> 
>> It is only in very odd cases (like this stress test) where the memory 
>> limitations of a 32-bit program would be relevant in compilation. 
> 
> RULE 2:
> 
> If GCC fails, RULE 1 applies.
> 

If you are doing a compile that takes more than 2 GB ram (within one 
compile), or hits other 32-bit limits, you are doing something very odd. 
  You /might/ be doing something useful, but it is certainly unusual. 
The /huge/ majority of C programmers will get through their entire 
programming lives without ever having an issue with this.

If you are doing C++ programming with huge code bases, or you are doing 
large link-time optimisation builds, then you are going to need a lot of 
ram when compiling.  But not for compiling any realistic C file.

> 
>> And on Windows, 32-bit programs are typically faster than 64-bit 
>> versions.
> 
> Sure, but all 32 bit processes under windows can use up to 4GB of RAM if 
> they turn on the LARGEADDRESSAWARE flag in the executable header.
> 
> gcc does NOT turn ON that flag, and doesn't even use the 2GB available 
> to it but...

So what?  Why would anyone care?

> 
> RULE 3
> Technical resons do not count, because RULE 1 always applies.
> 

Pull your head out your backside, switch off your rampant paranoia, and 
read what I wrote.

[toc] | [prev] | [next] | [standalone]

#140712

On 07/02/2019 07:32, David Brown wrote:
> On 07/02/2019 07:13, jacobnavia wrote:

>> If GCC fails, RULE 1 applies.
>>
> 
> If you are doing a compile that takes more than 2 GB ram (within one 
> compile), or hits other 32-bit limits, you are doing something very odd. 
>   You /might/ be doing something useful, but it is certainly unusual. 
> The /huge/ majority of C programmers will get through their entire 
> programming lives without ever having an issue with this.

You seem to be an apologist for gcc.

If there is a version of gcc for (eg) x64 that targets x64, then there 
is no reason for gcc itself not to take advantage of that by building 
itself as an x64 executable. After all it will be run under a 64-bit OS, 
and most other programs will be 64-bit too.

Yes, a 32-bit executable may run a few percent faster, unless it's doing 
lots of 64-bit arithmetic, but it's already clear with gcc that 
compilation speed is not a priority.

Why put an arbitrary limit on the sizes of programs that can be 
compiled, when removing the limit is so simple?

(I bet if it was the other way around, and you were comparing a full 
64-bit gcc against a 32-bit compiler X, you would say that X was just a 
toy if it fell over at the first 100Kloc program.)

[toc] | [prev] | [next] | [standalone]

#140716

On 07/02/2019 13:06, Bart wrote:
> On 07/02/2019 07:32, David Brown wrote:
>> On 07/02/2019 07:13, jacobnavia wrote:
> 
>>> If GCC fails, RULE 1 applies.
>>>
>>
>> If you are doing a compile that takes more than 2 GB ram (within one
>> compile), or hits other 32-bit limits, you are doing something very
>> odd.   You /might/ be doing something useful, but it is certainly
>> unusual. The /huge/ majority of C programmers will get through their
>> entire programming lives without ever having an issue with this.
> 
> You seem to be an apologist for gcc.

It has /nothing/ to do with gcc.

> 
> If there is a version of gcc for (eg) x64 that targets x64, then there
> is no reason for gcc itself not to take advantage of that by building
> itself as an x64 executable. After all it will be run under a 64-bit OS,
> and most other programs will be 64-bit too.

Absolute nonsense.

If there is a version of gcc (or any compiler) designed to run on a
64-bit host, and it is as simple and convenient to make a 64-bit build
as a 32-bit build, and the result is as fast, then it makes sense to
make a 64-bit build.  If a 32-bit build is more convenient or faster,
then it makes sense to make a 32-bit build.

The size of the target is irrelevant.  The /processor/ on the target,
and its similarity to the processor on the host, is irrelevant.


In the Linux world, 64 bit has been around for a long time, it is
efficient, and programs can usually be built as 32-bit or 64-bit (and
for various processors).  It is natural to distribute a tool such as a
compiler in the native size for the OS.  Indeed, in my experience
pre-built toolchains (that are not part of the distribution) are now
often 64-bit only.  That applies to many embedded toolchains, which may
be for processors from 8-bit to 64-bit in size.

In the Windows world, 32-bit Windows is still common.  And for a lot of
programs, supporting 64-bit builds means work - there is a lot of source
code that assumes 32-bit sizes.  So if you are distributing binaries for
Windows, you have to make 32-bit builds anyway, which will work for all
Windows users (32-bit or 64-bit).  Then you have to ask yourself, is it
worth the effort making a 64-bit build?  It can mean extra work porting,
it means extra work testing, it might be slower for the end user anyway.
 So why bother?  If there is not an obvious answer (such as "people
regularly need to compile multi-MB source files, so we need support for
many GB of memory"), then why bother?


I have seen this for many compilers, for many targets.


> 
> Yes, a 32-bit executable may run a few percent faster, unless it's doing
> lots of 64-bit arithmetic, but it's already clear with gcc that
> compilation speed is not a priority.

Agreed.  But note that it is not "gcc" or the gcc developers that are
making and distributing the toolchain here - it is TDM (whoever they are).

> 
> Why put an arbitrary limit on the sizes of programs that can be
> compiled, when removing the limit is so simple?
> 

It is simple for gcc itself - the code there has worked fine for 64-bit
hosts for a couple of decades.  I don't know what else TDM provides with
their builds - there may be issues there.  But it is the TDM folks that
make this decision, not the gcc folks.

(You /do/ understand the difference here, don't you?  It has only been
explained to you several dozen times before.)

I agree that there is no point in having an arbitrary limit on sizes if
it gains nothing - but equally I don't see a point in making an extra
effort to extend limits far beyond what would normally be useful.


> (I bet if it was the other way around, and you were comparing a full
> 64-bit gcc against a 32-bit compiler X, you would say that X was just a
> toy if it fell over at the first 100Kloc program.)
> 

You can bet that if you like - you have a warped view of me anyway.  I'd
expect compilers to handle 100 kloc source, but only if it was sensible
code - I'm not particularly concerned how they handle a 100K line
functions, or 100K tiny functions.

[toc] | [prev] | [next] | [standalone]

#140719

On 07/02/2019 13:08, David Brown wrote:
> On 07/02/2019 13:06, Bart wrote:

> The size of the target is irrelevant.  The /processor/ on the target,
> and its similarity to the processor on the host, is irrelevant.

I said gcc for x64 that targets x64. On such PCs, you wouldn't be using 
a cross compiler.

Look, I've had this exact same problem with gcc, which generated x64 
code but was itself x32 (ie. x86-32), which meant it crashed out when 
total memory usage approached 2GB.

Why did I have a 32-bit version? Usually, if there is a choice on a 
download page, you will select a 64-bit program if your system is 
64-bit. So in that case there can't have been a choice.

> In the Windows world, 32-bit Windows is still common.  And for a lot of
> programs, supporting 64-bit builds means work - there is a lot of source
> code that assumes 32-bit sizes.  So if you are distributing binaries for
> Windows, you have to make 32-bit builds anyway, which will work for all
> Windows users (32-bit or 64-bit).  Then you have to ask yourself, is it
> worth the effort making a 64-bit build?

It that case why does ANYONE bother with 64-bit programs?

If a program as complex as the gcc C compiler (and it would have been 
the same for all the extra files like g++.exe), which could conceivably 
be applied to any scale of application, considers a 2GB RAM ceiling for 
ALL objects in the running program adequate, why would any other?

(BTW, are /you/ running a 64-bit version of gcc on an x64 machine? If 
so, why, since a 32-bit version would be faster?

On Windows, the two bytes at offset 0x84 (132) of gcc.exe file will be 
0x4C 0x01 for 32-bit, and 0x64 0x86 for 64-bit. Sorry don't know about ELF.)

>  It can mean extra work porting,

Rubbish. According to just about every expert on this group, a decently 
written program should compile to any target with little problem, 
especially if it just does file i/o.

> You can bet that if you like - you have a warped view of me anyway.  I'd
> expect compilers to handle 100 kloc source, but only if it was sensible
> code - I'm not particularly concerned how they handle a 100K line
> functions, or 100K tiny functions.

Nevertheless, given a real 100K line program, you might want to know how 
different compilers cope. If you have a way of synthesising more 
realistic programs large enough to be challenging, then I would be 
interested.

Here's a reminder of my original tests along these lines:

   https://github.com/sal55/mx/blob/master/compilertest.txt

 From that you will see that, while most compilers dealt fine with 
100Kloc in one function, DMC, DMD (D), PicoC, Rust, Nim, FPC (Pascal) 
and FBC (Basic) had real problems.

[toc] | [prev] | [next] | [standalone]

#140728

On 07/02/2019 14:49, Bart wrote:
> On 07/02/2019 13:08, David Brown wrote:
>> On 07/02/2019 13:06, Bart wrote:
> 
>> The size of the target is irrelevant.  The /processor/ on the target,
>> and its similarity to the processor on the host, is irrelevant.
> 
> I said gcc for x64 that targets x64. On such PCs, you wouldn't be using
> a cross compiler.

So what?

gcc x86 always supports 32-bit and 64-bit code generation, using "-m32"
or "-m64" flags.  It doesn't matter if the compiler is built as a 32-bit
binary, or a 64-bit binary - it can still generate for both targets.
The same applies to other x86 compilers (icc, MSVC, clang at least), the
same applies to MIPS compilers, PPC compilers, ARM compilers.

> 
> Look, I've had this exact same problem with gcc, which generated x64
> code but was itself x32 (ie. x86-32), which meant it crashed out when
> total memory usage approached 2GB.

If you /need/ to compile such huge code that 2 GB working ram is not
enough for the compiler, then I agree you need a 64-bit build of the
compiler.  That is totally and completely independent of the target -
whether you are compiling for x86-32, x86-64, msp430 16-bit, RISCV
128-bit, or anything else.

How many people need a C compiler that can work with such large programs
for pure compilation?  I'd guess, roughly, zero.  Testing to destruction
on how big nonsense files you can compile does not count as "need".

> 
> Why did I have a 32-bit version? Usually, if there is a choice on a
> download page, you will select a 64-bit program if your system is
> 64-bit. So in that case there can't have been a choice.
> 

I have no idea why you have a 32-bit compiler.  I have no idea where you
got it from, or what choices you had.  (If you had followed my advice
about gcc for Windows, you'd have a 64-bit compiler.)

>> In the Windows world, 32-bit Windows is still common.  And for a lot of
>> programs, supporting 64-bit builds means work - there is a lot of source
>> code that assumes 32-bit sizes.  So if you are distributing binaries for
>> Windows, you have to make 32-bit builds anyway, which will work for all
>> Windows users (32-bit or 64-bit).  Then you have to ask yourself, is it
>> worth the effort making a 64-bit build?
> 
> It that case why does ANYONE bother with 64-bit programs?
> 

Because for some people, and some programs, it /does/ make a difference.
 For C++ compilation, or for large LTO builds, it is useful to have lots
of memory.  You have to be somewhat masochistic to want to build
programs like Firefox or LibreOffice with LTO on Windows, but certainly
you'd use a 64-bit compiler build for the job.

> If a program as complex as the gcc C compiler (and it would have been
> the same for all the extra files like g++.exe), which could conceivably
> be applied to any scale of application, considers a 2GB RAM ceiling for
> ALL objects in the running program adequate, why would any other?
> 

Perhaps it is because compilation of C code does not take a lot of
memory?  At least, it does not take /that/ much memory.  On the other
hand, a video editing program will use everything it can get hold of, as
will many games.  (And, for reasons unknown to me, web browsers.)

> (BTW, are /you/ running a 64-bit version of gcc on an x64 machine? If
> so, why, since a 32-bit version would be faster?

I am using Linux.  On Linux, 64-bit programs are usually faster.  Ask MS
why they picked a weird and slow calling convention that reduces speed
compared to Linux, don't ask me.  (And ask others why 32-bit x86 Linux
has a particularly slow calling convention compared to 32-bit Windows.)
 But usually, I don't actually care.

> 
> On Windows, the two bytes at offset 0x84 (132) of gcc.exe file will be
> 0x4C 0x01 for 32-bit, and 0x64 0x86 for 64-bit. Sorry don't know about
> ELF.)

Use "file" on *nix, or msys2, to tell you the file type.

> 
>>  It can mean extra work porting,
> 
> Rubbish. According to just about every expert on this group, a decently
> written program should compile to any target with little problem,
> especially if it just does file i/o.

I don't believe anyone has said anything like that.  But even if it were
true, what percentage of programs are "decently written" ?

> 
>> You can bet that if you like - you have a warped view of me anyway.  I'd
>> expect compilers to handle 100 kloc source, but only if it was sensible
>> code - I'm not particularly concerned how they handle a 100K line
>> functions, or 100K tiny functions.
> 
> Nevertheless, given a real 100K line program, you might want to know how
> different compilers cope. If you have a way of synthesising more
> realistic programs large enough to be challenging, then I would be
> interested.

Real 100k line programs have a better mixture of function sizes, and
will compile faster than the artificial tests.  (But as I say, I would
expect compilers - 32-bit or not - to handle a real 100kloc program
without trouble.)

> 
> Here's a reminder of my original tests along these lines:
> 
>   https://github.com/sal55/mx/blob/master/compilertest.txt
> 
> From that you will see that, while most compilers dealt fine with
> 100Kloc in one function, DMC, DMD (D), PicoC, Rust, Nim, FPC (Pascal)
> and FBC (Basic) had real problems.
> 

[toc] | [prev] | [next] | [standalone]

#140746

Le 07/02/2019 à 14:08, David Brown a écrit :
> In the Windows world, 32-bit Windows is still common.


Windows Vista started in 2006 and offered the first 64 bit windows. 
Since then, 64 bit systems are the norm since there is since 2004 no 
intel CPU is 32 bit.

Thirteen years later there are still some windows XP machines (maybe 
20%) but those machines are unlikely to run any new software.

[toc] | [prev] | [next] | [standalone]

Page 1 of 3  [1] 2 3  Next page →

Back to top | Article view | comp.lang.c

csiph-web