Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #106011 > unrolled thread

C Compilers QoI

Back to article view | Back to comp.lang.c

Contents

  C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 13:01 +0000
    Re: C Compilers QoI mark.bluemel@gmail.com - 2017-03-14 06:16 -0700
    Re: C Compilers QoI already5chosen@yahoo.com - 2017-03-14 06:42 -0700
    Re: C Compilers QoI "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2017-03-14 06:56 -0700
      Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 15:58 +0000
        Re: C Compilers QoI "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2017-03-14 09:22 -0700
        Re: C Compilers QoI scott@slp53.sl.home (Scott Lurndal) - 2017-03-14 16:54 +0000
          Re: C Compilers QoI supercat@casperkitty.com - 2017-03-14 10:24 -0700
            Re: C Compilers QoI scott@slp53.sl.home (Scott Lurndal) - 2017-03-14 17:58 +0000
              Re: C Compilers QoI supercat@casperkitty.com - 2017-03-14 14:12 -0700
                Re: C Compilers QoI Robert Wessel <robertwessel2@yahoo.com> - 2017-03-14 16:34 -0500
                Re: C Compilers QoI Robert Wessel <robertwessel2@yahoo.com> - 2017-03-14 17:36 -0500
                  Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 23:06 +0000
                  Re: C Compilers QoI supercat@casperkitty.com - 2017-03-14 16:20 -0700
                Re: C Compilers QoI antispam@math.uni.wroc.pl - 2017-03-15 02:32 +0000
                  Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-15 12:49 +0000
                  Re: C Compilers QoI supercat@casperkitty.com - 2017-03-15 11:56 -0700
                    Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-15 23:09 +0000
                      Re: C Compilers QoI scott@slp53.sl.home (Scott Lurndal) - 2017-03-16 18:17 +0000
                        Re: C Compilers QoI supercat@casperkitty.com - 2017-03-16 11:47 -0700
                          Re: C Compilers QoI Ian Collins <ian-news@hotmail.com> - 2017-03-17 07:50 +1300
                            Re: C Compilers QoI supercat@casperkitty.com - 2017-03-16 12:59 -0700
                              Re: C Compilers QoI David Brown <david.brown@hesbynett.no> - 2017-03-16 21:24 +0100
                                Re: C Compilers QoI supercat@casperkitty.com - 2017-03-16 14:23 -0700
                                  Re: C Compilers QoI Ian Collins <ian-news@hotmail.com> - 2017-03-17 11:30 +1300
                                    Re: C Compilers QoI Ian Collins <ian-news@hotmail.com> - 2017-03-17 11:38 +1300
                                      Re: C Compilers QoI Gareth Owen <gwowen@gmail.com> - 2017-03-17 05:07 +0000
                                        Re: C Compilers QoI Ian Collins <ian-news@hotmail.com> - 2017-03-17 20:26 +1300
                                          Re: C Compilers QoI Gareth Owen <gwowen@gmail.com> - 2017-03-17 07:59 +0000
                                    Re: C Compilers QoI supercat@casperkitty.com - 2017-03-16 16:14 -0700
                                      Re: C Compilers QoI Robert Wessel <robertwessel2@yahoo.com> - 2017-03-16 22:54 -0500
                                        Re: C Compilers QoI supercat@casperkitty.com - 2017-03-17 09:01 -0700
                                      Re: C Compilers QoI Ian Collins <ian-news@hotmail.com> - 2017-03-26 14:16 +1300
                                        Re: C Compilers QoI supercat@casperkitty.com - 2017-03-26 14:19 -0700
                                          Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-26 22:43 +0100
                                            Re: C Compilers QoI GOTHIER Nathan <nathan.gothier@gmail.com> - 2017-03-27 01:28 +0200
                                            Re: C Compilers QoI supercat@casperkitty.com - 2017-03-26 22:07 -0700
                                              Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-27 11:28 +0100
                                                Re: C Compilers QoI supercat@casperkitty.com - 2017-03-27 07:38 -0700
                                                  Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-27 16:23 +0100
                                                    Re: C Compilers QoI supercat@casperkitty.com - 2017-03-27 08:55 -0700
                                                      Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-27 17:01 +0100
                                                        Re: C Compilers QoI supercat@casperkitty.com - 2017-03-27 10:03 -0700
                                                    Re: C Compilers QoI GOTHIER Nathan <nathan.gothier@gmail.com> - 2017-03-27 17:58 +0200
                                          Re: C Compilers QoI already5chosen@yahoo.com - 2017-03-27 03:54 -0700
                                            Re: C Compilers QoI supercat@casperkitty.com - 2017-03-27 08:46 -0700
                                              Re: C Compilers QoI scott@slp53.sl.home (Scott Lurndal) - 2017-03-27 17:47 +0000
                                                Re: C Compilers QoI supercat@casperkitty.com - 2017-03-27 11:48 -0700
                                                  Re: C Compilers QoI Robert Wessel <robertwessel2@yahoo.com> - 2017-03-27 15:37 -0500
                                                    Re: C Compilers QoI supercat@casperkitty.com - 2017-03-27 14:46 -0700
                                                      Re: C Compilers QoI Robert Wessel <robertwessel2@yahoo.com> - 2017-03-27 20:28 -0500
                                                        Re: C Compilers QoI supercat@casperkitty.com - 2017-03-28 01:18 -0700
                                                          Re: C Compilers QoI Robert Wessel <robertwessel2@yahoo.com> - 2017-03-28 18:24 -0500
                                                            Re: C Compilers QoI supercat@casperkitty.com - 2017-03-29 07:40 -0700
                                                              Re: C Compilers QoI David Brown <david.brown@hesbynett.no> - 2017-03-29 17:05 +0200
                                                                Re: C Compilers QoI supercat@casperkitty.com - 2017-03-29 11:07 -0700
                                                              Re: C Compilers QoI antispam@math.uni.wroc.pl - 2017-03-31 04:04 +0000
                                                                Re: C Compilers QoI bartc <bc@freeuk.com> - 2017-03-31 10:53 +0100
                                                                  Re: C Compilers QoI Malcolm McLean <malcolm.mclean5@btinternet.com> - 2017-03-31 07:35 -0700
                                                                    Re: C Compilers QoI supercat@casperkitty.com - 2017-03-31 08:07 -0700
                                                                  Re: C Compilers QoI supercat@casperkitty.com - 2017-03-31 07:54 -0700
                                                                Re: C Compilers QoI supercat@casperkitty.com - 2017-03-31 07:43 -0700
                              Re: C Compilers QoI Ian Collins <ian-news@hotmail.com> - 2017-03-17 11:21 +1300
                                Re: C Compilers QoI Ian Collins <ian-news@hotmail.com> - 2017-03-17 11:24 +1300
                        Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-16 22:15 +0000
                          Re: C Compilers QoI scott@slp53.sl.home (Scott Lurndal) - 2017-03-17 12:31 +0000
                        Re: C Compilers QoI already5chosen@yahoo.com - 2017-03-16 15:24 -0700
                          Re: C Compilers QoI supercat@casperkitty.com - 2017-03-16 16:03 -0700
                          Re: C Compilers QoI scott@slp53.sl.home (Scott Lurndal) - 2017-03-17 12:39 +0000
                            Re: C Compilers QoI already5chosen@yahoo.com - 2017-03-17 08:27 -0700
                    Re: C Compilers QoI antispam@math.uni.wroc.pl - 2017-03-16 01:35 +0000
                      Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-16 02:18 +0000
                        Re: C Compilers QoI antispam@math.uni.wroc.pl - 2017-03-18 00:45 +0000
                          Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-18 13:12 +0000
                            Re: C Compilers QoI antispam@math.uni.wroc.pl - 2017-03-18 20:31 +0000
                              Re: C Compilers QoI supercat@casperkitty.com - 2017-03-18 17:31 -0700
                      Re: C Compilers QoI supercat@casperkitty.com - 2017-03-16 09:22 -0700
            Re: C Compilers QoI Nick Bowler <nbowler@draconx.ca> - 2017-03-24 16:13 +0000
              Re: C Compilers QoI supercat@casperkitty.com - 2017-03-24 12:32 -0700
                Re: C Compilers QoI Robert Wessel <robertwessel2@yahoo.com> - 2017-03-24 22:13 -0500
                  Re: C Compilers QoI supercat@casperkitty.com - 2017-03-25 13:05 -0700
                    Re: C Compilers QoI Robert Wessel <robertwessel2@yahoo.com> - 2017-03-27 15:44 -0500
                      Re: C Compilers QoI supercat@casperkitty.com - 2017-03-27 14:51 -0700
                        Re: C Compilers QoI Robert Wessel <robertwessel2@yahoo.com> - 2017-03-27 20:31 -0500
          Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 17:26 +0000
            Re: C Compilers QoI Andrew Haley <andrew29@littlepinkcloud.invalid> - 2017-03-15 12:02 -0500
    Re: C Compilers QoI David Brown <david.brown@hesbynett.no> - 2017-03-14 15:00 +0100
      Re: C Compilers QoI already5chosen@yahoo.com - 2017-03-14 07:36 -0700
        Re: C Compilers QoI David Brown <david.brown@hesbynett.no> - 2017-03-14 15:57 +0100
        Re: C Compilers QoI Andrey Tarasevich <andreytarasevich@hotmail.com> - 2017-03-14 08:11 -0700
          Re: C Compilers QoI BGB <cr88192@hotmail.com> - 2017-03-29 23:51 -0500
      Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 15:28 +0000
        Re: C Compilers QoI David Brown <david.brown@hesbynett.no> - 2017-03-14 21:41 +0100
    Re: C Compilers QoI Keith Thompson <kst-u@mib.org> - 2017-03-14 09:26 -0700
      Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 18:23 +0000
        Re: C Compilers QoI Keith Thompson <kst-u@mib.org> - 2017-03-14 12:05 -0700
    Re: C Compilers QoI already5chosen@yahoo.com - 2017-03-14 09:35 -0700
      Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 17:08 +0000
        Re: C Compilers QoI already5chosen@yahoo.com - 2017-03-14 10:24 -0700
          Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 17:34 +0000
          Re: C Compilers QoI already5chosen@yahoo.com - 2017-03-14 11:55 -0700
            Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 19:03 +0000
              Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 19:06 +0000
              Re: C Compilers QoI already5chosen@yahoo.com - 2017-03-14 12:10 -0700
                Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 21:14 +0000
                  Re: C Compilers QoI already5chosen@yahoo.com - 2017-03-14 14:42 -0700
                  Re: C Compilers QoI supercat@casperkitty.com - 2017-03-14 14:53 -0700
                    Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 23:20 +0000
                      Re: C Compilers QoI antispam@math.uni.wroc.pl - 2017-03-15 02:43 +0000
    Re: C Compilers QoI jacobnavia <jacob@jacob.remcomp.fr> - 2017-03-14 19:39 +0100
      Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 19:10 +0000
    Re: C Compilers QoI antispam@math.uni.wroc.pl - 2017-03-14 19:29 +0000
    Re: C Compilers QoI Noob <root@127.0.0.1> - 2017-03-15 09:27 +0100
    Re: C Compilers QoI ed_davis2@yahoo.com - 2017-03-16 15:10 -0700
      Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-16 23:19 +0000

Page 1 of 6  [1] 2 3 4 5 6  Next page →

#106011 — C Compilers QoI

I was doing some very routine tests but came across surprising issues 
with a few of my C compilers that I didn't expect.

This was the test code:

  int main(void) {
     int a,b,c,d;

     a = b+c*d;
  }

except that the 'a=b+c*d;' line was repeated a million times. (Also 
those lines are in an include file, but I don't think that matters.)

------------------------------

This is what I found when I wanted to find out how long it took to 
compile, unoptimised, to 32-bit code (my machine has 7GB memory):

Pelles C: out of memory after 12 seconds

Lccwin32: error at line 930000 or so after 5 seconds (presumably out of 
memory)

DMC: Out of memory after four and a half *minutes*. (Even though it only 
used about 40MB. So maybe requesting memory but not using it? I don't know.)

gcc/tdm: out of memory after 30 seconds

All the above, other than DMC, had a peak memory usage of about 2GB 
presumably they were all 32-bit apps.

Some did complete:

MSVC 2008: in about 8 seconds; peak memory 0.6GB

Tiny C: in one second; peak memory difficult to see but between 0 and 
20MB (ie. max 0.02GB).

------------------------------

When I targeted 64-bit code, then Pelles C and gcc/tdm were the same (so 
I guess were were still 32-bit apps).

Lccwin64, however, completed, and in just over 10 seconds. Peak memory 
2.5GB.

And Tiny C still did it effortlessly in one second!

------------------------------

Anyway, people will say, use a more up-to-date compiler built as a 
64-bit application. But with two of my compilers that target 64-bit, I 
didn't even know they were 32-bit apps. If they used memory more 
efficiently, then they would have worked (MSVC only used 0.6GB; Tiny C 
apparently uses no memory at all!).

(I was trying to compare the performance of my own project, which I'd 
been having trouble keeping to the throughput I wanted. It looks like 
it'll be on a par with Tiny C, but memory usage is much higher at around 
0.5GB when compiled as 32-bit. That might be a consequence of having an 
extra pass.)

-- 
bartc

[toc] | [next] | [standalone]

#106012

On Tuesday, 14 March 2017 13:01:34 UTC, Bart  wrote:
> I was doing some very routine tests but came across surprising issues 
> with a few of my C compilers that I didn't expect.
> 
> This was the test code:
> 
>   int main(void) {
>      int a,b,c,d;
> 
>      a = b+c*d;
>   }
> 
> except that the 'a=b+c*d;' line was repeated a million times.

"Doctor! When I do this it hurts...."

[toc] | [prev] | [next] | [standalone]

#106014

On Tuesday, March 14, 2017 at 3:01:34 PM UTC+2, Bart wrote:
> I was doing some very routine tests but came across surprising issues 
> with a few of my C compilers that I didn't expect.
> 
> This was the test code:
> 
>   int main(void) {
>      int a,b,c,d;
> 
>      a = b+c*d;
>   }
> 
> except that the 'a=b+c*d;' line was repeated a million times. (Also 
> those lines are in an include file, but I don't think that matters.)
> 


Did you try the same with a million lines of non-dead code?

> ------------------------------
> 
> This is what I found when I wanted to find out how long it took to 
> compile, unoptimised, to 32-bit code (my machine has 7GB memory):
> 
> Pelles C: out of memory after 12 seconds
> 
> Lccwin32: error at line 930000 or so after 5 seconds (presumably out of 
> memory)
> 
> DMC: Out of memory after four and a half *minutes*. (Even though it only 
> used about 40MB. So maybe requesting memory but not using it? I don't know.)
> 
> gcc/tdm: out of memory after 30 seconds
> 
> All the above, other than DMC, had a peak memory usage of about 2GB 
> presumably they were all 32-bit apps.
> 
> Some did complete:
> 
> MSVC 2008: in about 8 seconds; peak memory 0.6GB
> 
> Tiny C: in one second; peak memory difficult to see but between 0 and 
> 20MB (ie. max 0.02GB).
> 
> ------------------------------
> 
> When I targeted 64-bit code, then Pelles C and gcc/tdm were the same (so 
> I guess were were still 32-bit apps).
> 
> Lccwin64, however, completed, and in just over 10 seconds. Peak memory 
> 2.5GB.
> 
> And Tiny C still did it effortlessly in one second!
> 
> ------------------------------
> 
> Anyway, people will say, use a more up-to-date compiler built as a 
> 64-bit application. But with two of my compilers that target 64-bit, I 
> didn't even know they were 32-bit apps. If they used memory more 
> efficiently, then they would have worked (MSVC only used 0.6GB; Tiny C 
> apparently uses no memory at all!).
> 
> (I was trying to compare the performance of my own project, which I'd 
> been having trouble keeping to the throughput I wanted. It looks like 
> it'll be on a par with Tiny C, but memory usage is much higher at around 
> 0.5GB when compiled as 32-bit. That might be a consequence of having an 
> extra pass.)
> 
> -- 
> bartc

[toc] | [prev] | [next] | [standalone]

#106016

On Tuesday, March 14, 2017 at 9:01:34 AM UTC-4, Bart wrote:
> I was doing some very routine tests but came across surprising issues 
> with a few of my C compilers that I didn't expect.
> 
> This was the test code:
> 
>   int main(void) {
>      int a,b,c,d;
> 
>      a = b+c*d;
>   }
> 
> except that the 'a=b+c*d;' line was repeated a million times. (Also 
> those lines are in an include file, but I don't think that matters.)
> 
> ------------------------------
> 
> This is what I found when I wanted to find out how long it took to 
> compile, unoptimised, to 32-bit code (my machine has 7GB memory):
> 
> Pelles C: out of memory after 12 seconds
> 
> Lccwin32: error at line 930000 or so after 5 seconds (presumably out of 
> memory)
> 
> DMC: Out of memory after four and a half *minutes*. (Even though it only 
> used about 40MB. So maybe requesting memory but not using it? I don't know.)
> 
> gcc/tdm: out of memory after 30 seconds
> 
> All the above, other than DMC, had a peak memory usage of about 2GB 
> presumably they were all 32-bit apps.
> 
> Some did complete:
> 
> MSVC 2008: in about 8 seconds; peak memory 0.6GB
> 
> Tiny C: in one second; peak memory difficult to see but between 0 and 
> 20MB (ie. max 0.02GB).
> 
> ------------------------------
> 
> When I targeted 64-bit code, then Pelles C and gcc/tdm were the same (so 
> I guess were were still 32-bit apps).
> 
> Lccwin64, however, completed, and in just over 10 seconds. Peak memory 
> 2.5GB.
> 
> And Tiny C still did it effortlessly in one second!
> 
> ------------------------------
> 
> Anyway, people will say, use a more up-to-date compiler built as a 
> 64-bit application. But with two of my compilers that target 64-bit, I 
> didn't even know they were 32-bit apps. If they used memory more 
> efficiently, then they would have worked (MSVC only used 0.6GB; Tiny C 
> apparently uses no memory at all!).
> 
> (I was trying to compare the performance of my own project, which I'd 
> been having trouble keeping to the throughput I wanted. It looks like 
> it'll be on a par with Tiny C, but memory usage is much higher at around 
> 0.5GB when compiled as 32-bit. That might be a consequence of having an 
> extra pass.)
> 
> -- 
> bartc

The solution would be to break the one million line source file into
smaller files and compile them separately.  All of them will compile
properly then.

Such a task is really taxing well beyond the maximums a compiler
would be designed for.  A single source file with a million lines
is undoubtedly extremely rare.  And in the case of a real need for
such a things there would be work-arounds.

My goals with CAlive are to produce my own compiler which compile
code in the way I see fit.  I am not greatly concerned with how long
it takes to compile because that's a small part of overall design,
and because CAlive will use the RDC framework I'm designing, which
is a continuous compiler, constantly re-compiling small changes in
the background.  I also use a system which generates "inquirys" on
unknown code, so every bit of code you attempt to compile will not
fail to compile, but will simply reach that point in flow and signal
an "inquiry" which will trap to the debugger, or exit the app at
that point.

Your goals are admirable, Bart.  But I would like to suggest you
stop comparing your products to other tools.  It's one thing to
compare them for functionality and correctness of generated code,
but for operation, compile time, memory utilization, etc., unless
you are trying to produce a product to compete with others in that
area and not in other areas where your tool has obvious advantages,
then I'm thinking those other metrics really shouldn't matter.

You are a talented developer, Bart.  I would advise moving forward
in your vision, and when you encounter legitimate show-stopper
things, bring them up.  But until then, press on and produce the
best product you're capable of, with whatever help you're able to
get in the way of design input and actual hands-on development.

I look forward to seeing your product when it is completed.  I
think you should call it TBC (for [T]he [B]art [C]ompiler). :-)

    tbc -W4 mysource.tbc -o myfile.exe

Thank you,
Rick C. Hodgin

[toc] | [prev] | [next] | [standalone]

#106045

On 14/03/2017 13:56, Rick C. Hodgin wrote:
> On Tuesday, March 14, 2017 at 9:01:34 AM UTC-4, Bart wrote:
>> I was doing some very routine tests but came across surprising issues
>> with a few of my C compilers that I didn't expect.
>>
>> This was the test code:
>>
>>   int main(void) {
>>      int a,b,c,d;
>>
>>      a = b+c*d;
>>   }
>>
>> except that the 'a=b+c*d;' line was repeated a million times. (Also
>> those lines are in an include file, but I don't think that matters.)

> The solution would be to break the one million line source file into
> smaller files and compile them separately.  All of them will compile
> properly then.

If the problem was actually to compile the code. But I was just 
comparing throughput.

I just happened to choose a test input that was a bit much for some of 
the compilers. And suprisingly so: the .c input is 8MB, the .obj output 
is 11-13MB, but my machine has 7000MB of RAM.

It's like a compiler running on a 64KB microcomputer running into 
difficulties on a 9-line source file!

Anyway, the real purpose was to compare with my own compiler, but with 
smaller files than 1Mloc, it's harder to get precise timings (so that I 
can see the effect of incremental changes). But I might need to drop 
down to 0.25Mloc when testing other compilers.

> Such a task is really taxing well beyond the maximums a compiler
> would be designed for.  A single source file with a million lines
> is undoubtedly extremely rare.  And in the case of a real need for
> such a things there would be work-arounds.

One of my test inputs is a file called gcc.c, which is an aggregation of 
the files comprising gcc's 'cc1' component (AIUI, the core of gcc).

gcc.c has 750K lines. With include files, nearly 0.9Mloc are involved. 
And this is real code with much more complex statements and expressions, 
and with macros used everywhere.

Is there any reason why such a 22MB file can't be compiled on a machine 
with several thousand MB of RAM?

(Using gcc, I can only compile that on Linux, but it gives a type error 
on line 691763 so it can't complete. So I won't know whether it could 
have compiled within my virtual Linux which may be memory-constrained.)

> I look forward to seeing your product when it is completed.  I
> think you should call it TBC (for [T]he [B]art [C]ompiler). :-)
>
>     tbc -W4 mysource.tbc -o myfile.exe

Thanks, that's a very good idea! I'd been stuck with 'bcc' but that can 
be confused with Borland's product. However the compile line would be this:

    tbc mysource

This compiles mysource.c (normal C code, not a new language this time!), 
and will run the program straightaway.

-- 
bartc

[toc] | [prev] | [next] | [standalone]

#106050

On Tuesday, March 14, 2017 at 11:58:48 AM UTC-4, Bart wrote:
> On 14/03/2017 13:56, Rick C. Hodgin wrote:
> Anyway, the real purpose was to compare with my own compiler, but with 
> smaller files than 1Mloc, it's harder to get precise timings (so that I 
> can see the effect of incremental changes). But I might need to drop 
> down to 0.25Mloc when testing other compilers.

That's my point ... apart from appeasing personal interests, I'm not
sure what bearing that info would have on your project.  I suppose if
you found your compiler was compiling x lines of code per second, that
might be worth worrying over.  But if it's in the ballpark, it would
seem to be a non-issue compared to the truly vast and nearly overwhelming
series of other tasks you have to perform correctly to get your compiler
to work.

I know in my progress, I am able to move forward, but with so many
other things tugging at my time ... it's been slow going.  I also
have some other things occurring in my life which impact my ability
to use what free time I do have.  But, I keep pressing on. :-)

> > Such a task is really taxing well beyond the maximums a compiler
> > would be designed for.  A single source file with a million lines
> > is undoubtedly extremely rare.  And in the case of a real need for
> > such a things there would be work-arounds.
> 
> One of my test inputs is a file called gcc.c, which is an aggregation of 
> the files comprising gcc's 'cc1' component (AIUI, the core of gcc).
> 
> gcc.c has 750K lines. With include files, nearly 0.9Mloc are involved. 
> And this is real code with much more complex statements and expressions, 
> and with macros used everywhere.
> 
> Is there any reason why such a 22MB file can't be compiled on a machine 
> with several thousand MB of RAM?

Agreed completely. :-)  I'm actually quite curious what CAlive will
do when I get it done.  So far my AAlive (sort of a higher-level
assembler) is working well.

I plan to go through three modules:  CAlive to BAlive to AAlive.

BAlive will be a very fundamental / rudimentary typed version of the
CAlive source code, and will be a stand-alone language on its own
somewhat similar to B in that everything's completely flexible as
everything is based only on fundamental operations and data sizes.

AAlive is basically an assembler.

So, compilation will go:  C->B->A, which is CAlive to BAlive to AAlive.

The "Alive" reference is to it being a dynamic language with the new
equivalent of Edit-and-Continue, something I call "LiveCode."

This three-part source-code design will impact performance, but at
each level I'm able to perform source code debugging, as well as
diffs in compilation to see what's actually changed, and then only
re-compile those lines and/or that portion.

It will funnel through to a reconstructed ABI which is then patched
at runtime, with a set of rules determining what is applicable at
runtime, and what requires a full restart.  My goal is to make all
internal code completely reconfigurable.  We'll see though... lots
of work.

> (Using gcc, I can only compile that on Linux, but it gives a type error 
> on line 691763 so it can't complete. So I won't know whether it could 
> have compiled within my virtual Linux which may be memory-constrained.)
> 
> > I look forward to seeing your product when it is completed.  I
> > think you should call it TBC (for [T]he [B]art [C]ompiler). :-)
> >
> >     tbc -W4 mysource.tbc -o myfile.exe
> 
> Thanks, that's a very good idea! I'd been stuck with 'bcc' but that can 
> be confused with Borland's product. However the compile line would be this:
> 
>     tbc mysource
> 
> This compiles mysource.c (normal C code, not a new language this time!), 
> and will run the program straightaway.

I think so.  Using tbc will make it its own thing.  I like it.

Thank you,
Rick C. Hodgin

[toc] | [prev] | [next] | [standalone]

#106056

BartC <bc@freeuk.com> writes:
>On 14/03/2017 13:56, Rick C. Hodgin wrote:
>> On Tuesday, March 14, 2017 at 9:01:34 AM UTC-4, Bart wrote:
>>> I was doing some very routine tests but came across surprising issues
>>> with a few of my C compilers that I didn't expect.
>>>
>>> This was the test code:
>>>
>>>   int main(void) {
>>>      int a,b,c,d;
>>>
>>>      a = b+c*d;
>>>   }
>>>
>>> except that the 'a=b+c*d;' line was repeated a million times. (Also
>>> those lines are in an include file, but I don't think that matters.)
>
>> The solution would be to break the one million line source file into
>> smaller files and compile them separately.  All of them will compile
>> properly then.
>
>If the problem was actually to compile the code. But I was just 
>comparing throughput.
>
>I just happened to choose a test input that was a bit much for some of 
>the compilers. And suprisingly so: the .c input is 8MB, the .obj output 
>is 11-13MB, but my machine has 7000MB of RAM.
>
>It's like a compiler running on a 64KB microcomputer running into 
>difficulties on a 9-line source file!

No, any good compiler would build an in-memory expression tree for
the function, so it can do proper optimization prior to generating
code for the function.   Your function main, has a million expresssions
in it, for which it needs some number of tree-bytes to represent.  Thus
you exceeded memory limits.

[toc] | [prev] | [next] | [standalone]

#106060

On Tuesday, March 14, 2017 at 11:54:30 AM UTC-5, Scott Lurndal wrote:
> BartC writes:
> >It's like a compiler running on a 64KB microcomputer running into 
> >difficulties on a 9-line source file!
> 
> No, any good compiler would build an in-memory expression tree for
> the function, so it can do proper optimization prior to generating
> code for the function.   Your function main, has a million expresssions
> in it, for which it needs some number of tree-bytes to represent.  Thus
> you exceeded memory limits.

A compiler for a small machine would benefit from building a tree when it
has enough space to do so, but a quality compiler for such a machine should
be able to generate working code even when it doesn't.  If a machine needs
to have arguments pushed right-to-left and a compiler runs out of memory
holding information about the code for earlier arguments having previously
generate 1,233 internal labels, it should be able to generate something like:

       jmp     L1234
   L1235:
       ...all the code for the earlier arguments processed thus far
       jmp     L1236
   L1234:

followed by the code to handle the later arguments, and finally

       jmp     L1235
   L1236:
       call    _theFunction

Obviously the code would be bigger and slower than it would be if the
compiler hadn't run out of memory, but once the compiler has spilled out
the code for the earlier parameters the only thing it would need to remember
is that it before it performs the function call it will need to output a
jump to L1235 and a branch target label L1236.  If there are multiple nested
function calls within an expression a compiler might need to keep track of
the label number associated with each, but that's a tiny burden compared with
having to buffer actual code.

[toc] | [prev] | [next] | [standalone]

#106065

supercat@casperkitty.com writes:
>On Tuesday, March 14, 2017 at 11:54:30 AM UTC-5, Scott Lurndal wrote:
>> BartC writes:
>> >It's like a compiler running on a 64KB microcomputer running into 
>> >difficulties on a 9-line source file!
>> 
>> No, any good compiler would build an in-memory expression tree for
>> the function, so it can do proper optimization prior to generating
>> code for the function.   Your function main, has a million expresssions
>> in it, for which it needs some number of tree-bytes to represent.  Thus
>> you exceeded memory limits.
>
>A compiler for a small machine would benefit from building a tree when it
>has enough space to do so,

A programmer writing code to be compiled by a compiler for a small
machine won't be submitting million-line single functions to it.

[toc] | [prev] | [next] | [standalone]

#106093

On Tuesday, March 14, 2017 at 12:58:45 PM UTC-5, Scott Lurndal wrote:
> supercat writes:
> >A compiler for a small machine would benefit from building a tree when it
> >has enough space to do so,
> 
> A programmer writing code to be compiled by a compiler for a small
> machine won't be submitting million-line single functions to it.

Probably not a million, but there would not have been anything unusual
about trying to compile programs or even functions that would be too large
for the compiler to keep in memory.  One of the design goals of C from
the beginning was that a compiler wouldn't need to keep a whole lot of
information in memory during compilation.  A compiler could read some
source, generate some output, and then promptly forget almost everything
about the source it had just processed save for symbol definitions which
were still in scope, or branch labels associated with any nested block
constructs.

Back when machines were smaller, it would have been considered obvious that
a quality compiler should be able to handle programs whose source tree would
be larger than the amount of RAM available to the compiler.  Commonplace
computers have enough memory that compilers have less need to handle programs
whose source trees are bigger than memory, but if simple compilers can handle
such programs it shouldn't be unreasonable to expect that fancier ones should
be able to do so as well.

[toc] | [prev] | [next] | [standalone]

#106096

On Tue, 14 Mar 2017 14:12:18 -0700 (PDT), supercat@casperkitty.com
wrote:

>On Tuesday, March 14, 2017 at 12:58:45 PM UTC-5, Scott Lurndal wrote:
>> supercat writes:
>> >A compiler for a small machine would benefit from building a tree when it
>> >has enough space to do so,
>> 
>> A programmer writing code to be compiled by a compiler for a small
>> machine won't be submitting million-line single functions to it.
>
>Probably not a million, but there would not have been anything unusual
>about trying to compile programs or even functions that would be too large
>for the compiler to keep in memory.  One of the design goals of C from
>the beginning was that a compiler wouldn't need to keep a whole lot of
>information in memory during compilation.  A compiler could read some
>source, generate some output, and then promptly forget almost everything
>about the source it had just processed save for symbol definitions which
>were still in scope, or branch labels associated with any nested block
>constructs.
>
>Back when machines were smaller, it would have been considered obvious that
>a quality compiler should be able to handle programs whose source tree would
>be larger than the amount of RAM available to the compiler.  Commonplace
>computers have enough memory that compilers have less need to handle programs
>whose source trees are bigger than memory, but if simple compilers can handle
>such programs it shouldn't be unreasonable to expect that fancier ones should
>be able to do so as well.


I don't think it's an unreasonable limitation at all.  First, haveing
two modes of operation immediately makes development more difficult -
you may well have two modes of operation.  And how you pick the
transition point then becomes quite problematic - or quite arbitrary.

And then, it's clear that all the compilers being discussed can handle
multi-hundred-thourands line functions, and frankly, I can see little
pointing expecting more.  It's also likely that the copmiler choking
on some of these very large functions might well be able to handle
source modules much larger than that, but split into individula
functions of more reasonable size.

[toc] | [prev] | [next] | [standalone]

#106099

On Tue, 14 Mar 2017 14:12:18 -0700 (PDT), supercat@casperkitty.com
wrote:

>On Tuesday, March 14, 2017 at 12:58:45 PM UTC-5, Scott Lurndal wrote:
>> supercat writes:
>> >A compiler for a small machine would benefit from building a tree when it
>> >has enough space to do so,
>> 
>> A programmer writing code to be compiled by a compiler for a small
>> machine won't be submitting million-line single functions to it.
>
>Probably not a million, but there would not have been anything unusual
>about trying to compile programs or even functions that would be too large
>for the compiler to keep in memory.  One of the design goals of C from
>the beginning was that a compiler wouldn't need to keep a whole lot of
>information in memory during compilation.  A compiler could read some
>source, generate some output, and then promptly forget almost everything
>about the source it had just processed save for symbol definitions which
>were still in scope, or branch labels associated with any nested block
>constructs.
>
>Back when machines were smaller, it would have been considered obvious that
>a quality compiler should be able to handle programs whose source tree would
>be larger than the amount of RAM available to the compiler.  Commonplace
>computers have enough memory that compilers have less need to handle programs
>whose source trees are bigger than memory, but if simple compilers can handle
>such programs it shouldn't be unreasonable to expect that fancier ones should
>be able to do so as well.


Sorry, hit send too early on first post, then had a PC issue needing a
reboot…

I don't think it's an unreasonable limitation at all.  First, having
two modes of operation immediately makes development more difficult -
you may well have two modes of operation, both of which you have to
validate.  And how you pick the transition point then becomes quite
problematic (after you run out of heap space tends to be a bad place
for many programs to try to recover) - or quite arbitrary.

And then, it's clear that all the compilers being discussed can handle
multi-hundred-thousand line functions, and frankly, I can see little
pointing expecting more.  It's also likely that the compiler choking
on some of these very large functions might well be able to handle
source modules much larger than that, but split into individual
functions of more reasonable size.

The limitations of the 50, 60s, 70s and 80s, really shouldn't impact
designs *today*.

[toc] | [prev] | [next] | [standalone]

#106100

On 14/03/2017 22:36, Robert Wessel wrote:

> And then, it's clear that all the compilers being discussed can handle
> multi-hundred-thousand line functions, and frankly, I can see little
> pointing expecting more.  It's also likely that the compiler choking
> on some of these very large functions might well be able to handle
> source modules much larger than that, but split into individual
> functions of more reasonable size.

I tried another test with 12,500 identical functions as shown below, 
with only the function name being stepped.

The blank line is included, to make 100,000 lines in total to make 
things a bit easier. To start with.

-----------------------------------

Results to compile with no optimising:

Pelles C:  aborted after a minute or so (I got fed up of waiting)
lccwin64:  5.5 seconds
DMC:       0.5 seconds
Tiny C:    0.2 seconds
gcc:       18 seconds
MSVC:      1 or so seconds

-----------------------------------
Next, I had 125,000 functions to make one million lines:

lccwin64:  aborted after a couple of minutes
DMC:       Out of memory
Tiny C:    1.3 seconds
gcc:       aborted after several minutes
MSVC:      12 seconds


It's clear than some compilers find these little tests troublesome. 
Either they can't handle it, or take so long to compile (where you are 
left wondering if they have just hung, or might eventually produce 
results given enough time) that they are not usable.

Of my tests today, two compilers have stood out: MSVC, and Tiny C.

Waiting 5 minutes or maybe 10 minutes for gcc to compile a program (or 
for it to eventually give up) when MSVC takes 12 seconds and Tiny C a 
fraction of that, is not really on, no matter what David Brown says.

With this function test, there were no problems with memory (not even 
with DMC, although that's what it reported).

-----------------------------------

void fn1(void) {
int a,b=2,c=3,d=4;
a=b+c*d;
a=a*b^c;
a=a+a+a;
a=d-c-a;
}

void fn2(void) {
int a,b=2,c=3,d=4;
a=b+c*d;
....

-- 
bartc

[toc] | [prev] | [next] | [standalone]

#106101

On Tuesday, March 14, 2017 at 5:36:27 PM UTC-5, robert...@yahoo.com wrote:
> On Tue, 14 Mar 2017 14:12:18 -0700 (PDT), supercat wrote:
> >Back when machines were smaller, it would have been considered obvious that
> >a quality compiler should be able to handle programs whose source tree would
> >be larger than the amount of RAM available to the compiler.  Commonplace
> >computers have enough memory that compilers have less need to handle programs
> >whose source trees are bigger than memory, but if simple compilers can handle
> >such programs it shouldn't be unreasonable to expect that fancier ones should
> >be able to do so as well.
> 
> Sorry, hit send too early on first post, then had a PC issue needing a
> reboot…
> 
> I don't think it's an unreasonable limitation at all.  First, having
> two modes of operation immediately makes development more difficult -
> you may well have two modes of operation, both of which you have to
> validate.  And how you pick the transition point then becomes quite
> problematic (after you run out of heap space tends to be a bad place
> for many programs to try to recover) - or quite arbitrary.

Even if a compiler has access to enough memory to perform certain kinds of
whole-program optimization, that doesn't mean that the benefits of doing so
will be sufficient to justify the costs.  Localized optimizations can offer
huge benefit, cheaply.  As optimizations become further reaching, they get
more expensive but in many cases offer decreasing benefits.  While it may be
cool to have a compiler that can dig up some optimizations that others would
miss, having a faster build time so programmers can afford to spend more time
writing efficient algorithms would be more helpful.

[toc] | [prev] | [next] | [standalone]

#106111

supercat@casperkitty.com wrote:
> 
> Back when machines were smaller, it would have been considered obvious that
> a quality compiler should be able to handle programs whose source tree would
> be larger than the amount of RAM available to the compiler.  Commonplace
> computers have enough memory that compilers have less need to handle programs
> whose source trees are bigger than memory, but if simple compilers can handle
> such programs it shouldn't be unreasonable to expect that fancier ones should
> be able to do so as well.

My toy compiler (for a tiny pseudo Pascal) can compile Bart's 1000000
lines in 15s.  It generates assembler source, and assembly takes
another 20s.  Generated code is awful: 12 machine instructions and
four temporary variables per source line.  I could improve output
a bit, but anything worth calling "optimization" would increase
memory use during compilation.  In fact, adding essential usablity
features like better error reporting or debug info requires storing
extra information which would increase memory use.

In another post you write that whole-program optimization is
not needed.  Whole-program probably not, but whole-function
optimization may be essential to generate correct and reasonable
code for large functions.  Namely various offsets may cross
limits supported by instructions.  Simple cases can be
deferred to assembler but on say ARM you need to have data
space for constant pool reachable by short offset -- compiler
may be forced to split function into pieces and insert extra
jumps around data.  Or reserve register to serve as base
register for constant pool.  IIUC register allocation is
frequently done on optimistic basis, assuming that some
problemetic situations will not occur.  If optimistic
approach works it leads to better allocation.  But if
optimistic approach fails one have to re-run register
allocation in reserving some registers to handle
problem.  Ability to re-run means storing all relevant
data.

In old days compiler would not allocate variable in
register except for explicit register declaration.
Common subexpression elimiantion was limited to
single expression.  We now expect much more, in
particular moving invariat computation out of loops
and good use of registers.  In case of linear
sequence of inctructions register allocation
is easy, but gets more tricky in presence of
interesing control flow.  And even in linear
case one may wish to reorder instructions for
better register allocation and/or better
scheduling.

Now consider why one would submit 1000000 line
function to the compiler.  Such functions are
hard to understand and maintain by hand.
So the likely reason is that they are output
of a generator.  In such case generator
presumably has more information then compiler
about resonable ways to split it.  Anyway,
the main reason to keep such function as
a single unit would be to enable better
optiomization.  Now, if compiler have to disable
some optimizations to handle such function,
then this defeats the purpose.

-- 
                              Waldek Hebisch

[toc] | [prev] | [next] | [standalone]

#106130

On 15/03/2017 02:32, antispam@math.uni.wroc.pl wrote:
> supercat@casperkitty.com wrote:
>>
>> Back when machines were smaller, it would have been considered obvious that
>> a quality compiler should be able to handle programs whose source tree would
>> be larger than the amount of RAM available to the compiler.  Commonplace
>> computers have enough memory that compilers have less need to handle programs
>> whose source trees are bigger than memory, but if simple compilers can handle
>> such programs it shouldn't be unreasonable to expect that fancier ones should
>> be able to do so as well.
>
> My toy compiler (for a tiny pseudo Pascal) can compile Bart's 1000000
> lines in 15s.  It generates assembler source, and assembly takes
> another 20s.

My own current compiler ['A'] (for a static language like C) took 107 
seconds to produce .asm output on the equivalent 1Mloc. Not great, but I 
have a good excuse, because the language it's written in is interpreted! 
(And I wasn't concerned about efficiency at the time.)

However, assembling the resulting 6M lines of asm took 230 seconds. Now 
this is crazy: presumably the assembler (Nasm) is written in C, and the 
task of assembling (I have done it) is much simpler than compiling a 
high level language, so why is it so slow? What on earth could it be doing?

I experimented with tokenising the 6M lines of asm source, and it took 
only 1.3 seconds; so Nasm is spending another 228.7 seconds doing... who 
knows?

> Generated code is awful: 12 machine instructions and
> four temporary variables per source line.

I used to do most of my coding in interpreted code, which is slower than 
the worst native code you could possibly generate.

And it works perfectly well with the right mix (of interpreted vs. 
compiled). So that might not be as important as you think.

Even 100% interpreted can work. Take that interpreted compiler 'A' I 
mentioned above. It used to be byte-compiled with another interpreted 
compiler. Byte-code compiling all modules (of 'A') took 1.8 seconds. All 
interpreted. How long does it take to compile all modules of a typical C 
compiler (other than tiny C with tiny C)?

Designing programs efficiently can be more important than the speed of 
the language you're using.

BTW that byte-code compiler was given the ultra-fast treatment last year 
by rewriting it in compiled code. Now, all modules of compiler 'A' 
[23Kloc] can be byte-code compiled in as little as 15 ms. It's scary!

These exercises are useful because it puts the spotlight on cumbersome 
tools such as Nasm and gcc and questions why there are so inefficient. 
David Brown says gcc is doing invaluable analyses, but what analysis 
could Nasm be doing on 1000000 repetitions of stuff like this:

  ;           a:=b+c*d
        mov       A0,   [Dframe+start.b]
        mov       A1,   [Dframe+start.c]
        imul      A1,   [Dframe+start.d]
        add       A0,   A1
        mov       [Dframe+start.a], A0

> Now consider why one would submit 1000000 line
> function to the compiler.

That's very easy to answer. I didn't want to use a function at all, but 
C doesn't allow statements outside a function. I hadn't expected it to 
be a big deal. Is it?


-- 
bartc

[toc] | [prev] | [next] | [standalone]

#106158

On Tuesday, March 14, 2017 at 9:32:34 PM UTC-5, anti...@math.uni.wroc.pl wrote:
> supercat wrote:
> > Back when machines were smaller, it would have been considered obvious that
> > a quality compiler should be able to handle programs whose source tree would
> > be larger than the amount of RAM available to the compiler.  Commonplace
> > computers have enough memory that compilers have less need to handle programs
> > whose source trees are bigger than memory, but if simple compilers can handle
> > such programs it shouldn't be unreasonable to expect that fancier ones should
> > be able to do so as well.
> 
> My toy compiler (for a tiny pseudo Pascal) can compile Bart's 1000000
> lines in 15s.  It generates assembler source, and assembly takes
> another 20s.  Generated code is awful: 12 machine instructions and
> four temporary variables per source line.  I could improve output
> a bit, but anything worth calling "optimization" would increase
> memory use during compilation.  In fact, adding essential usablity
> features like better error reporting or debug info requires storing
> extra information which would increase memory use.

Going after low-hanging fruit is cheap and doesn't require much extra
memory.  For example, instead of having a function to generate code to
put an effective address into EBX, one could pass a pointer to a structure
that describes the effective address as one of several forms, and have
code that would perform operations on [EBX] accept that structure and
generate code for the effective address indicated therein.  If one does
likewise for code which would place value operands in EAX, then something
like:

   p->member += 3;

would be improved from:

   mov ebx,&p
   mov ebx,[ebx]
   add ebx,4 ; member offset
   mov eax,3
   add [dword ebx],eax

to

   mov ebx,[p]
   add [dword ebx],3

A huge improvement in code quality, but relatively cheap.

> In another post you write that whole-program optimization is
> not needed.  Whole-program probably not, but whole-function
> optimization may be essential to generate correct and reasonable
> code for large functions.  Namely various offsets may cross
> limits supported by instructions.  Simple cases can be
> deferred to assembler but on say ARM you need to have data
> space for constant pool reachable by short offset -- compiler
> may be forced to split function into pieces and insert extra
> jumps around data.

I think the ARM assembler has a directive which instructs the assembler to
generate a jump ahead and produce a table for previous ldr= instructions if
it's necessary to do so, and otherwise do nothing and generate no code.  On
some assemblers for the 8051 and 8088 (Keil's assembler and Borland's TASM,
respectively) and likely some other chips as well there are options to
replace "jnz foo" with "jz $+5 / jmp foo" when needed to allow longer branch
range.  If a compiler's going to need to track the size of generated code it
may as well generate machine code itself without an assembler.  If memory is
tight, the compiler could generate a file containing a "patchable" format
along with a second file containing a list of patches and relocation points.
Those files could then  be processed by a utility that wouldn't need to know
anything about C, and could be much faster than would be possible if it had
to process assembly source text.

>                       Or reserve register to serve as base
> register for constant pool.  IIUC register allocation is
> frequently done on optimistic basis, assuming that some
> problemetic situations will not occur.  If optimistic
> approach works it leads to better allocation.  But if
> optimistic approach fails one have to re-run register
> allocation in reserving some registers to handle
> problem.  Ability to re-run means storing all relevant
> data.

A compiler will generally need to see an entire loop before deciding upon
the optimal register allocation of variables at the start, but the larger
a loop is the less relative cost there will be for guessing wrong.  At worst,
the compiler will have to shuffle some variables around before branching back
to the start of the loop.  If a compiler can hold off on generating the code
for the start of the loop until it knows the best arrangement, that will allow
better code than if it has to start generating code before it knows what's
best, but if the compiler defaults to "safe" choices when generating code in
such circumstances, it will be able to work around those choices later on.

> In old days compiler would not allocate variable in
> register except for explicit register declaration.
> Common subexpression elimiantion was limited to
> single expression.  We now expect much more, in
> particular moving invariat computation out of loops
> and good use of registers.  In case of linear
> sequence of inctructions register allocation
> is easy, but gets more tricky in presence of
> interesing control flow.  And even in linear
> case one may wish to reorder instructions for
> better register allocation and/or better
> scheduling.

Of course, but such things have a certain cost.  For some kinds of code, the
payoffs may be huge, making such optimizations clearly worthwhile.  For other
kinds of code, the payoffs may be much smaller.

> Now consider why one would submit 1000000 line
> function to the compiler.  Such functions are
> hard to understand and maintain by hand.
> So the likely reason is that they are output
> of a generator.

Indeed.

> In such case generator
> presumably has more information then compiler
> about resonable ways to split it.  Anyway,
> the main reason to keep such function as
> a single unit would be to enable better
> optiomization.  Now, if compiler have to disable
> some optimizations to handle such function,
> then this defeats the purpose.

I'd say that the most common reason to keep such a function as a single unit
would be a lack of any particular reason to split it.  If one wants to have
a program generate C code to e.g. draw an object described in a vector-
graphics file, and one passes it a file containing a single million-vector
object, the output of the program would most likely be a huge C function that
would draw the entire object.  If each vector in the original object led to
one line of generated C code, that would be a million-line function.  Should
such a program split large files into a sequence of smaller functions?  On
what basis should it do so?

I'd suggest that if someone writing such a code generator were worried about
the quality of the machine code that would be generated from the resulting
C code, the code generator would be better placed than a C compiler to handle
many non-localized optimizations.  If a group of vectors precisely matches
another, but shifted by some amount, it may be worthwhile for the utility that
converts the vector-graphics file into C code to generate a function to draw
that group of vectors with a given offset, and then replace each place where
that sequence appears with a call to that function.  Such optimization,
however, should really be the job of the code generator, rather than the job
of the C compiler.

[toc] | [prev] | [next] | [standalone]

#106202

On 15/03/2017 18:56, supercat@casperkitty.com wrote:

> Going after low-hanging fruit is cheap and doesn't require much extra
> memory.  For example, instead of having a function to generate code to
> put an effective address into EBX, one could pass a pointer to a structure
> that describes the effective address as one of several forms, and have
> code that would perform operations on [EBX] accept that structure and
> generate code for the effective address indicated therein.  If one does
> likewise for code which would place value operands in EAX, then something
> like:
>
>    p->member += 3;
>
> would be improved from:
>
>    mov ebx,&p
>    mov ebx,[ebx]
>    add ebx,4 ; member offset
>    mov eax,3
>    add [dword ebx],eax

I don't understand what you mean about a function, above. But the code 
TCC32 generates for such a member access is:

     mov     eax,[p]
     add     eax,4
     mov     ecx,[eax]
     add     ecx,3
     mov     [eax],ecx

> to
>
>    mov ebx,[p]
>    add [dword ebx],3

The code I generate is more like this (but with a +4 offset you may have 
missed). But which one is faster on x86 is anyone's guess (maybe 
read-modify-write instructions are slower).

> I'd suggest that if someone writing such a code generator were worried about
> the quality of the machine code that would be generated from the resulting
> C code, the code generator would be better placed than a C compiler to handle
> many non-localized optimizations.  If a group of vectors precisely matches
> another, but shifted by some amount, it may be worthwhile for the utility that
> converts the vector-graphics file into C code to generate a function to draw
> that group of vectors with a given offset, and then replace each place where
> that sequence appears with a call to that function.  Such optimization,
> however, should really be the job of the code generator, rather than the job
> of the C compiler.

The size of an individual function shouldn't be an issue, unless it 
infringes on some hard limits of a compiler. It would be better QoI 
however if the limits were determined by hardware resources rather than 
some arbitrary number. (I have plenty of such limits in my own programs, 
but I'm not at the minute looking at creating a robust, professional 
product.)

But if there are no specific limits, there might be practical ones if 
some aspect has O(n-squared) behaviour for example, which is another QoI 
matter.

(I found exactly that in my own code yesterday, when testing those 
125,000 functions. Actually it couldn't even manage 5000 functions 
because of this (due to thousands of duplicate 'a', 'b', 'c' and 'd' 
variables). Fortunately that was easy to fix.

But I wouldn't have known about the problem if it wasn't for these 
'totally pointless' tests!)

-- 
Bartc

[toc] | [prev] | [next] | [standalone]

#106259

BartC <bc@freeuk.com> writes:

>
>I don't understand what you mean about a function, above. But the code 
>TCC32 generates for such a member access is:
>
>     mov     eax,[p]
>     add     eax,4
>     mov     ecx,[eax]
>     add     ecx,3
>     mov     [eax],ecx
>
>> to
>>
>>    mov ebx,[p]
>>    add [dword ebx],3
>
>The code I generate is more like this (but with a +4 offset you may have 
>missed). But which one is faster on x86 is anyone's guess (maybe 
>read-modify-write instructions are slower).

If you are writing a compiler, and you're not familiar with the
Intel Software/Architecture Optimization Guides, which tell you _exactly_
how fast each instruction is, then you're not doing it right
(hint:  The add instruction above, assuming a hit in L1, is a
single cycle instruction (with, perhaps a load-to-use penalty
of three or four cycles depending on where in the pipeline you
are)).

That guide makes many recommendations to compiler writers on how
to generate optimally performing code, from simple things like
using xor to clear a register to how to most effectively schedule
instructions to keep all the out-of-order pipelines active.  Another
recommendation is to use ADD/SUB instead of INC/DEC (due to potential
false dependencies on the processor flags), or to use LEA instead
of ADD/MUL in some situations.

[toc] | [prev] | [next] | [standalone]

#106262

On Thursday, March 16, 2017 at 1:18:08 PM UTC-5, Scott Lurndal wrote:
> If you are writing a compiler, and you're not familiar with the
> Intel Software/Architecture Optimization Guides, which tell you _exactly_
> how fast each instruction is, then you're not doing it right
> (hint:  The add instruction above, assuming a hit in L1, is a
> single cycle instruction (with, perhaps a load-to-use penalty
> of three or four cycles depending on where in the pipeline you
> are)).

That's only really relevant if the person who is using the compiler knows
what kind of hardware will be used to run the generated code.  Otherwise
code orderings which improve performance on one processor may make it worse
on another.

Of course, people often use compilers to generate code for the purposes of
executing code themselves, but there are many other cases where code is
generated for the purpose of being distributed as an executable.  Some
people may view with disdain the idea that people would want to run code
they didn't compile themselves, but outside of certain narrow circles that's
the normal way things are done.

[toc] | [prev] | [next] | [standalone]

Page 1 of 6  [1] 2 3 4 5 6  Next page →

Back to top | Article view | comp.lang.c

csiph-web