Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.compilers > #1740 > unrolled thread

Alternative C compilers on x86_64 Linux?

Back to article view | Back to comp.compilers

Contents

  Alternative C compilers on x86_64 Linux? arnold@skeeve.com (Aharon Robbins) - 2016-09-02 03:01 +0000
    Re: Alternative C compilers on x86_64 Linux? jacobnavia <jacob@jacob.remcomp.fr> - 2016-09-05 00:20 +0200
      Re: Alternative C compilers on x86_64 Linux? alexfrunews@gmail.com - 2016-09-05 17:26 -0700
        Re: diagnosing C errors, was Alternative C compilers on x86_64 Linux? BartC <bc@freeuk.com> - 2016-09-09 12:31 +0100
          Re: diagnosing C errors, was Alternative C compilers on x86_64 Linux? Kaz Kylheku <221-501-9011@kylheku.com> - 2016-09-09 16:06 +0000
          Re: diagnosing C errors, was Alternative C compilers on x86_64 Linux? Hans-Peter Diettrich <DrDiettrich1@netscape.net> - 2016-09-10 00:38 +0200
    Re: Alternative C compilers on x86_64 Linux? Nemo <nemo@invalid.invalid> - 2016-09-04 18:32 -0400
    Re: Alternative C compilers on x86_64 Linux? Kaz Kylheku <221-501-9011@kylheku.com> - 2016-09-04 23:48 +0000
    Re: Alternative C compilers on x86_64 Linux? BartC <bc@freeuk.com> - 2016-09-05 13:19 +0100
      Re: Alternative C compilers on x86_64 Linux? Kaz Kylheku <221-501-9011@kylheku.com> - 2016-09-06 02:15 +0000
        Re: Alternative C compilers on x86_64 Linux? BartC <bc@freeuk.com> - 2016-09-06 21:35 +0100
          Re: dead code or otherwise, was Alternative C compilers on x86_64 Linux? George Neuner <gneuner2@comcast.net> - 2016-09-08 11:58 -0400
            Re: dead code or otherwise, was Alternative C compilers on x86_64 Linux? Kaz Kylheku <221-501-9011@kylheku.com> - 2016-09-08 22:54 +0000
              Re: dead code or otherwise, was Alternative C compilers on x86_64 Linux? George Neuner <gneuner2@comcast.net> - 2016-09-09 00:51 -0400
                Re: dead code or otherwise, was Alternative C compilers on x86_64 Linux? Kaz Kylheku <221-501-9011@kylheku.com> - 2016-09-09 06:05 +0000
                  Re: dead code or otherwise, was Alternative C compilers on x86_64 Linux? George Neuner <gneuner2@comcast.net> - 2016-09-09 14:02 -0400
                    Re: dead code or otherwise, was Alternative C compilers on x86_64 Linux? Kaz Kylheku <221-501-9011@kylheku.com> - 2016-09-09 19:57 +0000
        Re: duplicate cases, was Alternative C compilers on x86_64 Linux? Andy Walker <anw@cuboid.co.uk> - 2016-09-07 17:01 +0100
    Re: Alternative C compilers on x86_64 Linux? Florian Weimer <fw@deneb.enyo.de> - 2016-09-05 20:06 +0200
    Re: Alternative C compilers on x86_64 Linux? alexfrunews@gmail.com - 2016-09-05 17:22 -0700
      Re: Alternative C compilers on x86_64 Linux? Kaz Kylheku <221-501-9011@kylheku.com> - 2016-09-06 02:18 +0000
    Re: Alternative C compilers on x86_64 Linux? rockbrentwood@gmail.com - 2016-09-07 17:00 -0700
    Re: Alternative C compilers on x86_64 Linux? arnold@skeeve.com (Aharon Robbins) - 2016-09-12 18:42 +0000
      Re: Alternative C compilers on x86_64 Linux? BartC <bc@freeuk.com> - 2016-09-12 22:37 +0100
        Re: Alternative C compilers on x86_64 Linux? arnold@skeeve.com (Aharon Robbins) - 2016-09-13 17:35 +0000
          Re: Alternative C compilers on x86_64 Linux? arnold@skeeve.com (Aharon Robbins) - 2016-09-14 18:14 +0000
    Re: Alternative C compilers on x86_64 Linux? rugxulo@gmail.com - 2016-09-26 15:29 -0700
      Re: Alternative C compilers on x86_64 Linux? arnold@skeeve.com (Aharon Robbins) - 2016-09-27 05:40 +0000
        Re: Alternative C compilers on x86_64 Linux? rugxulo@gmail.com - 2016-09-27 16:36 -0700
          Re: Alternative C compilers on x86_64 Linux? BartC <bc@freeuk.com> - 2016-09-28 18:16 +0100
            Re: Alternative C compilers on x86_64 Linux? BartC <bc@freeuk.com> - 2016-09-29 14:03 +0100
              Re: Alternative C compilers on x86_64 Linux? George Neuner <gneuner2@comcast.net> - 2016-09-29 22:57 -0400
              Re: Alternative C compilers on x86_64 Linux? Hans-Peter Diettrich <DrDiettrich1@netscape.net> - 2016-09-30 05:03 +0200
                Re: Alternative C compilers on x86_64 Linux? BartC <bc@freeuk.com> - 2016-09-30 12:00 +0100
                  Re: Alternative C compilers on x86_64 Linux? Hans-Peter Diettrich <DrDiettrich1@netscape.net> - 2016-09-30 20:15 +0200
                    Re: Alternative C compilers on x86_64 Linux? BartC <bc@freeuk.com> - 2016-10-01 22:17 +0100
              Re: Alternative C compilers on x86_64 Linux? BartC <bc@freeuk.com> - 2016-09-30 12:28 +0100
                Re: Alternative C compilers on x86_64 Linux? BartC <bc@freeuk.com> - 2016-10-17 22:49 +0100
          Re: Alternative C compilers on x86_64 Linux? George Neuner <gneuner2@comcast.net> - 2016-09-28 21:24 -0400
        Re: Alternative C compilers on x86_64 Linux? George Neuner <gneuner2@comcast.net> - 2016-09-28 01:01 -0400
          Re: Alternative C compilers on x86_64 Linux? arnold@skeeve.com (Aharon Robbins) - 2016-09-29 02:55 +0000
        Re: Alternative C compilers on x86_64 Linux? arnold@skeeve.com - 2016-09-29 05:30 -0600
          Re: Alternative C compilers on x86_64 Linux? arnold@skeeve.com (Aharon Robbins) - 2016-09-30 10:13 +0000

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

#1749

On 2016-09-06, alexfrunews@gmail.com <alexfrunews@gmail.com> wrote:
> On Sunday, September 4, 2016 at 12:02:33 PM UTC-7, Aharon Robbins wrote:
>> Can I get recommendations for other (free) C compilers besides GCC and CLANG?
>> I've been using the revived PCC for gawk development since it's faster
>> than GCC, but recently it's developed a bug where it won't compile the
>> current (valid) code.
>>
>> LCC seems to be 32 bit only and requires very manual configuration.
>>
>> TinyCC is blindingly fast, and can compile gawk, but is broken in that
>> it won't diagnose duplicate case statements inside switch. The developers
>> don't consider this a problem. So I refuse to use it.
>>
>> In short, I'm looking for a faster compiler that actually works.
>
> What is preventing you from doing most of development using, say, that
> same TinyCC and then, when you think you're done with the round of
> changes (bugfixes, improvements, new features) recompiling the coded
> with gcc or clang and rerunning the tests to make sure there are no
> issues (warnings, real bugs) missed because of the limitations of the
> fast compiler? Given the nature of C, I think, it's always beneficial
> to try your code with different compilers (and on different platforms)
> to uncover portability and other problems.

If TinyCC is lacking basic ISO-C-required diagnostics, it means you
basically cannot make a single git commit without running the code
through another compiler.

Or you have to to hoard up a bunch of commits, then validate with
the other compiler, then interactively rebase them to fix all the things
that TinyCC didn't catch, and then publish the commits.

Can you say, PITA?

You need a compiler with great diagnostics for development.  (More
than just the ISO C required ones, but those would be nice, for
starters).

[toc] | [prev] | [next] | [standalone]

#1753

On Sunday, September 4, 2016 at 2:02:33 PM UTC-5, Aharon Robbins wrote:
> LCC seems to be 32 bit only and requires very manual configuration.

LCC-WIN 32 and apparently 64 by Jacob Navia appeared to have spruced
up LCC with elements drawn from Watcom to make it a bona fide Windows
C compiler; rather than one that piggybacks on another compiler's
library or the system library. It accepts a bit more than C; and Navia
has been trying to push it into a form of C sufficiently enhanced as
to make C++ unnecessary; while still firmly rooted in that which makes
C distinctively C. The one way you can tell however that it is not in
an active life cycle is that a large examples and demo archive that
has been separately posted for it by others has not undergone any
substantial change in almost a decade. LCC and LCC-WIN's main drawback
is that they are NOT doing the types of analyses that a good compiler
ought to be doing but appear to be more akin to byte-code compilers.
You can see it by the run times compared to GCC on Linux on the same
CPU.

[toc] | [prev] | [next] | [standalone]

#1765

In article <16-09-001@comp.compilers>, Aharon Robbins <arnold@skeeve.com> wrote:
>Can I get recommendations for other (free) C compilers besides GCC and CLANG?
>I've been using the revived PCC for gawk development since it's faster
>than GCC, but recently it's developed a bug where it won't compile the
>current (valid) code.
>
>LCC seems to be 32 bit only and requires very manual configuration.
>
>TinyCC is blindingly fast, and can compile gawk, but is broken in that
>it won't diagnose duplicate case statements inside switch. The developers
>don't consider this a problem. So I refuse to use it.
>
>In short, I'm looking for a faster compiler that actually works.

Thanks for all the replies. Some short replies to the relevant
answers:

To Jacob Navia - I'm sorry it's rough selling C compilers. I understand
your complaints. As others pointed out, duplicate cases is a constraint
violation.  Why would I used a compiler that didn't catch missing
semicolons or allowed only two expressions in a for(;;) ?  Same thing
applies to case statements.

As Kaz further pointed out, you can't just compile with it and then
recompile with gcc before committing. Big PITA, and I actually ended
up one time making a bad commit because tcc didn't catch duplicate cases.

Nemo - Intel and Oracle compilers - I'm sure they're good, but I'm
looking for FAST compilation. I often do 'make distclean' and then
configure and make many times over in one session. Fast compiles
make a big difference. I'd use GCC or clang for anything I wanted to
install, but while developing, speed is wonderful, but correctness
beats it.

Kaz suggested C++ - that's a lot of work for my code base. It might
get me more type checking but it won't get me compilation speed. C++
compilers can be slower than C compilers.

Florian - golang compilers. Worth checking out.

Privately, I was pointed at nwcc, but it fails to compile gawk.

Privately by someone else, I was suggested to just fix tcc myself.
It turns out that I was able to add duplicate case checking in about
an hour's work and just under 100 lines of code. But I have the
background and experience for that; I suspect someone else would
have had a harder time.

I will see about trying to get the upstream to accept it. And I hope
that PCC will eventually get fixed too.

Thanks all,

Arnold
--
Aharon (Arnold) Robbins 		arnold AT skeeve DOT com

[toc] | [prev] | [next] | [standalone]

#1767

On 12/09/2016 19:42, Aharon Robbins wrote:

> Privately by someone else, I was suggested to just fix tcc myself.
> It turns out that I was able to add duplicate case checking in about
> an hour's work and just under 100 lines of code. But I have the
> background and experience for that; I suspect someone else would
> have had a harder time.

Do you use TCC to compile itself, or something like gcc?

I'm just interested in whether it partly relies on gcc to get its speed!

--
Bartc

[toc] | [prev] | [next] | [standalone]

#1769

In article <16-09-028@comp.compilers>, BartC  <bc@freeuk.com> wrote:
>On 12/09/2016 19:42, Aharon Robbins wrote:
>> Privately by someone else, I was suggested to just fix tcc myself.
>> It turns out that I was able to add duplicate case checking in about
>> an hour's work and just under 100 lines of code. But I have the
>> background and experience for that; I suspect someone else would
>> have had a harder time.
>
>Do you use TCC to compile itself, or something like gcc?
>
>I'm just interested in whether it partly relies on gcc to get its speed!

The default configure + make compiles tcc with gcc.
--
Aharon (Arnold) Robbins 		arnold AT skeeve DOT com

[toc] | [prev] | [next] | [standalone]

#1771

>BartC  <bc@freeuk.com> wrote:
>>Do you use TCC to compile itself, or something like gcc?
>>
>>I'm just interested in whether it partly relies on gcc to get its speed!

Aharon Robbins <arnold@skeeve.com> wrote:
>The default configure + make compiles tcc with gcc.

I should point out that a large part of tcc's speed comes from the
fact that it generates object code directly, instead of going
through the assembler.  It might even have its own internal 'ld',
I'm not sure.
--
Aharon (Arnold) Robbins 		arnold AT skeeve DOT com
[Oh, like WATFOR and the original Dartmough BASIC. -John]

[toc] | [prev] | [next] | [standalone]

#1772

Hi,

On Sunday, September 4, 2016 at 2:02:33 PM UTC-5, Aharon Robbins wrote:
>
> Can I get recommendations for other (free) C compilers besides
> GCC and CLANG?
>
> In short, I'm looking for a faster compiler that actually works.

I'm not sure what miracle advice you expected to get from here.

The first thing you should do is check your build system. Maybe there
isn't much leeway there, but you should definitely something better
(simpler?) than Autotools. Try using Dash instead of Bash. Also, at
the very least, you should make your C modules as small as possible
and only use expensive optimizations on crucial files (instead of
blindly using "-O2" for everything).

If modern GCC is a bottleneck, use an older version from an old
distro (in a VM perhaps). If that isn't good advice, then try
to do rebuilds atop RAM disk (md or whatever *nix calls it).

What distro/GCC are you using anyways? Surely you've heard of
"-Og". (If you have a semi-recent Intel machine, try Clear Linux.)

Have you ever tried ccache or distcc? I haven't, but I assume that
something like that would help. I'm sure this has been attempted
before.

Honestly, I did a very naive build of Gawk myself (under old Lucid
Puppy Linux, admittedly not hosted in RAM but on hard disk). It
didn't seem that slow, but I was using "make -j4". So the obvious
question is how many cores does your cpu have? Are you using an
old laptop? Seriously, I hate advice like this, but you may honestly
get more gains from getting a newer machine with more cores. Heck,
some server machines these days have 20+ (although I admit that most
home versions don't have nearly that many).

Can't you use GNU's build server or something? Maybe I'm wrong, I
don't know the details, but I would assume they have something you
can ssh into for fast builds. Or maybe that's only for release
testing and not development?

I dunno, this kind of problem is very complex and naive at the same
time.

[toc] | [prev] | [next] | [standalone]

#1773

Hi. Thanks for your note.

In article <16-09-033@comp.compilers>,  <rugxulo@gmail.com> wrote:
>Hi,
>
>On Sunday, September 4, 2016 at 2:02:33 PM UTC-5, Aharon Robbins wrote:
>>
>> Can I get recommendations for other (free) C compilers besides
>> GCC and CLANG?
>>
>> In short, I'm looking for a faster compiler that actually works.
>
>I'm not sure what miracle advice you expected to get from here.

I develop locally. My system is Ubuntu 16.04 with gcc 5.4 on a Skylake
Core i5. /proc/cpuinfo claims 4 CPUs, so there are likely two hyperthreaded
physical cores and there's plenty of RAM.  The machine is quite fast,
even though it's using a conventional disk.

I often have to go through this cycle:

	make distclean
	./bootstrap.sh	# set modification times on some files
	./configure && make && make check

For example, when merging into the (too-many) different branches
and encountering a conflict.

The time difference between using tcc on the one hannd and GCC + make
-j on the other is quite noticeable; tcc + make -j is even faster.
Doing many builds an hour can happen, and a faster compiler saves me time.
tcc also makes a VERY noticeable difference in the time it takes
to run configure.

When I posted, tcc wasn't an option since it didn't check for duplicate
case labels, and PCC had stopped working for me.  So I was seeking an
additional, fast compiler.

Dorking with the Makefile to only use -O on certain files isn't really
an option; autotools sets things up to compile everything the same way
and for the shipped tarball that is the right option.

Similarly, I don't really wish to switch off the autotools; I've too
much time and experience invested in them. They work, crufty as they are,
and I have not had to invest any real time in keeping things up to date
with respect to them.

Since I originally posted, someone suggested that I just fix tcc on
my own. I was able to do this with less than 2 hours work so now
I'm back to being fat, dumb and happy. :-)

Thanks,

Arnold
--
Aharon (Arnold) Robbins 		arnold AT skeeve DOT com

[toc] | [prev] | [next] | [standalone]

#1774

Hi,

On Tuesday, September 27, 2016 at 1:36:38 PM UTC-5, Aharon Robbins wrote:
>
> In article <16-09-033@comp.compilers>,  <rugxulo@hates.spam> wrote:
> >
> >On Sunday, September 4, 2016 at 2:02:33 PM UTC-5, Aharon Robbins wrote:
> >>
> >> Can I get recommendations for other (free) C compilers besides
> >> GCC and CLANG?
> >>
> >> In short, I'm looking for a faster compiler that actually works.

... that runs on (and targets?) Linux/AMD64 with ELF, C99/POSIX,
widechar/Unicode, etc.

There are a lot of other compilers (and OSes), but they can't all
support literally everything.

> >I'm not sure what miracle advice you expected to get from here.
>
> I develop locally. My system is Ubuntu 16.04 with gcc 5.4 on a
> Skylake Core i5. /proc/cpuinfo claims 4 CPUs, so there are likely
> two hyperthreaded physical cores and there's plenty of RAM.
> The machine is quite fast, even though it's using a conventional
> disk.

Four cpus is still good, better than average. (But even my old
Westmere has that. So where's our "thousands of cores", Intel??
They sure do overhype SMP on every street corner! Humbug!)

And Skylake is very new. (I can't even pretend to remember all
the supported instruction sets.)

Honestly, I'd have been almost surprised if someone like you didn't
already use a "modern" (ugh) setup like this.

I'm not really familiar with tmpfs, but you can presumably use it
to speed some things up.

There was a small regression in GCC 5.x, so I think 6.x is indeed
slightly faster, but it's probably not quite the speedup you wanted.

> I often have to go through this cycle:
>
> 	make distclean
> 	./bootstrap.sh	# set modification times on some files
> 	./configure && make && make check
>
> For example, when merging into the (too-many) different branches
> and encountering a conflict.

The bottleneck could really be something simple like the makefile
itself, but I realize that rewriting or replacing that isn't usually
easy.

> The time difference between using tcc on the one hand and GCC + make
> -j on the other is quite noticeable; tcc + make -j is even faster.
> Doing many builds an hour can happen, and a faster compiler saves me time.
> tcc also makes a VERY noticeable difference in the time it takes
> to run configure.

I hate all that configure stuff (as I'm not really *nix savvy), but
can't you make a site-wide cache of certain things (that you know
will absolutely never change)?

http://www.gnu.org/software/automake/manual/html_node/config_002esite.html

> When I posted, tcc wasn't an option since it didn't check for duplicate
> case labels, and PCC had stopped working for me.  So I was seeking an
> additional, fast compiler.

Regarding PCC, you act like it just started to break. How exactly?
This is why you never upgrade anything unless forced!  :-)
So just stick with (stable) older version there (or is that
Debian or Ubuntu's fault?).

TCC hasn't had a release in recent years either. It's fast because
it does everything in one pass, and it does include it's own
assembler and linker. It doesn't optimize well, though.

I guess you know that GCC has slowed down tremendously since the
old days (2.7.x vs. 2.95.x vs. 3.x). Of course, using Skylake,
you shouldn't have any excuse (except binaries compiled for weaker
targets). It may not be reasonable, but you really should give
ClearLinux a shot.

> Dorking with the Makefile to only use -O on certain files isn't really
> an option; autotools sets things up to compile everything the same way
> and for the shipped tarball that is the right option.

If you're going to let AutoTools (or anything else) dictate everything
for you, then you're going to have to live with slow compiles.

> Similarly, I don't really wish to switch off the autotools; I've too
> much time and experience invested in them. They work, crufty as they are,
> and I have not had to invest any real time in keeping things up to date
> with respect to them.

Great, but if that's your bottleneck, then you'll have to fix it.

> Since I originally posted, someone suggested that I just fix tcc on
> my own. I was able to do this with less than 2 hours work so now
> I'm back to being fat, dumb and happy. :-)

There's always room for improvement (overall).

[toc] | [prev] | [next] | [standalone]

#1776

On 28/09/2016 00:36, rugxulo@gmail.com wrote:

> TCC hasn't had a release in recent years either. It's fast because
> it does everything in one pass, and it does include it's own
> assembler and linker. It doesn't optimize well, though.

Compilers can be fast anyway. I think we've just got used to very slow
ones such as gcc. I've been using my own /interpreted/ compiler and it
was still double the speed of gcc!

I was reminded of how fast they can be when I recently developed a new
byte-code compiler, and managed up to a million lines per second
compilation speed (one test was 1.5Mlps). And that was on a low-end
5-year-old PC, running on a single core.

That was two passes; a compiler to native code would use an extra pass,
and probably the code-generating is a bit fiddlier. But I would estimate
source to in-memory native-code generation at at least 500K lines per
second on the same machine, if I was to give that compiler the same
treatment.

(And because this is not for C, I can use all of that throughput instead
of wasting it repeatedly compiling the same header files.)

However it is so fast that it would be necessary to consider carefully
what to do with the output: invoking an external assembler or linker
would be like hitting a brick wall. So it would need to generate an
entire executable directly or prepare code to run in-memory.

These fast compilers (the byte-code one I've completed, and the possible
native code one) will only compile an entire project at once. (Because
of dependencies, parallelising would be trickier. But at the minute it's
fast enough: the new byte-code compiler can entirely re-compile my
current native-code compiler from scratch in 0.03 seconds.)

But with C projects then yes, the range of fast tools is limited. It
doesn't help if you also become dependent on things such as 'configure'
or 'make' (I work in Windows where there is less of that).

Maybe it needs people with experience of fast graphics or rendering, who
know how to speed things up, and apply them to C compilers! The C
language does put some obstacles in the way (the same header needs to
processed for the 50th time in case something comes out different), but
I think there is plenty that can be done.

TCC does a good job, but it's a shame about the code generation. (My own
native code compiler has a naive non-optimising code generator but is
still much better than TCC's code.)

(One one test of the fast compiler, my own code can manage 800Klps.
Converting to C intermediate code then putting it through gcc -O3 gets
it up to 1Mlps. But compiling with TCC gets it down to 350Klps (TCC is
not good with switch statements). Not so bad, but still ...)

--
Bartc
[Back when I was publishing the Journal of C Language of Translation, people did some interesting
stuff with C header files, saving a precompiled version for the usual case that subsequent
runs don't change any preprocessor stuff that would affect the code. -John]

[toc] | [prev] | [next] | [standalone]

#1781

On 28/09/2016 18:16, BartC wrote:
> On 28/09/2016 00:36, rugxulo@gmail.com wrote:
>
>> TCC hasn't had a release in recent years either. It's fast because
>> it does everything in one pass, and it does include it's own
>> assembler and linker. It doesn't optimize well, though.

> (And because this is not for C, I can use all of that throughput instead
> of wasting it repeatedly compiling the same header files.)

> These fast compilers (the byte-code one I've completed, and the possible
> native code one) will only compile an entire project at once.

> The C
> language does put some obstacles in the way (the same header needs to
> processed for the 50th time in case something comes out different), but
> I think there is plenty that can be done.
>
> TCC does a good job, but it's a shame about the code generation. (My own
> native code compiler has a naive non-optimising code generator but is
> still much better than TCC's code.)
>
> (One one test of the fast compiler, my own code can manage 800Klps.
> Converting to C intermediate code then putting it through gcc -O3 gets
> it up to 1Mlps. But compiling with TCC gets it down to 350Klps (TCC is
> not good with switch statements). Not so bad, but still ...)

> [Back when I was publishing the Journal of C Language of Translation,
> people did some interesting
> stuff with C header files, saving a precompiled version for the usual
> case that subsequent
> runs don't change any preprocessor stuff that would affect the code. -John]

Yes, George Neuner's post has a link to how gcc's precompiled headers
can be used. Maybe, if a compiler is already sluggish, they can make a
difference.

But I don't know if they would help to get the fastest speeds: loading
and decoding a precompiled header file doesn't sound that much less work
to me than just parsing a normal header anyway.

Most C compilers seem to support multiple C source files as input. This
is similar to my scheme of compiling multiple sources one after the
other. So if the input to a compiler looks like this:

    gcc -c A.c B.c C.c

where each of A, B and C include the same header file H.h, then once H.h
has been processed for A.c, the results of that (symbol tables and so
on) could be re-used for B and C without needing to process either H.h
or H.pch again.

Of course, a compiler can be so slow, and/or a header can be so large,
than even processing H.h /once/ can be a bottleneck! Then maybe
precompiled headers might be an easier option than making the compiler
itself faster, which is never going to happen in the case of gcc.

(I can tokenise C source code at some 10M lines per second on my PC
(this excludes symbol table lookups; just raw tokenising). But gcc might
process the same source at only 10K lines per second, even excluding
dealing with headers.

That's around a thousand times slower. People like to excuse it by
pointing to its superior code generation as the reason, but I think
that's only a small part of it. For a start, you can turn off
optimisation and it's still pretty slow. I think it's just too big and
complex.)

--
Bartc
[Tokenizing 10M lines/sec is pretty impressive.  In compilers that don't do heavy optimization
the lexer is usually the slowest part since it's the only thing that has to touch each
character of the source code individually. -John]

[toc] | [prev] | [next] | [standalone]

#1782

On Thu, 29 Sep 2016 14:03:55 +0100, BartC <bc@freeuk.com> wrote:

>Most C compilers seem to support multiple C source files as input. This
>is similar to my scheme of compiling multiple sources one after the
>other. So if the input to a compiler looks like this:
>
>    gcc -c A.c B.c C.c
>
>where each of A, B and C include the same header file H.h, then once H.h
>has been processed for A.c, the results of that (symbol tables and so
>on) could be re-used for B and C without needing to process either H.h
>or H.pch again.

But it isn't done that way: the compilation of each .c file is treated
independently.  A common header file will be (re)processed for each
code file that includes it.

Also recall that the standard headers form a nesting hierarchy: some
of the more common headers indirectly pull in many others.  Nesting
also is common in wizard generated code, and in C++ template
libraries, etc.


A pre-compiled header contains already digested data that can be
plugged (more or less) directly into the compiler's internal data
structures.  Loading a .pch file very often is much faster than
(re)compiling its constituent source files, and the time savings grow
with the number of files involved.

It works best when you have many code files all of which include a
common *set* of headers.  You extract the common includes from the
code files into a single common header file.  In each code file you
then include that one common header.  Then precompile it.

For a project with just a handful of files it really doesn't matter
... but with complex projects with hundreds of source files I've
gotten ~80% reductions in build time by using precompiled headers.  I
have seen 45 minute clean builds turn into <10 minutes.

George

[toc] | [prev] | [next] | [standalone]

#1783

BartC schrieb:

> (I can tokenise C source code at some 10M lines per second on my PC
> (this excludes symbol table lookups; just raw tokenising). But gcc might
> process the same source at only 10K lines per second, even excluding
> dealing with headers.

IMO it's not the header files, which slow down compilation, but the
preprocessor macros which require to look up and optionally expand every
token. Almost every language has to allow for external references, which
are read from some shared file. Next comes the amount of declarations in
the standard C header files, which require much memory and can cause
swapping, even if only a very small subset of all declarations is
actually used in source code. In so far I don't think that it's fair or
meaningful to compare a full blown compiler with a bare tokenizer.

DoDi

[toc] | [prev] | [next] | [standalone]

#1786

On 30/09/2016 04:03, Hans-Peter Diettrich wrote:
> BartC schrieb:
>
>> (I can tokenise C source code at some 10M lines per second on my PC ...
>
> IMO it's not the header files, which slow down compilation, but the
> preprocessor macros which require to look up and optionally expand every
> token. ...

> In so far I don't think that it's fair or
> meaningful to compare a full blown compiler with a bare tokenizer.

I seem to remember some comment in this group that tokenising accounts
for a big chunk of a compiler's runtime (50% or something).

While it is true that doing a full compile will take longer than just
raw tokenising, should that factor be of the order of 1000 times longer,
or three magnitudes?

I was investigating whether a reasonable working compiler could be
developed working at between 1 and 2 magnitudes slow-down from the raw
tokenising speed.

Probably one magnitude is a little optimistic, for C source anyway (with
preprocessing and stuff also needed), but two magnitudes is easily on
the cards. I think Tiny C is within that range.

--
Bartc

[toc] | [prev] | [next] | [standalone]

#1788

BartC schrieb:
> On 30/09/2016 04:03, Hans-Peter Diettrich wrote:
>> BartC schrieb:
>>> (I can tokenise C source code at some 10M lines per second on my PC ...
>>
>> IMO it's not the header files, which slow down compilation, but the
>> preprocessor macros which require to look up and optionally expand every
>> token. ...
>
>> In so far I don't think that it's fair or
>> meaningful to compare a full blown compiler with a bare tokenizer.
>
> I seem to remember some comment in this group that tokenising accounts
> for a big chunk of a compiler's runtime (50% or something).

This seems to be a reasonable figure for C, including all those nasty
tasks which have to be done before a token can be passed on to the parser.

> While it is true that doing a full compile will take longer than just
> raw tokenising, should that factor be of the order of 1000 times longer,
> or three magnitudes?

Find out yourself. Replace the grammar of your tokenizer by the C
grammar, and test again. I'd wonder if it would not reach the speed of
your tokenizer. Then add the preprocessor with file inclusion, macro
definition, recognition and expansion, conditional compilation, and test
again. Then add Ansi and Unicode string literals and symbol tables, and
test again.

DoDi

[toc] | [prev] | [next] | [standalone]

#1791

On 30/09/2016 19:15, Hans-Peter Diettrich wrote:
> BartC schrieb:
>> I seem to remember some comment in this group that tokenising accounts
>> for a big chunk of a compiler's runtime (50% or something).
>
> This seems to be a reasonable figure for C, including all those nasty
> tasks which have to be done before a token can be passed on to the parser.
>
>> While it is true that doing a full compile will take longer than just
>> raw tokenising, should that factor be of the order of 1000 times longer,
>> or three magnitudes?
>
> Find out yourself.

I already know the answer. It /doesn't/ take that much longer.

  Replace the grammar of your tokenizer by the C
> grammar, and test again. I'd wonder if it would not reach the speed of
> your tokenizer. Then add the preprocessor with file inclusion, macro
> definition, recognition and expansion, conditional compilation, and test
> again. Then add Ansi and Unicode string literals and symbol tables, and
> test again.

I don't need to do this work because it's already been done by Tiny C.

Using a monolithic, working test program of 22K lines (25K lines
including standard headers for tcc; 27K using gcc) Tiny C compiled it in
no more than 0.07 seconds (at least 360K lps).

gcc took from 2.2 seconds (unoptimised), to over 8 seconds (optimised).
3-12K lps. (Probably it strained the global optimiser having a large
single module as usually the difference between -O0/-O3 is smaller .)

Tiny C must be parsing the same C grammar and expanding the same macros
as gcc. So whatever it takes to do that, can presumably be done at some
speed faster than 350Klps.

[toc] | [prev] | [next] | [standalone]

#1787

On 29/09/2016 14:03, BartC wrote:

> [Tokenizing 10M lines/sec is pretty impressive.  In compilers that don't
> do heavy optimization
> the lexer is usually the slowest part since it's the only thing that has
> to touch each
> character of the source code individually. -John]

This is what I was putting to the test. Actually I struggled to recreate
that benchmark, but in the end managed to process actual C source (a
monolithic file containing all CPython sources) at some 9.7Mlps. Figures
do depend on the source style.

On another desktop (also cheap, but with Intel rather than AMD), it
managed 11.5Mlps. With non-C sources, which has less 'busy' syntax, up
to nearly 13Mlps.

Figures exclude file-loading (although that made little difference on
the second machine).

Probably some more throughput can be squeezed by applying some ASM (or
by writing in native C and using more options rather than using
atrocious-looking intermediate code), but at the moment this isn't a
bottleneck.

(This test has some bits missing, for example it parses floating point
numbers but doesn't convert the character sequences to actual values,
but on the input I used, floating point figures very little.

On the 1Mlps I quoted for an actual working compiler (although for
in-memory byte-code), that includes file-loading but expects the OS to
have cached the file as will normally be the case.

This compiler is case-insensitive which slows down the tokeniser a tiny
bit. That's one tiny advantage of compiling C!)

--
Bartc

[toc] | [prev] | [next] | [standalone]

#1796

On 30/09/2016 12:28, BartC wrote:
> On 29/09/2016 14:03, BartC wrote:
>
>> [Tokenizing 10M lines/sec is pretty impressive.  In compilers that don't
>> do heavy optimization
>> the lexer is usually the slowest part since it's the only thing that has
>> to touch each
>> character of the source code individually. -John]
>
> This is what I was putting to the test. Actually I struggled to recreate
> that benchmark, but in the end managed to process actual C source (a
> monolithic file containing all CPython sources) at some 9.7Mlps. Figures
> do depend on the source style.

I've put a version of that test program here as a C file:

https://github.com/bartg/langs/blob/master/clex.c

When I ran this on an original Raspberry Pi that struggled to do
anything fast (gcc ran at 500 lines per second, unoptimised), it managed
1.3M lines per second.


--
Bartc

[toc] | [prev] | [next] | [standalone]

#1777

On Tue, 27 Sep 2016 16:36:29 -0700 (PDT), rugxulo@gmail.com wrote:

>Four cpus is still good, better than average. (But even my old
>Westmere has that.

True.  And newer isn't always better.  Recently Intel has been focused
more on power consumption than on performance.  You can have more
cores, but they run slower.

>So where's our "thousands of cores", Intel??
>They sure do overhype SMP on every street corner! Humbug!)

You can have 72 weak cores in the Xeon Phi 7290.
http://www.intel.com/content/www/us/en/processors/xeon/xeon-phi-detail.html

Or you can have 24 strong cores in the E7-8890
http://www.intel.com/content/www/us/en/processors/xeon/xeon-processor-e7-family.html


Depends on how much you want to pay.
<grin>


>And Skylake is very new. (I can't even pretend to remember all
>the supported instruction sets.)

Skylake has been generally available for more than a year.  Its
successor, Kaby Lake, is already available in selected [high priced]
systems and is expected to be generally available by the holidays this
year.


>The bottleneck could really be something simple like the makefile
>itself, but I realize that rewriting or replacing that isn't usually
>easy.
>
>   :
>
>> Dorking with the Makefile to only use -O on certain files isn't really
>> an option; autotools sets things up to compile everything the same way
>> and for the shipped tarball that is the right option.
>
>If you're going to let AutoTools (or anything else) dictate everything
>for you, then you're going to have to live with slow compiles.

There likely is opportunity to improve the make file, but slow builds
with GCC generally are the result of design patterns that produce
[too] many small source files, and/or coding patterns that prevent use
of precompiled headers [there are a LOT of caveats there].
https://gcc.gnu.org/onlinedocs/gcc/Precompiled-Headers.html

George

[toc] | [prev] | [next] | [standalone]

#1775

On Tue, 27 Sep 2016 05:40:55 -0000 (UTC), arnold@skeeve.com (Aharon
Robbins) wrote:

>I develop locally. My system is Ubuntu 16.04 with gcc 5.4 on a Skylake
>Core i5. /proc/cpuinfo claims 4 CPUs, so there are likely two hyperthreaded
>physical cores and there's plenty of RAM.

The i5 has 4 non-HT cores.


>The time difference between using tcc on the one hand and GCC + make
>-j on the other is quite noticeable; tcc + make -j is even faster.
>Doing many builds an hour can happen, and a faster compiler saves me time.
>tcc also makes a VERY noticeable difference in the time it takes
>to run configure.

If you really have plenty of RAM, why not create a ramdisk?   Or put
in an SSD for the tmp filesystem?


>Since I originally posted, someone suggested that I just fix tcc on
>my own. I was able to do this with less than 2 hours work so now
>I'm back to being fat, dumb and happy. :-)

As long as you don't expect the same level of optimization.  TCC is
nice for quick development turn-around, but it doesn't produce the
fastest code.

George

[toc] | [prev] | [next] | [standalone]

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

Back to top | Article view | comp.compilers

csiph-web