Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.arch.embedded > #24128

Re: A comparison of C implementations for the STM8

Show all headers | View raw

(This reply is mainly for the OP - I am just replying to Don's post
because his post provides convenient spots for my comments to add to his
useful points.)

On 26/05/16 01:54, Don Y wrote:
> On 5/25/2016 12:39 PM, Don Y wrote:
>> On 5/25/2016 12:20 PM, Philipp Klaus Krause wrote:
>>> The STM8 is an 8-bit microcontroller architecture targeted by varioues C
>>> implementations.
> 
>>> http://www.colecovision.eu/stm8/compilers.shtml
>>
>> As only a *cursory* read (shopping day, today):
>> - any extensions (nonstandard) that a developer might find useful
>> - add some whitespace after the final table (e.g., a footer)
>> - opine as to why speed/size tradeoffs seem insignificant
>> - yellow on white is kinda brutal on the eyes (red, green, blue?)
>> - quality of libraries delivered with each
>> - "invent" some bit-oriented benchmarks (toggling an "I/O", etc.)
>> - library source availability (rebuild for different optimization)
>> - why *you* choose to use SDCC ("honestly")
>>
>> I'll try to give it a more thorough review after my wallet has
>> been emptied...
> 
> Please don't take this as being overly critical; just pointing out
> questions that an engaged reader could come away asking himself.
> Keep or discard as much or as little as you choose...
> 
> First, I think you need to ask yourself WHY you are doing this as
> that can color *how* you are doing it.  I.e., if you are evangelizing for
> SDCC, then you might want to be disproportionately more critical (to
> yourself) of things you say *for* SDCC (to ensure your bias isn't
> coloring the questions you are asking).

Good points.

Also consider how much effort you want to put into this.  How many
people are actually going to look at your page?  Many (such as me) are
just curious, but it is only really relevant for people who are picking
STM8 devices for a new project and want guidance in choosing tools -
that is not a big market these days.  If you were looking at ARM
compilers, or at least AVR or PIC tools, it would be different.  So
treat suggestions here with that in view - spend the effort if you are
enjoying the investigation, but feel entirely free to tell me or Don
that our ideas are nice but you don't want to spend the time on it.

> 
> [OTOH, there's nothing wrong with pointing out a product's strengths,
> if they truly are!]
> 
> Your appraisal of standards compliance seems very subjective.  What
> exactly does "far ahead of the competition" mean?  Number of features
> correctly implemented?  How do you decide which of those features
> are important vs. others that aren't supported?
> 
> Is it practical to tabulate the differences in UNSUPPORTED features
> for each compiler?  (i.e., omit the stuff they *do* and highlight
> the things they *don't* -- or do "nonconformingly")
> 

Standards compliance is important to some people, but for many small
embedded projects, code is written entirely for the one device and one
compiler.  It is a lot more useful to know that a compiler has an extra
feature for efficient handling of arrays of bits, than to know if it has
full C99 support (including, for example, wide character functions).

So a table with information based on features rather than just standards
would be useful (though more work, of course).  A rough figure (or set
of up to five stars as Don suggested) for how much of the C standards
are covered could be nice.  But it would also be useful with rows for
things like inline functions, mixed declarations and data,
recursion/re-entrant functions, true boolean types, data in flash,
interrupt support, inline assembly support, intrinsic functions, etc.

What about library features, device header files, code samples, etc.?

Are there any cases of incorrect code generation?


>    "Some compilers instead offer non-standard versions of these
>    types with reduced precision..."
> 
> You've got a table, there.  Why not indicate WHICH compilers?
> (i.e., does nonstandard appear as a "yes" or as a "no"?  How
> do I differentiate between a "NONSTANDARD no" and a "MISSING no")
> 
> Presumably, enumerations and bitfields are supported by each?
> Or, do those have to be preprocessor hacks?
> 
> [Speaking of "preprocessor", differences between their implementations?]
> 
> In your Code Quality assessment, you claim:
>    "the floating-point parts, especially C library functions, dominate
>    the benchmark"
> But, given that you've just previously claimed this support is tenuous
> among the different compilers, how do I determine what the consequences
> of those implementations are in your assessment?    I.e., if it is
> a "hacked" data type it could make the results look better *or* worse!
> 
> You note that:
>    "for the others, they were modified as necessary to make them compile"
> and, later:
>    "Cosmic generates wrong code for Dhrystone"
> I.e., was this a consequence of your "modifications"?  (perhaps incorrect)

As I understand it, Dhrystone itself has bugs and undefined behaviour.
(That's not uncommon - a SPEC benchmark was optimised to an empty loop
by a development version of gcc, due to its undefined behaviour.)

Generally speaking, Dhrystone and other such benchmarks written decades
ago for mainframes and minicomputers are completely useless as
comparisons in small microcontrollers.  The code is so different from
the sort of code that will be used on these devices, that it offers no
useful information.

I am not sure if there are easily available ready-made benchmark code
snippets that /are/ useful for such devices.  But I would be more
interested in microbenchmarks or even samples of generated code.  How
well does the compiler handle basic arithmetic?  What does it do for
samples like these:

uint8_t test1(uint8_t a, uint8_t b) {
	return (a * 3) + b;
}

uint32_t test2(uint16_t a, uint8_t b) {
	return ((uint32_t) a) * b;
}

Does it make good code for a switch?  If you write low-level  bit-banged
SPI routines, how fast can it run them (testing bit and I/O access,
loops, and perhaps inlining of pin on/off functions)?

> 
> You might elaborate on the sort of modifications that were necessary
> given that these are all trying to be "standards compliant" (at least
> for your comparison).  I.e., syntax errors?  data sizes?  etc.
> 
> Your statement:
>     "Most software for embedded systems does not use floating-point
> arithmetic"
> might be misleading.  Often, something isn't used simply because it isn't
> PROVIDED in an efficient manner.  (where "efficient" is defined by the
> application's constraints).  I.e., replacing a floating point operation
> with one using fixed point on larger (wider) data types might be a
> valid comparison in many implementations.
> 

It is certainly true that on most small embedded systems, floating point
is rare or non-existent, and its performance is a minor issue (because
you assume that it is ridiculously slow, and only use it when that is
okay).  But some benchmarks of a few basic operations can be a helpful
guide.

> The Summary table is highly subjective.  And, you might want to move to
> a different representation of "score":  "-" suggests incomplete/inadequate
> whereas "one star" vs. "five stars" might be a better relative assessment.
> (likewise the bias introduced by red == fail, green == pass)
> 
> You might also want to comment on tweeks that are offered in the libraries
> available (e.g., trimmed down printf()'s, how/if the file oriented
> operations
> are addressed -- in systems that don't have "files", support for
> reentrancy,
> how clean the startup code is, etc.)
> 
> [BTW, I assume your benchmark's discount or normalize any crt0 costs
> in each of the above metrics?]
> 
> Rock on!

Back to comp.arch.embedded | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

A comparison of C implementations for the STM8 Philipp Klaus Krause <pkk@spth.de> - 2016-05-25 21:20 +0200
  Re: A comparison of C implementations for the STM8 Tim Wescott <tim@seemywebsite.com> - 2016-05-25 14:35 -0500
  Re: A comparison of C implementations for the STM8 Don Y <blockedofcourse@foo.invalid> - 2016-05-25 12:39 -0700
    Re: A comparison of C implementations for the STM8 Don Y <blockedofcourse@foo.invalid> - 2016-05-25 16:54 -0700
      Re: A comparison of C implementations for the STM8 David Brown <david.brown@hesbynett.no> - 2016-05-26 09:20 +0200
        Re: A comparison of C implementations for the STM8 Don Y <blockedofcourse@foo.invalid> - 2016-05-26 09:50 -0700
        Re: A comparison of C implementations for the STM8 Philipp Klaus Krause <pkk@spth.de> - 2016-05-27 10:59 +0200
          Re: A comparison of C implementations for the STM8 Philipp Klaus Krause <pkk@spth.de> - 2016-06-17 10:55 +0200
      Re: A comparison of C implementations for the STM8 Philipp Klaus Krause <pkk@spth.de> - 2016-05-26 15:22 +0200
    Re: A comparison of C implementations for the STM8 Philipp Klaus Krause <pkk@spth.de> - 2016-05-26 14:54 +0200

csiph-web