Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.arch.embedded > #27625 > unrolled thread

A new benchmark suitable for small systems: stdcbench

Back to article view | Back to comp.arch.embedded

Contents

  A new benchmark suitable for small systems: stdcbench Philipp Klaus Krause <pkk@spth.de> - 2018-02-07 16:49 +0100
    Re: A new benchmark suitable for small systems: stdcbench raimond.dragomir@gmail.com - 2018-02-08 22:36 -0800
      Re: A new benchmark suitable for small systems: stdcbench Philipp Klaus Krause <pkk@spth.de> - 2018-02-09 20:48 +0100
        Re: A new benchmark suitable for small systems: stdcbench Paul Rubin <no.email@nospam.invalid> - 2018-02-09 13:28 -0800
          Re: A new benchmark suitable for small systems: stdcbench Stef <stef33d@yahooI-N-V-A-L-I-D.com.invalid> - 2018-02-10 01:26 +0100
          Re: A new benchmark suitable for small systems: stdcbench Philipp Klaus Krause <pkk@spth.de> - 2018-02-10 18:47 +0100
            Re: A new benchmark suitable for small systems: stdcbench Tauno Voipio <tauno.voipio@notused.fi.invalid> - 2018-02-10 21:32 +0200
              Re: A new benchmark suitable for small systems: stdcbench Philipp Klaus Krause <pkk@spth.de> - 2018-02-10 20:40 +0100
          Re: A new benchmark suitable for small systems: stdcbench Philipp Klaus Krause <pkk@spth.de> - 2018-02-20 14:00 +0100
            Re: A new benchmark suitable for small systems: stdcbench raimond.dragomir@gmail.com - 2018-02-20 05:13 -0800
              Re: A new benchmark suitable for small systems: stdcbench Philipp Klaus Krause <pkk@spth.de> - 2018-02-20 14:38 +0100
                Re: A new benchmark suitable for small systems: stdcbench Philipp Klaus Krause <pkk@spth.de> - 2018-02-20 15:27 +0100
                  Re: A new benchmark suitable for small systems: stdcbench Jack <jack4747@gmail.com> - 2018-02-20 07:14 -0800
                    Re: A new benchmark suitable for small systems: stdcbench Philipp Klaus Krause <pkk@spth.de> - 2018-02-20 17:30 +0100
                      Re: A new benchmark suitable for small systems: stdcbench Philipp Klaus Krause <pkk@spth.de> - 2018-03-05 19:59 +0100
                  Re: A new benchmark suitable for small systems: stdcbench Philipp Klaus Krause <pkk@spth.de> - 2018-02-20 20:58 +0100
                    Re: A new benchmark suitable for small systems: stdcbench raimond.dragomir@gmail.com - 2018-02-20 23:37 -0800
                      Re: A new benchmark suitable for small systems: stdcbench Philipp Klaus Krause <pkk@spth.de> - 2018-02-21 09:29 +0100
                      Re: A new benchmark suitable for small systems: stdcbench David Brown <david.brown@hesbynett.no> - 2018-02-21 11:01 +0100
            Re: A new benchmark suitable for small systems: stdcbench Tauno Voipio <tauno.voipio@notused.fi.invalid> - 2018-02-20 21:37 +0200
              Re: A new benchmark suitable for small systems: stdcbench raimond.dragomir@gmail.com - 2018-02-20 23:42 -0800
        Re: A new benchmark suitable for small systems: stdcbench raimond.dragomir@gmail.com - 2018-02-10 22:48 -0800
          Re: A new benchmark suitable for small systems: stdcbench Paul Rubin <no.email@nospam.invalid> - 2018-02-11 13:22 -0800
        stdcbench STM8 compiler comparison Philipp Klaus Krause <pkk@spth.de> - 2018-02-11 19:20 +0100
          Re: stdcbench STM8 compiler comparison Grant Edwards <invalid@invalid.invalid> - 2018-02-11 23:29 +0000
            Re: stdcbench STM8 compiler comparison Philipp Klaus Krause <pkk@spth.de> - 2018-02-12 07:13 +0100

Page 2 of 2 — ← Prev page 1 [2]

#27656

marți, 20 februarie 2018, 21:37:33 UTC+2, Tauno Voipio a scris:
> On 20.2.18 15:00, Philipp Klaus Krause wrote:
> > Am 09.02.2018 um 22:28 schrieb Paul Rubin:
> >> Philipp Klaus Krause <pkk@spth.de> writes:
> >>> Output for a 98 Mhz C8051F120 (compiled via sdcc -mmcs51 --model-large
> >>
> >> Was that really supposed to say 98 mhz?
> >>
> >> Can you say the code size for the different compiler outputs?
> >>
> >> Could you do the AVR8 the and MSP430 with gcc, if you happen to have
> >> those available?  Would the ARM Cortex M0 be getting outside the
> >> intended range of this benchmark?
> >>
> >> Thanks!
> >>
> > 
> > Here's a first reuslt from an ARM Cortex-M0 (the STM32F051R8 at 48 Mhz)
> > compiled using GCC 6.3.1 with -O2 and using newlib-nano:
> > 
> > stdcbench 0.3
> > stdcbench c90base score: 1141
> > stdcbench c90lib score: 651
> > stdcbench final score: 1792
> > 
> > Per clock cycle, this Cortex-M0 with GCC gets about twice the score
> > compared to an STM8 with IAR.
> > Interesting is the large difference between the c90base and the c90lib
> > score. I guess newlib-nanao is optimized for code size at the expense of
> > speed (even though it is surprising to see that much of a difference vs.
> > the situation for the STM8).
> > 
> > Philipp
> > 
> 
> GCC -Os does wonders compared to the standard compiled newlib.
> 
> -- 
> 
> -TV

Speaking of that, we always take the newlib/nelib-nano/whatever-lib for granted.
An interesting possibility would be to compile the same lib bench without lib calls
(of course, another version of the program doing the exact same thing but with no lib calls)
Just to see how we stand. It would be quite a good indicator of the
performance of the lib.

[toc] | [prev] | [next] | [standalone]

#27640

vineri, 9 februarie 2018, 21:48:34 UTC+2, Philipp Klaus Krause a scris:
> Am 09.02.2018 um 07:36 schrieb raimond.dragomir@gmail.com:
> > Nice.
> > One observation though: the STM8 score seems too low. I mean, it would
> > be difficult to compare systems that have scores like that (11,12,15 etc.)
> > I know STM8 and I know it's quite powerfull. I even use these (and some
> > AVRs) at a much lower frequency (5MHz).
> > 
> > What I'm trying to say is that the score for such a system (STM8/AVR8/16MHz)
> > should be in a 100-1000 range.
> > 
> > So please scale the scores up! (But take care that the lsb digits to not be noise!).
> > 
> > I don't care if the PC scores would be millions...
> > 
> 
> I agree. The previous resolution often was insufficient to even see the
> effect of compiler optimizations. In version 0.3, I did a bit of
> rebalancing and rescaling of scores.
> 
> Output for the 16 Mhz STM8AF5288 (compiled via sdcc -mstm8
> --opt-code-speed --max-allocs-per-node 10000):
> 
> stdcbench 0.3
> stdcbench c90base score: 109
> stdcbench c90lib score: 88
> stdcbench final score: 197
> 
> Output for the 16 Mhz STM8AF5288 (compiled via sdcc -mstm8
> --opt-code-size --max-allocs-per-node 10000):
> 
> stdcbench 0.3
> stdcbench c90base score: 107
> stdcbench c90lib score: 87
> stdcbench final score: 194
> 
> Output for a 98 Mhz C8051F120 (compiled via sdcc -mmcs51 --model-large
> --stack-auto --opt-code-size --max-allocs-per-node 10000):
> 
> stdcbench 0.3
> stdcbench c90base score: 96
> stdcbench final score: 96
> 
> Philipp
> 
> P.S.: The reason the c90lib module is not enabled for the C8051F120 is
> that it runs out of stack space.

Now it's better :)
This kind of benchmark is very interesting. Without it you can only
have a "feeling" about the power of an architecture, and only if you have
much experience with it. And of course it much depends on the application.

For example, it seems that the STM8S 16MHz performs better than the C8051
at 100MHz. This is not a surprise for me. I have worked a long time with
8051 and I know very well what is it capable of. For example, an 8051 is
almost unbeatable for small control applications of under 8K program
size and max. 256 bytes of internal ram. But if you step this line and your
program goes bigger, and especially if you need bigger ram and start to use
the XRAM, the efficiency goes down rapidly. The 8051 just doesn't scale
well in the addressing range. In the 8K/256 range is probably the best 8bitter,
in the 64K/64K range is probably the worst :) Here the program size is not 
a direct factor, it usually depends on how much ram you need. My experience
is that you can "grow" your program up to 8K and still use only the internal
256 bytes ram.

But of course, this benchmark is not suppose to reveal this kind of things...

[toc] | [prev] | [next] | [standalone]

#27642

raimond.dragomir@gmail.com writes:
> For example, it seems that the STM8S 16MHz performs better than the C8051
> at 100MHz... an 8051 is almost unbeatable for small control applications
> of under 8K program size and max. 256 bytes of internal ram.

I've always had the impression the 8051 was not well suited for commonly
used C coding styles and datatypes.  It was always intended to be
programmed in assembler, has good support for single-bit operations but not
much for 16 bit (usual C int type), etc.

The STM8 is interesting.  I found out about it fairly recently and got some
of the small STM8S103F3 boards for various purposes.  In small cheap 8
bitters it's often quite attractive compared with AVR and the like.  I'm
not sure what else is out there that's comparable, except maybe PIC.  I see
from the colecovision site that Philipp Klaus Krause did most of the SDCC
back end, so thanks Philipp!

[toc] | [prev] | [next] | [standalone]

#27641 — stdcbench STM8 compiler comparison

Here is a small comparison of STM8 results with various current
compilers (all done on the STM8AF5288).

SDCC 3.7.0 RC1 with optimization for code size (-mstm8 --opt-code-size
--max-allocs-per-node 100000), binary size 20953 B:

stdcbench 0.3

stdcbench c90base score: 106
stdcbench c90lib score: 87
stdcbench final score: 193

SDCC 3.7.0 RC1 with optimization for code speed  (-mstm8
--opt-code-speed --max-allocs-per-node 100000), binary size 21083 B:

stdcbench 0.3
stdcbench c90base score: 109
stdcbench c90lib score: 88
stdcbench final score: 197

IAR 3.10.1.201 with optimization for code size, binary size 24288 B:

stdcbench 0.3
stdcbench c90base score: 117
stdcbench c90lib score: 71
stdcbench final score: 188

IAR 3.10.1.201 with optimization for code speed, binary size 27268 B:

stdcbench 0.3
stdcbench c90base score: 197
stdcbench c90lib score: 100
stdcbench final score: 297

Cosmic 4.4.4 with optimization for code size:

stdcbench 0.3
stdcbench c90base score: 116
stdcbench final score: 116

Cosmic 4.4.4 with optimization for code speed:

stdcbench 0.3
stdcbench c90base score: 123
stdcbench final score: 123

For Cosmic 4.4.4, the c90lib module was disabled, since Cosmic 4.4.4
doesn't provide qsort() in the standard library. The Raisonance compiler
was not included in the comparison due to dificulties getting an
evaluation license.

These results are quite interesting when compared to Dhrystone and
Coremark (see http://www.colecovision.eu/stm8/compilers.shtml). In
particular, while SDCC is ahead in Dhrystone and Coremark scores, it
apparently falls behing in stdcbench scores. On the other hand, SDCC
seems to do better in code size for stdcbench.

Philipp

[toc] | [prev] | [next] | [standalone]

#27643 — Re: stdcbench STM8 compiler comparison

On 2018-02-11, Philipp Klaus Krause <pkk@spth.de> wrote:
> Here is a small comparison of STM8 results with various current
> compilers (all done on the STM8AF5288).
>
> SDCC 3.7.0 RC1 with optimization for code size (-mstm8 --opt-code-size
> --max-allocs-per-node 100000), binary size 20953 B:
>
> stdcbench 0.3
>
> stdcbench c90base score: 106
> stdcbench c90lib score: 87
> stdcbench final score: 193

Are bigger scores better or worse?

--
Grant

[toc] | [prev] | [next] | [standalone]

#27644 — Re: stdcbench STM8 compiler comparison

Am 12.02.2018 um 00:29 schrieb Grant Edwards:
> On 2018-02-11, Philipp Klaus Krause <pkk@spth.de> wrote:
>> Here is a small comparison of STM8 results with various current
>> compilers (all done on the STM8AF5288).
>>
>> SDCC 3.7.0 RC1 with optimization for code size (-mstm8 --opt-code-size
>> --max-allocs-per-node 100000), binary size 20953 B:
>>
>> stdcbench 0.3
>>
>> stdcbench c90base score: 106
>> stdcbench c90lib score: 87
>> stdcbench final score: 193
> 
> Are bigger scores better or worse?
> 
> --
> Grant
> 

Bigger scores are better.

Philipp

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.arch.embedded

csiph-web