Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.arch.embedded > #27625

A new benchmark suitable for small systems: stdcbench

Show all headers | View raw

For benchmarking C implementations, the there are a few benchmarks, but
they all have their problems. Many benchmarks have memory requirements
that are far too high or need functionality not necessarily available.
Some are quite one-sided in what they measure (e.g. Whetstone,
Dhrystone, Coremark).

So, I deciced to write a new benchmark, stdcbench. I wanted it to be
suitable for small systems (4KB of RAM, about 32 KB of Flash). There is
a trade-off here, since all the data and code will fit easily into
caches on bigger systems, but IMO it is worth it.

The current version consists of 2 modules, which on typical systems
should contribute about equally to the score.

c90base:
It benchmarks a commonly-implemented subset of what the standard
requires for freestanding implementations of C90. It consists of three
submodules:
1) Huffman/RLE decompression (adapted from real-world code)
2) Integer matrix multiplication (synthetic)
3) Insertion sort (adapted from real-world code)

c90lib:
Benchmarks the standard library.
I consists of two submodules:
1) Computation of lnlc-width (adapted from real-world code).
2) Peephole optimizer (simplified from real-world code).


C99 features (e.g. bool, restrict) are used where available, but not
necessary.

So far, stdcbench seems to achieve the goals: benchmark a wide range of
important standard c functionality, without giving too much emphasis to
any particular aspect.

Scores are reported for each module and as total.

Example output from a i7-7500U-based system (benchmark compiled with GCC
7.2.0 using -O2 -march=native):

stdcbench 0.2
stdcbench c90base score: 7827
stdcbench c90lib score: 6548
stdcbench final score: 14375

Example output from a STM8AF5288 at 16 Mhz (benchmark compiled with SDCC
3.6.9 using -mstm8 --opt-code-speed --max-allocs-per-node 10000):

stdcbench 0.2
stdcbench c90base score: 6
stdcbench c90lib score: 6
stdcbench final score: 12

Future plans for the benchmark:

1) Come up with module(s) for floating-point performance. What matters
for embedded systems? How should correctness be verified for floating-point?
2) Find out why the c90lib module hangs on C8051F120 (possible compiler
bug).
3) State run/reporting rules.
4) Benchmark a few interesting systems


I am looking forward to comments from you.

http://stdcbench.org/

Philipp

Back to comp.arch.embedded | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

A new benchmark suitable for small systems: stdcbench Philipp Klaus Krause <pkk@spth.de> - 2018-02-07 16:49 +0100
  Re: A new benchmark suitable for small systems: stdcbench raimond.dragomir@gmail.com - 2018-02-08 22:36 -0800
    Re: A new benchmark suitable for small systems: stdcbench Philipp Klaus Krause <pkk@spth.de> - 2018-02-09 20:48 +0100
      Re: A new benchmark suitable for small systems: stdcbench Paul Rubin <no.email@nospam.invalid> - 2018-02-09 13:28 -0800
        Re: A new benchmark suitable for small systems: stdcbench Stef <stef33d@yahooI-N-V-A-L-I-D.com.invalid> - 2018-02-10 01:26 +0100
        Re: A new benchmark suitable for small systems: stdcbench Philipp Klaus Krause <pkk@spth.de> - 2018-02-10 18:47 +0100
          Re: A new benchmark suitable for small systems: stdcbench Tauno Voipio <tauno.voipio@notused.fi.invalid> - 2018-02-10 21:32 +0200
            Re: A new benchmark suitable for small systems: stdcbench Philipp Klaus Krause <pkk@spth.de> - 2018-02-10 20:40 +0100
        Re: A new benchmark suitable for small systems: stdcbench Philipp Klaus Krause <pkk@spth.de> - 2018-02-20 14:00 +0100
          Re: A new benchmark suitable for small systems: stdcbench raimond.dragomir@gmail.com - 2018-02-20 05:13 -0800
            Re: A new benchmark suitable for small systems: stdcbench Philipp Klaus Krause <pkk@spth.de> - 2018-02-20 14:38 +0100
              Re: A new benchmark suitable for small systems: stdcbench Philipp Klaus Krause <pkk@spth.de> - 2018-02-20 15:27 +0100
                Re: A new benchmark suitable for small systems: stdcbench Jack <jack4747@gmail.com> - 2018-02-20 07:14 -0800
                Re: A new benchmark suitable for small systems: stdcbench Philipp Klaus Krause <pkk@spth.de> - 2018-02-20 17:30 +0100
                Re: A new benchmark suitable for small systems: stdcbench Philipp Klaus Krause <pkk@spth.de> - 2018-03-05 19:59 +0100
                Re: A new benchmark suitable for small systems: stdcbench Philipp Klaus Krause <pkk@spth.de> - 2018-02-20 20:58 +0100
                Re: A new benchmark suitable for small systems: stdcbench raimond.dragomir@gmail.com - 2018-02-20 23:37 -0800
                Re: A new benchmark suitable for small systems: stdcbench Philipp Klaus Krause <pkk@spth.de> - 2018-02-21 09:29 +0100
                Re: A new benchmark suitable for small systems: stdcbench David Brown <david.brown@hesbynett.no> - 2018-02-21 11:01 +0100
          Re: A new benchmark suitable for small systems: stdcbench Tauno Voipio <tauno.voipio@notused.fi.invalid> - 2018-02-20 21:37 +0200
            Re: A new benchmark suitable for small systems: stdcbench raimond.dragomir@gmail.com - 2018-02-20 23:42 -0800
      Re: A new benchmark suitable for small systems: stdcbench raimond.dragomir@gmail.com - 2018-02-10 22:48 -0800
        Re: A new benchmark suitable for small systems: stdcbench Paul Rubin <no.email@nospam.invalid> - 2018-02-11 13:22 -0800
      stdcbench STM8 compiler comparison Philipp Klaus Krause <pkk@spth.de> - 2018-02-11 19:20 +0100
        Re: stdcbench STM8 compiler comparison Grant Edwards <invalid@invalid.invalid> - 2018-02-11 23:29 +0000
          Re: stdcbench STM8 compiler comparison Philipp Klaus Krause <pkk@spth.de> - 2018-02-12 07:13 +0100

csiph-web