Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!1.eu.feeder.erje.net!weretis.net!feeder4.news.weretis.net!feeder5.news.weretis.net!news.solani.org!.POSTED!not-for-mail
From: Philipp Klaus Krause <pkk@spth.de>
Newsgroups: comp.lang.c,comp.benchmarks
Subject: Re: A new benchmark
Followup-To: comp.lang.c
Date: Wed, 7 Feb 2018 15:02:44 +0100
Organization: solani.org
Lines: 56
Message-ID: <p5f0u4$5fi$1@solani.org>
References: <p4cjpp$g2c$1@solani.org>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Trace: solani.org 1518012165 5618 eJwNwoERwEAEBMCWHndEOcHrv4RkZ2ku3gGng8u1vP/hTVabpoHznBLRnAyldBfq3fCeAc4HFlURCA== (7 Feb 2018 14:02:45 GMT)
X-Complaints-To: abuse@news.solani.org
NNTP-Posting-Date: Wed, 7 Feb 2018 14:02:45 +0000 (UTC)
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1
X-NNTP-Posting-Host: eJwFwYEBwCAIA7CXQGiZ54jY/09YgqDzVhJMCFKtS81r4pvpijQz1SxE6CR1em9/6Qe57fUPKiURYg==
Content-Language: en-US
X-User-ID: eJwNwgURwEAMADBLZZCzkn8Jv0uUDa1dTE30fntJx7POAChOVVsICB8pd/hNRGWBZl5OxwMZeBEK
In-Reply-To: <p4cjpp$g2c$1@solani.org>
Cancel-Lock: sha1:ZdkSAbG9ThTGDh6zGhUfjGAZtPo=
Xref: csiph.com comp.lang.c:126412 comp.benchmarks:53

stdcbench has improved substantially since my first post to comp.lang.c.

The current version consists of 2 modules, which on typical systems
should contribute about equally to the score.

c90base:
It benchmarks a commonly-implemented subset of what the standard
requires for freestanding implementations of C90. It consists of three
submodules:
1) Huffman/RLE decompression (adapted from real-world code)
2) Integer matrix multiplication (synthetic)
3) Insertion sort (adapted from real-world code)

c90lib:
Benchmarks the standard library.
I consists of two submodules:
1) Computation of lnlc-width (adapted from real-world code)
2) Peephole optimizer (simplified from real-world code)

C99 features (e.g. bool, restrict) are used where available, but not
necessary.

So far, stdcbench seems to achieve the goals: benchmark a wide range of
important standard c functionality, without giving too much emphasis to
any particular aspect.

Scores are reported for each module and as total.

Example output from a i7-7500U-based system (benchmark compiled with GCC
7.2.0 using -O2 -march=native):

stdcbench 0.2
stdcbench c90base score: 7827
stdcbench c90lib score: 6548
stdcbench final score: 14375

Example output from a STM8AF5288 at 16 Mhz (benchmark compiled with SDCC
3.6.9 using -mstm8 --opt-code-speed --max-allocs-per-node 10000):

stdcbench 0.2
stdcbench c90base score: 6
stdcbench c90lib score: 6
stdcbench final score: 12

stdcbench already seems quite reasonable for benchmarking integer
performance. However, there is still work to do:

1) Come up with module(s) for floating-point performance. What matters
for embedded systems? How should correctness be verified for floating-point?
2) Find out why the c90lib module hangs on C8051F120 (possible compiler
bug).
3) State run/reporting rules.
4) Benchmark a few interesting systems
6) Create a website or write a paper about the benchmark and the results.

Philipp