Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #125802

Re: A new benchmark

Show all headers | View raw

Am 25.01.2018 um 22:42 schrieb supercat@casperkitty.com:
> On Thursday, January 25, 2018 at 6:50:19 AM UTC-6, Philipp Klaus Krause wrote:
>> * Gives a balanced reflection of performance.
>> 	- No single functionality should dominate the results.
>> * Gives a single number as final score.
> 
> The performance of many real-world applications will be dominated by one or
> at most a small number of "functionalities", though different programs'
> performance will be dominated by different functionalities.  A benchmark
> that does a different mix of operations from what an application requires
> is unlikely to predict how well different machines will perform when running
> that application.

Of course, a benchmark score will not be representative of performance
of many individual applications.

But there are still important use-cases for benchmarks. They do give a
general idea fo how C implementations perform. Two examples:

When choosign an implementation, typically people cannot compare all
implementations with their own application. They might be able to
compare a select few, but otherwise they would have to rely on other
parameters, such as the clock speed of the processor, or benchmark
scores. And typically, a benchmark score will give a better idea than
raw hardware parameters.

When deciding if a certain optimization is worth adding to an
implementation. The benefits of an optimization have to be weighted
against potential regressions for other use-cases, and against added
implementation complexity. Of course an implementation might decide,
that some application is so important, that the benefits of some
optimization that is very useful there are worth it. But what about
otehr optimizations that don't seem to implement any individual
killer-application much? A benchmark can be useful to help with the
decision.

> Further, in many cases I would suggest that there should be a distinction
> made between how well an implementation performs when given "100% portable"
> code to do a task, versus how well it could perform when given code which
> is tailored around that implementation's strength and weaknesses.  Of course,
> any benchmark intended to measure that would need to be configurable based
> on the target platform's features.
> 

I disagree somewhat here. Even for embedded targets, typically one wants
to write portable code (or at least have the non-portable bits isolated
somewhere). Yes, one might identify a certain bottleneck, and then try
to improve performance there by using non-portable constructs. But an
implementation that doesn't need a lot of non-portable stuff to be
efficient is still better.
Also, fortunately, most implementation I can think of are able to get
very good performance with portable code.¹

Philipp

¹ The only counterexample that comes to my mind are implementation for
architectures like MCS-51, where accesses via generic pointers go
through some helper funtion. On the other hand, when it is known, to
which named address space they point, code can be generated directly.

Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

A new benchmark Philipp Klaus Krause <pkk@spth.de> - 2018-01-25 13:50 +0100
  Re: A new benchmark "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2018-01-25 08:25 -0500
    Re: A new benchmark Philipp Klaus Krause <pkk@spth.de> - 2018-01-25 14:45 +0100
      Re: A new benchmark fir <profesor.fir@gmail.com> - 2018-01-25 06:41 -0800
        Re: A new benchmark Philipp Klaus Krause <pkk@spth.de> - 2018-01-25 15:52 +0100
          Re: A new benchmark fir <profesor.fir@gmail.com> - 2018-01-25 07:02 -0800
            Re: A new benchmark Philipp Klaus Krause <pkk@spth.de> - 2018-01-26 15:22 +0100
  Re: A new benchmark bartc <bc@freeuk.com> - 2018-01-25 16:09 +0000
  Re: A new benchmark Robert Wessel <robertwessel2@yahoo.com> - 2018-01-25 13:00 -0600
    Re: A new benchmark Philipp Klaus Krause <pkk@spth.de> - 2018-01-26 10:14 +0100
      Re: A new benchmark Robert Wessel <robertwessel2@yahoo.com> - 2018-01-26 10:56 -0600
        Re: A new benchmark supercat@casperkitty.com - 2018-01-26 13:07 -0800
          Re: A new benchmark Robert Wessel <robertwessel2@yahoo.com> - 2018-01-27 13:13 -0600
  Re: A new benchmark supercat@casperkitty.com - 2018-01-25 13:42 -0800
    Re: A new benchmark Philipp Klaus Krause <pkk@spth.de> - 2018-01-26 11:19 +0100
      Re: A new benchmark supercat@casperkitty.com - 2018-01-26 08:17 -0800

csiph-web