Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #32215

Re: method micro benachmarking with reflection

Show key headers only | View raw

On 7/30/2015 1:30 PM, e.d.programmer@gmail.com wrote:
> On Thursday, July 30, 2015 at 11:42:29 AM UTC-4, Eric Sosman wrote:
>>       One of the articles I linked to suggests doing "several"
>> times that many loop executions ...
>>
>>       ... in the *warmup* phase, before serious measurements.
>
> Right.  It helps if you call the method before getting the initial timestamp to exclude the one time (classloader, public class variable creation) stuff.

     The point (and a repeated warning in the links provided) is that
"call the method" is not enough of a warmup.  You must

     - Call the method enough times to be sure that the JIT has finished
       converting it to native code,

     - Call *exactly* the same methods during the timing phase as you
       did during the warmup, with exactly the same classes loaded and
       so on, lest the JIT change its mind,

     - Check that nothing unexpected happened during the timing phase.

> The 10000 was a random number.  It depends how long it takes to run the method once, and how long you want to spend testing it.  If 10000 calls only takes a few minutes you could do 100000 calls to be more accurate.  If 10000 calls takes 10 hours, do 100000 calls if you have a week to test...

     If 10000 calls takes a few minutes, it seems to me that we're no
longer in the realm of "micro benachmarking [sic]."  Microbenchmarks
are rather different from tests of system capacity, throughput, or
latency.  All are difficult, all present challenges of their very own.

-- 
esosman@comcast-dot-net.invalid
"Don't be afraid of work. Make work afraid of you." -- TLM

Back to comp.lang.java.programmer | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

method micro benachmarking with reflection philipp.kraus.flashpixx@gmail.com - 2015-07-30 03:41 -0700
  Re: method micro benachmarking with reflection Arne Vajhøj <arne@vajhoej.dk> - 2015-07-30 07:16 -0400
    Re: method micro benachmarking with reflection Arne Vajhøj <arne@vajhoej.dk> - 2015-07-30 21:52 -0400
      Re: method micro benachmarking with reflection Arne Vajhøj <arne@vajhoej.dk> - 2015-07-30 21:55 -0400
  Re: method micro benachmarking with reflection Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-07-30 08:23 -0400
    Re: method micro benachmarking with reflection e.d.programmer@gmail.com - 2015-07-30 06:00 -0700
      Re: method micro benachmarking with reflection Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-07-30 09:30 -0400
        Re: method micro benachmarking with reflection e.d.programmer@gmail.com - 2015-07-30 08:26 -0700
          Re: method micro benachmarking with reflection Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-07-30 11:42 -0400
            Re: method micro benachmarking with reflection e.d.programmer@gmail.com - 2015-07-30 10:30 -0700
              Re: method micro benachmarking with reflection Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-07-30 14:20 -0400
        Re: method micro benachmarking with reflection Jeff Higgins <jeff@invalid.invalid> - 2015-07-30 16:08 -0400
    Re: method micro benachmarking with reflection philipp.kraus.flashpixx@gmail.com - 2015-07-30 06:11 -0700
    Re: method micro benachmarking with reflection v_borchert@despammed.com (Volker Borchert) - 2015-07-30 21:33 +0000
      Re: method micro benachmarking with reflection Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-07-30 18:18 -0400
        Re: method micro benachmarking with reflection Arne Vajhøj <arne@vajhoej.dk> - 2015-07-30 20:13 -0400
  Re: method micro benachmarking with reflection Jeff Higgins <jeff@invalid.invalid> - 2015-07-30 17:11 -0400
  Re: method micro benachmarking with reflection v_borchert@despammed.com (Volker Borchert) - 2015-07-30 21:53 +0000
  Re: method micro benachmarking with reflection Sebastian <news@seyweiler.dyndns.org> - 2015-08-04 21:44 +0200

csiph-web