Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #32211

Re: method micro benachmarking with reflection

Show key headers only | View raw

On 7/30/2015 9:00 AM, e.d.programmer@gmail.com wrote:
> On Thursday, July 30, 2015 at 8:23:58 AM UTC-4, Eric Sosman wrote:
>> On 7/30/2015 6:41 AM, philipp.kraus.flashpixx@gmail.com wrote:
>>> Hello,
>>>
>>> I have got a complex system and I would like to add some microbenchmarks to the application.
>>> So I need the runtime of a method call, but I don't want to add to each method a time counter. Can I add via reflection API a structure, that adds the micro benchmarks automatically? E.g.
>>>
>>> registerbenchmark( myclass.class, mymethod-handle ).
>>>
>>> This calls should do on any object
>>>
>>> myclass x = new myclass();
>>>
>>> timer start
>>> x.mymethod();
>>> timer stop
>>>
>>> So I would like to do register all classes & methods on the application startup and on any instance of a class the benchmark code should be run. So I need an event, which is called on each method call.
>>>
>>> Can I register an event-handler at the JVM to get each method call? Does there exists some frameworks for microbenchmarking?
>>
>>       (Disclaimer: I've never done this ...)
>>
>>       I think that java.lang.instrument package can be used for this
>> sort of thing.  It lets you change a method's byte code on the fly,
>> so you can insert calls to start and stop timers or whatever.
>>
>>       BUT: Unless a method has "macroscopic" run time, you'll have a
>> hard time finding a sufficiently fine-grained timer.  Nothing Java
>> itself supplies will suffice (System.nanoTime() may look promising,
>> but even though it gives a result in nanoseconds it seldom measures
>> that precisely).  Using JNI, you might be able to access chip-specific
>> low-level cycle counters or some such -- but even then, you'd have to
>> take it on faith that all the cycles between a method's start and end
>> were consumed by the method; what if the garbage collector decided to
>> preempt the method's thread for a few milliseconds?
>>
>>       Final thought: Re-read the disclaimer ...
>>
>> --
>> esosman@comcast-dot-net.invalid
>> "Don't be afraid of work. Make work afraid of you." -- TLM

     Aside: Posters with good netiquette don't quote signatures.

	http://alt-usage-english.org/posting_quotes.html
	http://www.html-faq.com/etiquette/?quoting

> To account for unknown variance on a performance test you should perform the test many times, depending on the length of the test.  If it only takes a second to complete the method once you can call the method 60 times and your test should take about a minute.  If it can take 1/10 of a second you can run the test 600 times if you want it to finish in a minute.  Always run your test once first because you don't want to call it 100 times if one call takes 60 minutes.  If it completes right away try 10 times then maybe 100 or 10000 times depending how long it runs and how long you want to spend testing it.  The more times you can call it the more you should eliminate the variance.  A garbage collect on one call out of 1000 is irrelevant.

     It's much harder for Java, *much* harder.  One GC per thousand calls
might be irrelevant (I'm not confident it is, but won't try to argue),
but what about one JIT per thousand calls?  That is, if the first five
hundred calls are interpreted and the next five hundred run as native
code, what happens to the timing?  And if the code is then re-JITted
(as it may be, and frequently is), resulting in a different native-code
translation, what can one say about the five hundred calls not yet
measured?

     Here are a few informative (and cautionary, and perhaps even
discouraging!) articles on Java microbenchmarking:

http://www.azulsystems.com/events/javaone_2002/microbenchmarks.pdf
https://wiki.openjdk.java.net/display/HotSpot/MicroBenchmarks
http://www.ibm.com/developerworks/java/library/j-jtp02225/index.html

(The first is a collection of presentation slides, and may be a bit
telegraphic since there's no presenter to talk about them.  The last
is by Brian Goetz, and used to be hosted on an Oracle Wiki until
Oracle in their wisdom shut the whole thing down, but we're fortunate
that IBM cached a copy.)

-- 
esosman@comcast-dot-net.invalid
"Don't be afraid of work. Make work afraid of you." -- TLM

Back to comp.lang.java.programmer | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

method micro benachmarking with reflection philipp.kraus.flashpixx@gmail.com - 2015-07-30 03:41 -0700
  Re: method micro benachmarking with reflection Arne Vajhøj <arne@vajhoej.dk> - 2015-07-30 07:16 -0400
    Re: method micro benachmarking with reflection Arne Vajhøj <arne@vajhoej.dk> - 2015-07-30 21:52 -0400
      Re: method micro benachmarking with reflection Arne Vajhøj <arne@vajhoej.dk> - 2015-07-30 21:55 -0400
  Re: method micro benachmarking with reflection Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-07-30 08:23 -0400
    Re: method micro benachmarking with reflection e.d.programmer@gmail.com - 2015-07-30 06:00 -0700
      Re: method micro benachmarking with reflection Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-07-30 09:30 -0400
        Re: method micro benachmarking with reflection e.d.programmer@gmail.com - 2015-07-30 08:26 -0700
          Re: method micro benachmarking with reflection Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-07-30 11:42 -0400
            Re: method micro benachmarking with reflection e.d.programmer@gmail.com - 2015-07-30 10:30 -0700
              Re: method micro benachmarking with reflection Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-07-30 14:20 -0400
        Re: method micro benachmarking with reflection Jeff Higgins <jeff@invalid.invalid> - 2015-07-30 16:08 -0400
    Re: method micro benachmarking with reflection philipp.kraus.flashpixx@gmail.com - 2015-07-30 06:11 -0700
    Re: method micro benachmarking with reflection v_borchert@despammed.com (Volker Borchert) - 2015-07-30 21:33 +0000
      Re: method micro benachmarking with reflection Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-07-30 18:18 -0400
        Re: method micro benachmarking with reflection Arne Vajhøj <arne@vajhoej.dk> - 2015-07-30 20:13 -0400
  Re: method micro benachmarking with reflection Jeff Higgins <jeff@invalid.invalid> - 2015-07-30 17:11 -0400
  Re: method micro benachmarking with reflection v_borchert@despammed.com (Volker Borchert) - 2015-07-30 21:53 +0000
  Re: method micro benachmarking with reflection Sebastian <news@seyweiler.dyndns.org> - 2015-08-04 21:44 +0200

csiph-web