Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #32206 > unrolled thread

method micro benachmarking with reflection

Back to article view | Back to comp.lang.java.programmer

Contents

  method micro benachmarking with reflection philipp.kraus.flashpixx@gmail.com - 2015-07-30 03:41 -0700
    Re: method micro benachmarking with reflection Arne Vajhøj <arne@vajhoej.dk> - 2015-07-30 07:16 -0400
      Re: method micro benachmarking with reflection Arne Vajhøj <arne@vajhoej.dk> - 2015-07-30 21:52 -0400
        Re: method micro benachmarking with reflection Arne Vajhøj <arne@vajhoej.dk> - 2015-07-30 21:55 -0400
    Re: method micro benachmarking with reflection Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-07-30 08:23 -0400
      Re: method micro benachmarking with reflection e.d.programmer@gmail.com - 2015-07-30 06:00 -0700
        Re: method micro benachmarking with reflection Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-07-30 09:30 -0400
          Re: method micro benachmarking with reflection e.d.programmer@gmail.com - 2015-07-30 08:26 -0700
            Re: method micro benachmarking with reflection Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-07-30 11:42 -0400
              Re: method micro benachmarking with reflection e.d.programmer@gmail.com - 2015-07-30 10:30 -0700
                Re: method micro benachmarking with reflection Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-07-30 14:20 -0400
          Re: method micro benachmarking with reflection Jeff Higgins <jeff@invalid.invalid> - 2015-07-30 16:08 -0400
      Re: method micro benachmarking with reflection philipp.kraus.flashpixx@gmail.com - 2015-07-30 06:11 -0700
      Re: method micro benachmarking with reflection v_borchert@despammed.com (Volker Borchert) - 2015-07-30 21:33 +0000
        Re: method micro benachmarking with reflection Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-07-30 18:18 -0400
          Re: method micro benachmarking with reflection Arne Vajhøj <arne@vajhoej.dk> - 2015-07-30 20:13 -0400
    Re: method micro benachmarking with reflection Jeff Higgins <jeff@invalid.invalid> - 2015-07-30 17:11 -0400
    Re: method micro benachmarking with reflection v_borchert@despammed.com (Volker Borchert) - 2015-07-30 21:53 +0000
    Re: method micro benachmarking with reflection Sebastian <news@seyweiler.dyndns.org> - 2015-08-04 21:44 +0200

#32206 — method micro benachmarking with reflection

Hello,

I have got a complex system and I would like to add some microbenchmarks to the application.
So I need the runtime of a method call, but I don't want to add to each method a time counter. Can I add via reflection API a structure, that adds the micro benchmarks automatically? E.g.

registerbenchmark( myclass.class, mymethod-handle ).

This calls should do on any object

myclass x = new myclass();

timer start
x.mymethod();
timer stop

So I would like to do register all classes & methods on the application startup and on any instance of a class the benchmark code should be run. So I need an event, which is called on each method call.

Can I register an event-handler at the JVM to get each method call? Does there exists some frameworks for microbenchmarking?

Hope for any comment, thanks a lot

Phil

[toc] | [next] | [standalone]

#32207

On 7/30/2015 6:41 AM, philipp.kraus.flashpixx@gmail.com wrote:
> I have got a complex system and I would like to add some microbenchmarks to the application.
> So I need the runtime of a method call, but I don't want to add to each method a time counter. Can I add via reflection API a structure, that adds the micro benchmarks automatically? E.g.
>
> registerbenchmark( myclass.class, mymethod-handle ).
>
> This calls should do on any object
>
> myclass x = new myclass();
>
> timer start
> x.mymethod();
> timer stop
>
> So I would like to do register all classes & methods on the application startup and on any instance of a class the benchmark code should be run. So I need an event, which is called on each method call.
>
> Can I register an event-handler at the JVM to get each method call? Does there exists some frameworks for microbenchmarking?
>
> Hope for any comment, thanks a lot

Sounds like a good case for AOP like AspectJ.

Arne

[toc] | [prev] | [next] | [standalone]

#32223

On 7/30/2015 7:16 AM, Arne Vajhøj wrote:
> On 7/30/2015 6:41 AM, philipp.kraus.flashpixx@gmail.com wrote:
>> I have got a complex system and I would like to add some
>> microbenchmarks to the application.
>> So I need the runtime of a method call, but I don't want to add to
>> each method a time counter. Can I add via reflection API a structure,
>> that adds the micro benchmarks automatically? E.g.
>>
>> registerbenchmark( myclass.class, mymethod-handle ).
>>
>> This calls should do on any object
>>
>> myclass x = new myclass();
>>
>> timer start
>> x.mymethod();
>> timer stop
>>
>> So I would like to do register all classes & methods on the
>> application startup and on any instance of a class the benchmark code
>> should be run. So I need an event, which is called on each method call.
>>
>> Can I register an event-handler at the JVM to get each method call?
>> Does there exists some frameworks for microbenchmarking?
>>
>> Hope for any comment, thanks a lot
>
> Sounds like a good case for AOP like AspectJ.

Example:

TimerTool.aj

import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;

import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;

@Aspect
public class TimerTool {
     private static Map<String,Long> store = new 
ConcurrentHashMap<String,Long>();
     public static void register(String signature) {
         store.put(signature, 0L);
     }
     public static long getOne(String signature) {
         return store.get(signature);
     }
     public static Iterable<Map.Entry<String,Long>> getAll() {
         return store.entrySet();
     }
     @Pointcut("call(* *.*()) && !within(TimerTool) ")
     public void mbmcall() {}
     @Around(value = "mbmcall()")
     public Object time(ProceedingJoinPoint pjp) {
         long t1 = System.nanoTime();
         Object res = pjp.proceed();
         long t2 = System.nanoTime();
         if(store.containsKey(pjp.getSignature().toString())) {
             store.put(pjp.getSignature().toString(), 
store.get(pjp.getSignature().toString()) + (t2 - t1));
         }

         return res;
     }
}

MBM.java

import java.util.Map;

public class MBM {
     public void m1() throws InterruptedException {
         Thread.sleep(1);
     }
     public void m2() throws InterruptedException {
         Thread.sleep(2);
     }
     public void m3() throws InterruptedException {
         Thread.sleep(3);
     }
     public static void main(String[] args) throws Exception {
         TimerTool.register("void MBM.m1()");
         TimerTool.register("void MBM.m3()");
         MBM o = new MBM();
         for(int i = 0; i < 1000; i++) {
             o.m1();
             o.m2();
             o.m3();
         }
         for(Map.Entry<String,Long> e : TimerTool.getAll()) {
             System.out.printf("%s : %.3f\n", e.getKey(), e.getValue() / 
1000000000.0);
         }
     }
}

Example output:

void MBM.m3() : 2.997
void MBM.m1() : 0.994

It seems very close to what you asked for.

Arne

[toc] | [prev] | [next] | [standalone]

#32224

On 7/30/2015 9:52 PM, Arne Vajhøj wrote:
> On 7/30/2015 7:16 AM, Arne Vajhøj wrote:
>> On 7/30/2015 6:41 AM, philipp.kraus.flashpixx@gmail.com wrote:
>>> I have got a complex system and I would like to add some
>>> microbenchmarks to the application.
>>> So I need the runtime of a method call, but I don't want to add to
>>> each method a time counter. Can I add via reflection API a structure,
>>> that adds the micro benchmarks automatically? E.g.
>>>
>>> registerbenchmark( myclass.class, mymethod-handle ).
>>>
>>> This calls should do on any object
>>>
>>> myclass x = new myclass();
>>>
>>> timer start
>>> x.mymethod();
>>> timer stop
>>>
>>> So I would like to do register all classes & methods on the
>>> application startup and on any instance of a class the benchmark code
>>> should be run. So I need an event, which is called on each method call.
>>>
>>> Can I register an event-handler at the JVM to get each method call?
>>> Does there exists some frameworks for microbenchmarking?
>>>
>>> Hope for any comment, thanks a lot
>>
>> Sounds like a good case for AOP like AspectJ.
>
> Example:

> Example output:
>
> void MBM.m3() : 2.997
> void MBM.m1() : 0.994
>
> It seems very close to what you asked for.

Whether it is what you want is another discussion.

As explained in detail by other posters then getting
useful information out of this is highly problematic.

It worked somewhat in my examples because my methods
took milliseconds to run.

I suspect that if they only took a few microseconds then
the results would be random like.

Arne

[toc] | [prev] | [next] | [standalone]

#32208

On 7/30/2015 6:41 AM, philipp.kraus.flashpixx@gmail.com wrote:
> Hello,
>
> I have got a complex system and I would like to add some microbenchmarks to the application.
> So I need the runtime of a method call, but I don't want to add to each method a time counter. Can I add via reflection API a structure, that adds the micro benchmarks automatically? E.g.
>
> registerbenchmark( myclass.class, mymethod-handle ).
>
> This calls should do on any object
>
> myclass x = new myclass();
>
> timer start
> x.mymethod();
> timer stop
>
> So I would like to do register all classes & methods on the application startup and on any instance of a class the benchmark code should be run. So I need an event, which is called on each method call.
>
> Can I register an event-handler at the JVM to get each method call? Does there exists some frameworks for microbenchmarking?

     (Disclaimer: I've never done this ...)

     I think that java.lang.instrument package can be used for this
sort of thing.  It lets you change a method's byte code on the fly,
so you can insert calls to start and stop timers or whatever.

     BUT: Unless a method has "macroscopic" run time, you'll have a
hard time finding a sufficiently fine-grained timer.  Nothing Java
itself supplies will suffice (System.nanoTime() may look promising,
but even though it gives a result in nanoseconds it seldom measures
that precisely).  Using JNI, you might be able to access chip-specific
low-level cycle counters or some such -- but even then, you'd have to
take it on faith that all the cycles between a method's start and end
were consumed by the method; what if the garbage collector decided to
preempt the method's thread for a few milliseconds?

     Final thought: Re-read the disclaimer ...

-- 
esosman@comcast-dot-net.invalid
"Don't be afraid of work. Make work afraid of you." -- TLM

[toc] | [prev] | [next] | [standalone]

#32209

On Thursday, July 30, 2015 at 8:23:58 AM UTC-4, Eric Sosman wrote:
> On 7/30/2015 6:41 AM, philipp.kraus.flashpixx@gmail.com wrote:
> > Hello,
> >
> > I have got a complex system and I would like to add some microbenchmarks to the application.
> > So I need the runtime of a method call, but I don't want to add to each method a time counter. Can I add via reflection API a structure, that adds the micro benchmarks automatically? E.g.
> >
> > registerbenchmark( myclass.class, mymethod-handle ).
> >
> > This calls should do on any object
> >
> > myclass x = new myclass();
> >
> > timer start
> > x.mymethod();
> > timer stop
> >
> > So I would like to do register all classes & methods on the application startup and on any instance of a class the benchmark code should be run. So I need an event, which is called on each method call.
> >
> > Can I register an event-handler at the JVM to get each method call? Does there exists some frameworks for microbenchmarking?
> 
>      (Disclaimer: I've never done this ...)
> 
>      I think that java.lang.instrument package can be used for this
> sort of thing.  It lets you change a method's byte code on the fly,
> so you can insert calls to start and stop timers or whatever.
> 
>      BUT: Unless a method has "macroscopic" run time, you'll have a
> hard time finding a sufficiently fine-grained timer.  Nothing Java
> itself supplies will suffice (System.nanoTime() may look promising,
> but even though it gives a result in nanoseconds it seldom measures
> that precisely).  Using JNI, you might be able to access chip-specific
> low-level cycle counters or some such -- but even then, you'd have to
> take it on faith that all the cycles between a method's start and end
> were consumed by the method; what if the garbage collector decided to
> preempt the method's thread for a few milliseconds?
> 
>      Final thought: Re-read the disclaimer ...
> 
> -- 
> esosman@comcast-dot-net.invalid
> "Don't be afraid of work. Make work afraid of you." -- TLM

To account for unknown variance on a performance test you should perform the test many times, depending on the length of the test.  If it only takes a second to complete the method once you can call the method 60 times and your test should take about a minute.  If it can take 1/10 of a second you can run the test 600 times if you want it to finish in a minute.  Always run your test once first because you don't want to call it 100 times if one call takes 60 minutes.  If it completes right away try 10 times then maybe 100 or 10000 times depending how long it runs and how long you want to spend testing it.  The more times you can call it the more you should eliminate the variance.  A garbage collect on one call out of 1000 is irrelevant.

[toc] | [prev] | [next] | [standalone]

#32211

On 7/30/2015 9:00 AM, e.d.programmer@gmail.com wrote:
> On Thursday, July 30, 2015 at 8:23:58 AM UTC-4, Eric Sosman wrote:
>> On 7/30/2015 6:41 AM, philipp.kraus.flashpixx@gmail.com wrote:
>>> Hello,
>>>
>>> I have got a complex system and I would like to add some microbenchmarks to the application.
>>> So I need the runtime of a method call, but I don't want to add to each method a time counter. Can I add via reflection API a structure, that adds the micro benchmarks automatically? E.g.
>>>
>>> registerbenchmark( myclass.class, mymethod-handle ).
>>>
>>> This calls should do on any object
>>>
>>> myclass x = new myclass();
>>>
>>> timer start
>>> x.mymethod();
>>> timer stop
>>>
>>> So I would like to do register all classes & methods on the application startup and on any instance of a class the benchmark code should be run. So I need an event, which is called on each method call.
>>>
>>> Can I register an event-handler at the JVM to get each method call? Does there exists some frameworks for microbenchmarking?
>>
>>       (Disclaimer: I've never done this ...)
>>
>>       I think that java.lang.instrument package can be used for this
>> sort of thing.  It lets you change a method's byte code on the fly,
>> so you can insert calls to start and stop timers or whatever.
>>
>>       BUT: Unless a method has "macroscopic" run time, you'll have a
>> hard time finding a sufficiently fine-grained timer.  Nothing Java
>> itself supplies will suffice (System.nanoTime() may look promising,
>> but even though it gives a result in nanoseconds it seldom measures
>> that precisely).  Using JNI, you might be able to access chip-specific
>> low-level cycle counters or some such -- but even then, you'd have to
>> take it on faith that all the cycles between a method's start and end
>> were consumed by the method; what if the garbage collector decided to
>> preempt the method's thread for a few milliseconds?
>>
>>       Final thought: Re-read the disclaimer ...
>>
>> --
>> esosman@comcast-dot-net.invalid
>> "Don't be afraid of work. Make work afraid of you." -- TLM

     Aside: Posters with good netiquette don't quote signatures.

	http://alt-usage-english.org/posting_quotes.html
	http://www.html-faq.com/etiquette/?quoting

> To account for unknown variance on a performance test you should perform the test many times, depending on the length of the test.  If it only takes a second to complete the method once you can call the method 60 times and your test should take about a minute.  If it can take 1/10 of a second you can run the test 600 times if you want it to finish in a minute.  Always run your test once first because you don't want to call it 100 times if one call takes 60 minutes.  If it completes right away try 10 times then maybe 100 or 10000 times depending how long it runs and how long you want to spend testing it.  The more times you can call it the more you should eliminate the variance.  A garbage collect on one call out of 1000 is irrelevant.

     It's much harder for Java, *much* harder.  One GC per thousand calls
might be irrelevant (I'm not confident it is, but won't try to argue),
but what about one JIT per thousand calls?  That is, if the first five
hundred calls are interpreted and the next five hundred run as native
code, what happens to the timing?  And if the code is then re-JITted
(as it may be, and frequently is), resulting in a different native-code
translation, what can one say about the five hundred calls not yet
measured?

     Here are a few informative (and cautionary, and perhaps even
discouraging!) articles on Java microbenchmarking:

http://www.azulsystems.com/events/javaone_2002/microbenchmarks.pdf
https://wiki.openjdk.java.net/display/HotSpot/MicroBenchmarks
http://www.ibm.com/developerworks/java/library/j-jtp02225/index.html

(The first is a collection of presentation slides, and may be a bit
telegraphic since there's no presenter to talk about them.  The last
is by Brian Goetz, and used to be hosted on an Oracle Wiki until
Oracle in their wisdom shut the whole thing down, but we're fortunate
that IBM cached a copy.)

-- 
esosman@comcast-dot-net.invalid
"Don't be afraid of work. Make work afraid of you." -- TLM

[toc] | [prev] | [next] | [standalone]

#32212

On Thursday, July 30, 2015 at 9:30:40 AM UTC-4, Eric Sosman wrote:
>      It's much harder for Java, *much* harder.  One GC per thousand calls
> might be irrelevant (I'm not confident it is, but won't try to argue),
> but what about one JIT per thousand calls?  That is, if the first five
> hundred calls are interpreted and the next five hundred run as native
> code, what happens to the timing?  And if the code is then re-JITted
> (as it may be, and frequently is), resulting in a different native-code
> translation, what can one say about the five hundred calls not yet
> measured?

I haven't tried this microediting myself, I just get the timestamp then run a loop to call the method 10000 times then get the timestamp again and compare.
Normally if it does call a GC at some point you do want that in your performance test because you don't want to test the method in a vacuum, you want to see how long it takes for end users..

[toc] | [prev] | [next] | [standalone]

#32213

On 7/30/2015 11:26 AM, e.d.programmer@gmail.com wrote:
> On Thursday, July 30, 2015 at 9:30:40 AM UTC-4, Eric Sosman wrote:
>>       It's much harder for Java, *much* harder.  One GC per thousand calls
>> might be irrelevant (I'm not confident it is, but won't try to argue),
>> but what about one JIT per thousand calls?  That is, if the first five
>> hundred calls are interpreted and the next five hundred run as native
>> code, what happens to the timing?  And if the code is then re-JITted
>> (as it may be, and frequently is), resulting in a different native-code
>> translation, what can one say about the five hundred calls not yet
>> measured?
>
> I haven't tried this microediting myself, I just get the timestamp then run a loop to call the method 10000 times then get the timestamp again and compare.
> Normally if it does call a GC at some point you do want that in your performance test because you don't want to test the method in a vacuum, you want to see how long it takes for end users..

     One of the articles I linked to suggests doing "several"
times that many loop executions ...

     ... in the *warmup* phase, before serious measurements.

-- 
esosman@comcast-dot-net.invalid
"Don't be afraid of work. Make work afraid of you." -- TLM

[toc] | [prev] | [next] | [standalone]

#32214

On Thursday, July 30, 2015 at 11:42:29 AM UTC-4, Eric Sosman wrote:
>      One of the articles I linked to suggests doing "several"
> times that many loop executions ...
> 
>      ... in the *warmup* phase, before serious measurements.

Right.  It helps if you call the method before getting the initial timestamp to exclude the one time (classloader, public class variable creation) stuff.
The 10000 was a random number.  It depends how long it takes to run the method once, and how long you want to spend testing it.  If 10000 calls only takes a few minutes you could do 100000 calls to be more accurate.  If 10000 calls takes 10 hours, do 100000 calls if you have a week to test...

[toc] | [prev] | [next] | [standalone]

#32215

On 7/30/2015 1:30 PM, e.d.programmer@gmail.com wrote:
> On Thursday, July 30, 2015 at 11:42:29 AM UTC-4, Eric Sosman wrote:
>>       One of the articles I linked to suggests doing "several"
>> times that many loop executions ...
>>
>>       ... in the *warmup* phase, before serious measurements.
>
> Right.  It helps if you call the method before getting the initial timestamp to exclude the one time (classloader, public class variable creation) stuff.

     The point (and a repeated warning in the links provided) is that
"call the method" is not enough of a warmup.  You must

     - Call the method enough times to be sure that the JIT has finished
       converting it to native code,

     - Call *exactly* the same methods during the timing phase as you
       did during the warmup, with exactly the same classes loaded and
       so on, lest the JIT change its mind,

     - Check that nothing unexpected happened during the timing phase.

> The 10000 was a random number.  It depends how long it takes to run the method once, and how long you want to spend testing it.  If 10000 calls only takes a few minutes you could do 100000 calls to be more accurate.  If 10000 calls takes 10 hours, do 100000 calls if you have a week to test...

     If 10000 calls takes a few minutes, it seems to me that we're no
longer in the realm of "micro benachmarking [sic]."  Microbenchmarks
are rather different from tests of system capacity, throughput, or
latency.  All are difficult, all present challenges of their very own.

-- 
esosman@comcast-dot-net.invalid
"Don't be afraid of work. Make work afraid of you." -- TLM

[toc] | [prev] | [next] | [standalone]

#32216

On 07/30/2015 09:30 AM, Eric Sosman wrote:
<snip>
>      Here are a few informative (and cautionary, and perhaps even
> discouraging!) articles on Java microbenchmarking:
>

How NOT To Write A Microbenchmark
(or Lies, Damn Lies,and Microbenchmarks)
> http://www.azulsystems.com/events/javaone_2002/microbenchmarks.pdf

MicroBenchmarks
(So You Want to Write a Micro-Benchmark)
> https://wiki.openjdk.java.net/display/HotSpot/MicroBenchmarks

Java theory and practice:
(Anatomy of a flawed microbenchmark)
> http://www.ibm.com/developerworks/java/library/j-jtp02225/index.html
>
> (The first is a collection of presentation slides, and may be a bit
> telegraphic since there's no presenter to talk about them.  The last
> is by Brian Goetz, and used to be hosted on an Oracle Wiki until
> Oracle in their wisdom shut the whole thing down, but we're fortunate
> that IBM cached a copy.)
>

Another of presentation slides:
Java Microbenchmark Harness
(the lesser of two evils)
<http://shipilev.net/talks/devoxx-Nov2013-benchmarking.pdf>

Interesting how so many articles on the subject have subtitles!
Thanks for the post - piqued my interest.

[toc] | [prev] | [next] | [standalone]

#32210

Am Donnerstag, 30. Juli 2015 14:23:58 UTC+2 schrieb Eric Sosman:
> On 7/30/2015 6:41 AM, philipp.kraus.flashpixx@gmail.com wrote:
> > Hello,
> >
> > I have got a complex system and I would like to add some microbenchmarks to the application.
> > So I need the runtime of a method call, but I don't want to add to each method a time counter. Can I add via reflection API a structure, that adds the micro benchmarks automatically? E.g.
> >
> > registerbenchmark( myclass.class, mymethod-handle ).
> >
> > This calls should do on any object
> >
> > myclass x = new myclass();
> >
> > timer start
> > x.mymethod();
> > timer stop
> >
> > So I would like to do register all classes & methods on the application startup and on any instance of a class the benchmark code should be run. So I need an event, which is called on each method call.
> >
> > Can I register an event-handler at the JVM to get each method call? Does there exists some frameworks for microbenchmarking?
> 
>      (Disclaimer: I've never done this ...)
> 
>      I think that java.lang.instrument package can be used for this
> sort of thing.  It lets you change a method's byte code on the fly,
> so you can insert calls to start and stop timers or whatever.

great that helps

> 
>      BUT: Unless a method has "macroscopic" run time, you'll have a
> hard time finding a sufficiently fine-grained timer.  Nothing Java
> itself supplies will suffice (System.nanoTime() may look promising,
> but even though it gives a result in nanoseconds it seldom measures
> that precisely).  Using JNI, you might be able to access chip-specific
> low-level cycle counters or some such -- but even then, you'd have to
> take it on faith that all the cycles between a method's start and end
> were consumed by the method; what if the garbage collector decided to
> preempt the method's thread for a few milliseconds?

For this case I would like to use PAPI http://icl.cs.utk.edu/papi/overview/

[toc] | [prev] | [next] | [standalone]

#32218

Eric Sosman wrote:

> Using JNI, you might be able to access chip-specific
> low-level cycle counters or some such -- but even then, you'd have to
> take it on faith that all the cycles between a method's start and end
> were consumed by the method; what if the garbage collector decided to
> preempt the method's thread for a few milliseconds?

And what about the cost of going native? Or has that become a myth?

-- 

"I'm a doctor, not a mechanic." Dr Leonard McCoy <mccoy@ncc1701.starfleet.fed>
"I'm a mechanic, not a doctor." Volker Borchert  <v_borchert@despammed.com>

[toc] | [prev] | [next] | [standalone]

#32220

On 7/30/2015 5:33 PM, Volker Borchert wrote:
> Eric Sosman wrote:
>
>> Using JNI, you might be able to access chip-specific
>> low-level cycle counters or some such -- but even then, you'd have to
>> take it on faith that all the cycles between a method's start and end
>> were consumed by the method; what if the garbage collector decided to
>> preempt the method's thread for a few milliseconds?
>
> And what about the cost of going native? Or has that become a myth?

     I don't know what that cost is, nowadays.  However, if used
once at the start and once at the end of each instrumented method
it should add roughly the same overhead to each, inflating each
method's time by about the same amount.  I think it likely that
the other sources of interference already mentioned (JIT and re-JIT,
GC, preemption and migration of threads, ...) present greater hazards
to meaningful measurement.

     Besides: How do you imagine Java's own methods get the time?
 From the source supplied with the JDK:

	public final class System {
	    ...
	    public static native long currentTimeMillis();
	    ...
	    public static native long nanoTime();
	    ...
	}

Of course, the JVM might special-case "its own" native methods,
but still: A trip into the O/S seems difficult to avoid.

-- 
esosman@comcast-dot-net.invalid
"Don't be afraid of work. Make work afraid of you." -- TLM

[toc] | [prev] | [next] | [standalone]

#32221

On 7/30/2015 6:18 PM, Eric Sosman wrote:
> On 7/30/2015 5:33 PM, Volker Borchert wrote:
>> Eric Sosman wrote:
>>
>>> Using JNI, you might be able to access chip-specific
>>> low-level cycle counters or some such -- but even then, you'd have to
>>> take it on faith that all the cycles between a method's start and end
>>> were consumed by the method; what if the garbage collector decided to
>>> preempt the method's thread for a few milliseconds?
>>
>> And what about the cost of going native? Or has that become a myth?
>
>      I don't know what that cost is, nowadays.  However, if used
> once at the start and once at the end of each instrumented method
> it should add roughly the same overhead to each, inflating each
> method's time by about the same amount.  I think it likely that
> the other sources of interference already mentioned (JIT and re-JIT,
> GC, preemption and migration of threads, ...) present greater hazards
> to meaningful measurement.
>
>      Besides: How do you imagine Java's own methods get the time?
>  From the source supplied with the JDK:
>
>      public final class System {
>          ...
>          public static native long currentTimeMillis();
>          ...
>          public static native long nanoTime();
>          ...
>      }
>
> Of course, the JVM might special-case "its own" native methods,
> but still: A trip into the O/S seems difficult to avoid.

I think the JNI overhead depends a lot on the API complexity.

Those methods are very easy.

The C JNI code to access complex object graphs and
convert to/from Java String can become large.

It is sort of reflection style.

Arne

[toc] | [prev] | [next] | [standalone]

#32217

On 07/30/2015 06:41 AM, philipp.kraus.flashpixx@gmail.com wrote:
<snip>
>Does there exists some frameworks for microbenchmarking?
<snip>

Seem to be more than one.
Have you tried a web search?

Here is one.

JMH Resources
<http://psy-lob-saw.blogspot.com/p/jmh-related-posts.html>

[toc] | [prev] | [next] | [standalone]

#32219

philipp.kraus.flashpixx@gmail.com wrote:

> Hope for any comment, thanks a lot

What about using a (possibly commercial) profiler?
You might even find one that allows to instrument selectively.
Or you could hack an open source one to your liking.

-- 

"I'm a doctor, not a mechanic." Dr Leonard McCoy <mccoy@ncc1701.starfleet.fed>
"I'm a mechanic, not a doctor." Volker Borchert  <v_borchert@despammed.com>

[toc] | [prev] | [next] | [standalone]

#32259

Am 30.07.2015 12:41, schrieb philipp.kraus.flashpixx@gmail.com:
> Hello,
>
> I have got a complex system and I would like to add some microbenchmarks to the application.
> So I need the runtime of a method call, but I don't want to add to each method a time counter. Can I add via reflection API a structure, that adds the micro benchmarks automatically? E.g.
>
> registerbenchmark( myclass.class, mymethod-handle ).
>
> This calls should do on any object
>
> myclass x = new myclass();
>
> timer start
> x.mymethod();
> timer stop
>
> So I would like to do register all classes & methods on the application startup and on any instance of a class the benchmark code should be run. So I need an event, which is called on each method call.
>
> Can I register an event-handler at the JVM to get each method call? Does there exists some frameworks for microbenchmarking?
>
> Hope for any comment, thanks a lot
>
> Phil
>
Nowadays, I shoould say, the accepted standard for microbenchmarking on 
the JVM is JMH. It's part of OpenJDK:

http://openjdk.java.net/projects/code-tools/jmh/

And it will become part of the standard, see JEP 230:

http://openjdk.java.net/jeps/230

It's also quite easy to use.

-- Sebastian

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.java.programmer

csiph-web