Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #28632

Re: NIO not so hot

Show all headers | View raw

On Sunday, June 1, 2014 11:15:01 AM UTC+1, Robert Klemme wrote:
> On 31.05.2014 11:29, Rupert Smith wrote:
> 
> 
> 
> > Try using a direct byte buffer which has been pre-allocated. Direct
> 
> > buffers are allocated outside the Java heap (using malloc()?), so the
> 
> > allocation cost is high. They only really provide a performance boost
> 
> > when re-used.
> 
> 
> 
> That of course depends on the usage scenario, e.g. the frequency of 
> 
> allocation etc.  If you serve long lasting connections the cost of 
> 
> allocating and freeing a DirectByteBuffer is negligible and other 
> 
> reasons may gain more weight in the decision to use a direct or heap 
> 
> buffer (e.g. whether access to the byte[] can make things faster as I 
> 
> assume is the case in my decoding tests posted upthread).

The allocation cost is unfortunately not negligable. Allocation cost within the heap is very low, because it is easy to do. Outside the heap with a malloc() type algorithm can be considerably slower, because free blocks may need to be searched for.

We could try:

ByteBuffer.allocate() in a loop and see.

 If you are servicing a long running connection, create a direct buffer big enough to handle it, and re-use it on subsequent reads. In the case of the FIX engine I wrote, this model worked well, because FIX is ASCII (that is a price would be "1.234" as ASCII characters), and needs to be decoded into binary. So I would read some ASCII into the buffer, then decode into a binary form, then get some more bytes into the buffer once the original ones were consumed, and so on.

> 
> 
> > Also, if you dig into the internals you will find that a heap buffer
> 
> > reading a file or socket will copy bytes from a direct buffer anyway,
> 
> > and that Java does its own internal pooling/re-allocation of direct
> 
> > buffers.
> 
> 
> 
> Can you point me to more information about this?  Or are you referring 
> 
> to OpenJDK's source code?

Yes, I looked in the OpenJDK source code. You don't have to dig too far under socket.read() or socket.write() to find it.

> 
> 
> 
> > Often benchmarks will say heap buffers are faster, because they
> 
> > allocate buffer then read some data then allow buffer to be garbage
> 
> > collected.
> 
> 
> 
> I think heap byte buffers were faster in my tests (see upthread) not 
> 
> because of allocation and GC (this was not included in the time 
> 
> measurement) but rather because data would cross the boundary between 
> 
> non Java heap memory (where they arrive from the OS) to Java heap more 
> 
> infrequently because of the larger batches.  If you have to fetch 
> 
> individual bytes from a ByteBuffer off Java heap you have to make the 
> 
> transition much more frequent.

As I say, this does seem to have been optimized, although I admit I am a little unsure as to exactly how. It was certainly the case in 1.4 and maybe 1.5 that heap buffer array [] access was faster, and get()/set() was slow. I have seen benchmarks and run my own micro-benachmarks which suggest that get()/set() is now every bit as fast as the array access.

> 
> 
> 
> > In the heap buffer case, the internal direct buffer pool
> 
> > is being used. In the direct buffer case, a new one is being
> 
> > allocated each time, which is slow.
> 
> 
> 
> Can you point me to writing about that internal byte buffer pool in the 
> 
> JRE?  I could not find anything.
> 
> 
> 
> > I may be wrong but... are the byte get()/set() calls not trapped by
> 
> > some compiler intrinsics and optimized away?
> 
> 
> 
> DirectByteBuffer.get() contains a native call to fetch the byte - and I 
> 
> don't think the JIT will optimize away native calls.  The JRE just does 
> 
> not have any insights into what JNI calls do.

Exactly what I though, yet it does seem to be optimized.

> 
> 
> 
> > Allocation costs
> 
> > and memory copying need to be avoided as much as possible.
> 
> 
> 
> While I agree with that general tendency of the statement ("allocation 
> 
> costs") I believe nowadays one needs to be very careful with these 
> 
> statements.  For example, if you share immutable data structures across 
> 
> threads which require copying during manipulation that allocation and GC 
> 
> cost may very well be smaller than the cost of locking in a more 
> 
> traditional approach.

Indeed. In some situations we used mutable data structures accross threads, which of course is dangerous if the programmer does not know how to handle it, and difficult to get right even if they do.

Rupert

Back to comp.lang.java.programmer | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

NIO not so hot Roedy Green <see_website@mindprod.com.invalid> - 2014-05-17 04:28 -0700
  Re: NIO not so hot Robert Klemme <shortcutter@googlemail.com> - 2014-05-18 13:38 +0200
    Re: NIO not so hot Robert Klemme <shortcutter@googlemail.com> - 2014-05-18 13:38 +0200
      Re: NIO not so hot Robert Klemme <shortcutter@googlemail.com> - 2014-05-18 14:12 +0200
        Re: NIO not so hot Robert Klemme <shortcutter@googlemail.com> - 2014-05-18 19:09 +0200
          Re: NIO not so hot Roedy Green <see_website@mindprod.com.invalid> - 2014-05-26 00:51 -0700
            Re: NIO not so hot Robert Klemme <shortcutter@googlemail.com> - 2014-05-26 20:14 +0200
    Re: NIO not so hot Roedy Green <see_website@mindprod.com.invalid> - 2014-05-26 01:15 -0700
    Re: NIO not so hot Roedy Green <see_website@mindprod.com.invalid> - 2014-05-26 01:24 -0700
      Re: NIO not so hot Robert Klemme <shortcutter@googlemail.com> - 2014-05-26 23:50 +0200
  Re: NIO not so hot Rupert Smith <rupertlssmith@googlemail.com> - 2014-05-31 02:29 -0700
    Re: NIO not so hot Robert Klemme <shortcutter@googlemail.com> - 2014-06-01 12:15 +0200
      Re: NIO not so hot Rupert Smith <rupertlssmith@googlemail.com> - 2014-06-02 07:37 -0700
        Re: NIO not so hot Robert Klemme <shortcutter@googlemail.com> - 2014-06-03 23:07 +0200
          Re: NIO not so hot Rupert Smith <rupertlssmith@googlemail.com> - 2014-06-06 01:32 -0700
      Re: NIO not so hot Rupert Smith <rupertlssmith@googlemail.com> - 2014-06-02 07:46 -0700

csiph-web