Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #32966 > unrolled thread

Performance JPA

Back to article view | Back to comp.lang.java.programmer

Contents

  Performance JPA Cecil Westerhof <Cecil@decebal.nl> - 2015-11-02 09:47 +0100
    Re: Performance JPA Silvio <silvio@internet.com> - 2015-11-02 14:23 +0100
      Re: Performance JPA Cecil Westerhof <Cecil@decebal.nl> - 2015-11-03 07:52 +0100
        Re: Performance JPA Martin Gregorie <martin@address-in-sig.invalid> - 2015-11-03 11:16 +0000
          Re: Performance JPA Cecil Westerhof <Cecil@decebal.nl> - 2015-11-03 19:11 +0100
            Re: Performance JPA Martin Gregorie <martin@address-in-sig.invalid> - 2015-11-03 19:15 +0000
              Re: Performance JPA Cecil Westerhof <Cecil@decebal.nl> - 2015-11-04 09:28 +0100
                Re: Performance JPA Cecil Westerhof <Cecil@decebal.nl> - 2015-11-10 08:07 +0100
                  Re: Performance JPA Silvio <silvio@internet.com> - 2015-11-10 10:35 +0100
                    Re: Performance JPA Cecil Westerhof <Cecil@decebal.nl> - 2015-11-11 06:20 +0100
                      Re: Performance JPA Arne Vajhøj <arne@vajhoej.dk> - 2015-11-11 09:32 -0500
                  Re: Performance JPA Arne Vajhøj <arne@vajhoej.dk> - 2015-11-10 19:56 -0500
                    Re: Performance JPA Cecil Westerhof <Cecil@decebal.nl> - 2015-11-11 06:50 +0100
                      Re: Performance JPA Martin Gregorie <martin@address-in-sig.invalid> - 2015-11-11 12:17 +0000
                      Re: Performance JPA Arne Vajhøj <arne@vajhoej.dk> - 2015-11-11 09:30 -0500
                        Re: Performance JPA Cecil Westerhof <Cecil@decebal.nl> - 2015-11-11 23:20 +0100
                    Re: Performance JPA Cecil Westerhof <Cecil@decebal.nl> - 2015-11-11 23:11 +0100
                      Re: Performance JPA Arne Vajhøj <arne@vajhoej.dk> - 2015-11-11 19:10 -0500
                        Re: Performance JPA Cecil Westerhof <Cecil@decebal.nl> - 2015-11-12 07:27 +0100
                          Re: Performance JPA Cecil Westerhof <Cecil@decebal.nl> - 2015-11-13 08:30 +0100
                            Re: Performance JPA Silvio <silvio@internet.com> - 2015-11-13 10:45 +0100
                              Re: Performance JPA Cecil Westerhof <Cecil@decebal.nl> - 2015-11-13 11:06 +0100
                                Re: Performance JPA Silvio <silvio@internet.com> - 2015-11-13 13:36 +0100
                                  Re: Performance JPA Cecil Westerhof <Cecil@decebal.nl> - 2015-11-20 09:04 +0100
                            Re: Performance JPA Arne Vajhøj <arne@vajhoej.dk> - 2015-11-13 21:46 -0500
                              Re: Performance JPA Cecil Westerhof <Cecil@decebal.nl> - 2015-11-20 09:09 +0100
            Re: Performance JPA Arne Vajhøj <arne@vajhoej.dk> - 2015-11-03 20:35 -0500
              Re: Performance JPA Cecil Westerhof <Cecil@decebal.nl> - 2015-11-04 09:35 +0100
                Re: Performance JPA Arne Vajhøj <arne@vajhoej.dk> - 2015-11-07 22:09 -0500
        Re: Performance JPA Silvio <silvio@internet.com> - 2015-11-03 12:29 +0100
          Re: Performance JPA Cecil Westerhof <Cecil@decebal.nl> - 2015-11-03 19:23 +0100
          Re: Performance JPA Arne Vajhøj <arne@vajhoej.dk> - 2015-11-03 20:44 -0500
            Re: Performance JPA Cecil Westerhof <Cecil@decebal.nl> - 2015-11-04 09:48 +0100
            Re: Performance JPA Silvio <silvio@internet.com> - 2015-11-04 10:23 +0100
              Re: Performance JPA Cecil Westerhof <Cecil@decebal.nl> - 2015-11-04 11:27 +0100
                Re: Performance JPA Silvio <silvio@internet.com> - 2015-11-04 12:42 +0100
                  Re: Performance JPA Arne Vajhøj <arne@vajhoej.dk> - 2015-11-07 22:18 -0500
                    Re: Performance JPA Silvio <silvio@internet.com> - 2015-11-09 10:43 +0100
                      Re: Performance JPA Cecil Westerhof <Cecil@decebal.nl> - 2015-11-20 09:13 +0100
              Re: Performance JPA Arne Vajhøj <arne@vajhoej.dk> - 2015-11-07 22:12 -0500
                Re: Performance JPA Silvio <silvio@internet.com> - 2015-11-09 10:22 +0100
                  Re: Performance JPA Arne Vajhøj <arne@vajhoej.dk> - 2015-11-10 21:35 -0500
                    Re: Performance JPA Silvio <silvio@internet.com> - 2015-11-11 11:29 +0100
        Re: Performance JPA Arne Vajhøj <arne@vajhoej.dk> - 2015-11-03 20:42 -0500
          Re: Performance JPA Cecil Westerhof <Cecil@decebal.nl> - 2015-11-04 11:19 +0100
            Re: Performance JPA Arne Vajhøj <arne@vajhoej.dk> - 2015-11-07 22:05 -0500
    Re: Performance JPA Arne Vajhøj <arne@vajhoej.dk> - 2015-11-02 20:08 -0500
      Re: Performance JPA Cecil Westerhof <Cecil@decebal.nl> - 2015-11-03 07:55 +0100
        Re: Performance JPA Arne Vajhøj <arne@vajhoej.dk> - 2015-11-03 20:45 -0500
    Re: Performance JPA Lew <lewbloch@gmail.com> - 2015-11-16 14:43 -0800

Page 3 of 3 — ← Prev page 1 2 [3]

#33036

On 11/08/2015 04:12 AM, Arne Vajhøj wrote:
> On 11/4/2015 4:23 AM, Silvio wrote:
>> On 11/04/2015 02:44 AM, Arne Vajhøj wrote:
>>> Really??
>>>
>>> I have never seen such a problem.
>>
>> I have seen tons of them. Code that used to work with a logical filter
>> that was easily represented as a WHERE-clause for SELECT, UPDATE or
>> DELETE statements was converted to something that was working with some
>> Set of entities. Every action on the complete set was then translated to
>> N calls on the individual entities. Sweet.
>
> Can you provide an example?
>
>>> em.createQuery is easy and generate an equivalent SQL to the one
>>> to chose if manually written.
>>>
>>
>> Sure, that is why I said it could be fixed by changing the code. But
>> IMHO it is a bit of a wart on the whole idea of having entities
>> represented as individual instances (which I generally consider a stupid
>> idea in the first place).
>
> ????
>
> Having a method that returns a collection of objects is basic
> feature in almost all data access code.
>
> Arne
>
>

No, it is not. Only if you are used to doing things to ORM (or ODBMS, if 
that still exists) way. Having an object that represents a set of 
database entities is not the same as having a set of objects that 
represent individual database entities.

Database actions like SELECT, UPDATE or DELETE are by definition 
operations on sets, not on individual entities. The former makes that 
trivial to represent, that latter makes it very hard. Once you have a 
collection of objects representing individual entities you have to write 
explicit code to do set operations.

But you have demonstrated many times to have a one track mind when it 
concerns ORMs so I am sure you will disagree.

[toc] | [prev] | [next] | [standalone]

#33089

On 11/9/2015 4:22 AM, Silvio wrote:
> On 11/08/2015 04:12 AM, Arne Vajhøj wrote:
>> On 11/4/2015 4:23 AM, Silvio wrote:
>>> On 11/04/2015 02:44 AM, Arne Vajhøj wrote:
>>>> em.createQuery is easy and generate an equivalent SQL to the one
>>>> to chose if manually written.
>>>
>>> Sure, that is why I said it could be fixed by changing the code. But
>>> IMHO it is a bit of a wart on the whole idea of having entities
>>> represented as individual instances (which I generally consider a stupid
>>> idea in the first place).
>>
>> ????
>>
>> Having a method that returns a collection of objects is basic
>> feature in almost all data access code.

> No, it is not. Only if you are used to doing things to ORM (or ODBMS, if
> that still exists) way.

Not true.

It is also common for non-ORM based data access code.

A few random examples:

* Spring JDBC - its JdbcTemplate.quert returns a list of object
* EAP/Fowler - Row Data Gateway and Data Mapper
* Core J2EE patterns - DAO
* Balus C's DAO demo at 
http://balusc.omnifaces.org/2008/07/dao-tutorial-data-layer.html

BTW, it should not really be necessary to argue this. Exposed API's
should be independent of implementation. That is very basic OO. And
one consequence of that is that the data access API should not depend on
whether the implementation use ORM or not.

>                         Having an object that represents a set of
> database entities is not the same as having a set of objects that
> represent individual database entities.
>
> Database actions like SELECT, UPDATE or DELETE are by definition
> operations on sets, not on individual entities. The former makes that
> trivial to represent, that latter makes it very hard. Once you have a
> collection of objects representing individual entities you have to write
> explicit code to do set operations.

Not true.

A single em.createQuery(sql).getResultList() return data for SELECT and
a single em.createQuery(sql).executeUpdate(0 does the updates for the rest.

> But you have demonstrated many times to have a one track mind when it
> concerns ORMs so I am sure you will disagree.

I am not really arguing about what I can agree with or disagree
with.

I am arguing about what is facts.

Arne

[toc] | [prev] | [next] | [standalone]

#33103

On 11/11/2015 03:35 AM, Arne Vajhøj wrote:
>
> Not true.
>
> It is also common for non-ORM based data access code.
>
> A few random examples:
>
> * Spring JDBC - its JdbcTemplate.quert returns a list of object
> * EAP/Fowler - Row Data Gateway and Data Mapper
> * Core J2EE patterns - DAO
> * Balus C's DAO demo at
> http://balusc.omnifaces.org/2008/07/dao-tutorial-data-layer.html
>
> BTW, it should not really be necessary to argue this. Exposed API's
> should be independent of implementation. That is very basic OO. And
> one consequence of that is that the data access API should not depend on
> whether the implementation use ORM or not.
>

OO is the cuplrit here. OO and relational databases only map cleanly if 
the data objects are in some way an abstraction of relational algebra 
(which basically means the objects represent relational expressions => 
sets). Only during iteration of such a set would (temporary) objects 
representing individual rows make sense.

But in many data access layers the data objects represent individual 
entities with possibly long lifetimes and holding some form of primary 
key. Such objects map poorly to set operations which results in 
inefficient database access for anything but the most basic scenarios 
(unless you execute explicit queries, see below).

In spite of what many programmers think OO is not about the measures you 
take to hide implementation details behind a bunch of classes. It is 
about designing the proper abstraction.

ORM is a terrible abstraction for an RDBMS and efficient access to it. 
It is merely an object persistence layer that happens to abuse an RDBMS 
for its storage needs. Nothing wrong with that if you are aware of this 
and understand its implications.

But when you (have to) do it the other way around and must retrofit an 
ORM onto an existing relational database and application(s) the result 
is often disappointing. That is where Cecil is faced with now.

>
> Not true.
>
> A single em.createQuery(sql).getResultList() return data for SELECT and
> a single em.createQuery(sql).executeUpdate(0 does the updates for the rest.
>

That IS explicit code. Executing explicit queries (SQL) is an obvious 
wart on the concept of an object persistence layer, albeit a very much 
needed wart because of the reasons stated above.

In the end things can always be fixed into something that is 
not-too-much-slower-than-necessary by inserting enough executeQuery 
calls because that takes the abstraction level to just above plain JDBC. 
It kind of defeats the purpose of the ORM but what must be done must be 
done.

>
> I am not really arguing about what I can agree with or disagree
> with.
>

Yes, you are and I am sure you know it. You just choose to defend ORMs 
blindly since that is a stance you have picked.

> I am arguing about what is facts.
>
> Arne
>
>

[toc] | [prev] | [next] | [standalone]

#32990

On 11/3/2015 1:52 AM, Cecil Westerhof wrote:
> On Monday  2 Nov 2015 14:23 CET, silvio@internet.com wrote:
>> But let's put that aside and see if we can fix your situation.
>> Please tell us what the typical database access from your
>> application looks like and which things are particularly slow in
>> comparison to how it was?
>
> At the moment I am told that everything is slower. I am told that a
> query that took at most 5 seconds now takes one to two minutes. But I
> asked them to measure the old situation and until now they did not do
> this. I hope to get today real data about this. But several pages now
> give a timeout.

Has anything else changed besides plain JDBC -> JPA?

If yes, then it is even harder.

> As I understood it until now, there should be a little overhead, but
> it can be worse I understand now. As said in the above rowmapper was
> used. Now the application uses em.find to read. For displaying a page
> only reads are done, so using em.find instead rs.get... seems to make
> a very big difference. Is there a reason for this and can there be
> something done about this?

If you are retrieving many objects, then you should not be using
em.find but em.createQuery.

Arne

[toc] | [prev] | [next] | [standalone]

#32999

On Wednesday  4 Nov 2015 02:42 CET, Arne Vajhøj wrote:

> On 11/3/2015 1:52 AM, Cecil Westerhof wrote:
>> On Monday  2 Nov 2015 14:23 CET, silvio@internet.com wrote:
>>> But let's put that aside and see if we can fix your situation.
>>> Please tell us what the typical database access from your
>>> application looks like and which things are particularly slow in
>>> comparison to how it was?
>>
>> At the moment I am told that everything is slower. I am told that a
>> query that took at most 5 seconds now takes one to two minutes. But
>> I asked them to measure the old situation and until now they did
>> not do this. I hope to get today real data about this. But several
>> pages now give a timeout.
>
> Has anything else changed besides plain JDBC -> JPA?
>
> If yes, then it is even harder.

Yes, there has been a complete overhaul of the application. But at
first it was said that the first fetch is a lot slower as when it is
fetched again. That was why we thought the culprit was JPA. That is
only in a certain case, so it could be a lot of things now I am
afraid.


>> As I understood it until now, there should be a little overhead,
>> but it can be worse I understand now. As said in the above
>> rowmapper was used. Now the application uses em.find to read. For
>> displaying a page only reads are done, so using em.find instead
>> rs.get... seems to make a very big difference. Is there a reason
>> for this and can there be something done about this?
>
> If you are retrieving many objects, then you should not be using
> em.find but em.createQuery.

I think we are doing that.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

[toc] | [prev] | [next] | [standalone]

#33018

On 11/4/2015 5:19 AM, Cecil Westerhof wrote:
> On Wednesday  4 Nov 2015 02:42 CET, Arne Vajhøj wrote:
>> On 11/3/2015 1:52 AM, Cecil Westerhof wrote:
>>> As I understood it until now, there should be a little overhead,
>>> but it can be worse I understand now. As said in the above
>>> rowmapper was used. Now the application uses em.find to read. For
>>> displaying a page only reads are done, so using em.find instead
>>> rs.get... seems to make a very big difference. Is there a reason
>>> for this and can there be something done about this?
>>
>> If you are retrieving many objects, then you should not be using
>> em.find but em.createQuery.
>
> I think we are doing that.

Then em.find should not be the problem.

Arne

[toc] | [prev] | [next] | [standalone]

#32978

On 11/2/2015 3:47 AM, Cecil Westerhof wrote:
> We needed to change from JDBC to JPA. It looks like the performance is
> 10 to 20 times as slow as it was with JDBC. I could understand JPA to
> be slower as JDBC, but not with this kind of difference. Are there any
> common error that would explain this kind of penalty?

With a factor 20 then JPA is doing something completely different from
what the JDBC is doing.

First thing you should do is to tell your JPA implementation
to log the SQL it executes, then you should be able to see
what the difference is.

Next step will be to fix it. But that will require details
on the difference, the JPA implementation, the model etc..

Arne

[toc] | [prev] | [next] | [standalone]

#32979

On Tuesday  3 Nov 2015 02:08 CET, Arne Vajhøj wrote:

> On 11/2/2015 3:47 AM, Cecil Westerhof wrote:
>> We needed to change from JDBC to JPA. It looks like the performance
>> is 10 to 20 times as slow as it was with JDBC. I could understand
>> JPA to be slower as JDBC, but not with this kind of difference. Are
>> there any common error that would explain this kind of penalty?
>
> With a factor 20 then JPA is doing something completely different
> from what the JDBC is doing.

This is the difference that I am told. I still wait on a confirmation.


> First thing you should do is to tell your JPA implementation
> to log the SQL it executes, then you should be able to see
> what the difference is.

We are going to do that.


> Next step will be to fix it. But that will require details
> on the difference, the JPA implementation, the model etc..

In my other response I explained the differences. When more
information is needed: let me know.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

[toc] | [prev] | [next] | [standalone]

#32992

On 11/3/2015 1:55 AM, Cecil Westerhof wrote:
> On Tuesday  3 Nov 2015 02:08 CET, Arne Vajhøj wrote:
>> First thing you should do is to tell your JPA implementation
>> to log the SQL it executes, then you should be able to see
>> what the difference is.
>
> We are going to do that.
>
>> Next step will be to fix it. But that will require details
>> on the difference, the JPA implementation, the model etc..
>
> In my other response I explained the differences. When more
> information is needed: let me know.

It did give some hints. And if it is just em.createQuery vs
em.find then problem solved.

But if it is a "real" JPA problem, then we will need a lot
more specific information.

Arne

[toc] | [prev] | [next] | [standalone]

#33137

On Monday, November 2, 2015 at 12:58:47 AM UTC-8, Cecil Westerhof wrote:
> We needed to change from JDBC to JPA. It looks like the performance is
> 10 to 20 times as slow as it was with JDBC. I could understand JPA to
> be slower as JDBC, but not with this kind of difference. Are there any
> common error that would explain this kind of penalty?

Yes, treating JPA as a wrapper for SQL and keeping 'DataManager' instances around for too long are the two biggest reasons that JPA performance would suck. I've seen JPA in high-performance applications, even with these mistakes, do very well on speed. But those are the two biggies.

-- 
Lew

[toc] | [prev] | [standalone]

Page 3 of 3 — ← Prev page 1 2 [3]

Back to top | Article view | comp.lang.java.programmer

csiph-web