Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.databases > #90 > unrolled thread

jTDS, trashes heap with w

Back to article view | Back to comp.lang.java.databases

Contents

  jTDS, trashes heap with w "Jan Burse" <jan.burse@THRWHITE.remove-dii-this> - 2011-04-27 15:21 +0000
    Re: jTDS, trashes heap wi "=?ISO-8859-1?Q?Arne_Vajh=" <=?iso-8859-1?q?arne_vajh=@THRWHITE.remove-dii-this> - 2011-04-27 15:21 +0000
      Re: jTDS, trashes heap wi "Jan Burse" <jan.burse@THRWHITE.remove-dii-this> - 2011-04-27 15:21 +0000
        Re: jTDS, trashes heap wi "=?ISO-8859-1?Q?Arne_Vajh=" <=?iso-8859-1?q?arne_vajh=@THRWHITE.remove-dii-this> - 2011-04-27 15:21 +0000
    Re: jTDS, trashes heap wi "Lew" <lew@THRWHITE.remove-dii-this> - 2011-04-27 15:21 +0000
      Re: jTDS, trashes heap wi "Jan Burse" <jan.burse@THRWHITE.remove-dii-this> - 2011-04-27 15:21 +0000
        Re: jTDS, trashes heap wi "Lew" <lew@THRWHITE.remove-dii-this> - 2011-04-27 15:21 +0000
          Re: jTDS, trashes heap wi "Jan Burse" <jan.burse@THRWHITE.remove-dii-this> - 2011-04-27 15:21 +0000
            Re: jTDS, trashes heap wi "Lew" <lew@THRWHITE.remove-dii-this> - 2011-04-27 15:21 +0000
              Re: jTDS, trashes heap wi "Jan Burse" <jan.burse@THRWHITE.remove-dii-this> - 2011-04-27 15:21 +0000
                Re: jTDS, trashes heap wi "Lew" <lew@THRWHITE.remove-dii-this> - 2011-04-27 15:21 +0000

#90 — jTDS, trashes heap with w

  To: comp.lang.java.databases
Dear All

I was just looking up the source code of the jTDS driver.
It seems that it is trashing the heap with java.lang
wrapper classes, even when one is using only primitive
types in his application.

For example, when I am doing, whereas the column
is integer:

    ResultSet set;
    set.getLong(1);

This amounts to a call to:

     public long getLong(int columnIndex) throws SQLException {
          return ((Long) Support.convert(con, getColumn(columnIndex), 

                   java.sql.Types.BIGINT, null)).longValue();
     }

During the above call a Long object will be created, as
can be seen from the convert implementation:

     case java.sql.Types.BIGINT:
       if (x == null) {
         return LONG_ZERO;
       } else if (x instanceof Long) {
         return x;
       } else if (x instanceof Byte) {
         return new Long(((Byte)x).byteValue() & 0xFF);
       } else if (x instanceof Number) {
         return new Long(((Number) x).longValue());
       } else if (x instanceof String) {
         return new Long(((String) x).trim());
       } else if (x instanceof Boolean) {
         return ((Boolean) x).booleanValue() ? LONG_ONE : LONG_ZERO;
       }
       break;

A solution would be to do the following
call on the result set:

    (long)set.getInt(1);

But what if I want to make the code, lets say column
size independent, and want to use a getLong() call
only.

Does this trashing happen in every JDBC driver?
Are there alternatives to jTDS, when accessing
MS SQL Databases?

Best Regards

---
 * Synchronet * The Whitehouse BBS --- whitehouse.hulds.com --- check it out free usenet!
--- Synchronet 3.15a-Win32 NewsLink 1.92
Time Warp of the Future BBS - telnet://time.synchro.net:24

[toc] | [next] | [standalone]

#91 — Re: jTDS, trashes heap wi

  To: comp.lang.java.databases
Jan Burse wrote:
> Are there alternatives to jTDS, when accessing
> MS SQL Databases?

The obvious one is to use the JDBC driver provided by
Microsoft.

Arne

---
 * Synchronet * The Whitehouse BBS --- whitehouse.hulds.com --- check it out free usenet!
--- Synchronet 3.15a-Win32 NewsLink 1.92
Time Warp of the Future BBS - telnet://time.synchro.net:24

[toc] | [prev] | [next] | [standalone]

#95 — Re: jTDS, trashes heap wi

  To: comp.lang.java.databases
Arne Vajhoj schrieb:
> Jan Burse wrote:
>> Are there alternatives to jTDS, when accessing
>> MS SQL Databases?
> 
> The obvious one is to use the JDBC driver provided by
> Microsoft.

There were some problems with the Microsoft driver
when I looked into it 2006. Some problems with opening
multiple cursors over the same database connection
session...

Are these solved now?

---
 * Synchronet * The Whitehouse BBS --- whitehouse.hulds.com --- check it out free usenet!
--- Synchronet 3.15a-Win32 NewsLink 1.92
Time Warp of the Future BBS - telnet://time.synchro.net:24

[toc] | [prev] | [next] | [standalone]

#97 — Re: jTDS, trashes heap wi

  To: comp.lang.java.databases
Jan Burse wrote:
> Arne Vajhoj schrieb:
>> Jan Burse wrote:
>>> Are there alternatives to jTDS, when accessing
>>> MS SQL Databases?
>>
>> The obvious one is to use the JDBC driver provided by
>> Microsoft.
> 
> There were some problems with the Microsoft driver
> when I looked into it 2006. Some problems with opening
> multiple cursors over the same database connection
> session...
> 
> Are these solved now?

I don't know because I don't know that specific problem.

But Microsoft actually updates the drivers frequently, so
it could be fixed.

SQLServer 2000 driver is at SP3 dated July 2004.

SQLServer 2005 driver is at version 1.2 dated October 2007.

SQLServer 2008 driver is promised soon.

Arne

---
 * Synchronet * The Whitehouse BBS --- whitehouse.hulds.com --- check it out free usenet!
--- Synchronet 3.15a-Win32 NewsLink 1.92
Time Warp of the Future BBS - telnet://time.synchro.net:24

[toc] | [prev] | [next] | [standalone]

#94 — Re: jTDS, trashes heap wi

  To: comp.lang.java.databases
Jan Burse wrote:
> Dear All
> 
> I was just looking up the source code of the jTDS driver.
> It seems that it is trashing the heap with java.lang
> wrapper classes, even when one is using only primitive
> types in his application.

Why do you call this "trashing the heap"?

> For example, when I am doing, whereas the column
> is integer:
> 
>    ResultSet set;
>    set.getLong(1);
> 
> This amounts to a call to:
> 
>     public long getLong(int columnIndex) throws SQLException {
>          return ((Long) Support.convert(con, getColumn(columnIndex),
>                   java.sql.Types.BIGINT, null)).longValue();
>     }
> 
> During the above call a Long object will be created, as
> can be seen from the convert implementation:

Yes, so?

> A solution would be to do the following
> call on the result set:

A solution to what?  You have yet to present a problem.

>    (long)set.getInt(1);
> 
> But what if I want to make the code, lets say column
> size independent, and want to use a getLong() call
> only.

Then just use that call.

> Does this trashing happen in every JDBC driver?

This is not "trashing".  Why do you call it that?

It is likely that all JDBC drivers convert SQL types to reference types in 
Java.  Every column is required to work with ResultSet#getObject() and to 
handle the possibility of a (SQL) NULL value in the column.  NULL is not in 
the domain of primitive values, so the driver really must use a reference type.

> Are there alternatives to jTDS, when accessing
> MS SQL Databases?

jTDS is a stable, performant, high-quality product.  I doubt you'll find better.

What is the problem?  You never stated a problem.

-- 
Lew

---
 * Synchronet * The Whitehouse BBS --- whitehouse.hulds.com --- check it out free usenet!
--- Synchronet 3.15a-Win32 NewsLink 1.92
Time Warp of the Future BBS - telnet://time.synchro.net:24

[toc] | [prev] | [next] | [standalone]

#96 — Re: jTDS, trashes heap wi

  To: comp.lang.java.databases
Lew schrieb:

> What is the problem?  You never stated a problem.
> 

Thrashing = unnecessarily filling the heap with objects,
and thus using up processor time, because of heap alloc
calls and gc calls.

Could for example in the case of getLong() be prevented
by a more elaborate implementation of Support.convert.
But anyway no pun intended on jTDS, I am using it all day.

Best Regards

---
 * Synchronet * The Whitehouse BBS --- whitehouse.hulds.com --- check it out free usenet!
--- Synchronet 3.15a-Win32 NewsLink 1.92
Time Warp of the Future BBS - telnet://time.synchro.net:24

[toc] | [prev] | [next] | [standalone]

#99 — Re: jTDS, trashes heap wi

  To: comp.lang.java.databases
Jan Burse wrote:
> Lew schrieb:
> 
>> What is the problem?  You never stated a problem.
>>
> 
> Thrashing = unnecessarily filling the heap with objects,
> and thus using up processor time, because of heap alloc
> calls and gc calls.

You make it sound so dire.  Allocating these objects takes next to no time, GC 
will be fast since they'll all be unreachable, and you don't have to worry 
about it.  Not only that, but Hotspot will probably optimize away the object 
calls in favor of primitives anyway, possibly enregistered.

All this fuss over a little heap allocation.  Tsk, tsk.

-- 
Lew

---
 * Synchronet * The Whitehouse BBS --- whitehouse.hulds.com --- check it out free usenet!
--- Synchronet 3.15a-Win32 NewsLink 1.92
Time Warp of the Future BBS - telnet://time.synchro.net:24

[toc] | [prev] | [next] | [standalone]

#100 — Re: jTDS, trashes heap wi

  To: Lew
Lew schrieb:
> You make it sound so dire.  Allocating these objects takes next to no 
> time, GC will be fast since they'll all be unreachable, and you don't 
> have to worry about it.  Not only that, but Hotspot will probably 
> optimize away the object calls in favor of primitives anyway, possibly 
> enregistered.

The hotspot is not infinitely smart, and alloc/gc is not negliable.
Try the following two codes:

Variant 1:

    Color red=new Color(0xff0000);
    g.setColor(red);
    for (int x=0; x<1000; x++) {
      for (int y=0; y<1000; y++) {
         g.drawRect(x,y,1,1);
      }
    }

Variant 2:

    for (int x=0; x<1000; x++) {
      for (int y=0; y<1000; y++) {
	Color red=new Color(0xff0000);
         g.setColor(red);
         g.drawRect(x,y,1,1);
      }
    }

There is a huge difference between the two
in performance.

Similarly one can waste performance in JDBC
drivers or what ever...

Best Regards

---
 * Synchronet * The Whitehouse BBS --- whitehouse.hulds.com --- check it out free usenet!
--- Synchronet 3.15a-Win32 NewsLink 1.92
Time Warp of the Future BBS - telnet://time.synchro.net:24

[toc] | [prev] | [next] | [standalone]

#101 — Re: jTDS, trashes heap wi

  To: comp.lang.java.databases
Jan Burse wrote:
> Lew schrieb:
>> You make it sound so dire.  Allocating these objects takes next to no 
>> time, GC will be fast since they'll all be unreachable, and you don't 
>> have to worry about it.  Not only that, but Hotspot will probably 
>> optimize away the object calls in favor of primitives anyway, possibly 
>> enregistered.
> 
> The hotspot is not infinitely smart, and alloc/gc is not negliable.
> Try the following two codes:
> 
> Variant 1:
> 
>    Color red=new Color(0xff0000);
>    g.setColor(red);
>    for (int x=0; x<1000; x++) {
>      for (int y=0; y<1000; y++) {
>         g.drawRect(x,y,1,1);
>      }
>    }
> 
> Variant 2:
> 
>    for (int x=0; x<1000; x++) {
>      for (int y=0; y<1000; y++) {
>     Color red=new Color(0xff0000);
>         g.setColor(red);
>         g.drawRect(x,y,1,1);
>      }
>    }
> 
> There is a huge difference between the two
> in performance.

Huge relative to what?  The JDBC calls through to the database?

I don't think so.

Furthermore, you give no evidence that the example you show is bound by 
allocation or GC performance, rather than the speed of graphics calls.  What 
does g.setColor() do?

By pulling the setColor() call inside the loop, you make it unclear whether 
it's that or the 'new' call that is causing your overhead.

You don't show how much difference allocation makes, nor GC, nor how it fits 
into the context of *database* calls, which are not CPU bound.

Again I say, you are performing premature optimization, absent any evidence 
that the allocation and GC are causing you any significant time overhead 
relative to your application.  I suggest that you *measure* the impact of the 
allocations and GCs, which will be very fast.

Are you really setting the color a million times in a row, vs. setting it once 
for a million operations?  Do you have a million calls into the database in a 
short time, thus motivating your optimization concerns?

Somehow I don't think so.

-- 
Lew

---
 * Synchronet * The Whitehouse BBS --- whitehouse.hulds.com --- check it out free usenet!
--- Synchronet 3.15a-Win32 NewsLink 1.92
Time Warp of the Future BBS - telnet://time.synchro.net:24

[toc] | [prev] | [next] | [standalone]

#109 — Re: jTDS, trashes heap wi

  To: comp.lang.java.databases
Lew schrieb:
> Do you have a million calls into the 
> database in a short time, thus motivating 
 > your optimization concerns?

I have approx. 1'000 DB calls per
item. And I have ca.
6'000 items.

The first version of the batch run
around 8 hours. Now I am down to
1 1/2 hours.

I am now using a custom memory cache
between the DB and the application.

Bye

---
 * Synchronet * The Whitehouse BBS --- whitehouse.hulds.com --- check it out free usenet!
--- Synchronet 3.15a-Win32 NewsLink 1.92
Time Warp of the Future BBS - telnet://time.synchro.net:24

[toc] | [prev] | [next] | [standalone]

#116 — Re: jTDS, trashes heap wi

  To: comp.lang.java.databases
Jan Burse wrote:
> Lew schrieb:
>> Do you have a million calls into the database in a short time, thus 
>> motivating 
>  > your optimization concerns?
> 
> I have approx. 1'000 DB calls per
> item. And I have ca.
> 6'000 items.

Interesting notation.  Where does it come from?

> The first version of the batch run
> around 8 hours. Now I am down to
> 1 1/2 hours.
> 
> I am now using a custom memory cache
> between the DB and the application.

Clearly you were I/O bound before, and memory allocation and GC were not the 
issue.

QED.

-- 
Lew

---
 * Synchronet * The Whitehouse BBS --- whitehouse.hulds.com --- check it out free usenet!
--- Synchronet 3.15a-Win32 NewsLink 1.92
Time Warp of the Future BBS - telnet://time.synchro.net:24

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.java.databases

csiph-web