Groups > comp.lang.java.programmer > #15496

Re: Controlling the Garbage Collector

From	Robert Klemme <shortcutter@googlemail.com>
Newsgroups	comp.lang.java.programmer
Subject	Re: Controlling the Garbage Collector
Date	2012-06-21 23:46 +0200
Message-ID	<a4hj1sFc30U1@mid.individual.net> (permalink)
References	(2 earlier) <6KSdnTwztJvtR0bSnZ2dnUVZ_t-dnZ2d@giganews.com> <jrgo11$3t6$1@dont-email.me> <_JmdnUu_Z-scJEHSnZ2dnUVZ_t6dnZ2d@giganews.com> <4fe27687$0$281$14726298@news.sunsite.dk> <K9mdnVvXpcl4-37SnZ2dnUVZ_q6dnZ2d@giganews.com>

Show all headers | View raw

On 21.06.2012 20:24, Aaron W. Hsu wrote:
> On Wed, 20 Jun 2012 21:19:01 -0400, Arne Vajhøj wrote:
>
>> On 6/16/2012 12:51 PM, Aaron W. Hsu wrote:
>
>>> In some languages (Chez Scheme) I can get quite explicit
>>> control over when and how garbage collection occurs, which can make it
>>> possible to do very fine grained things to avoid some of the corner
>>> case problems that can manifest in garbage collected languages.
>>
>> What do you mean by corner case?
>
> So, you mentioned that you thought real-time was not a corner case, but I
> guess I would consider it one compared to general purpose programming in
> the majority. However, there are other, less broad things where I have
> found it extremely helpful to have control of the collector.

Your statement illustrates one of the biggest problems of Java (or maybe 
even any GC'ed language): GC should be automatic but humans still need 
control (just count the collectors and all the JVM options for GC) 
because the automatisms are not good enough (yet).  Question is: will 
they ever be good enough that we can get by without settings - or at 
least a dramatic reduced number of settings?  I do not see that coming 
soon.  Sun took seven or eight years from the first experimental 
implementation of G1 to production ready.  Go figure.

> For instance, I implemented a number of weak pointer structures which
> required me to be able to extend what happened during collections. This
> can be done in user-space efficiently if there is an efficient way to
> extend the garbage collector and trigger events to occur on certain
> conditions.  This would be an example of a relatively obscure data
> structure that can be really useful in some situations.

Can you explain what you did at collection time?  I am asking because 
anytime I would prefer to do cleanup when I do not need an object any 
more.  This is the typical resource deallocation in a finally block or 
other piece of code which is invoked after some operation has finished 
(hooks for example).  Advantage is that you free your resources as early 
as possible and do not need to cope with the downsides of finalizers. 
Finalizers in Java have so many disadvantages (i.e. when making objects 
live again, effects on GC performance) that they are short of being 
deprecated - at least that's my impression from what I read.

> On the other side, there are times when I know that I am going to be
> doing a number of very small, little allocations all in a row, and I do
> not want the GC to run during this tight loop, because it will cause a
> noticeable hit. I would rather delay the collector, consuming more memory
> until the end when I can explicitly trigger the collector at that point.

But what do you gain if a GC pause occurs after this operation?  The 
time is spent either way.  Btw, depending on collector chosen you might 
not even notice that it is running because it works in separate threads.

> Or, I may know that some structure is going to be extremely long lived
> and I need to keep it out of the collector entirely.

Long lived objects which live shorter than the application (i.e. not 
classes) are actually the Achilles heel of GC because it is very hard to 
tune the collector in a way that it does not visit those long living 
objects too often and yet run often enough to ensure enough free memory 
is available.

> Finally, there are times when I want to do a large bulk allocation
> outside of the collector, and then selectively move certain things into
> the collected space, but still have a checked, high-level way of
> accessing data structures in the uncollected space.

I once mused about such a thing as well.  I you think a bit longer about 
this then you'll notice that it won't work: these objects still have to 
be visited because they may have references to other objects not in the 
uncollectable space.  You'll probably do not gain much - if anything at all.

> Now, all of these are corner cases to me because you can get by without
> them in most situations, unless you are trying to eek out all of the
> performance you can from your system.

Use many cores and G1.  At least try it out.

Kind regards

	robert

-- 
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

Thread

Recommendations for Lightweight Threading? "Aaron W. Hsu" <arcfide@sacrideo.us> - 2012-06-15 17:33 -0500
  Re: Recommendations for Lightweight Threading? markspace <-@.> - 2012-06-15 15:55 -0700
    Re: Recommendations for Lightweight Threading? "Aaron W. Hsu" <arcfide@sacrideo.us> - 2012-06-15 18:12 -0500
      Re: Recommendations for Lightweight Threading? markspace <-@.> - 2012-06-15 16:31 -0700
        Re: Recommendations for Lightweight Threading? "Aaron W. Hsu" <arcfide@sacrideo.us> - 2012-06-15 20:00 -0500
        Re: Recommendations for Lightweight Threading? Robert Klemme <shortcutter@googlemail.com> - 2012-06-16 14:39 +0200
          Re: Recommendations for Lightweight Threading? Daniel Pitts <newsgroup.nospam@virtualinfinity.net> - 2012-06-16 12:13 -0700
      Re: Recommendations for Lightweight Threading? Roedy Green <see_website@mindprod.com.invalid> - 2012-06-16 00:57 -0700
  Re: Recommendations for Lightweight Threading? Lew <lewbloch@gmail.com> - 2012-06-15 15:57 -0700
    Re: Recommendations for Lightweight Threading? "Aaron W. Hsu" <arcfide@sacrideo.us> - 2012-06-15 18:12 -0500
  Re: Recommendations for Lightweight Threading? Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-06-15 20:19 -0400
    Re: Recommendations for Lightweight Threading? "Aaron W. Hsu" <arcfide@sacrideo.us> - 2012-06-15 19:59 -0500
      Re: Recommendations for Lightweight Threading? Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-06-15 21:37 -0400
        Controlling the Garbage Collector "Aaron W. Hsu" <arcfide@sacrideo.us> - 2012-06-16 11:51 -0500
          Re: Controlling the Garbage Collector Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-06-16 14:24 -0400
            Re: Controlling the Garbage Collector markspace <-@.> - 2012-06-16 12:24 -0700
              Re: Controlling the Garbage Collector Daniel Pitts <newsgroup.nospam@virtualinfinity.net> - 2012-06-16 13:14 -0700
                Re: Controlling the Garbage Collector Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-06-16 16:35 -0400
                Re: Controlling the Garbage Collector "Aaron W. Hsu" <arcfide@sacrideo.us> - 2012-06-16 19:43 -0500
                Re: Controlling the Garbage Collector Wanja Gayk <brixomatic@yahoo.com> - 2012-06-23 13:36 +0200
                Re: Controlling the Garbage Collector Robert Klemme <shortcutter@googlemail.com> - 2012-06-23 15:39 +0200
                Re: Controlling the Garbage Collector markspace <-@.> - 2012-06-16 14:34 -0700
          Re: Controlling the Garbage Collector Robert Klemme <shortcutter@googlemail.com> - 2012-06-18 04:32 -0700
            Re: Controlling the Garbage Collector Roedy Green <see_website@mindprod.com.invalid> - 2012-06-24 14:31 -0700
          Re: Controlling the Garbage Collector Arne Vajhøj <arne@vajhoej.dk> - 2012-06-20 21:19 -0400
            Re: Controlling the Garbage Collector "Aaron W. Hsu" <arcfide@sacrideo.us> - 2012-06-21 13:24 -0500
              Re: Controlling the Garbage Collector Lew <lewbloch@gmail.com> - 2012-06-21 11:37 -0700
                Re: Controlling the Garbage Collector Fred Greer <fggreer@nospam.invalid> - 2012-06-21 21:20 +0000
              Re: Controlling the Garbage Collector Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-06-21 15:24 -0400
              Re: Controlling the Garbage Collector Robert Klemme <shortcutter@googlemail.com> - 2012-06-21 23:46 +0200
                Re: Controlling the Garbage Collector Jukka Lahtinen <jtfjdehf@hotmail.com.invalid> - 2012-06-25 15:28 +0300
                Re: Controlling the Garbage Collector Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-06-25 09:05 -0400
                Re: Controlling the Garbage Collector Andreas Leitgeb <avl@gamma.logic.tuwien.ac.at> - 2012-06-25 17:01 +0000
                Re: Controlling the Garbage Collector Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-06-25 13:45 -0400
                Re: Controlling the Garbage Collector Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-06-25 13:49 -0400
                Re: Controlling the Garbage Collector Robert Klemme <shortcutter@googlemail.com> - 2012-06-25 20:41 +0200
                Re: Controlling the Garbage Collector Andreas Leitgeb <avl@gamma.logic.tuwien.ac.at> - 2012-06-26 12:25 +0000
                Re: Controlling the Garbage Collector Robert Klemme <shortcutter@googlemail.com> - 2012-06-26 06:46 -0700
                Re: Controlling the Garbage Collector Andreas Leitgeb <avl@gamma.logic.tuwien.ac.at> - 2012-06-26 16:26 +0000
                Re: Controlling the Garbage Collector Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-06-26 13:07 -0400
                Re: Controlling the Garbage Collector Robert Klemme <shortcutter@googlemail.com> - 2012-06-26 22:28 +0200
                Re: Controlling the Garbage Collector Andreas Leitgeb <avl@gamma.logic.tuwien.ac.at> - 2012-06-26 23:49 +0000
                Re: Controlling the Garbage Collector Lew <lewbloch@gmail.com> - 2012-06-26 18:20 -0700
                Re: Controlling the Garbage Collector Highway to Hell <HtH49439112@gmail.com> - 2012-06-26 21:52 -0400
                Re: Controlling the Garbage Collector Gene Wirchenko <genew@ocis.net> - 2012-06-26 20:01 -0700
                Re: Controlling the Garbage Collector Highway to Hell <HtH49439112@gmail.com> - 2012-06-26 23:23 -0400
                Re: Controlling the Garbage Collector Gene Wirchenko <genew@ocis.net> - 2012-06-27 09:05 -0700
                Re: Controlling the Garbage Collector Martin Gregorie <martin@address-in-sig.invalid> - 2012-06-27 20:15 +0000
                Re: Controlling the Garbage Collector Gene Wirchenko <genew@ocis.net> - 2012-06-27 13:52 -0700
                Re: Controlling the Garbage Collector Lew <lewbloch@gmail.com> - 2012-06-27 13:41 -0700
                Re: Controlling the Garbage Collector Gene Wirchenko <genew@ocis.net> - 2012-06-27 19:02 -0700
                Re: Controlling the Garbage Collector Martin Gregorie <martin@address-in-sig.invalid> - 2012-06-28 21:45 +0000
                Re: [OT] Driver's license restrictions (Was: Controlling the Garbage Collector) Lew <lewbloch@gmail.com> - 2012-06-28 15:16 -0700
                Re: [OT] Driver's license restrictions (Was: Controlling the Garbage Collector) Martin Gregorie <martin@address-in-sig.invalid> - 2012-06-29 02:10 +0000
                Re: [OT] Driver's license restrictions (Was: Controlling the Garbage Collector) Gene Wirchenko <genew@ocis.net> - 2012-06-28 19:57 -0700
                Re: [OT] Driver's license restrictions glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2012-06-29 04:06 +0000
                Re: [OT] Driver's license restrictions Tim Slattery <Slattery_T@bls.gov> - 2012-06-29 08:35 -0400
                Re: [OT] Driver's license restrictions (Was: Controlling the Garbage Collector) Tim Slattery <Slattery_T@bls.gov> - 2012-06-29 08:33 -0400
                Re: Controlling the Garbage Collector Tim Slattery <Slattery_T@bls.gov> - 2012-06-29 08:30 -0400
                Re: Controlling the Garbage Collector Martin Gregorie <martin@address-in-sig.invalid> - 2012-06-29 23:04 +0000
                Re: Controlling the Garbage Collector Highway to Hell <HtH49439112@gmail.com> - 2012-06-27 16:53 -0400
                Re: Controlling the Garbage Collector Leif Roar Moldskred <leifm@dimnakorr.com> - 2012-06-27 11:32 -0500
                Re: Controlling the Garbage Collector Highway to Hell <HtH49439112@gmail.com> - 2012-06-27 16:54 -0400
                Re: Controlling the Garbage Collector Lew <lewbloch@gmail.com> - 2012-06-27 11:45 -0700
                Re: Controlling the Garbage Collector Highway to Hell <HtH49439112@gmail.com> - 2012-06-27 16:55 -0400
                Re: Controlling the Garbage Collector glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2012-06-27 02:06 +0000
                Re: Controlling the Garbage Collector Andreas Leitgeb <avl@gamma.logic.tuwien.ac.at> - 2012-06-27 16:34 +0000
                Re: Controlling the Garbage Collector Leif Roar Moldskred <leifm@dimnakorr.com> - 2012-06-27 11:45 -0500
                Re: Controlling the Garbage Collector Wanja Gayk <brixomatic@yahoo.com> - 2012-07-02 11:21 +0200
                Re: Controlling the Garbage Collector Lew <lewbloch@gmail.com> - 2012-06-26 13:52 -0700
                Re: Controlling the Garbage Collector Andreas Leitgeb <avl@gamma.logic.tuwien.ac.at> - 2012-06-26 23:40 +0000
                Re: Controlling the Garbage Collector Lew <lewbloch@gmail.com> - 2012-06-26 16:48 -0700
              Re: Controlling the Garbage Collector markspace <-@.> - 2012-06-21 15:15 -0700
                Re: Controlling the Garbage Collector Lew <lewbloch@gmail.com> - 2012-06-21 15:33 -0700
                Re: Controlling the Garbage Collector "Aaron W. Hsu" <arcfide@sacrideo.us> - 2012-06-21 21:24 -0500
              Re: Controlling the Garbage Collector markspace <-@.> - 2012-06-21 15:23 -0700
        Re: Recommendations for Lightweight Threading? Wanja Gayk <brixomatic@yahoo.com> - 2012-06-17 15:49 +0200
    Re: Recommendations for Lightweight Threading? Roedy Green <see_website@mindprod.com.invalid> - 2012-06-16 01:00 -0700
      Re: Recommendations for Lightweight Threading? Roedy Green <see_website@mindprod.com.invalid> - 2012-06-16 01:04 -0700
  Re: Recommendations for Lightweight Threading? Wanja Gayk <brixomatic@yahoo.com> - 2012-06-16 04:03 +0200
  Re: Recommendations for Lightweight Threading? Kevin McMurtrie <mcmurtrie@pixelmemory.us> - 2012-06-15 22:27 -0700
    Re: Recommendations for Lightweight Threading? Robert Klemme <shortcutter@googlemail.com> - 2012-06-16 14:39 +0200
      Re: Recommendations for Lightweight Threading? Kevin McMurtrie <mcmurtrie@pixelmemory.us> - 2012-06-18 19:37 -0700
        Re: Recommendations for Lightweight Threading? Robert Klemme <shortcutter@googlemail.com> - 2012-06-19 07:46 -0700

csiph-web