Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.java.programmer > #15311 > unrolled thread
| Started by | "Aaron W. Hsu" <arcfide@sacrideo.us> |
|---|---|
| First post | 2012-06-15 17:33 -0500 |
| Last post | 2012-06-19 07:46 -0700 |
| Articles | 20 on this page of 84 — 19 participants |
Back to article view | Back to comp.lang.java.programmer
Recommendations for Lightweight Threading? "Aaron W. Hsu" <arcfide@sacrideo.us> - 2012-06-15 17:33 -0500
Re: Recommendations for Lightweight Threading? markspace <-@.> - 2012-06-15 15:55 -0700
Re: Recommendations for Lightweight Threading? "Aaron W. Hsu" <arcfide@sacrideo.us> - 2012-06-15 18:12 -0500
Re: Recommendations for Lightweight Threading? markspace <-@.> - 2012-06-15 16:31 -0700
Re: Recommendations for Lightweight Threading? "Aaron W. Hsu" <arcfide@sacrideo.us> - 2012-06-15 20:00 -0500
Re: Recommendations for Lightweight Threading? Robert Klemme <shortcutter@googlemail.com> - 2012-06-16 14:39 +0200
Re: Recommendations for Lightweight Threading? Daniel Pitts <newsgroup.nospam@virtualinfinity.net> - 2012-06-16 12:13 -0700
Re: Recommendations for Lightweight Threading? Roedy Green <see_website@mindprod.com.invalid> - 2012-06-16 00:57 -0700
Re: Recommendations for Lightweight Threading? Lew <lewbloch@gmail.com> - 2012-06-15 15:57 -0700
Re: Recommendations for Lightweight Threading? "Aaron W. Hsu" <arcfide@sacrideo.us> - 2012-06-15 18:12 -0500
Re: Recommendations for Lightweight Threading? Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-06-15 20:19 -0400
Re: Recommendations for Lightweight Threading? "Aaron W. Hsu" <arcfide@sacrideo.us> - 2012-06-15 19:59 -0500
Re: Recommendations for Lightweight Threading? Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-06-15 21:37 -0400
Controlling the Garbage Collector "Aaron W. Hsu" <arcfide@sacrideo.us> - 2012-06-16 11:51 -0500
Re: Controlling the Garbage Collector Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-06-16 14:24 -0400
Re: Controlling the Garbage Collector markspace <-@.> - 2012-06-16 12:24 -0700
Re: Controlling the Garbage Collector Daniel Pitts <newsgroup.nospam@virtualinfinity.net> - 2012-06-16 13:14 -0700
Re: Controlling the Garbage Collector Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-06-16 16:35 -0400
Re: Controlling the Garbage Collector "Aaron W. Hsu" <arcfide@sacrideo.us> - 2012-06-16 19:43 -0500
Re: Controlling the Garbage Collector Wanja Gayk <brixomatic@yahoo.com> - 2012-06-23 13:36 +0200
Re: Controlling the Garbage Collector Robert Klemme <shortcutter@googlemail.com> - 2012-06-23 15:39 +0200
Re: Controlling the Garbage Collector markspace <-@.> - 2012-06-16 14:34 -0700
Re: Controlling the Garbage Collector Robert Klemme <shortcutter@googlemail.com> - 2012-06-18 04:32 -0700
Re: Controlling the Garbage Collector Roedy Green <see_website@mindprod.com.invalid> - 2012-06-24 14:31 -0700
Re: Controlling the Garbage Collector Arne Vajhøj <arne@vajhoej.dk> - 2012-06-20 21:19 -0400
Re: Controlling the Garbage Collector "Aaron W. Hsu" <arcfide@sacrideo.us> - 2012-06-21 13:24 -0500
Re: Controlling the Garbage Collector Lew <lewbloch@gmail.com> - 2012-06-21 11:37 -0700
Re: Controlling the Garbage Collector Fred Greer <fggreer@nospam.invalid> - 2012-06-21 21:20 +0000
Re: Controlling the Garbage Collector Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-06-21 15:24 -0400
Re: Controlling the Garbage Collector Robert Klemme <shortcutter@googlemail.com> - 2012-06-21 23:46 +0200
Re: Controlling the Garbage Collector Jukka Lahtinen <jtfjdehf@hotmail.com.invalid> - 2012-06-25 15:28 +0300
Re: Controlling the Garbage Collector Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-06-25 09:05 -0400
Re: Controlling the Garbage Collector Andreas Leitgeb <avl@gamma.logic.tuwien.ac.at> - 2012-06-25 17:01 +0000
Re: Controlling the Garbage Collector Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-06-25 13:45 -0400
Re: Controlling the Garbage Collector Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-06-25 13:49 -0400
Re: Controlling the Garbage Collector Robert Klemme <shortcutter@googlemail.com> - 2012-06-25 20:41 +0200
Re: Controlling the Garbage Collector Andreas Leitgeb <avl@gamma.logic.tuwien.ac.at> - 2012-06-26 12:25 +0000
Re: Controlling the Garbage Collector Robert Klemme <shortcutter@googlemail.com> - 2012-06-26 06:46 -0700
Re: Controlling the Garbage Collector Andreas Leitgeb <avl@gamma.logic.tuwien.ac.at> - 2012-06-26 16:26 +0000
Re: Controlling the Garbage Collector Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-06-26 13:07 -0400
Re: Controlling the Garbage Collector Robert Klemme <shortcutter@googlemail.com> - 2012-06-26 22:28 +0200
Re: Controlling the Garbage Collector Andreas Leitgeb <avl@gamma.logic.tuwien.ac.at> - 2012-06-26 23:49 +0000
Re: Controlling the Garbage Collector Lew <lewbloch@gmail.com> - 2012-06-26 18:20 -0700
Re: Controlling the Garbage Collector Highway to Hell <HtH49439112@gmail.com> - 2012-06-26 21:52 -0400
Re: Controlling the Garbage Collector Gene Wirchenko <genew@ocis.net> - 2012-06-26 20:01 -0700
Re: Controlling the Garbage Collector Highway to Hell <HtH49439112@gmail.com> - 2012-06-26 23:23 -0400
Re: Controlling the Garbage Collector Gene Wirchenko <genew@ocis.net> - 2012-06-27 09:05 -0700
Re: Controlling the Garbage Collector Martin Gregorie <martin@address-in-sig.invalid> - 2012-06-27 20:15 +0000
Re: Controlling the Garbage Collector Gene Wirchenko <genew@ocis.net> - 2012-06-27 13:52 -0700
Re: Controlling the Garbage Collector Lew <lewbloch@gmail.com> - 2012-06-27 13:41 -0700
Re: Controlling the Garbage Collector Gene Wirchenko <genew@ocis.net> - 2012-06-27 19:02 -0700
Re: Controlling the Garbage Collector Martin Gregorie <martin@address-in-sig.invalid> - 2012-06-28 21:45 +0000
Re: [OT] Driver's license restrictions (Was: Controlling the Garbage Collector) Lew <lewbloch@gmail.com> - 2012-06-28 15:16 -0700
Re: [OT] Driver's license restrictions (Was: Controlling the Garbage Collector) Martin Gregorie <martin@address-in-sig.invalid> - 2012-06-29 02:10 +0000
Re: [OT] Driver's license restrictions (Was: Controlling the Garbage Collector) Gene Wirchenko <genew@ocis.net> - 2012-06-28 19:57 -0700
Re: [OT] Driver's license restrictions glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2012-06-29 04:06 +0000
Re: [OT] Driver's license restrictions Tim Slattery <Slattery_T@bls.gov> - 2012-06-29 08:35 -0400
Re: [OT] Driver's license restrictions (Was: Controlling the Garbage Collector) Tim Slattery <Slattery_T@bls.gov> - 2012-06-29 08:33 -0400
Re: Controlling the Garbage Collector Tim Slattery <Slattery_T@bls.gov> - 2012-06-29 08:30 -0400
Re: Controlling the Garbage Collector Martin Gregorie <martin@address-in-sig.invalid> - 2012-06-29 23:04 +0000
Re: Controlling the Garbage Collector Highway to Hell <HtH49439112@gmail.com> - 2012-06-27 16:53 -0400
Re: Controlling the Garbage Collector Leif Roar Moldskred <leifm@dimnakorr.com> - 2012-06-27 11:32 -0500
Re: Controlling the Garbage Collector Highway to Hell <HtH49439112@gmail.com> - 2012-06-27 16:54 -0400
Re: Controlling the Garbage Collector Lew <lewbloch@gmail.com> - 2012-06-27 11:45 -0700
Re: Controlling the Garbage Collector Highway to Hell <HtH49439112@gmail.com> - 2012-06-27 16:55 -0400
Re: Controlling the Garbage Collector glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2012-06-27 02:06 +0000
Re: Controlling the Garbage Collector Andreas Leitgeb <avl@gamma.logic.tuwien.ac.at> - 2012-06-27 16:34 +0000
Re: Controlling the Garbage Collector Leif Roar Moldskred <leifm@dimnakorr.com> - 2012-06-27 11:45 -0500
Re: Controlling the Garbage Collector Wanja Gayk <brixomatic@yahoo.com> - 2012-07-02 11:21 +0200
Re: Controlling the Garbage Collector Lew <lewbloch@gmail.com> - 2012-06-26 13:52 -0700
Re: Controlling the Garbage Collector Andreas Leitgeb <avl@gamma.logic.tuwien.ac.at> - 2012-06-26 23:40 +0000
Re: Controlling the Garbage Collector Lew <lewbloch@gmail.com> - 2012-06-26 16:48 -0700
Re: Controlling the Garbage Collector markspace <-@.> - 2012-06-21 15:15 -0700
Re: Controlling the Garbage Collector Lew <lewbloch@gmail.com> - 2012-06-21 15:33 -0700
Re: Controlling the Garbage Collector "Aaron W. Hsu" <arcfide@sacrideo.us> - 2012-06-21 21:24 -0500
Re: Controlling the Garbage Collector markspace <-@.> - 2012-06-21 15:23 -0700
Re: Recommendations for Lightweight Threading? Wanja Gayk <brixomatic@yahoo.com> - 2012-06-17 15:49 +0200
Re: Recommendations for Lightweight Threading? Roedy Green <see_website@mindprod.com.invalid> - 2012-06-16 01:00 -0700
Re: Recommendations for Lightweight Threading? Roedy Green <see_website@mindprod.com.invalid> - 2012-06-16 01:04 -0700
Re: Recommendations for Lightweight Threading? Wanja Gayk <brixomatic@yahoo.com> - 2012-06-16 04:03 +0200
Re: Recommendations for Lightweight Threading? Kevin McMurtrie <mcmurtrie@pixelmemory.us> - 2012-06-15 22:27 -0700
Re: Recommendations for Lightweight Threading? Robert Klemme <shortcutter@googlemail.com> - 2012-06-16 14:39 +0200
Re: Recommendations for Lightweight Threading? Kevin McMurtrie <mcmurtrie@pixelmemory.us> - 2012-06-18 19:37 -0700
Re: Recommendations for Lightweight Threading? Robert Klemme <shortcutter@googlemail.com> - 2012-06-19 07:46 -0700
Page 2 of 5 — ← Prev page 1 [2] 3 4 5 Next page →
| From | Robert Klemme <shortcutter@googlemail.com> |
|---|---|
| Date | 2012-06-23 15:39 +0200 |
| Subject | Re: Controlling the Garbage Collector |
| Message-ID | <a4lv7qFga4U1@mid.individual.net> |
| In reply to | #15543 |
On 23.06.2012 13:36, Wanja Gayk wrote: > Maybe it's just because sometimes people don't like to give up control > they once had, just to be sure they could if they needed, even though in > 99.99% of all cases they don't. In my experience it's different: you need to have control because the automatic mechanisms in GC implementations still do not cope well with all use cases (at least those we had). Although that does not mean that for many use cases defaults are sufficient (which they are I believe). Kind regards robert -- remember.guy do |as, often| as.you_can - without end http://blog.rubybestpractices.com/
[toc] | [prev] | [next] | [standalone]
| From | markspace <-@.> |
|---|---|
| Date | 2012-06-16 14:34 -0700 |
| Subject | Re: Controlling the Garbage Collector |
| Message-ID | <jriu5a$mm5$1@dont-email.me> |
| In reply to | #15337 |
On 6/16/2012 1:14 PM, Daniel Pitts wrote: > On 6/16/12 12:24 PM, markspace wrote: >> On 6/16/2012 11:24 AM, Eric Sosman wrote: >>> Then I'd suggest Java may be ill-suited to your needs. >> >> >> For us production coders, that's true. Depending on how experimental >> Aaron's project is however, hacking the JVM may not be out of the >> question. What he's talking about certainly sounds more academic to me, >> so I'm sort of curious what methods he's going to use to achieve his >> goals. > It almost sounds like the Aaron may be better off using Event Loop style > architecture, with one event dispatch thread per CPU core. Allow the GC > to do what it does best, and have the operations be smaller and more > decoupled. > I dunno. Phase 1 of his little project might just be to implement a regular old multi-threaded program in Java (probably one with some specific characteristics that he's interested in). Phase 2 might be to modify his local JVM and show that his special GC/threading model produces some specific improvement(s), while running the same byte codes as phase 1. That's what I'd do, if I had the resources....
[toc] | [prev] | [next] | [standalone]
| From | Robert Klemme <shortcutter@googlemail.com> |
|---|---|
| Date | 2012-06-18 04:32 -0700 |
| Subject | Re: Controlling the Garbage Collector |
| Message-ID | <2f2646f4-a3ee-49c5-bc2d-1b44ef91d28c@googlegroups.com> |
| In reply to | #15332 |
On Saturday, June 16, 2012 6:51:13 PM UTC+2, Aaron W. Hsu wrote: > On Fri, 15 Jun 2012 21:37:32 -0400, Eric Sosman wrote: > > > Even if the thread you're interested in is careful not to create new > > object instances, object creation in other threads can trigger the > > garbage collector at pretty much any time. > > The garbage collector is one of the reasons that I have hesitated to move > to Java. In some languages (Chez Scheme) I can get quite explicit > control over when and how garbage collection occurs, which can make it > possible to do very fine grained things to avoid some of the corner case > problems that can manifest in garbage collected languages. Usually this > is not a problem at all, but I would like to have the control nonetheless. Be sure to try out G1 collector. In my brief tests so far it worked pretty well with regard to the target timings albeit at the expense of a bit of CPU overhead. Kind regards robert
[toc] | [prev] | [next] | [standalone]
| From | Roedy Green <see_website@mindprod.com.invalid> |
|---|---|
| Date | 2012-06-24 14:31 -0700 |
| Subject | Re: Controlling the Garbage Collector |
| Message-ID | <hp1fu715rd0lo2s417f1pqdplib1b3ih9b@4ax.com> |
| In reply to | #15376 |
On Mon, 18 Jun 2012 04:32:15 -0700 (PDT), Robert Klemme <shortcutter@googlemail.com> wrote, quoted or indirectly quoted someone who said : >Be sure to try out G1 collector. In my brief tests so far it worked pretty >well with regard to the target timings albeit at the expense of a bit of CPU overhead. There is also the GC in Jet. It does not seem to have the lurchiness of Oracle GC. see http://mindprod.com/jgloss/jet.html -- Roedy Green Canadian Mind Products http://mindprod.com Computers are machines that do exactly what you tell them, but they still can surprise you with the results. ~ Dr. Richard Dawkins 1941-03-26
[toc] | [prev] | [next] | [standalone]
| From | Arne Vajhøj <arne@vajhoej.dk> |
|---|---|
| Date | 2012-06-20 21:19 -0400 |
| Subject | Re: Controlling the Garbage Collector |
| Message-ID | <4fe27687$0$281$14726298@news.sunsite.dk> |
| In reply to | #15332 |
On 6/16/2012 12:51 PM, Aaron W. Hsu wrote:
> On Fri, 15 Jun 2012 21:37:32 -0400, Eric Sosman wrote:
>> Even if the thread you're interested in is careful not to create new
>> object instances, object creation in other threads can trigger the
>> garbage collector at pretty much any time.
>
> The garbage collector is one of the reasons that I have hesitated to move
> to Java. In some languages (Chez Scheme) I can get quite explicit
> control over when and how garbage collection occurs, which can make it
> possible to do very fine grained things to avoid some of the corner case
> problems that can manifest in garbage collected languages. Usually this
> is not a problem at all, but I would like to have the control nonetheless.
What do you mean by corner case?
GC and real time does usually not fit well (*), but I would not consider
real time a corner case. Some need it - many don't.
Arne
*) There is RTSJ (Real-Time Specification for Java) but it does not
seem to have that much traction.
[toc] | [prev] | [next] | [standalone]
| From | "Aaron W. Hsu" <arcfide@sacrideo.us> |
|---|---|
| Date | 2012-06-21 13:24 -0500 |
| Subject | Re: Controlling the Garbage Collector |
| Message-ID | <K9mdnVvXpcl4-37SnZ2dnUVZ_q6dnZ2d@giganews.com> |
| In reply to | #15476 |
On Wed, 20 Jun 2012 21:19:01 -0400, Arne Vajhøj wrote: > On 6/16/2012 12:51 PM, Aaron W. Hsu wrote: >> In some languages (Chez Scheme) I can get quite explicit >> control over when and how garbage collection occurs, which can make it >> possible to do very fine grained things to avoid some of the corner >> case problems that can manifest in garbage collected languages. > > What do you mean by corner case? So, you mentioned that you thought real-time was not a corner case, but I guess I would consider it one compared to general purpose programming in the majority. However, there are other, less broad things where I have found it extremely helpful to have control of the collector. For instance, I implemented a number of weak pointer structures which required me to be able to extend what happened during collections. This can be done in user-space efficiently if there is an efficient way to extend the garbage collector and trigger events to occur on certain conditions. This would be an example of a relatively obscure data structure that can be really useful in some situations. On the other side, there are times when I know that I am going to be doing a number of very small, little allocations all in a row, and I do not want the GC to run during this tight loop, because it will cause a noticeable hit. I would rather delay the collector, consuming more memory until the end when I can explicitly trigger the collector at that point. Or, I may know that some structure is going to be extremely long lived and I need to keep it out of the collector entirely. Finally, there are times when I want to do a large bulk allocation outside of the collector, and then selectively move certain things into the collected space, but still have a checked, high-level way of accessing data structures in the uncollected space. Now, all of these are corner cases to me because you can get by without them in most situations, unless you are trying to eek out all of the performance you can from your system. -- Aaron W. Hsu | arcfide@sacrideo.us | http://www.sacrideo.us Programming is just another word for the lost art of thinking.
[toc] | [prev] | [next] | [standalone]
| From | Lew <lewbloch@gmail.com> |
|---|---|
| Date | 2012-06-21 11:37 -0700 |
| Subject | Re: Controlling the Garbage Collector |
| Message-ID | <b6e4fa27-dc2a-441b-8135-323f29b8e98d@googlegroups.com> |
| In reply to | #15490 |
Aaron W. Hsu wrote: > Arne Vajhøj wrote: >> Aaron W. Hsu wrote: >>> In some languages (Chez Scheme) I can get quite explicit >>> control over when and how garbage collection occurs, which can make it >>> possible to do very fine grained things to avoid some of the corner >>> case problems that can manifest in garbage collected languages. >> >> What do you mean by corner case? > > So, you mentioned that you thought real-time was not a corner case, but I > guess I would consider it one compared to general purpose programming in > the majority. However, there are other, less broad things where I have > found it extremely helpful to have control of the collector. > > For instance, I implemented a number of weak pointer structures which WeakReference. > required me to be able to extend what happened during collections. This Object#finalize() WeakHashMap > can be done in user-space efficiently if there is an efficient way to > extend the garbage collector and trigger events to occur on certain > conditions. This would be an example of a relatively obscure data > structure that can be really useful in some situations. Java has the infrastructure for what you need. > On the other side, there are times when I know that I am going to be > doing a number of very small, little allocations all in a row, and I do > not want the GC to run during this tight loop, because it will cause a > noticeable hit. I would rather delay the collector, consuming more memory > until the end when I can explicitly trigger the collector at that point. Have you looked at the parameters to control GC? > Or, I may know that some structure is going to be extremely long lived > and I need to keep it out of the collector entirely. Are you absolutely certain you can do this better than the collector? I'm not. > Finally, there are times when I want to do a large bulk allocation > outside of the collector, and then selectively move certain things into > the collected space, but still have a checked, high-level way of > accessing data structures in the uncollected space. Not Java. > Now, all of these are corner cases to me because you can get by without > them in most situations, unless you are trying to eek out all of the > performance you can from your system. I'm not so sure that you could do what you want the way you say you want to. I feel highly confident that by appropriate choice of collectors and tuning their parameters you can achieve your performance goals. -- Lew
[toc] | [prev] | [next] | [standalone]
| From | Fred Greer <fggreer@nospam.invalid> |
|---|---|
| Date | 2012-06-21 21:20 +0000 |
| Subject | Re: Controlling the Garbage Collector |
| Message-ID | <js037a$nq9$1@dont-email.me> |
| In reply to | #15492 |
On Thu, 21 Jun 2012 11:37:09 -0700, Lew wrote: > Aaron W. Hsu wrote: >> Arne Vajhøj wrote: >>> Aaron W. Hsu wrote: >>>> In some languages (Chez Scheme) I can get quite explicit control over >>>> when and how garbage collection occurs, which can make it possible to >>>> do very fine grained things to avoid some of the corner case problems >>>> that can manifest in garbage collected languages. >>> >>> What do you mean by corner case? >> >> So, you mentioned that you thought real-time was not a corner case, but >> I guess I would consider it one compared to general purpose programming >> in the majority. However, there are other, less broad things where I >> have found it extremely helpful to have control of the collector. >> >> For instance, I implemented a number of weak pointer structures which > > WeakReference. > >> required me to be able to extend what happened during collections. This > > Object#finalize() > WeakHashMap > >> can be done in user-space efficiently if there is an efficient way to >> extend the garbage collector and trigger events to occur on certain >> conditions. This would be an example of a relatively obscure data >> structure that can be really useful in some situations. ReferenceQueue
[toc] | [prev] | [next] | [standalone]
| From | Eric Sosman <esosman@ieee-dot-org.invalid> |
|---|---|
| Date | 2012-06-21 15:24 -0400 |
| Subject | Re: Controlling the Garbage Collector |
| Message-ID | <jrvsds$bls$1@dont-email.me> |
| In reply to | #15490 |
On 6/21/2012 2:24 PM, Aaron W. Hsu wrote:
> [...]
> On the other side, there are times when I know that I am going to be
> doing a number of very small, little allocations all in a row, and I do
> not want the GC to run during this tight loop, because it will cause a
> noticeable hit. I would rather delay the collector, consuming more memory
> until the end when I can explicitly trigger the collector at that point.
Sounds like you might be better off collecting first, when there
are fewer objects to be discovered and analyzed. Also, since you've
said you're not speaking of real-time applications, I don't see why
you should be concerned about the precise timing of the "hit:" Before,
after, during, what difference does it make? (If you're worried about
GC roiling the caches, please provide actual measurements.)
> Or, I may know that some structure is going to be extremely long lived
> and I need to keep it out of the collector entirely.
Generational collectors do a pretty good imitation of this. It's
not perfect because it takes time for the GC to discover what's long-
lived and what's not, but it's pretty effective. Also, it has the
huge advantage of always being right; few programmers can match that
accuracy. (See also "memory leak.")
> Finally, there are times when I want to do a large bulk allocation
> outside of the collector, and then selectively move certain things into
> the collected space, but still have a checked, high-level way of
> accessing data structures in the uncollected space.
JNI, if you simply must.
> Now, all of these are corner cases to me because you can get by without
> them in most situations, unless you are trying to eek out all of the
> performance you can from your system.
"Eek" is right ...
--
Eric Sosman
esosman@ieee-dot-org.invalid
[toc] | [prev] | [next] | [standalone]
| From | Robert Klemme <shortcutter@googlemail.com> |
|---|---|
| Date | 2012-06-21 23:46 +0200 |
| Subject | Re: Controlling the Garbage Collector |
| Message-ID | <a4hj1sFc30U1@mid.individual.net> |
| In reply to | #15490 |
On 21.06.2012 20:24, Aaron W. Hsu wrote: > On Wed, 20 Jun 2012 21:19:01 -0400, Arne Vajhøj wrote: > >> On 6/16/2012 12:51 PM, Aaron W. Hsu wrote: > >>> In some languages (Chez Scheme) I can get quite explicit >>> control over when and how garbage collection occurs, which can make it >>> possible to do very fine grained things to avoid some of the corner >>> case problems that can manifest in garbage collected languages. >> >> What do you mean by corner case? > > So, you mentioned that you thought real-time was not a corner case, but I > guess I would consider it one compared to general purpose programming in > the majority. However, there are other, less broad things where I have > found it extremely helpful to have control of the collector. Your statement illustrates one of the biggest problems of Java (or maybe even any GC'ed language): GC should be automatic but humans still need control (just count the collectors and all the JVM options for GC) because the automatisms are not good enough (yet). Question is: will they ever be good enough that we can get by without settings - or at least a dramatic reduced number of settings? I do not see that coming soon. Sun took seven or eight years from the first experimental implementation of G1 to production ready. Go figure. > For instance, I implemented a number of weak pointer structures which > required me to be able to extend what happened during collections. This > can be done in user-space efficiently if there is an efficient way to > extend the garbage collector and trigger events to occur on certain > conditions. This would be an example of a relatively obscure data > structure that can be really useful in some situations. Can you explain what you did at collection time? I am asking because anytime I would prefer to do cleanup when I do not need an object any more. This is the typical resource deallocation in a finally block or other piece of code which is invoked after some operation has finished (hooks for example). Advantage is that you free your resources as early as possible and do not need to cope with the downsides of finalizers. Finalizers in Java have so many disadvantages (i.e. when making objects live again, effects on GC performance) that they are short of being deprecated - at least that's my impression from what I read. > On the other side, there are times when I know that I am going to be > doing a number of very small, little allocations all in a row, and I do > not want the GC to run during this tight loop, because it will cause a > noticeable hit. I would rather delay the collector, consuming more memory > until the end when I can explicitly trigger the collector at that point. But what do you gain if a GC pause occurs after this operation? The time is spent either way. Btw, depending on collector chosen you might not even notice that it is running because it works in separate threads. > Or, I may know that some structure is going to be extremely long lived > and I need to keep it out of the collector entirely. Long lived objects which live shorter than the application (i.e. not classes) are actually the Achilles heel of GC because it is very hard to tune the collector in a way that it does not visit those long living objects too often and yet run often enough to ensure enough free memory is available. > Finally, there are times when I want to do a large bulk allocation > outside of the collector, and then selectively move certain things into > the collected space, but still have a checked, high-level way of > accessing data structures in the uncollected space. I once mused about such a thing as well. I you think a bit longer about this then you'll notice that it won't work: these objects still have to be visited because they may have references to other objects not in the uncollectable space. You'll probably do not gain much - if anything at all. > Now, all of these are corner cases to me because you can get by without > them in most situations, unless you are trying to eek out all of the > performance you can from your system. Use many cores and G1. At least try it out. Kind regards robert -- remember.guy do |as, often| as.you_can - without end http://blog.rubybestpractices.com/
[toc] | [prev] | [next] | [standalone]
| From | Jukka Lahtinen <jtfjdehf@hotmail.com.invalid> |
|---|---|
| Date | 2012-06-25 15:28 +0300 |
| Subject | Re: Controlling the Garbage Collector |
| Message-ID | <m3zk7ro9sg.fsf@ipa.eternal-september.org> |
| In reply to | #15496 |
Robert Klemme <shortcutter@googlemail.com> writes: > Long lived objects which live shorter than the application (i.e. not > classes) are actually the Achilles heel of GC because it is very hard to > tune the collector in a way that it does not visit those long living > objects too often and yet run often enough to ensure enough free memory is It might be good to have a method to gell the GC that a certain long-lived object is no longer needed. For example, something like System.gc(Object old) where old is the object that has just been needed the last time. -- Jukka Lahtinen
[toc] | [prev] | [next] | [standalone]
| From | Eric Sosman <esosman@ieee-dot-org.invalid> |
|---|---|
| Date | 2012-06-25 09:05 -0400 |
| Subject | Re: Controlling the Garbage Collector |
| Message-ID | <js9nn1$lba$1@dont-email.me> |
| In reply to | #15576 |
On 6/25/2012 8:28 AM, Jukka Lahtinen wrote:
> Robert Klemme <shortcutter@googlemail.com> writes:
>
>> Long lived objects which live shorter than the application (i.e. not
>> classes) are actually the Achilles heel of GC because it is very hard to
>> tune the collector in a way that it does not visit those long living
>> objects too often and yet run often enough to ensure enough free memory is
>
> It might be good to have a method to gell the GC that a certain
> long-lived object is no longer needed.
> For example, something like
> System.gc(Object old)
> where old is the object that has just been needed the last time.
How would you call the method?
--
Eric Sosman
esosman@ieee-dot-org.invalid
[toc] | [prev] | [next] | [standalone]
| From | Andreas Leitgeb <avl@gamma.logic.tuwien.ac.at> |
|---|---|
| Date | 2012-06-25 17:01 +0000 |
| Subject | Re: Controlling the Garbage Collector |
| Message-ID | <slrnjuh6bd.u9l.avl@gamma.logic.tuwien.ac.at> |
| In reply to | #15577 |
Eric Sosman <esosman@ieee-dot-org.invalid> wrote:
> On 6/25/2012 8:28 AM, Jukka Lahtinen wrote:
>> Robert Klemme <shortcutter@googlemail.com> writes:
>>> Long lived objects which live shorter than the application (i.e. not
>>> classes) are actually the Achilles heel of GC because it is very hard to
>>> tune the collector in a way that it does not visit those long living
>>> objects too often and yet run often enough to ensure enough free memory is
>> It might be good to have a method to gell the GC that a certain
>> long-lived object is no longer needed.
>> For example, something like
>> System.gc(Object old)
>> where old is the object that has just been needed the last time.
> How would you call the method?
class MyRef<T> {
private T ref;
public T consume() { T tmp = ref; ref=null; return tmp; }
public MyRef(T t) { ref=t; }
}
[...somewhere else...]
BigObject bo=new BigObject();
// use bo ...
MyRef<BigObject> mrBo = new MyRef<>(bo); bo=null;
System.gc( mrBo.consume() );
;-)
Maybe slightly more interesting questions:
What would be the behaviour of that proposed call,
- if "old" *is* still (hard-)referenced from live objects?
- if there's still a weak/soft-reference on that object?
(pick one: Exception, force(for weak/soft-refs) or ignore?)
[toc] | [prev] | [next] | [standalone]
| From | Eric Sosman <esosman@ieee-dot-org.invalid> |
|---|---|
| Date | 2012-06-25 13:45 -0400 |
| Subject | Re: Controlling the Garbage Collector |
| Message-ID | <jsa838$vih$1@dont-email.me> |
| In reply to | #15580 |
On 6/25/2012 1:01 PM, Andreas Leitgeb wrote:
> Eric Sosman <esosman@ieee-dot-org.invalid> wrote:
>> On 6/25/2012 8:28 AM, Jukka Lahtinen wrote:
>>> Robert Klemme <shortcutter@googlemail.com> writes:
>>>> Long lived objects which live shorter than the application (i.e. not
>>>> classes) are actually the Achilles heel of GC because it is very hard to
>>>> tune the collector in a way that it does not visit those long living
>>>> objects too often and yet run often enough to ensure enough free memory is
>>> It might be good to have a method to gell the GC that a certain
>>> long-lived object is no longer needed.
>>> For example, something like
>>> System.gc(Object old)
>>> where old is the object that has just been needed the last time.
>> How would you call the method?
>
> class MyRef<T> {
> private T ref;
> public T consume() { T tmp = ref; ref=null; return tmp; }
> public MyRef(T t) { ref=t; }
> }
> [...somewhere else...]
> BigObject bo=new BigObject();
> // use bo ...
> MyRef<BigObject> mrBo = new MyRef<>(bo); bo=null;
> System.gc( mrBo.consume() );
> ;-)
;-)
> Maybe slightly more interesting questions:
> What would be the behaviour of that proposed call,
> - if "old" *is* still (hard-)referenced from live objects?
> - if there's still a weak/soft-reference on that object?
> (pick one: Exception, force(for weak/soft-refs) or ignore?)
Exactly: The JVM could not simply trust the System.gc(obj)
call to be telling the truth, because that would open the door
to all manner of attacks. Therefore the JVM would have to verify
that obj was unreachable (at whatever strength) before deciding
to dispose of it. That is, the JVM would have to run nearly all
of the garbage collector to determine obj's reachability -- so
why not just run the ordinary GC in the ordinary way? I'm at a
loss to imagine what benefit a System.gc(obj) would offer.
--
Eric Sosman
esosman@ieee-dot-org.invalid
[toc] | [prev] | [next] | [standalone]
| From | Eric Sosman <esosman@ieee-dot-org.invalid> |
|---|---|
| Date | 2012-06-25 13:49 -0400 |
| Subject | Re: Controlling the Garbage Collector |
| Message-ID | <jsa8c8$1jj$1@dont-email.me> |
| In reply to | #15581 |
On 6/25/2012 1:45 PM, Eric Sosman wrote:
>[...]
> Exactly: The JVM could not simply trust the System.gc(obj)
> call to be telling the truth, because that would open the door
> to all manner of attacks.[...]
... like
System.gc(Runtime.getRuntime());
;-)
--
Eric Sosman
esosman@ieee-dot-org.invalid
[toc] | [prev] | [next] | [standalone]
| From | Robert Klemme <shortcutter@googlemail.com> |
|---|---|
| Date | 2012-06-25 20:41 +0200 |
| Subject | Re: Controlling the Garbage Collector |
| Message-ID | <a4rpnjForkU1@mid.individual.net> |
| In reply to | #15581 |
On 25.06.2012 19:45, Eric Sosman wrote: > On 6/25/2012 1:01 PM, Andreas Leitgeb wrote: >> Maybe slightly more interesting questions: >> What would be the behaviour of that proposed call, >> - if "old" *is* still (hard-)referenced from live objects? >> - if there's still a weak/soft-reference on that object? >> (pick one: Exception, force(for weak/soft-refs) or ignore?) > > Exactly: The JVM could not simply trust the System.gc(obj) > call to be telling the truth, because that would open the door > to all manner of attacks. Not just attacks but simple errors as well. > Therefore the JVM would have to verify > that obj was unreachable (at whatever strength) before deciding > to dispose of it. That is, the JVM would have to run nearly all > of the garbage collector to determine obj's reachability -- so > why not just run the ordinary GC in the ordinary way? I'm at a > loss to imagine what benefit a System.gc(obj) would offer. I had exactly this kind of reasoning in mind when I said: > I once mused about such a thing as well. If you think a bit longer > about this then you'll notice that it won't work: these objects still > have to be visited because they may have references to other objects > not in the uncollectable space. You'll probably do not gain much - > if anything at all. Thank you for the elaboration, Eric and Andreas! I think a non uniform object memory model (e.g. some GCed, some manually managed) cannot get rid of the GC overhead even for the manual kind as long as it is allowed to have references between the different kinds of objects. Disallowing that however would make the distinction visible and you end up with a language with a non uniform object model. That would certainly make programmers' lives harder instead of easier. Kind regards robert -- remember.guy do |as, often| as.you_can - without end http://blog.rubybestpractices.com/
[toc] | [prev] | [next] | [standalone]
| From | Andreas Leitgeb <avl@gamma.logic.tuwien.ac.at> |
|---|---|
| Date | 2012-06-26 12:25 +0000 |
| Subject | Re: Controlling the Garbage Collector |
| Message-ID | <slrnjujaih.u9l.avl@gamma.logic.tuwien.ac.at> |
| In reply to | #15584 |
Robert Klemme <shortcutter@googlemail.com> wrote:
> Thank you for the elaboration, Eric and Andreas! I think a non uniform
> object memory model (e.g. some GCed, some manually managed) cannot get
> rid of the GC overhead even for the manual kind as long as it is allowed
> to have references between the different kinds of objects.
It is no problem for GC'ed objects to be referenced from manually
managed objects.
In the reverse case, it could mean that references to manually
collected items could become "stale". Of course that would have
to be a fundamentally different kind of "stale" as what we know
from C.
Imagine there was a class ManualReference, similar to WeakReference.
When a manually managed object was created, it would be transparently
wrapped by such a ManualReference. Any ref to such an object would
really be to a ManualReference that in turn holds the object, and
certain things done on the reference are passed to it's payload, like
accessing fields, or calling methods. Other stuff (assignment, or
passing around the value) would still be a local access on the variable
itself. That would surely need some changes in the JVM and/or the
compiler to work.
(I'm not proposing that. I just find it challenging to think it through.)
Now, if by some new mechanism (e.g. a new keyword "free"), a manually
managed object was freed, then the (transparent) ManualReference would
be simply cleared, and all further accesses cause a NullPointerException.
The ManualReference object itself would be normally GC-managed, thus
remain in memory at least until all references to it are indeed cleared.
Gist: I think it *could* be done, but it would surely require some
deep changes in Java, that are not justified by any so far identified
advantage.
PS: I once had a task where some explicit System.gc() really helped.
I don't know why it is so vaguely defined ("suggests", rather
than "requests" a best effort...), but it did appear to work
well, anyway. Most of the other usecases for manual control should
be dealt with by Java7's try-with-resources.
[toc] | [prev] | [next] | [standalone]
| From | Robert Klemme <shortcutter@googlemail.com> |
|---|---|
| Date | 2012-06-26 06:46 -0700 |
| Subject | Re: Controlling the Garbage Collector |
| Message-ID | <5d156697-1477-49c1-a098-f7bfa76783b4@googlegroups.com> |
| In reply to | #15596 |
On Tuesday, June 26, 2012 2:25:53 PM UTC+2, Andreas Leitgeb wrote: > Robert Klemme <shortcutter@googlemail.com> wrote: > > Thank you for the elaboration, Eric and Andreas! I think a non uniform > > object memory model (e.g. some GCed, some manually managed) cannot get > > rid of the GC overhead even for the manual kind as long as it is allowed > > to have references between the different kinds of objects. > > It is no problem for GC'ed objects to be referenced from manually > managed objects. I think it is. The point in separating object kinds is to reduce the number of objects GC has to look at in order to avoid looking at long lived old objects over and over again. But to determine the liveliness state of an "automatic" object GC would still have to look at the "manual" object _during each allocation_. Even worse, since references only point forward (otherwise we'd have introduced a tremendous memory - and probably CPU - hit for bidirectional references) it is necessary to look at _all_ "manual" objects reachable from roots. > In the reverse case, it could mean that references to manually > collected items could become "stale". Of course that would have > to be a fundamentally different kind of "stale" as what we know > from C. > > Imagine there was a class ManualReference, similar to WeakReference. > When a manually managed object was created, it would be transparently > wrapped by such a ManualReference. Any ref to such an object would > really be to a ManualReference that in turn holds the object, and > certain things done on the reference are passed to it's payload, like > accessing fields, or calling methods. Other stuff (assignment, or > passing around the value) would still be a local access on the variable > itself. That would surely need some changes in the JVM and/or the > compiler to work. > > (I'm not proposing that. I just find it challenging to think it through.) > > Now, if by some new mechanism (e.g. a new keyword "free"), a manually > managed object was freed, then the (transparent) ManualReference would > be simply cleared, and all further accesses cause a NullPointerException. > > The ManualReference object itself would be normally GC-managed, thus > remain in memory at least until all references to it are indeed cleared. You replaced the automatic collected object by another automatic collected object (the ManualReference). Again, nothing gained in terms of number of objects to look at for the GC. More abstract: you introduced a reference type which can be set to null at any time (i.e. when someone decides to "free" this object). Basically that's the same situation we have with WeakReference now and as consequence you need to check the reference for null on every access. Because of concurrency you need to be able to lock the reference for a certain duration (e.g. a method call) in order to avoid clearing during a logical operation. You would at least need to be able to to some atomic "check for null and use ref" otherwise the null check would be moot. With WeakReference you do that by copying the reference to an ordinary reference. The clearing mechanism would need to prevent clearing while the reference is locked / copied into a regular reference. As a consequence the effect of your "free" operation is not to actually remove the object but only to _schedule_ it for removal. This is a quite similar situation to "automatic" objects which have all references to them cleared (or GC'ed themselves) btw. Since we cannot impose a limit on the locking time and the actual time lag between "free" and last "unlock" can be quite high, we might not have gained much in term of control of memory usage. Even worse, someone else might come along and do a "lock" after "free" thus prolonging the locking period. The alternative would be to disallow "lock" after free but that comes with issues of it's own: then any "lock" operation could fail and the user of the language is prompted with the issue that he needs to deal with the exception originating from a failed lock. Maybe that could be salvaged by making all "lock" operations after "free" return null. That would be similar to WeakReference. "locking" of course would need to be reentrant so several methods of a class could cooperate and use the same "manual" member object. That means: a "lock" after "free" must return null only if it is the first "lock" of a thread, i.e. if the thread did not have the lock before. As another side effect we will have a situation where after a "free" some threads might still see an instance while others don't. Well, that can happen with WeakReference as well depending on when threads fetch the reference, so that might not be too big an issue. Now you would have to check the refcount of the "manual" object on every "unlock". This has a number of consequences - refcount checking might be imposed on the thread which uses a "manual" object creating CPU overhead - additional synchronization is necessary to avoid race conditions for "lock" and "unlock" - refcount checking does not work with circular references > Gist: I think it *could* be done, but it would surely require some > deep changes in Java, that are not justified by any so far identified > advantage. I don't even think there is an advantage. The - admittedly brief - analysis above shows that there is a ton of issues lurking and I believe allowing mixed references between "manual" and "automatic" objects introduces more complexity (in terms of code, runtime and probably also memory) instead of relieving us from the old object overhead. Kind regards robert
[toc] | [prev] | [next] | [standalone]
| From | Andreas Leitgeb <avl@gamma.logic.tuwien.ac.at> |
|---|---|
| Date | 2012-06-26 16:26 +0000 |
| Subject | Re: Controlling the Garbage Collector |
| Message-ID | <slrnjujol5.u9l.avl@gamma.logic.tuwien.ac.at> |
| In reply to | #15597 |
Robert Klemme <shortcutter@googlemail.com> wrote: > On Tuesday, June 26, 2012 2:25:53 PM UTC+2, Andreas Leitgeb wrote: >> Robert Klemme <shortcutter@googlemail.com> wrote: >> > Thank you for the elaboration, Eric and Andreas! I think a non uniform >> > object memory model (e.g. some GCed, some manually managed) cannot get >> > rid of the GC overhead even for the manual kind as long as it is allowed >> > to have references between the different kinds of objects. >> It is no problem for GC'ed objects to be referenced from manually >> managed objects. > I think it is. The point in separating object kinds is to reduce the > number of objects GC has to look at in order to avoid looking at long > lived old objects over and over again. Oh, seems I got the motivation wrong, then. If that was your point, then obviously my essay didn't address that. My motivation was having some large objects, whose memory could be explicitly freed, as soon as the memory was no longer needed in the program. >> [ManualReference to deal with possibly dangling pointers sanely] > Basically that's the same situation we have with WeakReference now and > as consequence you need to check the reference for null on every access. That's not the point. Seeing a null in a manual reference would happen only from a Bug: it indicates that you freed an object that still had refs on it, which shouldn't have happened in the first place. Theoretically, a hypothetical JVM might even throw a different exception for this case, such as a DanglingPointerException to be used only for debugging:-) I've snipped most of your comments, as I think they are just following from the misunderstanding... (lock, refcount,...) >> Gist: I think it *could* be done, but it would surely require some >> deep changes in Java, that are not justified by any so far identified >> advantage. > I don't even think there is an advantage. Well, so do I. It was still fun, thinking about it.
[toc] | [prev] | [next] | [standalone]
| From | Eric Sosman <esosman@ieee-dot-org.invalid> |
|---|---|
| Date | 2012-06-26 13:07 -0400 |
| Subject | Re: Controlling the Garbage Collector |
| Message-ID | <jscq8f$cu1$1@dont-email.me> |
| In reply to | #15599 |
On 6/26/2012 12:26 PM, Andreas Leitgeb wrote:
> [...]
> Oh, seems I got the motivation wrong, then. If that was your point,
> then obviously my essay didn't address that. My motivation was having
> some large objects, whose memory could be explicitly freed, as soon as
> the memory was no longer needed in the program.
And my question, still, is "Why?" At a guess, the intent is
to make the large objects' memory available for re-use right away,
without waiting for GC (and thereby, perhaps, reducing the number
of GC's required). But in all the scenarios so far, I think it's
been shown that the effort to support manual release safely is at
least as great as that of a GC, maybe more.
--
Eric Sosman
esosman@ieee-dot-org.invalid
[toc] | [prev] | [next] | [standalone]
Page 2 of 5 — ← Prev page 1 [2] 3 4 5 Next page →
Back to top | Article view | comp.lang.java.programmer
csiph-web