Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #14824 > unrolled thread

Proposed new Java feature

Back to article view | Back to comp.lang.java.programmer

Contents

  Proposed new Java feature "Mike Schilling" <mscottschilling@hotmail.com> - 2012-05-26 16:11 -0700
    Re: Proposed new Java feature Robert Klemme <shortcutter@googlemail.com> - 2012-05-27 13:32 +0200
      Re: Proposed new Java feature "Mike Schilling" <mscottschilling@hotmail.com> - 2012-05-27 11:14 -0700
        Re: Proposed new Java feature Robert Klemme <shortcutter@googlemail.com> - 2012-05-28 12:13 +0200
        Re: Proposed new Java feature v_borchert@despammed.com (Volker Borchert) - 2012-06-04 20:16 +0000
    Re: Proposed new Java feature markspace <-@.> - 2012-05-27 09:28 -0700
      Re: Proposed new Java feature "Mike Schilling" <mscottschilling@hotmail.com> - 2012-05-27 11:00 -0700
        Re: Proposed new Java feature Daniel Pitts <newsgroup.nospam@virtualinfinity.net> - 2012-05-27 11:46 -0700
          Re: Proposed new Java feature "Mike Schilling" <mscottschilling@hotmail.com> - 2012-05-27 12:04 -0700
            Re: Proposed new Java feature Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-05-27 15:30 -0400
              Re: Proposed new Java feature "Mike Schilling" <mscottschilling@hotmail.com> - 2012-05-27 12:59 -0700
                Re: Proposed new Java feature Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-05-27 17:51 -0400
                  Re: Proposed new Java feature markspace <-@.> - 2012-05-27 15:20 -0700
                    Re: Proposed new Java feature Robert Klemme <shortcutter@googlemail.com> - 2012-05-28 12:13 +0200
                      Re: Proposed new Java feature markspace <-@.> - 2012-05-28 08:28 -0700
                        Re: Proposed new Java feature Robert Klemme <shortcutter@googlemail.com> - 2012-05-28 19:29 +0200
                          Re: Proposed new Java feature markspace <-@.> - 2012-05-28 12:16 -0700
                            Re: Proposed new Java feature Robert Klemme <shortcutter@googlemail.com> - 2012-05-28 22:58 +0200
                              Re: Proposed new Java feature "Mike Schilling" <mscottschilling@hotmail.com> - 2012-05-28 21:40 -0700
                                Re: Proposed new Java feature Robert Klemme <shortcutter@googlemail.com> - 2012-05-29 22:52 +0200
                                  Re: Proposed new Java feature "Mike Schilling" <mscottschilling@hotmail.com> - 2012-05-29 18:53 -0700
                  Re: Proposed new Java feature Tom Anderson <twic@urchin.earth.li> - 2012-05-27 23:27 +0100
                    Re: Proposed new Java feature Robert Klemme <shortcutter@googlemail.com> - 2012-05-28 12:22 +0200
                  Re: Proposed new Java feature "Mike Schilling" <mscottschilling@hotmail.com> - 2012-05-27 21:43 -0700
                    Re: Proposed new Java feature Andreas Leitgeb <avl@gamma.logic.tuwien.ac.at> - 2012-05-29 14:32 +0000
    Re: Proposed new Java feature Tom Anderson <twic@urchin.earth.li> - 2012-05-27 18:43 +0100
    Re: Proposed new Java feature Kevin McMurtrie <mcmurtrie@pixelmemory.us> - 2012-06-02 12:41 -0700
      Re: Proposed new Java feature markspace <-@.> - 2012-06-02 15:55 -0700

Page 2 of 2 — ← Prev page 1 [2]

#14930

"Robert Klemme" <shortcutter@googlemail.com> wrote in message 
news:a2kr84F7kbU1@mid.individual.net...
> On 05/29/2012 06:40 AM, Mike Schilling wrote:
>> "Robert Klemme"<shortcutter@googlemail.com>  wrote in message
>> news:a2i77nFik2U1@mid.individual.net...
>>> The point is that with the global reference cleanup the specific code 
>>> does
>>> not have a chance to run because it does not know when.  The proper 
>>> point
>>> in time is just before or during remove().
>>
>> How would the cleanup code knew when to run during Thread exit?
>
> Obviously that would have to be implemented in class Thread using a 
> finally block.

Do the same with Threads returned to ThreadPools and we're good. 

[toc] | [prev] | [next] | [standalone]

#14842

On Sun, 27 May 2012, Eric Sosman wrote:

> On 5/27/2012 3:59 PM, Mike Schilling wrote:
>> "Eric Sosman"<esosman@ieee-dot-org.invalid>  wrote in message
>> news:jptvdf$1s5$1@dont-email.me...
>>> On 5/27/2012 3:04 PM, Mike Schilling wrote:
>>>> "Daniel Pitts"<newsgroup.nospam@virtualinfinity.net>   wrote in message
>>>> news:8Euwr.47425$On2.20024@newsfe16.iad...
>>>>> On 5/27/12 11:00 AM, Mike Schilling wrote:
>>>>>> "markspace"<-@.>    wrote in message news:jptkmp$vbg$1@dont-email.me...
>>>>>>> On 5/26/2012 4:11 PM, Mike Schilling wrote:
>>>>>>> 
>>>>>>>> Proposed feature: a static method on Thread that clears all 
>>>>>>>> ThreadLocals for the current thread.
>>>>>>> 
>>>>>>> At the end of the comments, there's a suggestion to use 
>>>>>>> ThreadLocal::remove(), with the implication that it allows the 
>>>>>>> thread local variable to be garbage collection.  Is there a reason 
>>>>>>> that doesn't work for you?
>>>>>> 
>>>>>> That acts on an individual ThreadLocal (and works quite well), but 
>>>>>> it doens't allow removing all ThreadLocals that might have been 
>>>>>> accumlated.
>>>>>
>>>>> You're basically saying "This type of resource can leak if not 
>>>>> cleared appropriately, so there should be a 'Release all resources' 
>>>>> method."
>>>>> 
>>>>> When paraphrased that way, does that make it clearer why it isn't a 
>>>>> good idea? It would be about the same as a "File.closeAll()" or a 
>>>>> "Socket.closeAll()" call.  Extremely dangerous and only a crutch for 
>>>>> not doing the right thing to begin with.
>>>> 
>>>> Or a "collect all unused memory" call . Clearly, that's a crutch for 
>>>> not keeping track of memory allocation properly in the first place. 
>>>> And the fact that files and sockets are closed when a process exits 
>>>> is yet another crutch.
>>> 
>>> I'm with Daniel.  Your code uses classes that you wrote, and classes 
>>> you got from somewhere else -- Sioux Unusuals, perhaps. And if those 
>>> classes use ThreadLocals for their own purposes, and you blithely 
>>> destroy them all, what happens then?  Or what about the class that 
>>> invoked your code, passing an InheritableThreadLocal? Is it a good 
>>> idea to change parts of your invoker's state that you don't 
>>> understand, that you aren't even aware of?
>> 
>> Consider the use case again.
>
>    I understood it fine the first time around, thanks.

It seems to me that you did not.

> If you think of anything new to add, pray do so.

Mike added this pithy statement of the goal:

  Or, to say it another way, when getting a Thread from a ThreadPool, it
  should be completely indistinguishable whether it's a recycled or
  brand-new thread.

At the moment, it is *not* possible to achieve that, because ThreadLocals 
will survive recycling.

One way of looking at a thread pool is that it provides a supply of fresh 
short-lived virtual threads, implemented on top of long-lived real 
threads. Every time we take a thread out of the pool, we want it to appear 
brand-new. Deleting all ThreadLocals when a real thread is recycled is not 
"extremely dangerous", because at that moment, the *virtual* thread is 
dying, and a new one is being created; that makes it an entirely 
appropriate thing to do.

It is certainly true that it is "only a crutch for not doing the right 
thing to begin with", but the sad fact is that programmers persist in not 
doing the right thing to begin with, and it is useful for framework 
writers to be able to limit the damage they can cause.

tom

-- 
Mathematics is the door and the key to the sciences. -- Roger Bacon

[toc] | [prev] | [next] | [standalone]

#14859

On 05/28/2012 12:27 AM, Tom Anderson wrote:
> On Sun, 27 May 2012, Eric Sosman wrote:
>
>> If you think of anything new to add, pray do so.
>
> Mike added this pithy statement of the goal:
>
> Or, to say it another way, when getting a Thread from a ThreadPool, it
> should be completely indistinguishable whether it's a recycled or
> brand-new thread.
>
> At the moment, it is *not* possible to achieve that, because
> ThreadLocals will survive recycling.

But that is actually a good thing: ThreadLocal's purpose is to attach 
state to a thread which can be accessed without synchronization (if not 
shared).  Consider attaching a database connection to a thread.  It 
would be disastrous for the connection itself to just clear the object 
reference through some global "remove all" and you would pay a high 
price for opening new connections over and over again.  If the database 
connection is supposed to be attached to a thread for only a certain 
amount of time (e.g. a method call) then it must be coded to be properly 
cleaned up; in case of a JDCB connection that means invoking commit() or 
rollback() and then close().  Only after that you can safely clear the 
reference.  But this entirely depends on the ThreadLocal at hand and 
needs to be done specifically for individual ThreadLocals - it cannot be 
done globally.

> One way of looking at a thread pool is that it provides a supply of
> fresh short-lived virtual threads, implemented on top of long-lived real
> threads. Every time we take a thread out of the pool, we want it to
> appear brand-new. Deleting all ThreadLocals when a real thread is
> recycled is not "extremely dangerous", because at that moment, the
> *virtual* thread is dying, and a new one is being created; that makes it
> an entirely appropriate thing to do.

It *is* dangerous as I have explained above.  The observable state of 
each of these virtual threads should of course be identical but that 
does not mean that the technical state must be identical (that's 
basically what caching is all about).  For example, in case of the DB 
connection you might open the connection in initialValue() or some other 
logic so the code using the ThreadLocal always sees a properly 
initialized connection.

> It is certainly true that it is "only a crutch for not doing the right
> thing to begin with", but the sad fact is that programmers persist in
> not doing the right thing to begin with, and it is useful for framework
> writers to be able to limit the damage they can cause.

Actually the global remove would cause more damage than it prevents. 
You cannot know about all the other ThreadLocals used in code and a 
framework writer who attempts to just clear state of other modules 
without even knowing them should be tarred and feathered.

Cheers

	robert

[toc] | [prev] | [next] | [standalone]

#14851

Eric Sosman wrote:
> On 5/27/2012 3:59 PM, Mike Schilling wrote:
>> "Eric Sosman"<esosman@ieee-dot-org.invalid>  wrote in message
>> news:jptvdf$1s5$1@dont-email.me...
>>> On 5/27/2012 3:04 PM, Mike Schilling wrote:
>>>> "Daniel Pitts"<newsgroup.nospam@virtualinfinity.net>   wrote in
>>>> message news:8Euwr.47425$On2.20024@newsfe16.iad...
>>>>> On 5/27/12 11:00 AM, Mike Schilling wrote:
>>>>>> "markspace"<-@.>    wrote in message
>>>>>> news:jptkmp$vbg$1@dont-email.me...
>>>>>>> On 5/26/2012 4:11 PM, Mike Schilling wrote:
>>>>>>>
>>>>>>>> Proposed feature: a static method on Thread that clears all
>>>>>>>> ThreadLocals
>>>>>>>> for
>>>>>>>> the current thread.
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I can see your points.  However, I don't have any real
>>>>>>> experience with ThreadLocal, and when a neophyte agrees with
>>>>>>> your argument, that's a red
>>>>>>> flag.
>>>>>>>
>>>>>>> Here's a blog where someone seems to have the same issue as you.
>>>>>>>
>>>>>>> <http://weblogs.java.net/blog/jjviana/archive/2010/06/10/threadlocal-thread-pool-bad-idea-or-dealing-apparent-glassfish-memor>
>>>>>>>
>>>>>>> At the end of the comments, there's a suggestion to use
>>>>>>> ThreadLocal::remove(), with the implication that it allows the
>>>>>>> thread local variable to be garbage collection.  Is there a
>>>>>>> reason that doesn't
>>>>>>> work for you?
>>>>>>
>>>>>> That acts on an individual ThreadLocal (and works quite well),
>>>>>> but it doens't allow removing all ThreadLocals that might have
>>>>>> been accumlated.
>>>>> You're basically saying "This type of resource can leak if not
>>>>> cleared appropriately, so there should be a 'Release all
>>>>> resources' method." When paraphrased that way, does that make it 
>>>>> clearer why it isn't
>>>>> a good idea? It would be about the same as a "File.closeAll()" or
>>>>> a "Socket.closeAll()" call.  Extremely dangerous and only a
>>>>> crutch for not doing the right thing to begin with.
>>>>
>>>> Or a "collect all unused memory" call . Clearly, that's a crutch
>>>> for not keeping track of memory allocation properly in the first
>>>> place.  And the fact that files and sockets are closed when a
>>>> process exits is yet another
>>>> crutch.
>>>
>>>      I'm with Daniel.  Your code uses classes that you wrote, and
>>> classes you got from somewhere else -- Sioux Unusuals, perhaps.
>>> And if those classes use ThreadLocals for their own purposes, and
>>> you blithely destroy them all, what happens then?  Or what about
>>> the class that invoked your code, passing an InheritableThreadLocal?
>>> Is it a good idea to change parts of your invoker's state that you
>>> don't understand, that you aren't even aware of?
>>
>> Consider the use case again.
>
>     I understood it fine the first time around, thanks.  If you
> think of anything new to add, pray do so.
>
>> [... reiteration of old material ...]
>
>     I notice that you do not address any of the issues Daniel or I
> raised; you just repeat what you "want."  I know three-year-olds
> who can do that, but I don't rush to grant their every whim.

Note that it makes your objections moot.  There's no way to enforce that the 
called code be "correct", and there is no invoker to worry about. 

[toc] | [prev] | [next] | [standalone]

#14886

Mike Schilling <mscottschilling@hotmail.com> wrote:
> Eric Sosman wrote:
>>     I notice that you do not address any of the issues Daniel or I
>> raised; you just repeat what you "want."  I know three-year-olds
>> who can do that, but I don't rush to grant their every whim.
> Note that it makes your objections moot.  There's no way to enforce that the 
> called code be "correct", and there is no invoker to worry about. 

Eric and Daniel's "issues" were as nonsensical as arguing against file
removal, for the risk of *all* the bytes of the file being lost as a
consequence.
We're *not* talking of deleting all files of the system, as that would
correspond to delete *every* threads' ThreadLocals.

The whole point of the proposal is to remove *all* the data attached to 
one respective thread under circumstances where that data would no longer
be relevant to the thread.  If some code relied on persistence of Thread-
Local data across two threads obtained from a pool-thread, then this code
is already broken.  And if one wants a cache that survives a single use
of a pool thread, then ThreadLocal would be a really goofy choice for it
(with or without the proposed feature).

Finally, the route of crude workarounds has already been entered by using
thread pools in the first place. In an ideal world, one would allocate
threads and discard them, and have the reuse of OS threads implemented
under the jvm hood, just like with memory.

[toc] | [prev] | [next] | [standalone]

#14832

On Sat, 26 May 2012, Mike Schilling wrote:

> Proposed feature: a static method on Thread that clears all ThreadLocals 
> for the current thread.  By 3 it's trivial to implement.  By 1 and 2 
> it's needed.  And ThreadPool implementations can simply call it directly 
> before putting the Thread back into the pool.

It's a good idea. Tomcat already has some sort of eldritch hack to do 
exactly this to request threads; it logs a warning if it finds undeleted 
entries. There's a bit about it here:

http://wiki.apache.org/tomcat/MemoryLeakProtection

They, and other container manufacturers, might be glad of an official way 
of doing this.

tom

-- 
Kein Mehrheit Fur Die Mitleid

[toc] | [prev] | [next] | [standalone]

#15009

In article <jprnu7$uv8$1@dont-email.me>,
 "Mike Schilling" <mscottschilling@hotmail.com> wrote:

> First a few observations:
> 
> 1. ThreadLocals may be, in general, an abomination, but there are situation 
> where they're the least of evils.  And if you're building a framework in 
> which third-party code runs (e.g. a webserver), there's no way to disallow 
> their use.
> 
> 2. ThreadLocals interact badly with ThreadPools, because the ThreadLocals 
> keep their value when the tyhread is put back into the pool.  This can lead 
> to leaks and even potential security issues.
> 
> 3. The current implementation of ThreadLocal is this: each thread contains a 
> map from the ThreadLocal instance to its value for that thread.  (This is 
> slightly less intiutive than giving a ThreadLocal instance a map from Thread 
> to value, but has the advantage that the map, which is only used within one 
> thread, needn't be synchronized.)
> 
> Proposed feature: a static method on Thread that clears all ThreadLocals for 
> the current thread.  By 3 it's trivial to implement.  By 1 and 2 it's 
> needed.  And ThreadPool implementations can simply call it directly before 
> putting the Thread back into the pool.

I find ThreadLocal useful for caching rapidly used utility objects that 
are stateful and expensive to initialize.  Formatters, regexp parsers, 
etc.  Despite that use, I do still hate the way ThreadLocal works.

A built-in ThreadLocalCache would be better.  It would behave like 
ThreadLocal except use a SoftReference for each element.  That would 
save me the trouble of writing my own when the thread count gets to be 
too high for having a few KB idle in each thread.  (It would be nice if 
SoftReference behaved better too)

Forcing the ThreadLocals out of a Thread would be bad.  There's still 
too much crap code floating around that uses ThreadLocal for parameter 
passing.  You'd get too much random behavior that's nearly impossible to 
debug.  It would be no worse to eliminate ThreadLocal support entirely.
-- 
I will not see posts from Google because I must filter them as spam

[toc] | [prev] | [next] | [standalone]

#15011

On 6/2/2012 12:41 PM, Kevin McMurtrie wrote:

> Forcing the ThreadLocals out of a Thread would be bad.  There's still
> too much crap code floating around that uses ThreadLocal for parameter
> passing.


Just because there's "crap out there" doesn't mean a new feature isn't 
useful or a good idea.  No one requires you to call "removeLocals" or 
whatever method in your code.  Obviously, no "crap out there" currently 
calls such a method, unless you're willing to muck with reflection.  So 
what harm does adding something useful do?  That someone *might* use it 
inappropriately?  Come on, that describes every single method in the 
Java API.

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.lang.java.programmer

csiph-web