Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #18521 > unrolled thread

Wormholes

Back to article view | Back to comp.lang.java.programmer

Contents

  Wormholes Roedy Green <see_website@mindprod.com.invalid> - 2012-09-03 18:12 -0700
    Re: Wormholes eric@invalid.com (EricF) - 2012-09-04 02:06 +0000
      Re: Wormholes Roedy Green <see_website@mindprod.com.invalid> - 2012-09-04 10:08 -0700
      Re: Wormholes Wanja Gayk <brixomatic@yahoo.com> - 2012-09-09 13:55 +0200
    Re: Wormholes Arne Vajhøj <arne@vajhoej.dk> - 2012-09-03 22:34 -0400
      Re: Wormholes Stuart <DerTopper@web.de> - 2012-09-04 12:15 +0200
    Re: Wormholes markspace <-@.> - 2012-09-03 21:00 -0700
      Re: Wormholes Robert Klemme <shortcutter@googlemail.com> - 2012-09-04 07:08 +0200
    Re: Wormholes Zermelo <zermelo@teletu.it.nospam.invalid> - 2012-09-04 08:47 +0200
      Re: Wormholes Lew <noone@lewscanon.com> - 2012-09-04 03:10 -0700
        Re: Wormholes Zermelo <zermelo@teletu.it.nospam.invalid> - 2012-09-04 16:09 +0200
          Re: Wormholes Gene Wirchenko <genew@ocis.net> - 2012-09-04 08:34 -0700
            Re: Wormholes Zermelo <zermelo@teletu.it.nospam.invalid> - 2012-09-04 17:46 +0200
              Re: Wormholes Daniel Pitts <newsgroup.nospam@virtualinfinity.net> - 2012-09-04 09:31 -0700
          Re: Wormholes Lew <lewbloch@gmail.com> - 2012-09-04 11:14 -0700
          Re: Wormholes Arne Vajhøj <arne@vajhoej.dk> - 2012-09-04 19:59 -0400
      Re: Wormholes Roedy Green <see_website@mindprod.com.invalid> - 2012-09-04 13:20 -0700
        Re: Wormholes Arne Vajhøj <arne@vajhoej.dk> - 2012-09-04 20:00 -0400
        Re: Wormholes Zermelo <zermelo@teletu.it.nospam.invalid> - 2012-09-05 09:01 +0200
        Re: Wormholes Zermelo <zermelo@teletu.it.nospam.invalid> - 2012-09-05 09:02 +0200
        Re: Wormholes Zermelo <zermelo@teletu.it.nospam.invalid> - 2012-09-05 09:02 +0200
          Re: Wormholes Zermelo <zermelo@teletu.it.nospam.invalid> - 2012-09-05 09:03 +0200
        Re: Wormholes markspace <-@.> - 2012-09-05 17:34 -0700
    Re: Wormholes Steven Simpson <ss@domain.invalid> - 2012-09-04 14:18 +0100
      Re: Wormholes Robert Klemme <shortcutter@googlemail.com> - 2012-09-05 21:51 +0200
        Re: Wormholes Steven Simpson <ss@domain.invalid> - 2012-09-05 23:26 +0100
          Re: Wormholes Andreas Leitgeb <avl@gamma.logic.tuwien.ac.at> - 2012-09-06 09:32 +0000
            Re: Wormholes Steven Simpson <ss@domain.invalid> - 2012-09-06 11:47 +0100
              Re: Wormholes Andreas Leitgeb <avl@gamma.logic.tuwien.ac.at> - 2012-09-06 15:10 +0000
                Re: Wormholes Steven Simpson <ss@domain.invalid> - 2012-09-06 20:12 +0100
                  Re: Wormholes Andreas Leitgeb <avl@gamma.logic.tuwien.ac.at> - 2012-09-06 21:08 +0000
    Re: Wormholes Nigel Wade <nmw@ion.le.ac.uk> - 2012-09-04 17:26 +0100
    Re: Wormholes Daniel Pitts <newsgroup.nospam@virtualinfinity.net> - 2012-09-04 09:28 -0700
    Re: Wormholes David Lamb <dalamb@cs.queensu.ca> - 2012-09-04 16:54 -0400

Page 2 of 2 — ← Prev page 1 [2]

#18552

Roedy Green <see_website@mindprod.com.invalid> writes:

> The techniques that I can think of  are default scope variable or pass
> value down call chain to where it is used.   Are there any others?

You can also overload method y and add an extra parameter.

[toc] | [prev] | [next] | [standalone]

#18553

ops sorry for the multi-post:)

[toc] | [prev] | [next] | [standalone]

#18562

On 9/4/2012 1:20 PM, Roedy Green wrote:
> On Tue, 04 Sep 2012 08:47:18 +0200, Zermelo
> <zermelo@teletu.it.nospam.invalid> wrote, quoted or indirectly quoted
> someone who said :
>
>> Add a static variable to x accessible to y.
>
> it would have to have default scope.  I hoping there was a technique


I don't think default scope would work.  Consider the case where 
multiple threads call the method in question.  If the second thread 
over-writes the first thread's value before it gets used, you'll have 
serious errors.

Something like a ThreadLocal is required here.  Something that simulates 
the call stack frame.


> that would behave like that, but which would not expose the variable
> to the entire package, when really only one method should use that
> variable.
>
> The techniques that I can think of  are default scope variable or pass
> value down call chain to where it is used.   Are there any others?


Is there any way you can show us the code?  I'm curious as to why it's 
so hard to refactor.

[toc] | [prev] | [next] | [standalone]

#18532

On 04/09/12 02:12, Roedy Green wrote:
> Somehow I must pass information down the long chain from x to y.  This
> means changing the signatures of all the intermediate methods, and
> adjusting code to the new way.  This can cause ripples incommensurate
> with the triviality of the change.

Assuming that you can't improve your structure or refactor, a 
ThreadLocal might be appropriate as your wormhole.

I've had to do this lately.  The intermediate steps were recursive calls 
through a hierarchy.  y was a custom implementation of a node nested in 
that hierarchy.  x started off the recursive call.

If I was only using the hierarchy once, for one call by x, I would have 
simply embedded the context in y.  But the hierarchy was to be re-used, 
and y needed a different context for each call by x.  Also, calls by x 
could be concurrent.

However, I could guarantee that any single call by x would invoke y by 
the same thread.  That allowed the context to be passed by a 
ThreadLocal, set by x just before the call.

I suppose you could do something similar without threads, e.g.:

   class Context {
     Map<ContextLocal<?>, Object> items = new IdentityHashMap<>();
   }

   class ContextLocal<T> {
     void set(Context ctxt, T val) {
       ctxt.put(this, val);
     }

     T get(Content ctxt) {
       return (T) ctxt.get(this);
     }
   }

Then you'd refactor once to pass Context through the chain.  x creates a 
static ContextLocal<MyContext> myContext, and one Context ctxt for each 
call, and sets up myContext.set(ctxt, value) before the call.  y calls 
myContext.get(ctxt) to get the value.

-- 
ss at comp dot lancs dot ac dot uk

[toc] | [prev] | [next] | [standalone]

#18558

On 04.09.2012 15:18, Steven Simpson wrote:
> On 04/09/12 02:12, Roedy Green wrote:
>> Somehow I must pass information down the long chain from x to y.  This
>> means changing the signatures of all the intermediate methods, and
>> adjusting code to the new way.  This can cause ripples incommensurate
>> with the triviality of the change.
>
> Assuming that you can't improve your structure or refactor, a
> ThreadLocal might be appropriate as your wormhole.
>
> I've had to do this lately.  The intermediate steps were recursive calls
> through a hierarchy.  y was a custom implementation of a node nested in
> that hierarchy.  x started off the recursive call.
>
> If I was only using the hierarchy once, for one call by x, I would have
> simply embedded the context in y.  But the hierarchy was to be re-used,
> and y needed a different context for each call by x.  Also, calls by x
> could be concurrent.
>
> However, I could guarantee that any single call by x would invoke y by
> the same thread.  That allowed the context to be passed by a
> ThreadLocal, set by x just before the call.

I knew finally someone would suggest ThreadLocal for this.  This might 
be even worse than global variables, especially since you pass hidden 
state which usually makes testing more difficult.  The proper approach 
would be to pass the state down the call chain.

IMHO the best usage for ThreadLocal is to cache state *inside a class* 
if calls may be concurrent and the cost of creating the state is 
significantly high.  But using it to pass information between classes 
because one wants to avoid adding method parameters is asking for trouble.

Also, you need to be aware that the lifetime of these objects can be 
quite long (there was a discussion about various aspects of ThreadLocal 
in light of thread pools here earlier).

Kind regards

	robert

-- 
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

[toc] | [prev] | [next] | [standalone]

#18561

On 05/09/12 20:51, Robert Klemme wrote:
> I knew finally someone would suggest ThreadLocal for this.  This might 
> be even worse than global variables, especially since you pass hidden 
> state which usually makes testing more difficult.

Quite; that's why I started with: "Assuming that you can't improve your 
structure or refactor, ..."  That is, others' advice is to be tried first.

> The proper approach would be to pass the state down the call chain.

You're probably right, but there's not enough information in the stated 
problem.  I'd recently experienced a specific version of the problem, 
and mentioned how it was solved.  For me, the "incommensurate ripples" 
would be a use-specific change in an API.

> IMHO the best usage for ThreadLocal is to cache state *inside a class* 
> if calls may be concurrent and the cost of creating the state is 
> significantly high.  But using it to pass information between classes 
> because one wants to avoid adding method parameters is asking for 
> trouble.

I'll be more specific with the example I gave.  Here's an abridged API 
for a hierarchical structure that can be serialized:

   abstract class Box {
     List<Box> children;
     abstract InputStream getFieldContent();

     final InputStream getChildContent() {
       List<InputStream> streams = new ArrayList<>(children.size());
       for (Box child : children)
         streams.add(child.getContent());
       return new SequenceInputStream(Collections.enumeration(streams));
     }

     final InputStream getContent() {
       return new SequenceInputStream(getFieldContent(), getChildContent());
     }
   }

Several library-defined extensions are provided, implementing 
getFieldContent() in various useful ways.

Outside the library, there's a user creating a custom box type, making a 
hierarchy including it, and caching it:

   class MyAppSpecBox extends Box {
     InputStream getFieldContent() {
       ...
     }
   }

   // Create hierarchy out of library Box extensions.
   Box root = ... ;

   // Add the custom box type somewhere in the hierarchy.
   Box myBox = new MyAppSpecBox();
   root.children.get(2).children.get(1).children.add(myBox);

   cache.store(key, root);

Fetch it later, and serialize it:

   Box root = cache.fetch(key);
   InputStream in = root.getContent();

Suppose we want MyAppSpecBox.getFieldContent() to use a context which is 
known only at the point of fetching from the cache.  We don't control 
the Box API, and even if we did, we couldn't add an application-specific 
parameter to the getContent() family of methods.  How would we add a 
generic one, one that would be usable by several users independently and 
simultaneously (other than the Context<T> class I suggested, which is 
just a variation on ThreadLocal<T>)?

If we could locate myBox from root, we could pass the context to it 
after fetching.  However, traversing the full hierarchy or even knowing 
the correct path seem clumsy ways to locate it.  Also, its storage of 
the context would not be thread-safe.

So, we throw in a ThreadLocal:

   static ThreadLocal<Context> context = ...;

   class MyAppSpecBox extends Box {
     InputStream getFieldContent() {
       Context ctxt = context.get();
       ...
     }
   }

We set it before invoking the hierarchy:

   Box root = cache.fetch(key);
   Context ctxt = new Context(...);
   context.set(ctxt);
   InputStream in = root.getContent();


> Also, you need to be aware that the lifetime of these objects can be 
> quite long (there was a discussion about various aspects of 
> ThreadLocal in light of thread pools here earlier).

That use of ThreadLocal was preserving state from one 'prong' of the 
stack to the next, presumably with no way to inject a 
ThreadLocal.set(null) at a common vertex of those prongs.  This use of 
ThreadLocal only pushes values up the stack, which allows us to be more 
rigorous:

   Box root = cache.fetch(key);
   Context ctxt = new Context(...);
   context.set(ctxt);
   try {
     InputStream in = root.getContent();
   } finally {
     context.set(null);
   }


Cheers,

Steven

-- 
ss at comp dot lancs dot ac dot uk

[toc] | [prev] | [next] | [standalone]

#18570

Steven Simpson <ss@domain.invalid> wrote:
> So, we throw in a ThreadLocal:
>
>    static ThreadLocal<Context> context = ...;
>
>    class MyAppSpecBox extends Box {
>      InputStream getFieldContent() {
>        Context ctxt = context.get();
>        ...
>      }
>    }
>
> We set it before invoking the hierarchy:
>
>    Box root = cache.fetch(key);
>    Context ctxt = new Context(...);
>    context.set(ctxt);
>    InputStream in = root.getContent();

What strikes me odd in this scenario is the coincidence of
two things:
   There's *only one* specific Box-subclass, that requires 
      specific extra information
   The code that kicks off processing of the object tree does
      know about this specific need and does know how to
      cater to it specifically (and the specific piece of 
      information is even available at that place).

That looks really bolted (or duct-taped) on.

At some point in future, such things tend to accumulate.

Then either you end up with "invoking" code, that sets up
a number of different ThreadLocal<...>s for those specific
Boxes (that may or may not actually show up in the tree),
or you still set up only one such ThreadLocal<Context>,
and a couple of specific Boxes use it.

in the former case it seems like the Box abstraction
completely missed the point.

In the latter case, it seems as if it might then turn out to
be reasonable to modify the Box baseclass to pass along the
Context directly (even if some Boxes still don't need it):

class Box {
       InputStream getContent(Context ctx) {
              child-loop { ... child.getContext(ctx); }
              ... getFieldContent(ctx);
              ...
       }
       InputStream getFieldContent(Context ctx) {
           getFieldContent(); // default-impl
       }
       InputStream getFieldContent() {return null;}
}

subclasses that need the context would override getFieldContent(Context)
the others (like the old ones) would still just override getFieldContent().

[toc] | [prev] | [next] | [standalone]

#18571

On 06/09/12 10:32, Andreas Leitgeb wrote:
> Steven Simpson <ss@domain.invalid> wrote:
>> So, we throw in a ThreadLocal:
>>
>>     static ThreadLocal<Context> context = ...;
>>
>>     class MyAppSpecBox extends Box {
>>       InputStream getFieldContent() {
>>         Context ctxt = context.get();
>>         ...
>>       }
>>     }
>>
>> We set it before invoking the hierarchy:
>>
>>     Box root = cache.fetch(key);
>>     Context ctxt = new Context(...);
>>     context.set(ctxt);
>>     InputStream in = root.getContent();
> What strikes me odd in this scenario is the coincidence of
> two things:
>     There's *only one* specific Box-subclass, that requires
>        specific extra information
>     The code that kicks off processing of the object tree does
>        know about this specific need and does know how to
>        cater to it specifically (and the specific piece of
>        information is even available at that place).

The odd part is that the custom box is lost in the structure before a 
context can be supplied to it, which is the result of caching the 
structure, and only knowing the context just when recovering the 
structure from the cache.  That means you either find the custom box to 
inform it of the context, or you leave the context somewhere the box 
will look.

> Then either you end up with "invoking" code, that sets up
> a number of different ThreadLocal<...>s for those specific
> Boxes (that may or may not actually show up in the tree),
> or you still set up only one such ThreadLocal<Context>,
> and a couple of specific Boxes use it.
>
> in the former case it seems like the Box abstraction
> completely missed the point.

I don't get what you mean, but the point of the Box abstraction is just 
to handle the aspects common to all boxes: they have some 
box-type-specific fields, followed by zero or more nested boxes. It's 
not aware of the application-specific Context.

> In the latter case, it seems as if it might then turn out to
> be reasonable to modify the Box baseclass to pass along the
> Context directly (even if some Boxes still don't need it):
>
> class Box {
>         InputStream getContent(Context ctx) {

This is out of the question - the library to which Box belongs knows 
nothing about the application that defines Context.


Cheers!

-- 
ss at comp dot lancs dot ac dot uk

[toc] | [prev] | [next] | [standalone]

#18572

Steven Simpson <ss@domain.invalid> wrote:
> The odd part is that the custom box is lost in the structure before a 
> context can be supplied to it, which is the result of caching the 
> structure, and only knowing the context just when recovering the 
> structure from the cache.  That means you either find the custom box to 
> inform it of the context, or you leave the context somewhere the box 
> will look.

I still think, the requirement itself is probably screwed.

Such screwed requirements just happen, and if they happen, then a
screwed implementation may seem like the correct answer - until
something harmless happens, that crashes and burns the screwed
solution and leaves you in the void searching for an alternative
screwed solution that survives at least until next time...

Back to the Box-example: as the Box-framework may not expect the
getContent() call to depend on any outer local context, it might
evaluate it earlier, and just return a cached result on your call.
This of course will crash and burn your approach, as you built on
the unfounded assumption that each Box's getFieldContent() wouldn't
be called beforehand.  If the Box-framework really gave such a promise,
then it had better provide a context parameter already, lest it be
screwed itself.

[toc] | [prev] | [next] | [standalone]

#18574

On 06/09/12 16:10, Andreas Leitgeb wrote:
> Back to the Box-example: as the Box-framework may not expect the
> getContent() call to depend on any outer local context, it might
> evaluate it earlier, and just return a cached result on your call.

No, not in this case.  The whole point of this framework is to create an 
internal structured representation of a file whose format is defined in 
terms of nested boxes - i.e. there's a parsing/deserialization phase too 
- and to allow those boxes to be manipulated.  After that, the point of 
the getContent() method is to serialize the hierarchy in its present 
state; getContent() has no advantage to cache anything, because the 
hierarchy is not ready to be serialized until getContent() is called.  
Its implementation is quite transparent - getFieldContent() will be 
called on each box as if by a depth-first traversal.  It is therefore 
relatively easy to offer informal guarantees about how a user's 
getFieldContent() will be invoked by getContent(), and the finality of 
Box's methods helps to prevent that from being undermined by subclasses.

Also, the extra context I'm providing does not affect the functional 
behaviour of the box structure.  It's there to regulate the arrival of 
bytes, so I can have the next batch already requested before the current 
batch has finished.

(Actually, it seems I haven't got that regulation quite right, so I've 
disabled it for now.  But I can add and remove it without changes to the 
box framework.)


> If the Box-framework really gave such a promise,
> then it had better provide a context parameter already, lest it be
> screwed itself.

The possibility of being screwed only comes if the getContent or 
getChildContent methods decide to call getFieldContent() indirectly by 
starting another thread.  For this framework, I don't foresee it 
happening.  Even if it did, I might be able still to make a lesser 
guarantee, that such threads are descendants of the original thread, so 
an InheritableThreadLocal would still work.

If even that's no good, and an extra parameter is required, what should 
it be, since the framework has no knowledge of how many or what types of 
contexts are required?  I already suggested a (different) Context and 
ContextLocal<T> pair of classes, but that is almost a dual of the 
ThreadLocal technique, but with the extra parameter made explicit, 
instead of implicitly being Thread.currentThread().  In other 
circumstances with fewer or weaker guarantees, I would do that, but I 
don't foresee ThreadLocal being a problem for this box framework.


-- 
ss at comp dot lancs dot ac dot uk

[toc] | [prev] | [next] | [standalone]

#18575

Steven Simpson <ss@domain.invalid> wrote:
> On 06/09/12 16:10, Andreas Leitgeb wrote:
>> Back to the Box-example: as the Box-framework may not expect the
>> getContent() call to depend on any outer local context, it might
>> evaluate it earlier, and just return a cached result on your call.
> No, not in this case.  The whole point of this framework is ...

That was more than I really cared to know about that Box-framework ;-)

> The possibility of being screwed only comes if the getContent or 
> getChildContent methods decide to call getFieldContent() indirectly by 
> starting another thread.  For this framework, I don't foresee it 
> happening.  Even if it did, I might be able still to make a lesser 
> guarantee, that such threads are descendants of the original thread, so 
> an InheritableThreadLocal would still work.

Still an unsafe assumption: Only after you changed ThreadLocal to 
InheritedThreadLocal and start to feel safe again, the framework
would be optimized further, to use pool-threads, of course ;-)

> If even that's no good, and an extra parameter is required, what should 
> it be, since the framework has no knowledge of how many or what types of 
> contexts are required?

Some Context class that contains/references the complete last-minute
knowledge of the site where getContent() is called from. It should
be the same instance for the whole tree of boxes.

Even if some of the "possible framework changes" won't happen with your
Box framework, they might do so for whatever framework the OP uses.

[toc] | [prev] | [next] | [standalone]

#18539

On 04/09/12 02:12, Roedy Green wrote:
> I run into this sort of problem fairly often.
>
> I have a method x that calls method y through a long chain of
> intermediate calls.
>
> I decide to add a feature.  Method x has access to information to
> decide if the feature should be implemented.  Method y is where the
> implementation is.
>

What about Properties? They might provide a suitably "global" solution 
without too much coupling. A class with its own internal static 
Properties variable, and static methods to set/get the properties. Then 
method x can set any property it likes, and method y can attempt to read 
any property it likes. Unset properties return null, or a default value 
if you implement a getProperty(key, default) method.

-- 
Nigel Wade

[toc] | [prev] | [next] | [standalone]

#18540

On 9/3/12 6:12 PM, Roedy Green wrote:
> I run into this sort of problem fairly often.
>
> I have a method x that calls method y through a long chain of
> intermediate calls.
>
> I decide to add a feature.  Method x has access to information to
> decide if the feature should be implemented.  Method y is where the
> implementation is.
>
> Somehow I must pass information down the long chain from x to y.  This
> means changing the signatures of all the intermediate methods, and
> adjusting code to the new way.  This can cause ripples incommensurate
> with the triviality of the change.
>
> I imagined some sort of wormhole to allow information in one place to
> simply be known somewhere else without the excess exposure of public
> variables or tediously arranging transport.  I read about Eiffel a
> long time ago. IIRC you could provide a list of precisely who could
> see any given information.
>
> I wondered if anyone had thoughts on ways of tackling such a problem.
>

If you arrange to pass-through a Parameter Object instead of a bunch of 
related parameters, this could solve your problem in many (not all) cases.

The primary benefit of a Parameter Object is that you can add new fields 
(if they have sensible defaults) without a rippling change to all 
existing users of the method which takes the Parameter Object.

There are other potential benefits as well. Your Parameter Object might 
have a nice "Builder" counterpart which makes it easier to specify 
optional parameters (using call-chaining for example).

A special case of the Parameter Object is often called the Context. 
Many types of frameworks pass around a Context object to all methods. 
The HttpServletRequest object is such an example (a.k.a The Request 
Context). This is a more generic form than Parameter Object, and I don't 
suggest it for most usages, as you can end up with name-collisions, type 
problems, and other problems that can't be diagnosed at compile time.

[toc] | [prev] | [next] | [standalone]

#18546

On 03/09/2012 9:12 PM, Roedy Green wrote:
> I run into this sort of problem fairly often.
>
> I have a method x that calls method y through a long chain of
> intermediate calls.
>
> I decide to add a feature.  Method x has access to information to
> decide if the feature should be implemented.  Method y is where the
> implementation is.

Are these methods in many separate classes and objects, or just one? If 
x, y, and all the intermediate c1..cN methods are in the same class, 
it's pretty easy to create a private (or protected) variable to be set 
by x and used by y (and presumably other equivalents of y if x has to 
invoke a different long chain of intermediate calls). Though this isn't 
thread-safe if multiple threads can call x.

If y is a method of a separate class, there ought to be some way to 
return the object z on which y would be invoked and have x invoke 
z.y(...). But perhaps for some reason you can't revise the code that way.

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.lang.java.programmer

csiph-web