Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #29460 > unrolled thread

dealing with ClassLoaders...

Back to article view | Back to comp.lang.java.programmer

Contents

  dealing with ClassLoaders... Andreas Leitgeb <avl@auth.logic.tuwien.ac.at> - 2014-07-31 19:40 +0000
    Re: dealing with ClassLoaders... Martin Gregorie <martin@address-in-sig.invalid> - 2014-07-31 20:44 +0000
      Re: dealing with ClassLoaders... Andreas Leitgeb <avl@auth.logic.tuwien.ac.at> - 2014-08-01 14:22 +0000
        Re: dealing with ClassLoaders... Andreas Leitgeb <avl@auth.logic.tuwien.ac.at> - 2014-08-01 21:10 +0000
          Re: dealing with ClassLoaders... Eric Sosman <esosman@comcast-dot-net.invalid> - 2014-08-01 17:32 -0400
            Re: dealing with ClassLoaders... Joerg Meier <joergmmeier@arcor.de> - 2014-08-02 01:21 +0200
            Re: dealing with ClassLoaders... Andreas Leitgeb <avl@auth.logic.tuwien.ac.at> - 2014-08-02 00:00 +0000
          Re: dealing with ClassLoaders... Martin Gregorie <martin@address-in-sig.invalid> - 2014-08-01 21:45 +0000
            Re: dealing with ClassLoaders... Andreas Leitgeb <avl@auth.logic.tuwien.ac.at> - 2014-08-01 23:14 +0000
              Re: dealing with ClassLoaders... Martin Gregorie <martin@address-in-sig.invalid> - 2014-08-02 00:35 +0000
                Re: dealing with ClassLoaders... Andreas Leitgeb <avl@auth.logic.tuwien.ac.at> - 2014-08-05 15:38 +0000
    Re: dealing with ClassLoaders... Silvio <silvio@internet.com> - 2014-08-01 13:31 +0200
      Re: dealing with ClassLoaders... Silvio <silvio@internet.com> - 2014-08-01 13:37 +0200
        Re: dealing with ClassLoaders... Andreas Leitgeb <avl@auth.logic.tuwien.ac.at> - 2014-08-01 16:06 +0000
    Re: dealing with ClassLoaders... "Chris Uppal" <chris.uppal@metagnostic.REMOVE-THIS.org> - 2014-08-01 13:19 +0100
      Re: dealing with ClassLoaders... Andreas Leitgeb <avl@auth.logic.tuwien.ac.at> - 2014-08-01 15:51 +0000
    Re: dealing with ClassLoaders... Josip Almasi <joe@vrspace.org> - 2014-08-01 15:55 +0200
      Re: dealing with ClassLoaders... Andreas Leitgeb <avl@auth.logic.tuwien.ac.at> - 2014-08-04 15:12 +0000
        Re: dealing with ClassLoaders... Mike Amling <mamling@chaff.us> - 2014-08-04 19:53 -0500
          Re: dealing with ClassLoaders... Andreas Leitgeb <avl@auth.logic.tuwien.ac.at> - 2014-08-05 15:16 +0000
        Re: dealing with ClassLoaders... Josip Almasi <joe@vrspace.org> - 2014-08-08 11:53 +0200
          Re: dealing with ClassLoaders... Andreas Leitgeb <avl@auth.logic.tuwien.ac.at> - 2014-08-08 18:46 +0000
            Re: dealing with ClassLoaders... Josip Almasi <joe@vrspace.org> - 2014-08-10 13:22 +0200
              Re: dealing with ClassLoaders... Andreas Leitgeb <avl@auth.logic.tuwien.ac.at> - 2014-08-12 17:16 +0000

Page 1 of 2  [1] 2  Next page →

#29460 — dealing with ClassLoaders...

All the classes of my app are principially loadable by the system's
default ClassLoader (i.e. they are in the java.class.path).  Some
Main class (mypackage.Main) of my app will even be loaded, and this
class will then create a new Thread with a custom ClassLoader that
is supposed to load all the other used classes of my app's packages
(while still delegating loading of newly used system classes to its
parent).

The classloader tutorials I skimmed so far seem to imply, that
a custom classloader is only appropriate to enable classloading
from sources that the system classloader cannot handle itself.

Is there a clean way to keep classes of a given package confined
to some custom classloader, in order to control the classes' and
their statics' scope/lifetime?

The outset is, that some of these classes contain static data.
(They had been designed as standalone application classes.)
Now, to improve invocation time, one VM is supposed to run all
the program instances. This is effectively a followup to a 
previous c.l.j.p thread of mine, where I asked about a certain
kind of "application servers".

If this is a no-go, then I'll have to redesign those classes to
get rid of static data, but that will be some effort and require
passing around context references or use ThreadLocal stuff, so
my hope is for classloaders to provide me with an easier way out.

[toc] | [next] | [standalone]

#29462

On Thu, 31 Jul 2014 19:40:18 +0000, Andreas Leitgeb wrote:

> All the classes of my app are principially loadable by the system's
> default ClassLoader (i.e. they are in the java.class.path).  Some Main
> class (mypackage.Main) of my app will even be loaded, and this class
> will then create a new Thread with a custom ClassLoader that is supposed
> to load all the other used classes of my app's packages (while still
> delegating loading of newly used system classes to its parent).
> 
> The classloader tutorials I skimmed so far seem to imply, that a custom
> classloader is only appropriate to enable classloading from sources that
> the system classloader cannot handle itself.
>
This sounds similar to something I've done a couple of times now when I 
wanted to put a number of related but separate programs and their 
supporting classes into a jar file and then, at runtime, select the one 
to be run via the command line.

The manifest Main-Class names single, top level launcher class. This is  
passed the name of the program to be run plus any options and arguments 
it needs via the argv[] array. The launcher class then does the following 
to launch the program:

   try
   {
      String fullAppName = PACKAGE + "." + argv[0];
      MAClassLoader  loader = new MAClassLoader();
      Class          target = loader.loadClass(fullAppName);
      MAApplication  app = (MAApplication)target.newInstance();
      String argList[] = new String[argv.length - 1];
      for (int i = 1; i < argv.length; i++)
         argList[i-1] = argv[i];
            
      app.run(argList);
   }
   catch (Exception e)
   {
      System.out.stderr.println(e.toString());
   }

I've omitted the rest of main() because it only does obvious, application-
specific stuff like showing help text when requested, validating the 
program name, etc. The program being run is responsible for validating 
the arguments passed to it.

MAApplication is a superclass which is extended by all the programs that 
can be run by the launcher. In this case they are all members of the same 
package so that the user only needs to input the program name. Here is 
MAClassLoader in its entirety:

package ma.internals;

/**
 * MAClassLoader loads the application.
 *
 * The only method used from the abstract parent ClassLoader
 * class is loadClass(String). As it is not abstract and
 * doesn't need to be overridden, this class definition is null.
 *
 * Its only purpose is to convert the abstract ClassLoader
 * declaration into a non-abstract class.
 */
public class MAClassLoader extends ClassLoader
{
   
}

I run programs with this type of command:

   java -jar /usr/java/jarlib/jarfile.jar program arguments....

where 'program' is the program to be run and the arguments will be passed 
to it by the launcher.

I hope this is useful info.


-- 
martin@   | Martin Gregorie
gregorie. | Essex, UK
org       |

[toc] | [prev] | [next] | [standalone]

#29486

Martin Gregorie <martin@address-in-sig.invalid> wrote:
> On Thu, 31 Jul 2014 19:40:18 +0000, Andreas Leitgeb wrote:
>> All the classes of my app are principially loadable by the system's
>> default ClassLoader (i.e. they are in the java.class.path).  Some Main
>> class (mypackage.Main) of my app will even be loaded, and this class
>> will then create a new Thread with a custom ClassLoader that is supposed
>> to load all the other used classes of my app's packages (while still
>> delegating loading of newly used system classes to its parent).
> This sounds similar to something I've done a couple of times now when I 
> wanted to put a number of related but separate programs and their 
> supporting classes into a jar file and then, at runtime, select the one 
> to be run via the command line.
> The manifest Main-Class names single, top level launcher class. This is  
> passed the name of the program to be run plus any options and arguments 
> it needs via the argv[] array. The launcher class then does the following 
> to launch the program:

  [ snip ]

> public class MAClassLoader extends ClassLoader
> {
  [ no snip ]
> }

> I hope this is useful info.

Yes, it was indeed useful information for me.
I hadn't realized, that ClassLoader already has all
the stuff in it. I feared the boring stuff (such
as obtaining byte[] out of jar-files) would be in
internal non-public classes.

I'll likely still override some of its methods, first, to make
sure that all other than my classes are still delegated to the
parent classloader, and second, to eventually cache the byte[]
definition of some of my classes that I need separately loaded,
but that will likely be used in many tasks at once.

In addition, I'll also pick the low hanging fruit among my
classes and un-static'ify or at least ThreadLocal'ize them so
I can have as many classes as possible loaded centrally.

Thanks a lot for your answer!

[toc] | [prev] | [next] | [standalone]

#29496

Andreas Leitgeb <avl@auth.logic.tuwien.ac.at> wrote:
> Martin Gregorie <martin@address-in-sig.invalid> wrote:
>> public class MAClassLoader extends ClassLoader
>> {
>   [ no snip ]
>> }
>> I hope this is useful info.

On further examination, I think it isn't as simple as that.
Essentially, this "empty" ClassLoader-subclass would pass 
everything through to the parent classloader (or the pre-
mordial one, absent a parent) and thus not be associated to
any of the classes loaded through it, thus provide no
isolation at all.

My approach now is to override loadClass(String,boolean):

   @Override // ClassLoader
   public Class<?> loadClass( String name, boolean resolve)
      throws ClassNotFoundException
   {
      Class<?> res = null;
      if (name != null && name.startsWith(PACKAGE_PREFIX)) {
         synchronized (this) {
            res = m_classes.get(name);
            if (res != null) { return res; }
            byte[] classData = null; // TODO: get class data
            res = defineClass(name,classData, 0, classData.length);
            m_classes.put(name, res);
         }
      } else { 
         // if this fails, then so be it:
         res = super.findSystemClass(name);
      }
      if (resolve && res != null) { resolveClass(res); }
      return res;
   }
   // m_classes is Map<String,Class<?>> initialized to new HashMap<>()

The hole that remains is at the TODO marker in the code snippet.
I'd like to use system code to pull the byte[] or ByteBuffer out
of whatever jar-file or unpacked directory the whole app is deployed
to.

The then-branch of the big "if" is to solve a should-be-simple task:
Use the system class loader's code to locate the class, but let "this"
classloader end up being associated with it.

How can I do that?

[toc] | [prev] | [next] | [standalone]

#29498

On 8/1/2014 5:10 PM, Andreas Leitgeb wrote:
> Andreas Leitgeb <avl@auth.logic.tuwien.ac.at> wrote:
>> Martin Gregorie <martin@address-in-sig.invalid> wrote:
>>> public class MAClassLoader extends ClassLoader
>>> {
>>    [ no snip ]
>>> }
>>> I hope this is useful info.
>
> On further examination, I think it isn't as simple as that.
> Essentially, this "empty" ClassLoader-subclass would pass
> everything through to the parent classloader (or the pre-
> mordial one, absent a parent) and thus not be associated to
> any of the classes loaded through it, thus provide no
> isolation at all.

     That conclusion may be worth re-examining.  Just because
MAClassLoader is a subclass of ClassLoader and uses methods
of the latter to do all its dirty work, it doesn't mean the
classes aren't loaded by MAClassLoader.  I'd suggest a quick
experiment: Load a class with MAClassLoader, instantiate an
instance thereof, and see what inst.getClass().getClassLoader()
says.

     It must be admitted, though, that ClassLoader is a bit
weird.  An abstract class all of whose methods are concrete?
Bizarre ...

     FWIW, I have successfully used

	private static class Loader extends ClassLoader {
	   private Class<?> load(byte[] classFile) {
	      Class<?> clown = defineClass(
	         null, classFile, 0, classFile.length);
	      resolveClass(clown);
	      return clown;
	   }
	}

... to load many distinct classes with identical package and
class names, which suggests there's at least some degree of
isolation present even though all the actual work is done by
ClassLoader itself.

-- 
esosman@comcast-dot-net.invalid

[toc] | [prev] | [next] | [standalone]

#29501

On Fri, 01 Aug 2014 17:32:40 -0400, Eric Sosman wrote:

>      That conclusion may be worth re-examining.  Just because
> MAClassLoader is a subclass of ClassLoader and uses methods
> of the latter to do all its dirty work, it doesn't mean the
> classes aren't loaded by MAClassLoader.  I'd suggest a quick
> experiment: Load a class with MAClassLoader, instantiate an
> instance thereof, and see what inst.getClass().getClassLoader()
> says.

Why not go ahead and text his actual use case instead ? class A { static {
System.out.println("I'm being loaded!"); } }, load it with MAClassLoader,
make a second MAClassLoader and load it again. Observe the console.

Liebe Gruesse,
		Joerg

-- 
Ich lese meine Emails nicht, replies to Email bleiben also leider
ungelesen.

[toc] | [prev] | [next] | [standalone]

#29502

Eric Sosman <esosman@comcast-dot-net.invalid> wrote:
> On 8/1/2014 5:10 PM, Andreas Leitgeb wrote:
>> Andreas Leitgeb <avl@auth.logic.tuwien.ac.at> wrote:
>>> Martin Gregorie <martin@address-in-sig.invalid> wrote:
>>>> public class MAClassLoader extends ClassLoader
>>>> {
>>>    [ no snip ]
>>>> }
>>>> I hope this is useful info.
>> On further examination, I think it isn't as simple as that.
>> Essentially, this "empty" ClassLoader-subclass would pass
>> everything through to the parent classloader (or the pre-
>> mordial one, absent a parent) and thus not be associated to
>> any of the classes loaded through it, thus provide no
>> isolation at all.
>
>      That conclusion may be worth re-examining.  Just because
> MAClassLoader is a subclass of ClassLoader and uses methods
> of the latter to do all its dirty work, it doesn't mean the
> classes aren't loaded by MAClassLoader.

You're right about the "Just because ...": That wasn't the ground for 
my claim. Instead, I drew my conclusion from what ClassLoader's
javadoc says about its own default implementation of findClass(),
namely: "The default implementation throws a ClassNotFoundException."

This (plus the documentation of loadClass()) means that any classname
passed to that class's loadClass() either gets loaded by its parent (not
in terms of inheritence, but in terms of ClassLoader.getParent()!) or it
inevitably will fail to be loaded at all.

There is a big difference between merely letting some superclass methods
do some work (which could still affect "this" instance), versus knowing
(from docs) that these superclass methods' work is merely delegating work
to a different *instance*, namely the ClassLoader's getParent().

>      It must be admitted, though, that ClassLoader is a bit
> weird.  An abstract class all of whose methods are concrete?

I think that's to allow the choice for which method to actually override,
without having to care about then-never-used dummy implementations of other
methods.

Normally, one would override findClass(), but if one overrides loadClass()
instead, and doesn't need findClass() in his implementation, then he doesn't
need to "implement" findClass().

>      FWIW, I have successfully used
> 	private static class Loader extends ClassLoader {
> 	   private Class<?> load(byte[] classFile) {
> 	      Class<?> clown = defineClass(
> 	         null, classFile, 0, classFile.length);
> 	      resolveClass(clown);
> 	      return clown;
> 	   }
> 	}
This is obviously quite different from Martin's MAClassLoader, in 
that your Loader instance itself does the defineClass() call and
thus the loaded class is associated with your Loader instance.

> ... to load many distinct classes with identical package and
> class names, which suggests there's at least some degree of
> isolation present even though all the actual work is done by
> ClassLoader itself.

In your code work is done in the instance itself, not delegated
to a *different* ClassLoader-*instance*.

[toc] | [prev] | [next] | [standalone]

#29499

On Fri, 01 Aug 2014 21:10:07 +0000, Andreas Leitgeb wrote:

> Andreas Leitgeb <avl@auth.logic.tuwien.ac.at> wrote:
>> Martin Gregorie <martin@address-in-sig.invalid> wrote:
>>> public class MAClassLoader extends ClassLoader {
>>   [ no snip ]
>>> }
>>> I hope this is useful info.
> 
> On further examination, I think it isn't as simple as that. Essentially,
> this "empty" ClassLoader-subclass would pass everything through to the
> parent classloader (or the pre-
> mordial one, absent a parent) and thus not be associated to any of the
> classes loaded through it, thus provide no isolation at all.
>
Your requirements would seem to be as bit different from mine, which were 
purely to package a number of programs (some Swing apps, some pure 
command-line apps, all accessing a single PostgreSQL database instance) 
into one convenient file and to be able to run them without needing to 
unpack the distribution file. There is almost never any need to run more 
than one of these programs at a time by any user/on any PC. So, as none 
of these ever share a JVM the most convenient way to do this was to pack 
them all into a jar file and use that very simple class loader as a way 
to select and run one of the programs in it. I do not have any of the 
your process isolation requirements which is why I can get away with such 
a simple class loader. Once the top level class defining the program to 
be run has been loaded I don't care what class loader is used to pull in 
the other classes it needs - in fact I'd be surprised if the default 
class loader *wasn't* used for that.  


-- 
martin@   | Martin Gregorie
gregorie. | Essex, UK
org       |

[toc] | [prev] | [next] | [standalone]

#29500

Martin Gregorie <martin@address-in-sig.invalid> wrote:
> On Fri, 01 Aug 2014 21:10:07 +0000, Andreas Leitgeb wrote:
>> Andreas Leitgeb <avl@auth.logic.tuwien.ac.at> wrote:
>>> Martin Gregorie <martin@address-in-sig.invalid> wrote:
>>>> public class MAClassLoader extends ClassLoader {
>>>   [ no snip ]
>>>> }
>>>> I hope this is useful info.
>> On further examination, I think it isn't as simple as that. Essentially,
>> this "empty" ClassLoader-subclass would pass everything through to the
>> parent classloader (or the pre-mordial one, absent a parent) and thus
>> not be associated to any of the classes loaded through it, thus provide no
>> isolation at all.
> Your requirements would seem to be as bit different from mine, which were 
> purely to package a number of programs (some Swing apps, some pure 
> command-line apps, all accessing a single PostgreSQL database instance) 
> into one convenient file and to be able to run them without needing to 
> unpack the distribution file.

That's the point. In your case there was no need for isolation, and
there wasn't really a point in creating this MAClassLoader at all.
Merely loading the starting class could have been done with Class.forName()
thus without any custom classloader. Essentially, what I had before the
requirement for multiple apps per VM came up.

I still appreciate that you posted it: That I got so far as to understand
what MAClassLoader does, was an important first learning step for me.

> Once the top level class defining the program to 
> be run has been loaded I don't care what class loader is used to pull in 
> the other classes it needs - in fact I'd be surprised if the default 
> class loader *wasn't* used for that.  

The default classloader *was* used for the other classes (no surprise so far),
but it was also used for that one dynamic starting class (which didn't matter
in your case, but may still be a theoretic surprise now in hindsight).

[toc] | [prev] | [next] | [standalone]

#29503

On Fri, 01 Aug 2014 23:14:01 +0000, Andreas Leitgeb wrote:

> That's the point. In your case there was no need for isolation, and
> there wasn't really a point in creating this MAClassLoader at all.
> Merely loading the starting class could have been done with
> Class.forName() thus without any custom classloader. Essentially, what I
> had before the requirement for multiple apps per VM came up.
>
I spent a few minutes trying to remember why I extended ClassLoader at 
all: the exact reason that now escapes me but I'm fairly sure that I was 
unable to load of the class via Class.forName(), probably because the 
classes I wanted to load and run are never explicitly instantiated within 
the outermost program, which IIRC was written in 2008, so that must have 
been Java 5 - it was certainly during the pre-Generics era. I don't 
usually write or extend library code unless I have to!  

Does this make any sort of sense?


-- 
martin@   | Martin Gregorie
gregorie. | Essex, UK
org       |

[toc] | [prev] | [next] | [standalone]

#29569

Martin Gregorie <martin@address-in-sig.invalid> wrote:
> On Fri, 01 Aug 2014 23:14:01 +0000, Andreas Leitgeb wrote:
>> That's the point. In your case there was no need for isolation, and
>> there wasn't really a point in creating this MAClassLoader at all.
>> Merely loading the starting class could have been done with
>> Class.forName() thus without any custom classloader. Essentially, what I
>> had before the requirement for multiple apps per VM came up.
> I spent a few minutes trying to remember why I extended ClassLoader at 
> all: the exact reason that now escapes me but I'm fairly sure that I was 
> unable to load of the class via Class.forName(), probably because the 
> classes I wanted to load and run are never explicitly instantiated within 
> the outermost program, which IIRC was written in 2008, so that must have 
> been Java 5 - it was certainly during the pre-Generics era. I don't 
> usually write or extend library code unless I have to!  
> Does this make any sort of sense?

I'm not enough into Java-Archaeology to analyze whether back in 2008
things were different enough to have required that solution. :-)

Also the apparent conflict between "Java 5" and "pre-Generics era"
might have an explanation outside the scope of pure technology.

Whatever it was, your sample helped me wrap my head around these
classloaders.

Thanks!

[toc] | [prev] | [next] | [standalone]

#29482

On 07/31/2014 09:40 PM, Andreas Leitgeb wrote:
> All the classes of my app are principially loadable by the system's
> default ClassLoader (i.e. they are in the java.class.path).  Some
> Main class (mypackage.Main) of my app will even be loaded, and this
> class will then create a new Thread with a custom ClassLoader that
> is supposed to load all the other used classes of my app's packages
> (while still delegating loading of newly used system classes to its
> parent).
>
> The classloader tutorials I skimmed so far seem to imply, that
> a custom classloader is only appropriate to enable classloading
> from sources that the system classloader cannot handle itself.
>
> Is there a clean way to keep classes of a given package confined
> to some custom classloader, in order to control the classes' and
> their statics' scope/lifetime?
>
> The outset is, that some of these classes contain static data.
> (They had been designed as standalone application classes.)
> Now, to improve invocation time, one VM is supposed to run all
> the program instances. This is effectively a followup to a
> previous c.l.j.p thread of mine, where I asked about a certain
> kind of "application servers".
>
> If this is a no-go, then I'll have to redesign those classes to
> get rid of static data, but that will be some effort and require
> passing around context references or use ThreadLocal stuff, so
> my hope is for classloaders to provide me with an easier way out.
>

Get rid of the static data. By definition static data limits how classes 
can be used. Just to get around using a context reference is not worth 
opening the can of worms they introduce. Threadlocals should only be 
used if all else fails which is usually the case when you have to write 
callback code that is called by a third party library that you can not 
modify to have it pass along context.

[toc] | [prev] | [next] | [standalone]

#29483

On 08/01/2014 01:31 PM, Silvio wrote:
> On 07/31/2014 09:40 PM, Andreas Leitgeb wrote:
>> All the classes of my app are principially loadable by the system's
>> default ClassLoader (i.e. they are in the java.class.path).  Some
>> Main class (mypackage.Main) of my app will even be loaded, and this
>> class will then create a new Thread with a custom ClassLoader that
>> is supposed to load all the other used classes of my app's packages
>> (while still delegating loading of newly used system classes to its
>> parent).
>>
>> The classloader tutorials I skimmed so far seem to imply, that
>> a custom classloader is only appropriate to enable classloading
>> from sources that the system classloader cannot handle itself.
>>
>> Is there a clean way to keep classes of a given package confined
>> to some custom classloader, in order to control the classes' and
>> their statics' scope/lifetime?
>>
>> The outset is, that some of these classes contain static data.
>> (They had been designed as standalone application classes.)
>> Now, to improve invocation time, one VM is supposed to run all
>> the program instances. This is effectively a followup to a
>> previous c.l.j.p thread of mine, where I asked about a certain
>> kind of "application servers".
>>
>> If this is a no-go, then I'll have to redesign those classes to
>> get rid of static data, but that will be some effort and require
>> passing around context references or use ThreadLocal stuff, so
>> my hope is for classloaders to provide me with an easier way out.
>>
>
> Get rid of the static data. By definition static data limits how classes
> can be used. Just to get around using a context reference is not worth
> opening the can of worms they introduce. Threadlocals should only be
> used if all else fails which is usually the case when you have to write
> callback code that is called by a third party library that you can not
> modify to have it pass along context.
>

I mean the can of worms class loaders introduce.

[toc] | [prev] | [next] | [standalone]

#29491

Silvio <silvio@internet.com> wrote:
> On 08/01/2014 01:31 PM, Silvio wrote:
>> On 07/31/2014 09:40 PM, Andreas Leitgeb wrote:
>>> Is there a clean way to keep classes of a given package confined
>>> to some custom classloader, in order to control the classes' and
>>> their statics' scope/lifetime?
>>
>> Get rid of the static data. By definition static data limits how classes
>> can be used. Just to get around using a context reference is not worth
>> opening the can of worms they introduce. Threadlocals should only be
>> used if all else fails which is usually the case when you have to write
>> callback code that is called by a third party library that you can not
>> modify to have it pass along context.
> I mean the can of worms class loaders introduce.

I agree on the ideal, but it'll take some time until arrive there...

Thanks for the answer!

[toc] | [prev] | [next] | [standalone]

#29484

Andreas Leitgeb wrote:

> Is there a clean way to keep classes of a given package confined
> to some custom classloader, in order to control the classes' and
> their statics' scope/lifetime?

It's a long time since I last messed with classloaders, but from memory the 
only safe way to ensure that your custom-classloader classes aren't loaded from 
the system default classloaders is to ensure that they are not on the classpath 
(or anywhere else the "main" classloaders look).

> The outset is, that some of these classes contain static data.
> (They had been designed as standalone application classes.)
> Now, to improve invocation time, one VM is supposed to run all
> the program instances.

If you just let your main "server" process load all the classes upfront (no 
custom classloaders at all) then you'll take the initialisation time hit just 
once as it stars.  If you use custom classloaders with one for each "client" 
you'll take a partial hit for every client instance, and you'll lose 
data-sharing between instances.  From what I remember of your earlier thread 
you wanted extremely fast startup for the "clients", in which case the 
just-get-it-over-with-when-the-server-starts approach might well make more 
sense (and be simpler too).

    -- chris 

[toc] | [prev] | [next] | [standalone]

#29490

Chris Uppal <chris.uppal@metagnostic.REMOVE-THIS.org> wrote:
> Andreas Leitgeb wrote:
>> Is there a clean way to keep classes of a given package confined
>> to some custom classloader, in order to control the classes' and
>> their statics' scope/lifetime?
> It's a long time since I last messed with classloaders, but from memory the 
> only safe way to ensure that your custom-classloader classes aren't loaded from 
> the system default classloaders is to ensure that they are not on the classpath 
> (or anywhere else the "main" classloaders look).

Well, I've heard that there were some crocodiles in the custom-classloader
swamp, but for now I'm positive that I can avoid getting eaten.

Static fields are a programming convenience at a highly volatile price.
One may get away with a peanuts-price if one is lucky, but eventually the
price may rise on each drop of an original assumption.

I had to drop some assumptions, but not yet all. As of now I think I can
still afford the price of some static fields, and will try to get rid of 
most others.  Eventually I even might be able to get rid of the rest
until I reach the ideal of no static mutable state. It just won't happen
at once, and I'll need custom classloaders during that (open-ended)
transition.

>> The outset is, that some of these classes contain static data.
>> (They had been designed as standalone application classes.)
>> Now, to improve invocation time, one VM is supposed to run all
>> the program instances.
>
> If you just let your main "server" process load all the classes upfront (no 
> custom classloaders at all) then you'll take the initialisation time hit just 
> once as it stars.

Indeed, but that would pre-require completion of abovementioned transition.

>  If you use custom classloaders with one for each "client" 
> you'll take a partial hit for every client instance, and you'll lose 
> data-sharing between instances.

data-sharing between clients is not an issue in my case, and I expect the
partial hit to gradually disappear along the path of transition.

>   From what I remember of your earlier thread 
> you wanted extremely fast startup for the "clients", in which case the 
> just-get-it-over-with-when-the-server-starts approach might well make more 
> sense (and be simpler too).

Extremely fast... yes, who doesn't? I think for now the jvm itself is the
bottleneck here, and those few extra classes loaded for a new client
will be neglectible for the time being...

Thanks for your answer, especially for the heads-up w.r.t dangers with
classloaders (classes accidentally getting loaded through system classloader).
I'd better afford some assertions in my code... kind of tripwires in some
classes that will ring the alarm-bells, if their classloader happens to be
identical to getSystemClassLoader().  The one thing worse than the problem
itself is not being aware of it happening.

[toc] | [prev] | [next] | [standalone]

#29485

On 07/31/2014 09:40 PM, Andreas Leitgeb wrote:
> All the classes of my app are principially loadable by the system's
> default ClassLoader (i.e. they are in the java.class.path).  Some
> Main class (mypackage.Main) of my app will even be loaded, and this
> class will then create a new Thread with a custom ClassLoader that
> is supposed to load all the other used classes of my app's packages
> (while still delegating loading of newly used system classes to its
> parent).
>
> The classloader tutorials I skimmed so far seem to imply, that
> a custom classloader is only appropriate to enable classloading
> from sources that the system classloader cannot handle itself.

Nah, it's also appropriate for context isolation.

> Is there a clean way to keep classes of a given package confined
> to some custom classloader, in order to control the classes' and
> their statics' scope/lifetime?

Yep, just don't break delegation model.
Model says something like, a classloader should ask parent classloader 
if it has already loaded the class.
So, make sure you respect that rule.
Also, make sure your loaders don't load the same packages.
Other than this, it's just
if (loaderPackage.equals(classPackage)) loadClass(cls);

BTW if you want to unload/reload classes, you need to remove reference 
to classloader that loaded them.

> The outset is, that some of these classes contain static data.
> (They had been designed as standalone application classes.)
> Now, to improve invocation time, one VM is supposed to run all
> the program instances. This is effectively a followup to a
> previous c.l.j.p thread of mine, where I asked about a certain
> kind of "application servers".
>
> If this is a no-go, then I'll have to redesign those classes to
> get rid of static data, but that will be some effort and require
> passing around context references or use ThreadLocal stuff, so
> my hope is for classloaders to provide me with an easier way out.

Yep, get rid of statics.
In the context of above, I'd say your custom classloader is logical 
source for singleton.

Regards...

[toc] | [prev] | [next] | [standalone]

#29557

Josip Almasi <joe@vrspace.org> wrote:
> On 07/31/2014 09:40 PM, Andreas Leitgeb wrote:
>> The classloader tutorials I skimmed so far seem to imply, that
>> a custom classloader is only appropriate to enable classloading
>> from sources that the system classloader cannot handle itself.
> Nah, it's also appropriate for context isolation.

But it seems like it doesn't really make it easy to use it
that way.

>> Is there a clean way to keep classes of a given package confined
>> to some custom classloader, in order to control the classes' and
>> their statics' scope/lifetime?
> Yep, just don't break delegation model.
> Model says something like, a classloader should ask parent classloader 
> if it has already loaded the class.
> So, make sure you respect that rule.

The twist in my case comes from the fact, that the parent *could* 
theoretically have loaded one of my special classes, but that would
just indicate a programming error (aka Bug), so I'd rather have the
compiler or at least the VM report a strange ClassCastException
than silently use one of my classes loaded by wrong classloader.

> Also, make sure your loaders don't load the same packages.
This I don't understand.
While my "special classes" happen to be in a specific package,
I don't see, why packages as such would be relevant. I'd expect
that even within the same package I should (in principle) be 
allowed to have some classes loaded through parent and others
through custom classloader.


My latest hack of loadClass now looks like this:

   @Override
   public Class<?> loadClass(String name, boolean resolve)
         throws ClassNotFoundException
   {
      Class<?> res = null;
      if (name != null && name.startsWith(PACKAGE_PREFIX)) {
         synchronized (this) {
            res = m_classes.get(name);
            if (res != null) { return res; }
            // name -> classData
            byte[] classData = new byte[1024*1024*10]; int len = 0;
            URL url = getSystemResource(name.replace('.', '/') + ".class");
            try {
               len = url.openConnection().getInputStream().read(classData);
            } catch (IOException e) {
               e.printStackTrace();
            }
            // if anything went wrong, let defineClass complain...
            res = defineClass(name, classData, 0, len);
            if (resolve && res != null) { resolveClass(res); }
            m_classes.put(name, res);
            return res;
         }
      } else {
         return super.loadClass(name,resolve);
      }
   }

This appears to work (from watching the debugger pass through it),
although it is obviously quick'n'dirty. Most obviously, I'd like 
to get rid of the 10M-sized array, but I don't think I can ask the
URL or the Stream about the exact size beforehand.

I'd welcome hints for how to improve the "classname -> byte[]" step.

> Yep, get rid of statics.
On my TODO list for later.

Thanks for your reply.

[toc] | [prev] | [next] | [standalone]

#29562

On 8/4/14 10:12 AM, Andreas Leitgeb wrote:
>              URL url=getSystemResource(name.replace('.', '/') + ".class");
>              byte[] classData = new byte[1024*1024*10]; int len = 0;
>              try {
>                 len = url.openConnection().getInputStream().read(classData);
>              } catch (IOException e) {
>                 e.printStackTrace();
>              }
     You could buffer to a ByteArrayOutputStream.

   InputStream source=getSystemResource(name.replace('.', '/') + 
".class").openConnection().getInputStream();
   ByteArrayOutputStream buffer=new ByteArrayOutputStream();
   try {
      for (int input; (input=source.read())>=0; ) {
         buffer.write(input);
      }
   } catch (IOException i) {
      Whatever
   }
   byte[] classData=buffer.toByteArray();

> This appears to work (from watching the debugger pass through it),
> although it is obviously quick'n'dirty. Most obviously, I'd like
> to get rid of the 10M-sized array, but I don't think I can ask the
> URL or the Stream about the exact size beforehand.

--Mike Amling
T25lIHJlYWQgZnJvbSBVUkwub3BlbkNvbm5lY3
Rpb24uZ2V0SW5wdXRTdHJlYW0gaXMgbm90IG5l
Y2Vzc2FyaWx5IHN1ZmZpY2llbnQu

[toc] | [prev] | [next] | [standalone]

#29568

Mike Amling <mamling@chaff.us> wrote:
> On 8/4/14 10:12 AM, Andreas Leitgeb wrote:
>>              URL url=getSystemResource(name.replace('.', '/') + ".class");
>>              byte[] classData = new byte[1024*1024*10]; int len = 0;
>>              try {
>>                 len = url.openConnection().getInputStream().read(classData);
>>              } catch (IOException e) {
>>                 e.printStackTrace();
>>              }
> You could buffer to a ByteArrayOutputStream.
>       for (int input; (input=source.read())>=0; ) {
>          buffer.write(input);
>       }

Thanks for your answer. You made me re-think my irrational
resistance against coding a loop for that task. :-)

Now, the snippet looks like this:

   byte[] chunk = new byte[1024]; int len = 0;
   URL url = getSystemResource(name.replace('.', '/') + ".class");
   try {
      InputStream is = url.openConnection().getInputStream();
      ByteArrayOutputStream os = new ByteArrayOutputStream();
      while ((len = is.read(chunk)) > 0) { os.write(chunk,0,len); }
      chunk = os.toByteArray();
   } catch (IOException e) { chunk = null; }
   // if anything went wrong, there'll be an Exception on next line
   //   (be it NPE at chunk.length or from defineClass)
   res = defineClass(name,chunk, 0, chunk.length);

I think, that's good enough for me now. I declare my problem
as solved. :-)

Thanks again to all who participated in this thread!

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.java.programmer

csiph-web