Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #23034

Re: email stop words

Show all headers | View raw

On 3/21/2013 6:24 AM, Eric Sosman wrote:
>
>      Integer count = map.get(word);
>      map.put(word, count == null ? 1 : count + 1);

Basically, yes.

>
> ... and that you switched to something more like
>
>      Integer count = map.get(word);
>      map.put(word, new Integer(count == null
>          ? 1 : count.intValue() + 1);
>

No, I made a Counter with a primitive and a reference to the word:

   Counter counter = map.get( word );
   if( counter == null ) {
     counter = new Counter();
     counter.word = word;
     counter.count = 1;
     map.put( word, counter );
   } else
     counter.count++;

> If so, the slowdown is probably due to increased memory pressure
> and garbage collection: `new' actually creates a new object every

Yeah, that's what I thought too.  Although since there's only as many 
Counters as there are Strings (words), I don't get why just making a 2x 
change would slow the system as horribly as it did.  There should be 
only 4 million Strings and therefore also 4 million Counters.  I can't 
figure out why that would be a problem.

> time, while auto-boxing uses (the equivalent of) Integer.valueOf().
> The latter maintains a pool of a couple hundred small-valued Integers
> and doles them out whenever needed, using `new' only for un-pooled
> values.

I think it would be worth it to change the JVM memory parameters from 
the defaults and see if that makes a difference.

Also, any thoughts on the best way to observe a GC that is thrashing? 
I'm really curious to pin this down to some sort of root cause.  I 
couldn't rule out a coding error somewhere either.

>      My suggestion would be to implement a Counter class that
> wraps a mutable integer value.  Then you'd use

Thanks, I'll take a look at this when I get a chance.  A good suggestion!

>      Or, you could just go back to auto-boxing.

Yes, A-B-A testing works.  Going back to auto-boxing restored the 
previous run times, so I'm fairly certain it's related to memory 
pressure or something similar.

Back to comp.lang.java.programmer | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

email stop words markspace <markspace@nospam.nospam> - 2013-03-20 16:40 -0700
  Re: email stop words Arne Vajhøj <arne@vajhoej.dk> - 2013-03-20 20:13 -0400
    Re: email stop words Lew <lewbloch@gmail.com> - 2013-03-20 17:21 -0700
      Re: email stop words Arne Vajhøj <arne@vajhoej.dk> - 2013-03-20 20:41 -0400
    Re: email stop words markspace <markspace@nospam.nospam> - 2013-03-20 17:21 -0700
      Re: email stop words lipska the kat <"nospam at neversurrender dot co dot uk"> - 2013-03-21 09:31 +0000
  Re: email stop words Joshua Cranmer 🐧 <Pidgeot18@verizon.invalid> - 2013-03-20 20:51 -0500
  Re: email stop words markspace <markspace@nospam.nospam> - 2013-03-20 19:41 -0700
    Re: email stop words Jukka Lahtinen <jtfjdehf@hotmail.com.invalid> - 2013-03-21 08:29 +0200
    Re: email stop words Eric Sosman <esosman@comcast-dot-net.invalid> - 2013-03-21 09:24 -0400
      Re: email stop words markspace <markspace@nospam.nospam> - 2013-03-21 09:33 -0700
        Re: email stop words Eric Sosman <esosman@comcast-dot-net.invalid> - 2013-03-21 14:15 -0400
    Re: email stop words Joerg Meier <joergmmeier@arcor.de> - 2013-03-21 14:29 +0100
    Re: email stop words Joshua Cranmer 🐧 <Pidgeot18@verizon.invalid> - 2013-03-21 15:38 -0500
      Re: email stop words markspace <markspace@nospam.nospam> - 2013-03-21 16:49 -0700
  Re: email stop words Fredrik Jonson <fredrik@jonson.org> - 2013-03-21 06:58 +0000

csiph-web