Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #24946

Re: java String split() does not work for delimiter "|" ?

Show all headers | View raw

Kevin McMurtrie <mcmurtrie@pixelmemory.us> writes:
> In article <i61Nt.55783$Su6.15816@fx16.iad>,
>  Arved Sandstrom <asandstrom2@eastlink.ca> wrote:
>> On 08/09/2013 03:46 AM, Kevin McMurtrie wrote:
>> > In article <4c416073-9f99-425e-9954-d65fe2ad1a76@googlegroups.com>,
>> >   Lew <lewbloch@gmail.com> wrote:
>> >
>> >> smart...@gmail.com wrote:
>> >>> StringTokenizer tokenizer = new StringTokenizer(content, "|");
>> >>> while(tokenizer.hasMoreTokens()){
>> >>>   _log.info("tokenizer.nextToken() : "+tokenizer.nextToken());
>> >>> }
>> >> "StringTokenizer is a legacy class that is retained for compatibility 
>> >> reasons although
>> >> its use is discouraged in new code. It is recommended that anyone seeking
>> >> this
>> >> functionality use the split method of String or the java.util.regex 
>> >> package instead."
>> >> http://docs.oracle.com/javase/7/docs/api/java/util/StringTokenizer.html
>> > Last time I checked, the performance of String.spit() sucked.  The
>> > JavaDoc up to 1.6 even says it sucks.  Hopefully they've fixed that
>> > before calling a simple and effective tool like StringTokenizer "legacy.
>> > Now if there was only a way to revert String.substring()'s performance
>> > in Java 1.7, I might try Oracle's version of Java.
>> I had to check that because I didn't remember ever seeing that the 
>> Javadoc for String.split saying that the performance sucked. Lo and 
>> behold, I don't see that language.
>> What's the basis for assessing the suckage of Java String.split? Doing 
>> millions of splits? And if the situation calls for industrial text 
>> processing, why use Java anyway? It's not the first language I'd think 
>> of for that purpose, it's cumbersome. And you can't ramp up your RAM?
>> I don't mind your comments about Java implementation performance, they 
>> are useful to followup. I just wonder what kind of Java programs you 
>> write where you find this kind of detail to be that important. Can't say 
>> I've ever in 15+ years seen a Java SE or EE project be significantly 
>> impacted by these considerations.
> String.split() delegates to the Pattern class.  The Pattern class 
> mentions that the form used in String is not efficient because it must 
> compile the regular expression on each use.
> Let me test...
> Java 1.6.0_51 on an old Mac gives me these relative times:
> splitNanos= 5341045000
> tokenizerNanos= 1934390000
> I hacked in a copy of 1.7.0_40-ea and got:
> splitNanos= 3299753000
> tokenizerNanos= 1675745000
> It's not HUGE, but don't think you should deprecate a class that's 2 
> times faster than the replacement.  String.split() is great for utility 
> use but the core code should use pre-compiled patterns or 
> StringTokenizer.
> Last time I checked, Oracle was still targeting big business.  Asking to 
> double the datacenter could get a whole Engineering team fired.

I can confirm that this does matter in business code. We got a 10%-20%
performance boost by avoiding split for certain use cases that used it a
lot, not just in micro-optimizing tests. The numbers from Kevin are
about what we had (although I personally wouldn't show that many decimal
places that suggest a higher degree of accuracy than is actually
reasonable).

Michael

Back to comp.lang.java.programmer | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Re: java String split() does not work for delimiter "|" ? smarty.ad4@gmail.com - 2013-08-08 05:07 -0700
  Re: java String split() does not work for delimiter "|" ? Lew <lewbloch@gmail.com> - 2013-08-08 13:16 -0700
    Re: java String split() does not work for delimiter "|" ? Kevin McMurtrie <mcmurtrie@pixelmemory.us> - 2013-08-08 23:46 -0700
      Re: java String split() does not work for delimiter "|" ? Arved Sandstrom <asandstrom2@eastlink.ca> - 2013-08-09 04:45 -0300
        Re: java String split() does not work for delimiter "|" ? Kevin McMurtrie <mcmurtrie@pixelmemory.us> - 2013-08-09 23:25 -0700
          Re: java String split() does not work for delimiter "|" ? Michael Jung <miju@golem.phantasia.org> - 2013-08-10 12:37 +0200
            Re: java String split() does not work for delimiter "|" ? Arved Sandstrom <asandstrom2@eastlink.ca> - 2013-08-10 22:33 -0300
              Re: java String split() does not work for delimiter "|" ? Michael Jung <miju@golem.phantasia.org> - 2013-08-11 11:12 +0200
                Re: java String split() does not work for delimiter "|" ? Joerg Meier <joergmmeier@arcor.de> - 2013-08-11 13:32 +0200
                Re: java String split() does not work for delimiter "|" ? Michael Jung <miju@golem.phantasia.org> - 2013-08-11 17:42 +0200
          Re: java String split() does not work for delimiter "|" ? Joerg Meier <joergmmeier@arcor.de> - 2013-08-10 15:34 +0200

csiph-web