Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #24948

Re: java String split() does not work for delimiter "|" ?

Show all headers | View raw

On 08/10/2013 07:37 AM, Michael Jung wrote:
> Kevin McMurtrie <mcmurtrie@pixelmemory.us> writes:
>> In article <i61Nt.55783$Su6.15816@fx16.iad>,
>>   Arved Sandstrom <asandstrom2@eastlink.ca> wrote:
>>> On 08/09/2013 03:46 AM, Kevin McMurtrie wrote:
>>>> In article <4c416073-9f99-425e-9954-d65fe2ad1a76@googlegroups.com>,
>>>>    Lew <lewbloch@gmail.com> wrote:
>>>>
>>>>> smart...@gmail.com wrote:
>>>>>> StringTokenizer tokenizer = new StringTokenizer(content, "|");
>>>>>> while(tokenizer.hasMoreTokens()){
>>>>>>    _log.info("tokenizer.nextToken() : "+tokenizer.nextToken());
>>>>>> }
>>>>> "StringTokenizer is a legacy class that is retained for compatibility
>>>>> reasons although
>>>>> its use is discouraged in new code. It is recommended that anyone seeking
>>>>> this
>>>>> functionality use the split method of String or the java.util.regex
>>>>> package instead."
>>>>> http://docs.oracle.com/javase/7/docs/api/java/util/StringTokenizer.html
>>>> Last time I checked, the performance of String.spit() sucked.  The
>>>> JavaDoc up to 1.6 even says it sucks.  Hopefully they've fixed that
>>>> before calling a simple and effective tool like StringTokenizer "legacy.
>>>> Now if there was only a way to revert String.substring()'s performance
>>>> in Java 1.7, I might try Oracle's version of Java.
>>> I had to check that because I didn't remember ever seeing that the
>>> Javadoc for String.split saying that the performance sucked. Lo and
>>> behold, I don't see that language.
>>> What's the basis for assessing the suckage of Java String.split? Doing
>>> millions of splits? And if the situation calls for industrial text
>>> processing, why use Java anyway? It's not the first language I'd think
>>> of for that purpose, it's cumbersome. And you can't ramp up your RAM?
>>> I don't mind your comments about Java implementation performance, they
>>> are useful to followup. I just wonder what kind of Java programs you
>>> write where you find this kind of detail to be that important. Can't say
>>> I've ever in 15+ years seen a Java SE or EE project be significantly
>>> impacted by these considerations.
>> String.split() delegates to the Pattern class.  The Pattern class
>> mentions that the form used in String is not efficient because it must
>> compile the regular expression on each use.
>> Let me test...
>> Java 1.6.0_51 on an old Mac gives me these relative times:
>> splitNanos= 5341045000
>> tokenizerNanos= 1934390000
>> I hacked in a copy of 1.7.0_40-ea and got:
>> splitNanos= 3299753000
>> tokenizerNanos= 1675745000
>> It's not HUGE, but don't think you should deprecate a class that's 2
>> times faster than the replacement.  String.split() is great for utility
>> use but the core code should use pre-compiled patterns or
>> StringTokenizer.
>> Last time I checked, Oracle was still targeting big business.  Asking to
>> double the datacenter could get a whole Engineering team fired.
>
> I can confirm that this does matter in business code. We got a 10%-20%
> performance boost by avoiding split for certain use cases that used it a
> lot, not just in micro-optimizing tests. The numbers from Kevin are
> about what we had (although I personally wouldn't show that many decimal
> places that suggest a higher degree of accuracy than is actually
> reasonable).
>
> Michael
>
I don't doubt that use of String.split is not always the optimal 
approach. From the sounds of it it's not often the optimal approach. But 
I'll bet that the large majority of the time using it is a "good enough" 
approach, because very often that extra 10-20 percent speed bump isn't 
actually needed.

Funny thing is, I can think of one ESB application of mine right now 
that needs to process a high volume of messages, and each message is 
composed of 10-20 lines each one of which may have multiple fields 
delimited by slashes...and I've been using String.split without 
problems. Having said that, this is a 24/7 "don't fail or shit rains 
down from the heavens" application, so I might try swapping out 
.split(), since it's not complicated logic and I know exactly what the 
delimiter is.

But I wouldn't eschew String.split as a rule. I doubt most apps care.

AHS

-- 
When a true genius appears, you can know him by this sign:
that all the dunces are in a confederacy against him.
-- Jonathan Swift

Back to comp.lang.java.programmer | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Re: java String split() does not work for delimiter "|" ? smarty.ad4@gmail.com - 2013-08-08 05:07 -0700
  Re: java String split() does not work for delimiter "|" ? Lew <lewbloch@gmail.com> - 2013-08-08 13:16 -0700
    Re: java String split() does not work for delimiter "|" ? Kevin McMurtrie <mcmurtrie@pixelmemory.us> - 2013-08-08 23:46 -0700
      Re: java String split() does not work for delimiter "|" ? Arved Sandstrom <asandstrom2@eastlink.ca> - 2013-08-09 04:45 -0300
        Re: java String split() does not work for delimiter "|" ? Kevin McMurtrie <mcmurtrie@pixelmemory.us> - 2013-08-09 23:25 -0700
          Re: java String split() does not work for delimiter "|" ? Michael Jung <miju@golem.phantasia.org> - 2013-08-10 12:37 +0200
            Re: java String split() does not work for delimiter "|" ? Arved Sandstrom <asandstrom2@eastlink.ca> - 2013-08-10 22:33 -0300
              Re: java String split() does not work for delimiter "|" ? Michael Jung <miju@golem.phantasia.org> - 2013-08-11 11:12 +0200
                Re: java String split() does not work for delimiter "|" ? Joerg Meier <joergmmeier@arcor.de> - 2013-08-11 13:32 +0200
                Re: java String split() does not work for delimiter "|" ? Michael Jung <miju@golem.phantasia.org> - 2013-08-11 17:42 +0200
          Re: java String split() does not work for delimiter "|" ? Joerg Meier <joergmmeier@arcor.de> - 2013-08-10 15:34 +0200

csiph-web