Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #28439

Re: byte stream vs char stream buffer

Show all headers | View raw

On 5/11/2014 12:34 PM, Robert Klemme wrote:

> 1. You are never transferring those bytes to Java land (i.e. into a byte
> or byte[] which is allocated on the heap) - data stays in native land.
>
> 2. You are not reading chars and hence also do not do the character
> decoding.


Yes, I knew the data was "mostly ascii" and therefore I didn't have to 
do character decoding.  An efficient UTF-8 converter shouldn't be much 
more complicated however.

I appear to be counting word lengths in the file, I'm not sure why at 
this point.  Some more found code:

          FileInputStream fins = new FileInputStream( path.toFile() );
          FileByteBufferInputStream fbbins =
                  new FileByteBufferInputStream( fins );

          int charRead;
          HashedHistogram histogram = new HashedHistogram();
          charRead = fbbins.read();
          StringBuilder sb = new StringBuilder();
          while( charRead != -1 )
          {
             if( charRead < 128 && !Character.isWhitespace( charRead ) ) {
                sb.append( (char) charRead );
                charRead = fbbins.read();
             } else {
                histogram.add( sb.toString() );
                sb.delete( 0, sb.length() );
                while( (Character.isWhitespace( (charRead = 
fbbins.read() )) ||
                        charRead >= 128) && charRead != -1 )
                {
                   // nothing
                }
             }
          }
          System.out.println( histogram.size() + " words" );
          Entry<Comparable,Integer>[] entries = 
histogram.getSortedEntries();
          System.out.println( "Bottom words:" );
          for( int i = 0; i < 20; i++ )
             System.out.println( entries[i].getKey()+", 
"+entries[i].getValue() );
          System.out.println( "Top words:" );
          for( int i = entries.length-1; i > entries.length-41; i-- )
             System.out.println( entries[i].getKey()+", 
"+entries[i].getValue() );


Kind of ugly, but that's what I have.

Back to comp.lang.java.programmer | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

byte stream vs char stream buffer Roedy Green <see_website@mindprod.com.invalid> - 2014-05-06 23:48 -0700
  Re: byte stream vs char stream buffer Andreas Leitgeb <avl@auth.logic.tuwien.ac.at> - 2014-05-08 13:30 +0000
    Re: byte stream vs char stream buffer Roedy Green <see_website@mindprod.com.invalid> - 2014-05-09 09:34 -0700
      Re: byte stream vs char stream buffer Roedy Green <see_website@mindprod.com.invalid> - 2014-05-10 13:18 -0700
        Re: byte stream vs char stream buffer Robert Klemme <shortcutter@googlemail.com> - 2014-05-10 23:24 +0200
          Re: byte stream vs char stream buffer Roedy Green <see_website@mindprod.com.invalid> - 2014-05-10 16:08 -0700
            Re: byte stream vs char stream buffer Robert Klemme <shortcutter@googlemail.com> - 2014-05-11 16:02 +0200
              Re: byte stream vs char stream buffer markspace <markspace@nospam.nospam> - 2014-05-11 11:50 -0700
                Re: byte stream vs char stream buffer markspace <markspace@nospam.nospam> - 2014-05-11 12:10 -0700
                Re: byte stream vs char stream buffer Robert Klemme <shortcutter@googlemail.com> - 2014-05-11 21:34 +0200
                Re: byte stream vs char stream buffer markspace <markspace@nospam.nospam> - 2014-05-11 12:55 -0700
                Re: byte stream vs char stream buffer Roedy Green <see_website@mindprod.com.invalid> - 2014-05-16 16:55 -0700
                Re: byte stream vs char stream buffer markspace <markspace@nospam.nospam> - 2014-05-16 17:36 -0700
                Re: byte stream vs char stream buffer Roedy Green <see_website@mindprod.com.invalid> - 2014-05-16 19:52 -0700
              Re: byte stream vs char stream buffer Roedy Green <see_website@mindprod.com.invalid> - 2014-05-16 16:37 -0700
          Re: byte stream vs char stream buffer Roedy Green <see_website@mindprod.com.invalid> - 2014-05-10 17:21 -0700
            Re: byte stream vs char stream buffer Martin Gregorie <martin@address-in-sig.invalid> - 2014-05-11 09:05 +0000
              Re: byte stream vs char stream buffer Roedy Green <see_website@mindprod.com.invalid> - 2014-05-12 02:07 -0700
                Re: byte stream vs char stream buffer Silvio <silvio@internet.com> - 2014-05-12 12:20 +0200
                Re: byte stream vs char stream buffer Roedy Green <see_website@mindprod.com.invalid> - 2014-05-16 20:19 -0700
                Re: byte stream vs char stream buffer Roedy Green <see_website@mindprod.com.invalid> - 2014-05-18 05:28 -0700
                Re: byte stream vs char stream buffer Silvio <silvio@internet.com> - 2014-05-18 23:54 +0200

csiph-web