Groups > comp.lang.java.programmer > #36973

Re: Anyone know anything about string encodings?

From	Arne Vajhøj <arne@vajhoej.dk>
Newsgroups	comp.lang.java.programmer
Subject	Re: Anyone know anything about string encodings?
Date	2017-07-20 20:18 -0400
Organization	Aioe.org NNTP Server
Message-ID	<okrh85$1iig$1@gioia.aioe.org> (permalink)
References	<e5731f8f-201f-43fb-99f4-46caac570c41@googlegroups.com>

Show all headers | View raw

On 7/20/2017 4:22 PM, Eric Douglas wrote:
> I get an internet stream, in this case a signature from the G Suite Google feed.
> It returns from this method:
> org.apache.http.HttpResponse.getEntity().getContent()
> 
> Then I try to pass it into org.apache.commons.io.IOUtils.toString().
> This method accepts a stream and a charset to return a String.
> 
> I tried to pass it a null and it took the default, which on our
> system came up with "windows-1252", and the new signatures looked wrong.
> I passed in "UTF-8" and they look right, at least for now. Will this
> always work, or how would I choose what charset it should convert with?

You implement the following logic:

read all or at least a few thousand bytes from stream to byte array
create temp string from byte array using ASCII
encoding = ISO-8859-1
if temp string contains charset info then
     encoding = that charset info
else if HTTP conten type has charset info then
     encoding = that charset info
end if
create string from byte array and remaining stream using found encoding

Arne

Back to comp.lang.java.programmer | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

Anyone know anything about string encodings? Eric Douglas <e.d.programmer@gmail.com> - 2017-07-20 13:22 -0700
  Re: Anyone know anything about string encodings? Joshua Cranmer 🐧 <Pidgeot18@verizon.invalid> - 2017-07-20 17:03 -0500
    Re: Anyone know anything about string encodings? Eric Douglas <e.d.programmer@gmail.com> - 2017-07-21 06:07 -0700
    Re: Anyone know anything about string encodings? Arne Vajhøj <arne@vajhoej.dk> - 2017-07-21 19:47 -0400
      Re: Anyone know anything about string encodings? Joshua Cranmer 🐧 <Pidgeot18@verizon.invalid> - 2017-07-22 12:20 -0500
        Re: Anyone know anything about string encodings? Arne Vajhøj <arne@vajhoej.dk> - 2017-07-22 16:40 -0400
          Re: Anyone know anything about string encodings? Joshua Cranmer 🐧 <Pidgeot18@verizon.invalid> - 2017-07-24 09:27 -0500
            Re: Anyone know anything about string encodings? Arne Vajhøj <arne@vajhoej.dk> - 2017-07-24 10:54 -0400
              Re: Anyone know anything about string encodings? Joshua Cranmer 🐧 <Pidgeot18@verizon.invalid> - 2017-07-24 16:04 -0500
                Re: Anyone know anything about string encodings? lewbloch@gmail.com - 2017-07-24 17:40 -0700
                Re: Anyone know anything about string encodings? Arne Vajhøj <arne@vajhoej.dk> - 2017-07-25 09:16 -0400
                Re: Anyone know anything about string encodings? Jukka Lahtinen <jtfjdehf@hotmail.com.invalid> - 2017-07-25 11:50 +0300
                Re: Anyone know anything about string encodings? Jukka Lahtinen <jtfjdehf@hotmail.com.invalid> - 2017-07-25 11:51 +0300
                Re: Anyone know anything about string encodings? Joshua Cranmer 🐧 <Pidgeot18@verizon.invalid> - 2017-07-25 09:39 -0500
                Re: Anyone know anything about string encodings? Arne Vajhøj <arne@vajhoej.dk> - 2017-07-25 14:34 -0400
                Re: Anyone know anything about string encodings? "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com> - 2017-07-25 13:07 +0200
                Re: Anyone know anything about string encodings? Arne Vajhøj <arne@vajhoej.dk> - 2017-07-25 09:12 -0400
                Re: Anyone know anything about string encodings? Jukka Lahtinen <jtfjdehf@hotmail.com.invalid> - 2017-07-25 18:05 +0300
                Re: Anyone know anything about string encodings? Arne Vajhøj <arne@vajhoej.dk> - 2017-07-25 14:31 -0400
                Re: Anyone know anything about string encodings? lewbloch@gmail.com - 2017-07-26 11:23 -0700
  Re: Anyone know anything about string encodings? Eric Sosman <esosman@comcast-dot-net.invalid> - 2017-07-20 18:09 -0400
    Re: Anyone know anything about string encodings? Eric Douglas <e.d.programmer@gmail.com> - 2017-07-21 06:09 -0700
      Re: Anyone know anything about string encodings? Arne Vajhøj <arne@vajhoej.dk> - 2017-07-21 19:46 -0400
  Re: Anyone know anything about string encodings? Arne Vajhøj <arne@vajhoej.dk> - 2017-07-20 20:18 -0400

csiph-web