Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #36978

Re: Anyone know anything about string encodings?

Show all headers | View raw

On 7/20/2017 6:03 PM, Joshua Cranmer 🐧 wrote:
> On 7/20/2017 3:22 PM, Eric Douglas wrote:
>> I get an internet stream, in this case a signature from the G Suite
>> Google feed. It returns from this method: 
>> org.apache.http.HttpResponse.getEntity().getContent()
>>
>> Then I try to pass it into org.apache.commons.io.IOUtils.toString(). 
>> This method accepts a stream and a charset to return a String.
>>
>> I tried to pass it a null and it took the default, which on our
>> system came up with "windows-1252", and the new signatures looked
>> wrong. I passed in "UTF-8" and they look right, at least for now.
>> Will this always work, or how would I choose what charset it should
>> convert with?

> It looks like you're getting the response body of an HTTP response. In 
> practice, the Content-Type of HTTP headers tends to be woefully 
> unreliable, and the charset parameter is often in center field of 
> miscommunication here. If you're working with HTML data, then the 
> general rule boils down to:
> 
> 1. Use the charset field within the document (if present)
> 2. Else, use the Content-Type header (if present)
> 3. Otherwise, play the game of charset roulette. 
> UTF-8-if-valid-else-Windows-1252 is usually the easiest way to win this 
> game.

I think my suggested algorithm handle your case 3 more
according to standards.

Arne

Back to comp.lang.java.programmer | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Anyone know anything about string encodings? Eric Douglas <e.d.programmer@gmail.com> - 2017-07-20 13:22 -0700
  Re: Anyone know anything about string encodings? Joshua Cranmer 🐧 <Pidgeot18@verizon.invalid> - 2017-07-20 17:03 -0500
    Re: Anyone know anything about string encodings? Eric Douglas <e.d.programmer@gmail.com> - 2017-07-21 06:07 -0700
    Re: Anyone know anything about string encodings? Arne Vajhøj <arne@vajhoej.dk> - 2017-07-21 19:47 -0400
      Re: Anyone know anything about string encodings? Joshua Cranmer 🐧 <Pidgeot18@verizon.invalid> - 2017-07-22 12:20 -0500
        Re: Anyone know anything about string encodings? Arne Vajhøj <arne@vajhoej.dk> - 2017-07-22 16:40 -0400
          Re: Anyone know anything about string encodings? Joshua Cranmer 🐧 <Pidgeot18@verizon.invalid> - 2017-07-24 09:27 -0500
            Re: Anyone know anything about string encodings? Arne Vajhøj <arne@vajhoej.dk> - 2017-07-24 10:54 -0400
              Re: Anyone know anything about string encodings? Joshua Cranmer 🐧 <Pidgeot18@verizon.invalid> - 2017-07-24 16:04 -0500
                Re: Anyone know anything about string encodings? lewbloch@gmail.com - 2017-07-24 17:40 -0700
                Re: Anyone know anything about string encodings? Arne Vajhøj <arne@vajhoej.dk> - 2017-07-25 09:16 -0400
                Re: Anyone know anything about string encodings? Jukka Lahtinen <jtfjdehf@hotmail.com.invalid> - 2017-07-25 11:50 +0300
                Re: Anyone know anything about string encodings? Jukka Lahtinen <jtfjdehf@hotmail.com.invalid> - 2017-07-25 11:51 +0300
                Re: Anyone know anything about string encodings? Joshua Cranmer 🐧 <Pidgeot18@verizon.invalid> - 2017-07-25 09:39 -0500
                Re: Anyone know anything about string encodings? Arne Vajhøj <arne@vajhoej.dk> - 2017-07-25 14:34 -0400
                Re: Anyone know anything about string encodings? "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com> - 2017-07-25 13:07 +0200
                Re: Anyone know anything about string encodings? Arne Vajhøj <arne@vajhoej.dk> - 2017-07-25 09:12 -0400
                Re: Anyone know anything about string encodings? Jukka Lahtinen <jtfjdehf@hotmail.com.invalid> - 2017-07-25 18:05 +0300
                Re: Anyone know anything about string encodings? Arne Vajhøj <arne@vajhoej.dk> - 2017-07-25 14:31 -0400
                Re: Anyone know anything about string encodings? lewbloch@gmail.com - 2017-07-26 11:23 -0700
  Re: Anyone know anything about string encodings? Eric Sosman <esosman@comcast-dot-net.invalid> - 2017-07-20 18:09 -0400
    Re: Anyone know anything about string encodings? Eric Douglas <e.d.programmer@gmail.com> - 2017-07-21 06:09 -0700
      Re: Anyone know anything about string encodings? Arne Vajhøj <arne@vajhoej.dk> - 2017-07-21 19:46 -0400
  Re: Anyone know anything about string encodings? Arne Vajhøj <arne@vajhoej.dk> - 2017-07-20 20:18 -0400

csiph-web