Path: csiph.com!aioe.org!.POSTED!not-for-mail From: =?UTF-8?Q?Arne_Vajh=c3=b8j?= Newsgroups: comp.lang.java.programmer Subject: Re: Anyone know anything about string encodings? Date: Thu, 20 Jul 2017 20:18:15 -0400 Organization: Aioe.org NNTP Server Lines: 29 Message-ID: References: NNTP-Posting-Host: PBuS014lhzVyHjLbXmo/ZQ.user.gioia.aioe.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: abuse@aioe.org User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 X-Notice: Filtered by postfilter v. 0.8.2 Content-Language: en-US Xref: csiph.com comp.lang.java.programmer:36973 On 7/20/2017 4:22 PM, Eric Douglas wrote: > I get an internet stream, in this case a signature from the G Suite Google feed. > It returns from this method: > org.apache.http.HttpResponse.getEntity().getContent() > > Then I try to pass it into org.apache.commons.io.IOUtils.toString(). > This method accepts a stream and a charset to return a String. > > I tried to pass it a null and it took the default, which on our > system came up with "windows-1252", and the new signatures looked wrong. > I passed in "UTF-8" and they look right, at least for now. Will this > always work, or how would I choose what charset it should convert with? You implement the following logic: read all or at least a few thousand bytes from stream to byte array create temp string from byte array using ASCII encoding = ISO-8859-1 if temp string contains charset info then encoding = that charset info else if HTTP conten type has charset info then encoding = that charset info end if create string from byte array and remaining stream using found encoding Arne