Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.java.programmer > #36980
| From | Joshua Cranmer đ§ <Pidgeot18@verizon.invalid> |
|---|---|
| Newsgroups | comp.lang.java.programmer |
| Subject | Re: Anyone know anything about string encodings? |
| Date | 2017-07-22 12:20 -0500 |
| Organization | A noiseless patient Spider |
| Message-ID | <ol019b$1q4$1@dont-email.me> (permalink) |
| References | <e5731f8f-201f-43fb-99f4-46caac570c41@googlegroups.com> <okr94s$e1e$1@dont-email.me> <oku3r0$m19$2@gioia.aioe.org> |
On 7/21/2017 6:47 PM, Arne VajhĂžj wrote: > I think my suggested algorithm handle your case 3 more > according to standards. For HTML, the standard more or less says "if there's no charset from the document itself or from the Content-Type, do whatever you want to." In practice, web browsers do different things. I believe IE tries to do autodetection. I know Firefox falls back onto a locale-dependent default encoding (e.g., en-US defaults to Windows-1252, but ja-JP would use Shift-JIS). I don't know about Chrome or Safari. Testing for UTF-8 is pretty much a safe first step because a) everyone already recommends using UTF-8 as the default anyways and b) it's very distinctive in its output format to the point that virtually all 8-bit non-UTF-8 text will fail the UTF-8 parsing. ICU, e.g., will say that text is UTF-8 with 50% confidence if it sees just 1 UTF-8 sequence and no invalid sequences moving to 100% confidence upon the third valid sequence. > > Arne > > -- Beware of bugs in the above code; I have only proved it correct, not tried it. -- Donald E. Knuth
Back to comp.lang.java.programmer | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Anyone know anything about string encodings? Eric Douglas <e.d.programmer@gmail.com> - 2017-07-20 13:22 -0700
Re: Anyone know anything about string encodings? Joshua Cranmer đ§ <Pidgeot18@verizon.invalid> - 2017-07-20 17:03 -0500
Re: Anyone know anything about string encodings? Eric Douglas <e.d.programmer@gmail.com> - 2017-07-21 06:07 -0700
Re: Anyone know anything about string encodings? Arne VajhĂžj <arne@vajhoej.dk> - 2017-07-21 19:47 -0400
Re: Anyone know anything about string encodings? Joshua Cranmer đ§ <Pidgeot18@verizon.invalid> - 2017-07-22 12:20 -0500
Re: Anyone know anything about string encodings? Arne VajhĂžj <arne@vajhoej.dk> - 2017-07-22 16:40 -0400
Re: Anyone know anything about string encodings? Joshua Cranmer đ§ <Pidgeot18@verizon.invalid> - 2017-07-24 09:27 -0500
Re: Anyone know anything about string encodings? Arne VajhĂžj <arne@vajhoej.dk> - 2017-07-24 10:54 -0400
Re: Anyone know anything about string encodings? Joshua Cranmer đ§ <Pidgeot18@verizon.invalid> - 2017-07-24 16:04 -0500
Re: Anyone know anything about string encodings? lewbloch@gmail.com - 2017-07-24 17:40 -0700
Re: Anyone know anything about string encodings? Arne VajhĂžj <arne@vajhoej.dk> - 2017-07-25 09:16 -0400
Re: Anyone know anything about string encodings? Jukka Lahtinen <jtfjdehf@hotmail.com.invalid> - 2017-07-25 11:50 +0300
Re: Anyone know anything about string encodings? Jukka Lahtinen <jtfjdehf@hotmail.com.invalid> - 2017-07-25 11:51 +0300
Re: Anyone know anything about string encodings? Joshua Cranmer đ§ <Pidgeot18@verizon.invalid> - 2017-07-25 09:39 -0500
Re: Anyone know anything about string encodings? Arne VajhĂžj <arne@vajhoej.dk> - 2017-07-25 14:34 -0400
Re: Anyone know anything about string encodings? "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com> - 2017-07-25 13:07 +0200
Re: Anyone know anything about string encodings? Arne VajhĂžj <arne@vajhoej.dk> - 2017-07-25 09:12 -0400
Re: Anyone know anything about string encodings? Jukka Lahtinen <jtfjdehf@hotmail.com.invalid> - 2017-07-25 18:05 +0300
Re: Anyone know anything about string encodings? Arne VajhĂžj <arne@vajhoej.dk> - 2017-07-25 14:31 -0400
Re: Anyone know anything about string encodings? lewbloch@gmail.com - 2017-07-26 11:23 -0700
Re: Anyone know anything about string encodings? Eric Sosman <esosman@comcast-dot-net.invalid> - 2017-07-20 18:09 -0400
Re: Anyone know anything about string encodings? Eric Douglas <e.d.programmer@gmail.com> - 2017-07-21 06:09 -0700
Re: Anyone know anything about string encodings? Arne VajhĂžj <arne@vajhoej.dk> - 2017-07-21 19:46 -0400
Re: Anyone know anything about string encodings? Arne VajhĂžj <arne@vajhoej.dk> - 2017-07-20 20:18 -0400
csiph-web