Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.php > #8790

Re: UTF-8 charset

Show all headers | View raw

M. Strobel wrote:

> Am 21.03.2013 15:31, schrieb Adrian Tuddenham:
>> M. Strobel <sorry_no_mail_here@nowhere.dee> wrote:
>>> Am 21.03.2013 00:13, schrieb The Cat in the Hat:
>>>> Christoph Becker <cmbecker69@gmx.de> wrote in
>>>> news:kide60$df1$1@speranza.aioe.org:
>>>>> The Cat in the Hat wrote:
>>>>>> How about omitting the smart quotes in your posts?
>>>>>>
>>>>>> Come back after you've figured out how to configure your newsreader
>>>>>> properly.
>>>>>
>>>>> Thomas' message was properly encoded as UTF-8.  Isn't that acceptable
>>>>> in this NG?
>>>>
>>>> Usenet was designed to use ASCII, not UTF-8. At best, it's poor
>>>> netiquette in *any* NG.
>>>
>>> Utter nonsense.

ACK.
 
>> There are some computers that cannot read UTF-8.

That would be computers and software older than 30 years now.  The Unicode 
Standard, version 2.0, and UTF-8, one the encodings for the character set 
thus specified, was published in 1992 CE.  All reasonably modern operating 
systems, in effect all commonly used ones, support Unicode and provide 
Unicode-capable fonts.  Many have made a Unicode encoding their default 
encoding; for example, NTFS encodes filenames using UTF-16, and UTF-8 has 
been the default locale encoding on GNU/Linux systems for several years now.

Thus, a major criticism of PHP is that as of version 5.4 it still has no 
native Unicode support, while other popular programming languages on the 
Web, like ECMAScript implementations and Python, have.

>> Always use the lowest common denominator if you want to communicate
>> effectively without excluding anyone.

Insisting on using *ancient* and therefore inherently *insecure* hardware 
and software is no excuse for claiming something mindbogglingly absurd like 
that using UTF-8 encoding would be “poor netiquette in *any* NG”.  First of 
all, netiquette (network etiquette) is less concerned with the technical 
aspects of messages but with the behavior of people towards each other thus 
exhibited.  Which makes this clearly a case of “the pot calling the kettle 
black”.

Speaking of netiquette, though, it *is* considered polite (on Usenet) to 
introduce oneself with one's full name.  People posting under unnecessarily 
abbreviated names, pseudonyms, and plain nick names, like “The Cat in the 
Hat”, are usually either frowned upon, laughed at, or ignored instead by 
longtime regulars (like me).  The same applies to people who violate 
technical standards or quasi-standards such as RFC 5536, along with the AUPs 
of their providers, by using non-addresses in address header field values:

<http://www.zedat.fu-berlin.de/NetNews-Regeln>
<http://www.kirchwitz.de/~amk/dni/netiquette> (based on a Big 8 original)

> More than 90% of the text is still readable if your usenet client only
> displays ASCII.
> 
> In the anglophone world people get easily to think ASCII is sufficient,
> but in a global world even 16-bit Unicode (Basic Multilingual Plane) is
> not.

Those are good points, however:

> Some languages use UCS-16 internally, and it is not enough for all uses
> and users.

This is a common misconception.  There is no “UCS-16” and there never was.
At most, there is or was (depending how you look at it), UCS-2 (Universal 
Character Set 2), to whose character set the Unicode character set from 
Unicode 2.0 on is byte-by-byte equivalent.

The underlying misconception here is confusing character set with character 
encoding, probably also common because of the “charset” parameter of 
Internet messages that despite its name declares *character encoding* of a 
message.

UCS-2 is/was a standard for a *character set along with a specific, 16-bit 
encoding* to encode all characters in the Basic Multilingual Plane (BMP), 
and *only* those.  This and the overhead for simple (purely US-ASCII-based) 
texts lead to the later success of the competing standard, Unicode.

See also: <http://en.wikipedia.org/wiki/UCS-16> (properly redirected to the 
UCS article, with an explanation why “UCS-16” is just wrong.)

*UTF*-16 (Unicode Transformation Format, 16-bit) is *another* encoding where 
each *code unit* of a sequence for a character has 16 bit; a character may 
be encoded using more than one code unit (the same is true for the other 
UTFs).  The character set thus encoded is the Unicode character set, and 
both are defined in the Unicode Standard.  That character set comprises, and 
its transformation formats can encode, a lot more than just the BMP, 
although of the subsets of Unicode the BMP remains the best supported one, 
also due to pre-installed font support.  At the moment, there are 1'114'112 
code points in Unicode (U+0000 to U+10FFFF), so in theory a text encoded 
with one of the UTFs could contain 1'114'112 different characters.  (In 
practice, some code point ranges in the BMP are reserved for surrogate 
characters to allow more than the 65536 potential characters of the BMP 
while keeping the encoding relatively simple and backwards-compatible.)
 
> So UTF-8 is THE solution. There must be some progress sometime.

ACK.

> ASCII, 7-bit encodings and octal are left behind.

US-ASCII *is* a 7-bit encoding.  “Octal”?

The most important thing about UTF-8 is that it is equivalent to US-ASCII 
for code points below U+0080 (one 8-bit code unit per character, the MSB is 
always 0, encodes the same characters as in US-ASCII).  Therefore, UTF-8, 
through Unicode, allows texts with the greatest possible range of characters 
(all written languages, even some extinct ones, common symbols, punctuation 
etc.) while using the least amount of memory when this range is _not_ fully 
used.

Actually, it should not be necessary to explain all this to people 
subscribed to a programming newsgroup, and to Web developers in particular, 
but there you are.

<http://www.joelonsoftware.com/articles/Unicode.html>
<http://unicode.org/faq/>


HTH

PointedEars
-- 
> If you get a bunch of authors […] that state the same "best practices"
> in any programming language, then you can bet who is wrong or right...
Not with javascript. Nonsense propagates like wildfire in this field.
  -- Richard Cornford, comp.lang.javascript, 2011-11-14

Back to comp.lang.php | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

mysql dynamic binding and pass-by-ref deprecated oldyork90 <oldyork90@yahoo.com> - 2013-03-19 20:49 -0700
  Re: mysql dynamic binding and pass-by-ref deprecated Jerry Stuckle <jstucklex@attglobal.net> - 2013-03-20 06:42 -0400
    Re: mysql dynamic binding and pass-by-ref deprecated oldyork90 <oldyork90@yahoo.com> - 2013-03-20 07:09 -0700
      Re: mysql dynamic binding and pass-by-ref deprecated Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2013-03-20 15:29 +0100
        Re: mysql dynamic binding and pass-by-ref deprecated The Cat in the Hat <doctor@suess.com> - 2013-03-20 21:58 +0000
          UTF-8 charset (was: mysql dynamic binding and pass-by-ref deprecated) Christoph Becker <cmbecker69@gmx.de> - 2013-03-20 23:45 +0100
            Re: UTF-8 charset (was: mysql dynamic binding and pass-by-ref deprecated) The Cat in the Hat <doctor@suess.com> - 2013-03-20 23:13 +0000
              Re: UTF-8 charset The Natural Philosopher <tnp@invalid.invalid> - 2013-03-20 23:31 +0000
              Re: UTF-8 charset "M. Strobel" <sorry_no_mail_here@nowhere.dee> - 2013-03-21 10:11 +0100
                Re: UTF-8 charset adrian@poppyrecords.invalid.invalid (Adrian Tuddenham) - 2013-03-21 14:31 +0000
                Re: UTF-8 charset The Natural Philosopher <tnp@invalid.invalid> - 2013-03-21 16:22 +0000
                Re: UTF-8 charset "M. Strobel" <sorry_no_mail_here@nowhere.dee> - 2013-03-21 18:05 +0100
                Re: UTF-8 charset Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2013-03-22 01:17 +0100
                Re: UTF-8 charset Christoph Becker <cmbecker69@gmx.de> - 2013-03-22 01:55 +0100
                Unicode support (was: UTF-8 charset) Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2013-03-22 02:16 +0100
                Re: Unicode support Christoph Becker <cmbecker69@gmx.de> - 2013-03-30 13:57 +0100
                Re: Unicode support Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2013-03-30 14:25 +0100
                Re: Unicode support The Natural Philosopher <tnp@invalid.invalid> - 2013-03-30 14:33 +0000
                Re: Unicode support Christoph Becker <cmbecker69@gmx.de> - 2013-03-30 15:55 +0100
                Re: Unicode support The Natural Philosopher <tnp@invalid.invalid> - 2013-03-30 18:04 +0000
                Re: Unicode support Christoph Becker <cmbecker69@gmx.de> - 2013-03-30 21:26 +0100
                Re: Unicode support The Natural Philosopher <tnp@invalid.invalid> - 2013-03-30 21:26 +0000
                Re: Unicode support Christoph Becker <cmbecker69@gmx.de> - 2013-03-30 23:14 +0100
                Re: Unicode support The Natural Philosopher <tnp@invalid.invalid> - 2013-03-30 22:29 +0000
                Re: Unicode support "M. Strobel" <sorry_no_mail_here@nowhere.dee> - 2013-03-31 14:10 +0200
                Re: Unicode support The Natural Philosopher <tnp@invalid.invalid> - 2013-03-31 13:41 +0100
                Re: Unicode support "M. Strobel" <sorry_no_mail_here@nowhere.dee> - 2013-03-31 19:12 +0200
                Re: Unicode support Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2013-03-30 20:18 +0100
                Re: Unicode support Christoph Becker <cmbecker69@gmx.de> - 2013-03-30 21:05 +0100
                Re: Unicode support Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2013-03-30 21:19 +0100
                Re: Unicode support "M. Strobel" <sorry_no_mail_here@nowhere.dee> - 2013-03-31 14:06 +0200
                Re: Unicode support "M. Strobel" <sorry_no_mail_here@nowhere.dee> - 2013-04-02 19:59 +0200
                Re: Unicode support The Natural Philosopher <tnp@invalid.invalid> - 2013-03-30 14:29 +0000
                Re: UTF-8 charset The Natural Philosopher <tnp@invalid.invalid> - 2013-03-22 10:50 +0000
                Re: UTF-8 charset "M. Strobel" <sorry_no_mail_here@nowhere.dee> - 2013-03-22 09:46 +0100
                Re: UTF-8 charset "M. Strobel" <sorry_no_mail_here@nowhere.dee> - 2013-03-21 18:36 +0100
                Re: UTF-8 charset [OT] adrian@poppyrecords.invalid.invalid (Adrian Tuddenham) - 2013-03-21 18:22 +0000
                Re: UTF-8 charset [OT] "Peter H. Coffin" <hellsop@ninehells.com> - 2013-03-21 15:00 -0500
                Re: UTF-8 charset [OT] Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2013-03-22 01:47 +0100
            Re: UTF-8 charset The Natural Philosopher <tnp@invalid.invalid> - 2013-03-20 23:20 +0000
              Re: UTF-8 charset Scott Johnson <noonehome@chalupasworld.com> - 2013-03-20 16:39 -0700
          Re: mysql dynamic binding and pass-by-ref deprecated Jerry Stuckle <jstucklex@attglobal.net> - 2013-03-20 23:29 -0400
          OT: Xnews mime-proxy (was: mysql dynamic binding and pass-by-ref deprecated) BootNic <bootnic.bounce@gmail.com> - 2013-03-21 10:53 -0400
        Re: mysql dynamic binding and pass-by-ref deprecated oldyork90 <oldyork90@yahoo.com> - 2013-03-21 06:43 -0700

csiph-web