Re: Unicode support

From	"M. Strobel" <sorry_no_mail_here@nowhere.dee>
Newsgroups	comp.lang.php
Subject	Re: Unicode support
Date	2013-03-31 19:12 +0200
Message-ID	<arr941F3rj2U1@mid.uni-berlin.de> (permalink)
References	(15 earlier) <kj6t3n$e1r$1@news.albasani.net> <kj6uci$of0$1@speranza.aioe.org> <kj79f3$9gg$1@news.albasani.net> <arqne6F2gmU1@mid.uni-berlin.de> <kj9ass$muh$1@news.albasani.net>

Show all headers | View raw

Am 31.03.2013 14:41, schrieb The Natural Philosopher:
> On 31/03/13 13:10, M. Strobel wrote:
>> Am 30.03.2013 19:04, schrieb The Natural Philosopher:
>>> On 30/03/13 14:55, Christoph Becker wrote:
>>>> The Natural Philosopher wrote:
>>>>
>>>>> On 30/03/13 13:25, Thomas 'PointedEars' Lahn wrote:
>>>>>
>>>>>> The same issue exists with characters outside the BMP in ECMAScript
>>>>>> implementations which uses 16-bit characters (usually one UTF-16 code
>>>>>> unit
>>>>>> per character).  But you can work around that rather efficiently.
>>>>>>
>>>>>
>>>>> The problem become 'what do you mean by strlen()' - the space the
>>>>> characters will occupy in an constant width font, or the storage
>>>>> allocated to the string?
>>>>>
>>>>> Mostly we are concerned with the latter.
>>>>
>>>> I am more concerned about the number of characters the string holds.
>>>> Say, I want to get the last character:
>>>>
>>>>     $str = '€';
>>>>     echo $str[2];
>>>>
>>>>> Because lack of precision in font reproduction, or even in guaranteeing
>>>>> which font may be selected, renders the former an 'open' question.
>>>>>
>>>>> strlen('€')===3 is in fact the correct answer.
>>>>
>>>> I suppose most *higher level languages* define the length of a string as
>>>> the number of characters the string holds.  Cf. ECMAScript's length
>>>> property and TCL's [string length].  Even PHP's mb_strlen() returns the
>>>> number of characters.
>>>>
>>>
>>> so what happens in a typographic ligature like 'ᴁ'?
>>>
>>> I think you are making a rod for your back here.
>>>
>>> The storage requirements are exact specific and useful.
>>>
>>> The concept of a 'character in a text string' is really not..and if you go deep into
>>> typography with kerning, leading,ligature and the like and the like you will
>>> understand why.
>>>
>>
>> Of course you need both, the storage requirements, and direct access to characters.
>> Maybe programming languages should use internally full 32 bit per char, or compress
>> the unicode string using a good library for access.
>>
>> C does not even know strings, just byte arrays
>>
> Thereby avoiding the problems completely by  not even pretending to solve them.
> 
> And you can always wrote a unicode_strlen() to any specification you want..the
> problem is..
> 
> ...given that many 'characters' may take LESS than a byte (ligature) or up to 3-4
> bytes (unicode character sets).... what specification?
> 

So what? How much space does a backspace take? And how much a DEL (127)? This has not
been a problem so far.

And a ligature can be decomposed. Of course every case has to be discussed, but this
is done AFAIK.

/Str.

Thread

mysql dynamic binding and pass-by-ref deprecated oldyork90 <oldyork90@yahoo.com> - 2013-03-19 20:49 -0700
  Re: mysql dynamic binding and pass-by-ref deprecated Jerry Stuckle <jstucklex@attglobal.net> - 2013-03-20 06:42 -0400
    Re: mysql dynamic binding and pass-by-ref deprecated oldyork90 <oldyork90@yahoo.com> - 2013-03-20 07:09 -0700
      Re: mysql dynamic binding and pass-by-ref deprecated Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2013-03-20 15:29 +0100
        Re: mysql dynamic binding and pass-by-ref deprecated The Cat in the Hat <doctor@suess.com> - 2013-03-20 21:58 +0000
          UTF-8 charset (was: mysql dynamic binding and pass-by-ref deprecated) Christoph Becker <cmbecker69@gmx.de> - 2013-03-20 23:45 +0100
            Re: UTF-8 charset (was: mysql dynamic binding and pass-by-ref deprecated) The Cat in the Hat <doctor@suess.com> - 2013-03-20 23:13 +0000
              Re: UTF-8 charset The Natural Philosopher <tnp@invalid.invalid> - 2013-03-20 23:31 +0000
              Re: UTF-8 charset "M. Strobel" <sorry_no_mail_here@nowhere.dee> - 2013-03-21 10:11 +0100
                Re: UTF-8 charset adrian@poppyrecords.invalid.invalid (Adrian Tuddenham) - 2013-03-21 14:31 +0000
                Re: UTF-8 charset The Natural Philosopher <tnp@invalid.invalid> - 2013-03-21 16:22 +0000
                Re: UTF-8 charset "M. Strobel" <sorry_no_mail_here@nowhere.dee> - 2013-03-21 18:05 +0100
                Re: UTF-8 charset Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2013-03-22 01:17 +0100
                Re: UTF-8 charset Christoph Becker <cmbecker69@gmx.de> - 2013-03-22 01:55 +0100
                Unicode support (was: UTF-8 charset) Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2013-03-22 02:16 +0100
                Re: Unicode support Christoph Becker <cmbecker69@gmx.de> - 2013-03-30 13:57 +0100
                Re: Unicode support Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2013-03-30 14:25 +0100
                Re: Unicode support The Natural Philosopher <tnp@invalid.invalid> - 2013-03-30 14:33 +0000
                Re: Unicode support Christoph Becker <cmbecker69@gmx.de> - 2013-03-30 15:55 +0100
                Re: Unicode support The Natural Philosopher <tnp@invalid.invalid> - 2013-03-30 18:04 +0000
                Re: Unicode support Christoph Becker <cmbecker69@gmx.de> - 2013-03-30 21:26 +0100
                Re: Unicode support The Natural Philosopher <tnp@invalid.invalid> - 2013-03-30 21:26 +0000
                Re: Unicode support Christoph Becker <cmbecker69@gmx.de> - 2013-03-30 23:14 +0100
                Re: Unicode support The Natural Philosopher <tnp@invalid.invalid> - 2013-03-30 22:29 +0000
                Re: Unicode support "M. Strobel" <sorry_no_mail_here@nowhere.dee> - 2013-03-31 14:10 +0200
                Re: Unicode support The Natural Philosopher <tnp@invalid.invalid> - 2013-03-31 13:41 +0100
                Re: Unicode support "M. Strobel" <sorry_no_mail_here@nowhere.dee> - 2013-03-31 19:12 +0200
                Re: Unicode support Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2013-03-30 20:18 +0100
                Re: Unicode support Christoph Becker <cmbecker69@gmx.de> - 2013-03-30 21:05 +0100
                Re: Unicode support Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2013-03-30 21:19 +0100
                Re: Unicode support "M. Strobel" <sorry_no_mail_here@nowhere.dee> - 2013-03-31 14:06 +0200
                Re: Unicode support "M. Strobel" <sorry_no_mail_here@nowhere.dee> - 2013-04-02 19:59 +0200
                Re: Unicode support The Natural Philosopher <tnp@invalid.invalid> - 2013-03-30 14:29 +0000
                Re: UTF-8 charset The Natural Philosopher <tnp@invalid.invalid> - 2013-03-22 10:50 +0000
                Re: UTF-8 charset "M. Strobel" <sorry_no_mail_here@nowhere.dee> - 2013-03-22 09:46 +0100
                Re: UTF-8 charset "M. Strobel" <sorry_no_mail_here@nowhere.dee> - 2013-03-21 18:36 +0100
                Re: UTF-8 charset [OT] adrian@poppyrecords.invalid.invalid (Adrian Tuddenham) - 2013-03-21 18:22 +0000
                Re: UTF-8 charset [OT] "Peter H. Coffin" <hellsop@ninehells.com> - 2013-03-21 15:00 -0500
                Re: UTF-8 charset [OT] Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2013-03-22 01:47 +0100
            Re: UTF-8 charset The Natural Philosopher <tnp@invalid.invalid> - 2013-03-20 23:20 +0000
              Re: UTF-8 charset Scott Johnson <noonehome@chalupasworld.com> - 2013-03-20 16:39 -0700
          Re: mysql dynamic binding and pass-by-ref deprecated Jerry Stuckle <jstucklex@attglobal.net> - 2013-03-20 23:29 -0400
          OT: Xnews mime-proxy (was: mysql dynamic binding and pass-by-ref deprecated) BootNic <bootnic.bounce@gmail.com> - 2013-03-21 10:53 -0400
        Re: mysql dynamic binding and pass-by-ref deprecated oldyork90 <oldyork90@yahoo.com> - 2013-03-21 06:43 -0700

csiph-web