Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.javascript > #19110

Re: Unicode support

Show all headers | View raw

Thomas 'PointedEars' Lahn wrote:
>
> Christoph Becker wrote:
> 
>> Thomas 'PointedEars' Lahn wrote:
>>> Christoph Becker wrote:
>>>> I suppose most *higher level languages* define the length of a string as
>>>> the number of characters the string holds.  Cf. ECMAScript's length
>>>> property and TCL's [string length].  Even PHP's mb_strlen() returns the
>>>> number of characters.
>>>
>>> AISB, unfortunately the “length” property of ECMAScript String instances
>>> is not a good example in that regard as it does _not_ mean the number of
>>> characters but the number of 16-bit units (UTF-16 code units, usually).
>>> That is only less obvious than with UTF-8 because all Unicode characters
>>> in the BMP can be encoded using only one such unit.  (I am working on a
> 
> “BMP” means “Basic Multilingual Plane” (U+0000 to U+FFFF).

I'm aware of that. :)

>>> workaround; you could call it “an mb_strlen() for ECMAScript
>>> implementations”.)
>>
>> ECMA-262, Edition 5.1, Section 15.5.5.1, states:
>>
>> | *length*
>> |
>> | The number of characters in the String value represented by this
>> | String object.
>>
>> IMO the spec is quite clear here,
> 
> It is.
> 
>> and so the implementations might be in error.
> 
> For a change, it is not.  You should look up the definition of “character” 
> in the Specification.  

I was not aware of that:

| Throughout the rest of this document, the phrase “code unit” and the
| word “character” will be used to refer to a 16-bit unsigned value
| used to represent a single 16-bit unit of text. The phrase “Unicode
| character” will be used to refer to the abstract linguistic or
| typographical unit represented by a single Unicode scalar value
| (which may be longer than 16 bits and thus may be represented by more
| than one code unit).

Thanks for pointing it out.  It's good to know that one has to take care
when e.g. using String.prototype.charAt() or
String.prototype.substring() on strings that may contain Unicode
characters outside the BMP.

> See also <news:2202673.rtQqbKup0V@PointedEars.de> pp. 
> (yes, in comp.lang.python).
> 
>> Anyway, I appreciate your working on a workaround; may come in handy. :)
> 
> At the latest after first contact ;-)
> 
> You will find it in 
> <http://PointEdears.de/websvn/listing.php?repname=JSX&path=%2Ftrunk%2Fstring%2F> 
> when it is ready for testing.

My feedreader will keep an eye on it. ;)

> F'up2 comp.lang.javascript

-- 
Christoph M. Becker

Back to comp.lang.javascript | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Re: Unicode support Christoph Becker <cmbecker69@gmx.de> - 2013-04-01 15:24 +0200
  Re: Unicode support Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2013-04-01 18:39 +0200
    Re: Unicode support Mike Duffy <Use.web.form@website_in_sig.com> - 2013-04-01 20:36 +0000
    Re: Unicode support Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2013-04-30 23:03 +0200
      Re: Unicode support Scott Sauyet <scott.sauyet@gmail.com> - 2013-05-02 05:27 -0700
        Re: Unicode support Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2013-05-03 17:59 +0200

csiph-web