Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.forth > #39647

Re: RfD: 1 chars = 1

Show all headers | View raw

Paul Rubin <no.email@nospam.invalid> writes:
>anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>> As a result, with a few exceptions (see below) Forth systems have
>> stayed with 8-bit characters, and they are not alone in this: UTF-8 is
>> the dominant file representation of Unicode, and most Unix software
>> handles Unicode also internally as UTF-8 (can anyone fill in here for
>> Windows).
>
>UTF-8 is a byte encoding of Unicode, but obviously some characters UTF-8
>representation use multiple bytes.  The most correct approach is
>probably not in the Forth spirit.  It's to have the char type be large
>enough to hold any codepoint (i.e. char = 21 bits or larger) and have a
>string type that can use a compact representation like UTF-8 instead of
>just being a vector of chars.

Look at the xchars wordset in Forth-2012.  The thing that holds code
points is an xchar on the stack.  The normal way to store it in memory
is as a string.  And actually it is pretty rare that you need to get
xchars on the stack.

>Windows (and Java, C#, and Objective C) use UTF-16 according to this:
>  
>  http://lucumr.pocoo.org/2014/1/9/ucs-vs-utf8/

For C# it's clear given where it comes from (and also since it started
out as a Java clone).

Objective C is more interesting.  Looking at the Wikipedia page, it
says that `Objective-C is a thin layer on top of C, and is often
called a "strict superset" of C, meaning that it should be possible to
compile any C program with an Objective-C compiler'.  And it gives
example code like

printf("\n");

or

const char* propertyName = property_getName(*thisProperty);

which I assume use the same char size as C on the same platform.
Maybe <http://lucumr.pocoo.org/2014/1/9/ucs-vs-utf8/> means the
character/string representation used by NeXTSTEP (first released 1989)
and its successors (MacOS X and iOS).

BTW, while Java uses a 16-bit char type, it has UTF-8 in various
places in .class files, and the Cacao Java compiler developed in our
groupd keeps this as UTF-8 (there is no reason to convert it to
UTF-16).

<http://lucumr.pocoo.org/2014/1/9/ucs-vs-utf8/> also says that "Ruby,
Go as well as Rust have decided on using UTF-8" and concludes "I'm
definitely expecting more languages to take the UTF-8 route in the
future and just provide more tools to deal with Unicode as part of the
standard library."

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2015: http://www.mpeforth.com/euroforth2015/euroforth2015.htm

Back to comp.lang.forth | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

RfD: 1 chars = 1 anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2015-06-08 17:34 +0000
  Re: RfD: 1 chars = 1 Paul Rubin <no.email@nospam.invalid> - 2015-06-08 10:59 -0700
    Re: RfD: 1 chars = 1 "Alex McDonald" <blog@rivadpm.com> - 2015-06-08 19:57 +0100
      Re: RfD: 1 chars = 1 Michael Barry <barrym95838@yahoo.com> - 2015-06-08 16:35 -0700
        Re: RfD: 1 chars = 1 anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2015-06-09 06:18 +0000
      Re: RfD: 1 chars = 1 Bernd Paysan <bernd.paysan@gmx.de> - 2015-06-09 23:35 +0200
        Re: RfD: 1 chars = 1 "Alex McDonald" <blog@rivadpm.com> - 2015-06-09 23:49 +0100
          Re: RfD: 1 chars = 1 Bernd Paysan <bernd.paysan@gmx.de> - 2015-06-10 01:41 +0200
            Re: RfD: 1 chars = 1 "Alex McDonald" <blog@rivadpm.com> - 2015-06-10 02:03 +0100
              Re: RfD: 1 chars = 1 anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2015-06-10 12:22 +0000
                Re: RfD: 1 chars = 1 "Alex McDonald" <blog@rivadpm.com> - 2015-06-11 14:50 +0100
              Re: RfD: 1 chars = 1 Bernd Paysan <bernd.paysan@gmx.de> - 2015-06-10 14:46 +0200
    Re: RfD: 1 chars = 1 anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2015-06-09 07:09 +0000
      Re: RfD: 1 chars = 1 Stefan Mauerhofer <smauerhofer@androsoft.ch> - 2015-06-09 01:52 -0700
      Re: RfD: 1 chars = 1 Bernd Paysan <bernd.paysan@gmx.de> - 2015-06-09 23:29 +0200
      Re: RfD: 1 chars = 1 David Thompson <dave.thompson2@verizon.net> - 2015-06-21 12:46 -0400
  Re: RfD: 1 chars = 1 Matthias Koch <matthias.koch@hot.uni-hannover.de> - 2015-06-09 12:06 +0200
    Re: RfD: 1 chars = 1 Mark Wills <markwills1970@gmail.com> - 2015-06-09 03:40 -0700
      Re: RfD: 1 chars = 1 Matthias Koch <matthias.koch@hot.uni-hannover.de> - 2015-06-09 12:45 +0200
        Re: RfD: 1 chars = 1 rickman <gnuarm@gmail.com> - 2015-06-09 07:17 -0400
          Re: RfD: 1 chars = 1 Matthias Koch <matthias.koch@hot.uni-hannover.de> - 2015-06-09 14:30 +0200
      Re: RfD: 1 chars = 1 "Elizabeth D. Rather" <erather@forth.com> - 2015-06-09 12:50 -0800
        Re: RfD: 1 chars = 1 "HAA" <someone@microsoft.com> - 2015-06-21 17:14 +1000
          Re: RfD: 1 chars = 1 "Elizabeth D. Rather" <erather@forth.com> - 2015-06-20 21:46 -1000
            Re: RfD: 1 chars = 1 "HAA" <someone@microsoft.com> - 2015-06-23 15:05 +1000
              Re: RfD: 1 chars = 1 "Elizabeth D. Rather" <erather@forth.com> - 2015-06-22 19:42 -1000
                Re: RfD: 1 chars = 1 "HAA" <someone@microsoft.com> - 2015-06-27 14:08 +1000

csiph-web