Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #164386

Re: UTF-8 strings in C

From Bonita Montero <Bonita.Montero@gmail.com>
Newsgroups comp.lang.c
Subject Re: UTF-8 strings in C
Date 2022-01-10 14:38 +0100
Organization A noiseless patient Spider
Message-ID <srhcti$tvd$1@dont-email.me> (permalink)
References <strlen-20220108201256@ram.dialup.fu-berlin.de> <srgj1m$p9s$1@dont-email.me> <srgqvn$5gn$1@gioia.aioe.org> <srh1kj$epo$1@dont-email.me> <srh3b9$q0m$1@dont-email.me>

Show all headers | View raw

Am 10.01.2022 um 11:55 schrieb Bonita Montero:
> Am 10.01.2022 um 11:26 schrieb Bonita Montero:
>> Am 10.01.2022 um 09:32 schrieb Mateusz Viste:
>>> 2022-01-10 at 07:17 +0100, Bonita Montero wrote:
>>>> Then write your own UTF-8 strlen().
>>>
>>> That wouldn't be CPU efficient. ...
> 
> Ok, it would be slower, but I think this is the fastest possible
> implementation (C++20):

This is more correct since it completely detects encoding-errors:

size_t utf8Strlen( char const *str )
{
     struct encode_t { size_t lenIncr, strIncr; };
     static encode_t const encodes[] =
     {
         { 1, 1 },
         { 0, 0 },
         { 1, 2 },
         { 1, 3 },
         { 1, 4 },
         { 0, 0 },
         { 0, 0 },
         { 0, 0 },
         { 0, 0 }
     };
     size_t len = 0;
     for( unsigned char c; (c = *str); )
     {
         encode_t const &enc = encodes[(size_t)countl_zero<unsigned 
char>( ~c )];
         if( !enc.lenIncr ) [[unlikely]]
             return -1;
         len += enc.lenIncr;
         for( char const *cpEnd = str + enc.strIncr; ++str != cpEnd; )
             if( ((unsigned char)*str & 0x0C0) != 0x080 ) [[unlikely]]
                 return -1;
     }
     return len;
}

Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Re: UTF-8 strings in C Bonita Montero <Bonita.Montero@gmail.com> - 2022-01-10 07:17 +0100
  Re: UTF-8 strings in C Mateusz Viste <mateusz@xyz.invalid> - 2022-01-10 09:32 +0100
    Re: UTF-8 strings in C Bonita Montero <Bonita.Montero@gmail.com> - 2022-01-10 11:26 +0100
      Re: UTF-8 strings in C Bonita Montero <Bonita.Montero@gmail.com> - 2022-01-10 11:55 +0100
        Re: UTF-8 strings in C Bonita Montero <Bonita.Montero@gmail.com> - 2022-01-10 14:38 +0100
          Re: UTF-8 strings in C om@iki.fi (Otto J. Makela) - 2022-01-14 14:26 +0200
            Re: UTF-8 strings in C om@iki.fi (Otto J. Makela) - 2022-01-14 16:07 +0200
              Re: UTF-8 strings in C Bonita Montero <Bonita.Montero@gmail.com> - 2022-01-14 15:42 +0100
              Re: UTF-8 strings in C Philipp Klaus Krause <pkk@spth.de> - 2022-01-15 09:08 +0100
                Re: UTF-8 strings in C om@iki.fi (Otto J. Makela) - 2022-01-22 11:44 +0200
                Re: UTF-8 strings in C Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2022-01-22 04:48 -0800
                Re: UTF-8 strings in C Richard Damon <Richard@Damon-Family.org> - 2022-01-22 08:27 -0500
                Re: UTF-8 strings in C James Kuyper <jameskuyper@alumni.caltech.edu> - 2022-01-22 14:25 -0500
                Re: UTF-8 strings in C om@iki.fi (Otto J. Makela) - 2022-01-27 13:32 +0200
                Re: UTF-8 strings in C om@iki.fi (Otto J. Makela) - 2022-01-27 13:33 +0200
                Re: UTF-8 strings in C James Kuyper <jameskuyper@alumni.caltech.edu> - 2022-01-27 11:08 -0500
                Re: UTF-8 strings in C Tim Rentsch <tr.17687@z991.linuxsc.com> - 2022-02-03 22:23 -0800
          Re: UTF-8 strings in C Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2022-01-14 04:41 -0800
            Re: UTF-8 strings in C Bonita Montero <Bonita.Montero@gmail.com> - 2022-01-14 14:43 +0100
              Re: UTF-8 strings in C Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2022-01-14 05:45 -0800
                Re: UTF-8 strings in C Mateusz Viste <mateusz@xyz.invalid> - 2022-01-14 15:10 +0100
                Re: UTF-8 strings in C Bonita Montero <Bonita.Montero@gmail.com> - 2022-01-14 15:43 +0100
                Re: UTF-8 strings in C Bart <bc@freeuk.com> - 2022-01-14 15:50 +0000
                Re: UTF-8 strings in C Bonita Montero <Bonita.Montero@gmail.com> - 2022-01-14 17:50 +0100
                Re: UTF-8 strings in C Mateusz Viste <mateusz@xyz.invalid> - 2022-01-14 17:17 +0100
                Re: UTF-8 strings in C Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2022-01-14 08:34 -0800
                Re: UTF-8 strings in C Mateusz Viste <mateusz@xyz.invalid> - 2022-01-14 17:42 +0100
                Re: UTF-8 strings in C Bonita Montero <Bonita.Montero@gmail.com> - 2022-01-14 17:51 +0100
                Re: UTF-8 strings in C Mateusz Viste <mateusz@xyz.invalid> - 2022-01-14 18:07 +0100
                Re: UTF-8 strings in C Bonita Montero <Bonita.Montero@gmail.com> - 2022-01-14 18:19 +0100
                Re: UTF-8 strings in C Tim Rentsch <tr.17687@z991.linuxsc.com> - 2022-02-07 09:02 -0800

csiph-web