Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.c++ > #82751
| From | Christian Gollwitzer <auriocus@gmx.de> |
|---|---|
| Newsgroups | comp.lang.c++ |
| Subject | Re: Can anyone improve this ? |
| Date | 2022-01-11 18:41 +0100 |
| Organization | A noiseless patient Spider |
| Message-ID | <srkfgp$34v$1@dont-email.me> (permalink) |
| References | <srjdtm$d9c$1@dont-email.me> <srjjt8$khb$1@gioia.aioe.org> <srkeor$u3h$1@dont-email.me> |
Am 11.01.22 um 18:28 schrieb Bonita Montero:
> Am 11.01.2022 um 10:50 schrieb Juha Nieminen:
>> How does it compare to the more straightforward:
>>
>> std::size_t utf8Strlen(const char *str)
>> {
>> std::size_t length = 0;
>> for(std::size_t index = 0; str[index]; ++index, ++length)
>> if((str[index] & 0xC0) == 0xC0)
>> while((str[index+1] & 0xC0) == 0x80)
>> ++index;
>> return length;
>> }
>
> The issue with that solution is that it doesn't correctly detect
> any kind of mis-formatted UTF-8-string. I get the number of chars
> preceding a header-char from the table and check if there are an
> according number of 0x80-headered chars.
>
I wrote this code once to check, if a string is valid UTF-8 - actually,
I used this platform to test-drive it:
https://leetcode.com/problems/utf-8-validation/ and it came off as the
fastest solution handed in so far:
==================================================================
enum utf8token { utf8lowbyte = 1, utf8doublet = 2, utf8triplet = 3,
utf8quadruplet = 4, utf8highbyte, utf8fail };
static utf8token utf8classify(unsigned char data) {
if ((data & 0x80) == 0) { return utf8lowbyte; }
if ((data & 0xC0) == 0x80) { return utf8highbyte;}
if ((data & 0xE0) == 0xC0) { return utf8doublet; }
if ((data & 0xF0) == 0xE0) { return utf8triplet; }
if ((data & 0xF8) == 0xF0) { return utf8quadruplet; }
return utf8fail;
}
static bool valid_utf8(const char* data, std::size_t dataSize) {
for (std::size_t i = 0; i < dataSize; i++) {
int codelength = utf8classify(static_cast<unsigned char>(data[i]));
if (codelength == utf8highbyte || codelength == utf8fail)
return false;
for (int j = 1; j<codelength; j++) {
// check for premature end of input
i++;
if (i >= dataSize) return false;
if (utf8classify(static_cast<unsigned char>(data[i])) !=
utf8highbyte)
return false;
}
}
return true;
}
==========================================================
It only returns true or false according to the problem definition, but I
think you can easily rework it to count the number of characters. I
believe all you have to do is increase a counter for every turn of the
outer loop.
Christian
Back to comp.lang.c++ | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Can anyone improve this ? Bonita Montero <Bonita.Montero@gmail.com> - 2022-01-11 09:08 +0100
Re: Can anyone improve this ? Juha Nieminen <nospam@thanks.invalid> - 2022-01-11 09:50 +0000
Re: Can anyone improve this ? Juha Nieminen <nospam@thanks.invalid> - 2022-01-11 11:07 +0000
Re: Can anyone improve this ? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2022-01-11 11:08 +0000
Re: Can anyone improve this ? Bonita Montero <Bonita.Montero@gmail.com> - 2022-01-11 18:28 +0100
Re: Can anyone improve this ? Christian Gollwitzer <auriocus@gmx.de> - 2022-01-11 18:41 +0100
Re: Can anyone improve this ? Bonita Montero <Bonita.Montero@gmail.com> - 2022-01-11 19:36 +0100
Re: Can anyone improve this ? Öö Tiib <ootiib@hot.ee> - 2022-01-11 21:54 -0800
Re: Can anyone improve this ? Bonita Montero <Bonita.Montero@gmail.com> - 2022-01-11 21:51 +0100
Re: Can anyone improve this ? Bonita Montero <Bonita.Montero@gmail.com> - 2022-01-12 14:19 +0100
Re: Can anyone improve this ? Bonita Montero <Bonita.Montero@gmail.com> - 2022-01-12 14:20 +0100
Re: Can anyone improve this ? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2022-01-12 14:14 +0000
Re: Can anyone improve this ? Bonita Montero <Bonita.Montero@gmail.com> - 2022-01-12 17:20 +0100
Re: Can anyone improve this ? "james...@alumni.caltech.edu" <jameskuyper@alumni.caltech.edu> - 2022-01-12 08:38 -0800
Re: Can anyone improve this ? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2022-01-12 16:55 +0000
Re: Can anyone improve this ? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2022-01-16 12:09 -0800
Re: Can anyone improve this ? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2022-01-16 12:12 -0800
Re: Can anyone improve this ? Öö Tiib <ootiib@hot.ee> - 2022-01-16 13:00 -0800
Re: Can anyone improve this ? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2022-01-17 10:13 -0800
Re: Can anyone improve this ? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2022-01-17 10:28 -0800
Re: Can anyone improve this ? "Alf P. Steinbach" <alf.p.steinbach@gmail.com> - 2022-01-17 21:51 +0100
Re: Can anyone improve this ? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2022-01-17 16:16 -0800
Re: Can anyone improve this ? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2022-01-17 22:43 -0800
Re: Can anyone improve this ? "Alf P. Steinbach" <alf.p.steinbach@gmail.com> - 2022-01-18 12:09 +0100
Re: Can anyone improve this ? Bonita Montero <Bonita.Montero@gmail.com> - 2022-01-18 09:18 +0100
csiph-web