Groups > comp.lang.c > #126205 > unrolled thread

Count string in string with loop increase 2, why is this not faster?

Started by	Bart Smissaert <bart.smissaert@gmail.com>
First post	2018-02-03 00:00 -0800
Last post	2018-02-03 22:03 +0000
Articles	20 on this page of 98 — 10 participants

Back to article view | Back to comp.lang.c

Page 4 of 5 — ← Prev page 1 2 3 [4] 5 Next page →

#126357

From	Bart Smissaert <bart.smissaert@gmail.com>
Date	2018-02-07 14:24 -0800
Message-ID	<f510802b-1b52-465d-871d-c86e344feb89@googlegroups.com>
In reply to	#126346

Managed to get it quite a bit faster still:

SQLITE_API int sqlite3_CountCharInString(const wchar_t *strSearch, int nCount, const wchar_t *strChar)
{
	int i;
	int total;
	total = 0;

	for (i = 0; i < nCount; i++)
	{
		if (*strSearch == *strChar) total++;
		total = total + (*strSearch == *strChar);
		strSearch++;
	}
	return total;
}

This is the line that speeds things up:

total = total + (*strSearch == *strChar);


RBS

[toc] | [prev] | [next] | [standalone]

#126365

From	Bart Smissaert <bart.smissaert@gmail.com>
Date	2018-02-07 14:44 -0800
Message-ID	<4410bfed-d828-477b-997e-be10f06391f4@googlegroups.com>
In reply to	#126357

Sorry, that should be:

SQLITE_API int sqlite3_CountCharInString(const wchar_t *strSearch, int nCount, const wchar_t *strChar)
{
	int i;
	int total;
	total = 0;

	for (i = 0; i < nCount; i++)
	{
		//if (*strSearch == *strChar) total++;
		total = total + (*strSearch == *strChar);
		strSearch++;
	}
	return total;
}


RBS

[toc] | [prev] | [next] | [standalone]

#126369

From	bartc <bc@freeuk.com>
Date	2018-02-07 22:59 +0000
Message-ID	<RwLeC.632366$9X.588558@fx09.am4>
In reply to	#126365

On 07/02/2018 22:44, Bart Smissaert wrote:
> Sorry, that should be:
> 
> SQLITE_API int sqlite3_CountCharInString(const wchar_t *strSearch, int nCount, const wchar_t *strChar)
> {
> 	int i;
> 	int total;
> 	total = 0;
> 
> 	for (i = 0; i < nCount; i++)
> 	{
> 		//if (*strSearch == *strChar) total++;
> 		total = total + (*strSearch == *strChar);

Yeah, I already noticed!

This form avoids a conditional jump in the code, which presumably is 
less efficient that always adding 0 or 1 to total on every character.

(I wrote the loop body as:

    total += (*strSearch++ == strChar);    // not *strchar

when I tested it. This reduced test time from 1.7 seconds to 1.2 seconds 
(neither test using *strChar). However when using *strChar (ie. a 
pointer to the find string character as you use), the faster timing was 
only 1.3 seconds.

All tested with gcc -O0 to eliminate spurious timings.)

-- 
bartc

[toc] | [prev] | [next] | [standalone]

#126373

From	Keith Thompson <kst-u@mib.org>
Date	2018-02-07 15:20 -0800
Message-ID	<lnzi4kz8sv.fsf@kst-u.example.com>
In reply to	#126357

Bart Smissaert <bart.smissaert@gmail.com> writes:
> Managed to get it quite a bit faster still:
>
> SQLITE_API int sqlite3_CountCharInString(const wchar_t *strSearch, int nCount, const wchar_t *strChar)
> {
> 	int i;
> 	int total;
> 	total = 0;
>
> 	for (i = 0; i < nCount; i++)
> 	{
> 		if (*strSearch == *strChar) total++;
> 		total = total + (*strSearch == *strChar);
> 		strSearch++;
> 	}
> 	return total;
> }
>
> This is the line that speeds things up:
>
> total = total + (*strSearch == *strChar);

Here:

    if (*strSearch == *strChar) total++;
    total = total + (*strSearch == *strChar);

you're doing the same thing twice.  Did you accidentally leave the old
line in place when you added the modified version?

That's more briefly written as:

    total += (*strSearch == *strChar);

(the parentheses aren't strictly necessary), but the generated code
should be the same.

I might write it like this:

    for (i = 0; i < nCount; i++, strSearch++) {
        total += (*strSearch == *strChar);
    }

since i++ and strSearch++ are both part of the iteration logic.  (Others
might run screaming at the sight of the comma operator being used like
that.)

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#126376

From	bartc <bc@freeuk.com>
Date	2018-02-07 23:31 +0000
Message-ID	<O_LeC.581732$5Z7.572440@fx29.am4>
In reply to	#126373

On 07/02/2018 23:20, Keith Thompson wrote:
> Bart Smissaert <bart.smissaert@gmail.com> writes:

>> 	for (i = 0; i < nCount; i++)

> I might write it like this:
> 
>      for (i = 0; i < nCount; i++, strSearch++) {
>          total += (*strSearch == *strChar);
>      }
> 
> since i++ and strSearch++ are both part of the iteration logic.

i isn't used, but it occurs three times here, and forces the comma 
operator to be used when people cram too much into a for-header as they 
tend to.

Since this is an N-times loop, I wrote it like this (nCount must be >= 0 
to start, or it can be unsigned):

     while (nCount--)
        ...

(With the increment added to the *strSearch term.)

   (Others
> might run screaming at the sight of the comma operator being used like
> that.)

Not so much the comma operator itself, as helping turn a simple loop 
into a scary looking one.

-- 
bartc

[toc] | [prev] | [next] | [standalone]

#126378

From	Bart Smissaert <bart.smissaert@gmail.com>
Date	2018-02-07 15:32 -0800
Message-ID	<521755ba-598e-4ac9-a6d7-5a3b690c9466@googlegroups.com>
In reply to	#126373

I like to keep things similar to VBA/VB6 for now as I am used to that. That is if the compiled code is does the same in any case. So for example:

total = total + (*strSearch == *strChar);

makes it more clear what is going on then:

total += (*strSearch == *strChar);

I take it this does the same thing.


RBS

[toc] | [prev] | [next] | [standalone]

#126389

From	Keith Thompson <kst-u@mib.org>
Date	2018-02-07 16:05 -0800
Message-ID	<lnpo5gz6ou.fsf@kst-u.example.com>
In reply to	#126378

Bart Smissaert <bart.smissaert@gmail.com> writes:
> I like to keep things similar to VBA/VB6 for now as I am used to
> that. That is if the compiled code is does the same in any case. So
> for example:
>
> total = total + (*strSearch == *strChar);
>
> makes it more clear what is going on then:
>
> total += (*strSearch == *strChar);
>
> I take it this does the same thing.

Yes, it does the same thing.  `total` is only evaluated once; that
doesn't matter in this case, but it could if the target were a more
complex expression or if it were volatile.

But in general, this:
    x += some_value;
is more idiomatic C than
    x = x + some_value;
and for that reason alone I prefer the "+=" form.

Having said that, there's nothing wrong with using the longer form.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#126309

From	Bart Smissaert <bart.smissaert@gmail.com>
Date	2018-02-05 15:29 -0800
Message-ID	<aecbccf9-7e7f-4673-b374-cfec1239c186@googlegroups.com>
In reply to	#126306

I also tried wmemcmp, but found it a bit slower than memcmp.

RBS

[toc] | [prev] | [next] | [standalone]

#126273

From	Richard Damon <Richard@Damon-Family.org>
Date	2018-02-04 17:41 -0500
Message-ID	<l_LdC.2573$iZ2.336@fx29.iad>
In reply to	#126269

On 2/4/18 4:53 PM, bartc wrote:
> On 04/02/2018 20:17, Bart Smissaert wrote:
>>> int sqlite3_CountStringInStringA(const WCHAR *strSearch, const  WCHAR
>> *strFind, int nSearch, int nFind);
>>
>> I tried that but get errors in compiling in MSVS 2012:
>>
>> Error    1    error C2143: syntax error : missing ')' before '*'    
>> C:\SQLite\311\sqlite3.c    3199    1    SQLite3_StdCall
> 
> I don't know what WCHAR is supposed to be. Try wchar_t instead.
> 
> If it still complains, try adding #include <wchar.h>
> 
> (But on my main compiler, wchar_t is just a typedef of 'short int'.)
> 

WCHAR is a windowism, which since he is dealing with VBA I am presuming 
he would be using. Windows uses WCHAR because they invented it before 
wchar_t was standardized, and it might be the wrong size. When using 
Unicode, the natural size of wchar_t would be an at least 21 bit type, 
likely 32 bits (so likely not short int).

[toc] | [prev] | [next] | [standalone]

#126277

From	bartc <bc@freeuk.com>
Date	2018-02-04 23:11 +0000
Message-ID	<yqMdC.472039$1x.346169@fx30.am4>
In reply to	#126273

On 04/02/2018 22:41, Richard Damon wrote:
> On 2/4/18 4:53 PM, bartc wrote:
>> On 04/02/2018 20:17, Bart Smissaert wrote:
>>>> int sqlite3_CountStringInStringA(const WCHAR *strSearch, const  WCHAR
>>> *strFind, int nSearch, int nFind);
>>>
>>> I tried that but get errors in compiling in MSVS 2012:
>>>
>>> Error    1    error C2143: syntax error : missing ')' before '*' 
>>> C:\SQLite\311\sqlite3.c    3199    1    SQLite3_StdCall
>>
>> I don't know what WCHAR is supposed to be. Try wchar_t instead.
>>
>> If it still complains, try adding #include <wchar.h>
>>
>> (But on my main compiler, wchar_t is just a typedef of 'short int'.)
>>
> 
> WCHAR is a windowism, which since he is dealing with VBA I am presuming 
> he would be using. Windows uses WCHAR because they invented it before 
> wchar_t was standardized, and it might be the wrong size. When using 
> Unicode, the natural size of wchar_t would be an at least 21 bit type, 
> likely 32 bits (so likely not short int).

WCHAR is declared using windows.h, and is described as a 16-bit value 
(and is a typedef of wchar_t).

5 of the Windows C compilers I can test right now (excluding MSVC) have 
sizeof(wchar_t) as 2. The above implies that the same applies to MSVC.

-- 
bartc

[toc] | [prev] | [next] | [standalone]

#126285

From	Richard Damon <Richard@Damon-Family.org>
Date	2018-02-04 23:45 -0500
Message-ID	<DjRdC.6654$BX2.5263@fx22.iad>
In reply to	#126277

On 2/4/18 6:11 PM, bartc wrote:
> On 04/02/2018 22:41, Richard Damon wrote:
>> On 2/4/18 4:53 PM, bartc wrote:
>>> On 04/02/2018 20:17, Bart Smissaert wrote:
>>>>> int sqlite3_CountStringInStringA(const WCHAR *strSearch, const  WCHAR
>>>> *strFind, int nSearch, int nFind);
>>>>
>>>> I tried that but get errors in compiling in MSVS 2012:
>>>>
>>>> Error    1    error C2143: syntax error : missing ')' before '*' 
>>>> C:\SQLite\311\sqlite3.c    3199    1    SQLite3_StdCall
>>>
>>> I don't know what WCHAR is supposed to be. Try wchar_t instead.
>>>
>>> If it still complains, try adding #include <wchar.h>
>>>
>>> (But on my main compiler, wchar_t is just a typedef of 'short int'.)
>>>
>>
>> WCHAR is a windowism, which since he is dealing with VBA I am 
>> presuming he would be using. Windows uses WCHAR because they invented 
>> it before wchar_t was standardized, and it might be the wrong size. 
>> When using Unicode, the natural size of wchar_t would be an at least 
>> 21 bit type, likely 32 bits (so likely not short int).
> 
> WCHAR is declared using windows.h, and is described as a 16-bit value 
> (and is a typedef of wchar_t).
> 
> 5 of the Windows C compilers I can test right now (excluding MSVC) have 
> sizeof(wchar_t) as 2. The above implies that the same applies to MSVC.
> 

The issue with windows, unicode and wchar is that windows got locked 
into the UCS-2/UTF-16 mode when UCS-2 would be a complete execution of 
Unicode. When Unicode expanded to 21 bits, and the 16 bit version became 
UTF-16 (with all the problems of UTF-8 of variable length codes) it was 
too hardwired in the ecosystem that the system process full unicode as a 
16 bit code, and ignore the issues (Western users likely can mostly live 
with using just the BMP) and there programs either just don't handle 
other characters totally right, or they forgo the advantages of a true 
wchar type that meet the expectation and take the inefficiecies of using 
UTF-16

[toc] | [prev] | [next] | [standalone]

#126276

From	Keith Thompson <kst-u@mib.org>
Date	2018-02-04 15:11 -0800
Message-ID	<lnlgg82vw2.fsf@kst-u.example.com>
In reply to	#126269

bartc <bc@freeuk.com> writes:
> On 04/02/2018 20:17, Bart Smissaert wrote:
>>> int sqlite3_CountStringInStringA(const WCHAR *strSearch, const  WCHAR
>> *strFind, int nSearch, int nFind);
>> 
>> I tried that but get errors in compiling in MSVS 2012:
>> 
>> Error	1	error C2143: syntax error : missing ')' before '*'	C:\SQLite\311\sqlite3.c	3199	1	SQLite3_StdCall
>
> I don't know what WCHAR is supposed to be. Try wchar_t instead.
>
> If it still complains, try adding #include <wchar.h>
>
> (But on my main compiler, wchar_t is just a typedef of 'short int'.)

WCHAR is a Windows-specific type defined in <windows.h>.  It appears
that both WCHAR and wchar_t are typedefs for unsigned short (*not* plain
short int).

If you're writing Windows-specific code, it probably makes sense to use
WCHAR, but either should work.  Just remember the required #include
<windows.h> (for WCHAR) or #include <wchar.h> (for wchar_t).

(Note that wchar_t is typically 32 bits on Unix-like systems.)

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#126280

From	bartc <bc@freeuk.com>
Date	2018-02-04 23:37 +0000
Message-ID	<NOMdC.1566374$OX.1165063@fx47.am4>
In reply to	#126276

On 04/02/2018 23:11, Keith Thompson wrote:
> bartc <bc@freeuk.com> writes:
>> On 04/02/2018 20:17, Bart Smissaert wrote:
>>>> int sqlite3_CountStringInStringA(const WCHAR *strSearch, const  WCHAR
>>> *strFind, int nSearch, int nFind);
>>>
>>> I tried that but get errors in compiling in MSVS 2012:
>>>
>>> Error	1	error C2143: syntax error : missing ')' before '*'	C:\SQLite\311\sqlite3.c	3199	1	SQLite3_StdCall
>>
>> I don't know what WCHAR is supposed to be. Try wchar_t instead.
>>
>> If it still complains, try adding #include <wchar.h>
>>
>> (But on my main compiler, wchar_t is just a typedef of 'short int'.)
> 
> WCHAR is a Windows-specific type defined in <windows.h>.  It appears
> that both WCHAR and wchar_t are typedefs for unsigned short (*not* plain
> short int).

Yeah, I lied a little; my main compiler is my own that uses unsigned 
short because char is unsigned. I assumed regular ones would use signed 
because char is usually signed.

If char and wchar_t are mixed signage, then that appears a little odd.

> (Note that wchar_t is typically 32 bits on Unix-like systems.)

So it's less memory efficient for the 99% of code where 32 bit 
characters are not needed.

-- 
bartc

[toc] | [prev] | [next] | [standalone]

#126283

From	Keith Thompson <kst-u@mib.org>
Date	2018-02-04 18:08 -0800
Message-ID	<lnh8qw2no5.fsf@kst-u.example.com>
In reply to	#126280

bartc <bc@freeuk.com> writes:
> On 04/02/2018 23:11, Keith Thompson wrote:
[...]
>> WCHAR is a Windows-specific type defined in <windows.h>.  It appears
>> that both WCHAR and wchar_t are typedefs for unsigned short (*not* plain
>> short int).
>
> Yeah, I lied a little; my main compiler is my own that uses unsigned 
> short because char is unsigned. I assumed regular ones would use signed 
> because char is usually signed.

Yeah, don't do that.  I've seen all 4 combinations of signed/unsigned
for plain char and wchar_t.

> If char and wchar_t are mixed signage, then that appears a little odd.

No argument.

[...]

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#126284

From	Richard Damon <Richard@Damon-Family.org>
Date	2018-02-04 23:12 -0500
Message-ID	<tQQdC.4451$fQ7.837@fx13.iad>
In reply to	#126280

On 2/4/18 6:37 PM, bartc wrote:
> On 04/02/2018 23:11, Keith Thompson wrote:
>> bartc <bc@freeuk.com> writes:
>>> On 04/02/2018 20:17, Bart Smissaert wrote:
>>>>> int sqlite3_CountStringInStringA(const WCHAR *strSearch, const  WCHAR
>>>> *strFind, int nSearch, int nFind);
>>>>
>>>> I tried that but get errors in compiling in MSVS 2012:
>>>>
>>>> Error    1    error C2143: syntax error : missing ')' before '*'    
>>>> C:\SQLite\311\sqlite3.c    3199    1    SQLite3_StdCall
>>>
>>> I don't know what WCHAR is supposed to be. Try wchar_t instead.
>>>
>>> If it still complains, try adding #include <wchar.h>
>>>
>>> (But on my main compiler, wchar_t is just a typedef of 'short int'.)
>>
>> WCHAR is a Windows-specific type defined in <windows.h>.  It appears
>> that both WCHAR and wchar_t are typedefs for unsigned short (*not* plain
>> short int).
> 
> Yeah, I lied a little; my main compiler is my own that uses unsigned 
> short because char is unsigned. I assumed regular ones would use signed 
> because char is usually signed.
> 
> If char and wchar_t are mixed signage, then that appears a little odd.
> 
>> (Note that wchar_t is typically 32 bits on Unix-like systems.)
> 
> So it's less memory efficient for the 99% of code where 32 bit 
> characters are not needed.
> 

char was made signed on some machine way back when because that seemed 
the best on that hardware, and for all 'real characters' (since they 
basically used ASCII) all values were still positive. History on those 
machines made it easiest to keep it that way, even when adopting later 
character sets that made some characters negative.

For use with UCS-2 or UTF-16, character values that would make signed 
short negative were expected, so it made sense to make it unsigned.

As to 32 bit being inefficient, if you really want the guarantees of 
wchar that ALL codes take a single code unit, the for unicode you need 
at least 21 bits, which normally means using a 32 bit storage element. 
Using UTF-16 to encode basically says you don't really want what wchar 
promises.

[toc] | [prev] | [next] | [standalone]

#126287

From	David Brown <david.brown@hesbynett.no>
Date	2018-02-05 13:08 +0100
Message-ID	<p59hg2$m76$1@dont-email.me>
In reply to	#126280

On 05/02/18 00:37, bartc wrote:
> On 04/02/2018 23:11, Keith Thompson wrote:
>> bartc <bc@freeuk.com> writes:
>>> On 04/02/2018 20:17, Bart Smissaert wrote:
>>>>> int sqlite3_CountStringInStringA(const WCHAR *strSearch, const  WCHAR
>>>> *strFind, int nSearch, int nFind);
>>>>
>>>> I tried that but get errors in compiling in MSVS 2012:
>>>>
>>>> Error    1    error C2143: syntax error : missing ')' before '*'   
>>>> C:\SQLite\311\sqlite3.c    3199    1    SQLite3_StdCall
>>>
>>> I don't know what WCHAR is supposed to be. Try wchar_t instead.
>>>
>>> If it still complains, try adding #include <wchar.h>
>>>
>>> (But on my main compiler, wchar_t is just a typedef of 'short int'.)
>>
>> WCHAR is a Windows-specific type defined in <windows.h>.  It appears
>> that both WCHAR and wchar_t are typedefs for unsigned short (*not* plain
>> short int).
> 
> Yeah, I lied a little; my main compiler is my own that uses unsigned
> short because char is unsigned. I assumed regular ones would use signed
> because char is usually signed.

I would recommend against that.  char on x86 is normally signed by
default - and there is plenty of code that assumes that is the case.
(IMHO, such code is badly written - if you need a char that is signed or
unsigned, you should specify exactly what you mean.  But such code
certainly exists.)

So if your compiler does things differently from others, you can expect
problems.

(On other processors, plain char may be signed or unsigned - it varies
for different targets.  While in theory different compilers on the same
target can have different defaults, it is usually fixed for a given
target processor.)

> 
> If char and wchar_t are mixed signage, then that appears a little odd.

The whole concept of a "char" being signed or unsigned is more than a
little odd to me - it is, I think, a historical accident.

> 
>> (Note that wchar_t is typically 32 bits on Unix-like systems.)
> 
> So it's less memory efficient for the 99% of code where 32 bit
> characters are not needed.
> 

The idea of 32-bit wchar_t, and utf-32 in general, is that one code
point (i.e., the 21-bit unicode number) is one code unit (a 32-bit
unsigned integer).  With utf-16, most Western code points fit in one
code unit, but some need two code units.  And with utf-8, only ASCII
characters fit within one code unit.

So if you have utf-8, you have a nice, compact encoding (for most
documents, it is the most compact utf encoding even for far eastern
languages), with no endian issues, and with direct compatibility with
ASCII characters, strings and functions.  But indexing or counting by
character, rather than by code unit, is a pain.

With utf-32 (32-bit wchar_t), you have a space-inefficient format and
you have endian complications.  But each code unit matches a code point
(which is /almost/ the same as a Unicode character), making it a lot
easier to handle substrings, indexing, and other manipulation.

With utf-16 (16-bit wchar_t), you have all the disadvantages of utf-8
and those of utf-32, and none of the advantages of either.  It is the
worst of both worlds.

MS made this decision at a time when 2-byte character sets were
considered advanced and "international", and UCS-16 was enough for
almost all purposes.  Java, QT and Python that made the same decision.
It turned out to be a bad idea in the end, and the *nix world got it
right with utf-8 almost everywhere, and utf-32 where needed.  Whether
*nix got it right through laziness, luck or good planning is a matter of
opinion.

[toc] | [prev] | [next] | [standalone]

#126289

From	Richard Damon <Richard@Damon-Family.org>
Date	2018-02-05 08:24 -0500
Message-ID	<zWYdC.6443$_V4.1675@fx16.iad>
In reply to	#126287

On 2/5/18 7:08 AM, David Brown wrote:
> MS made this decision at a time when 2-byte character sets were
> considered advanced and "international", and UCS-16 was enough for
> almost all purposes.  Java, QT and Python that made the same decision.
> It turned out to be a bad idea in the end, and the *nix world got it
> right with utf-8 almost everywhere, and utf-32 where needed.  Whether
> *nix got it right through laziness, luck or good planning is a matter of
> opinion.

I do wonder if the difference was that in *nix land, everything was a 
'file', or more properly a stream of bytes. With this view, UTF-8 is a 
natural encoding, as it is a byte stream.

MSDOS/Windows much more thought of different sort of things and 
different. Since one major use in windows of Unicode was filenames, and 
the long filename into fixed size chunks, UCS-2 was a natural fit, so a 
character wouldn't get split between chunks (this was part of keeping it 
compatible with the older short 8.3 filenames).

Unix on the other hand generally kept the filename all as one string, so 
it wasn't a problem, and UTF-8 is generally more compact (or at least no 
worse) for most names than UCS-2/UTF-16.

[toc] | [prev] | [next] | [standalone]

#126291

From	David Brown <david.brown@hesbynett.no>
Date	2018-02-05 14:39 +0100
Message-ID	<p59mqa$psu$1@dont-email.me>
In reply to	#126289

On 05/02/18 14:24, Richard Damon wrote:
> On 2/5/18 7:08 AM, David Brown wrote:
>> MS made this decision at a time when 2-byte character sets were
>> considered advanced and "international", and UCS-16 was enough for
>> almost all purposes.  Java, QT and Python that made the same decision.
>> It turned out to be a bad idea in the end, and the *nix world got it
>> right with utf-8 almost everywhere, and utf-32 where needed.  Whether
>> *nix got it right through laziness, luck or good planning is a matter of
>> opinion.
> 
> I do wonder if the difference was that in *nix land, everything was a
> 'file', or more properly a stream of bytes. With this view, UTF-8 is a
> natural encoding, as it is a byte stream.
> 
> MSDOS/Windows much more thought of different sort of things and
> different. Since one major use in windows of Unicode was filenames, and
> the long filename into fixed size chunks, UCS-2 was a natural fit, so a
> character wouldn't get split between chunks (this was part of keeping it
> compatible with the older short 8.3 filenames).
> 
> Unix on the other hand generally kept the filename all as one string, so
> it wasn't a problem, and UTF-8 is generally more compact (or at least no
> worse) for most names than UCS-2/UTF-16.

That could well be part of it.  In particular, DOS/Windows gives some
meaning to parts of the file name - there are lots of characters that
are disallowed, filenames that are reserved for special purposes (like
"COM1" for a serial port - where *nix uses device nodes rather than file
names), and of course it uses the extension to determine what to do with
files.  In the *nix world, the file name is just a string - only "/" and
"\0" are special in any way.  Supporting utf-8 rather than plain ASCII
did not need any changes at all to most of *nix - it is only in user
interaction that it is relevant.  But DOS/Windows needs to be able to
break up the filename in parts, and fixed size chunks makes that a lot
easier.

[toc] | [prev] | [next] | [standalone]

#126296

From	Robert Wessel <robertwessel2@yahoo.com>
Date	2018-02-05 10:37 -0600
Message-ID	<0e1h7dhc1j836ur7l9pkstabas2iq2856l@4ax.com>
In reply to	#126289

On Mon, 5 Feb 2018 08:24:47 -0500, Richard Damon
<Richard@Damon-Family.org> wrote:

>On 2/5/18 7:08 AM, David Brown wrote:
>> MS made this decision at a time when 2-byte character sets were
>> considered advanced and "international", and UCS-16 was enough for
>> almost all purposes.  Java, QT and Python that made the same decision.
>> It turned out to be a bad idea in the end, and the *nix world got it
>> right with utf-8 almost everywhere, and utf-32 where needed.  Whether
>> *nix got it right through laziness, luck or good planning is a matter of
>> opinion.
>
>I do wonder if the difference was that in *nix land, everything was a 
>'file', or more properly a stream of bytes. With this view, UTF-8 is a 
>natural encoding, as it is a byte stream.
>
>MSDOS/Windows much more thought of different sort of things and 
>different. Since one major use in windows of Unicode was filenames, and 
>the long filename into fixed size chunks, UCS-2 was a natural fit, so a 
>character wouldn't get split between chunks (this was part of keeping it 
>compatible with the older short 8.3 filenames).
>
>Unix on the other hand generally kept the filename all as one string, so 
>it wasn't a problem, and UTF-8 is generally more compact (or at least no 
>worse) for most names than UCS-2/UTF-16.

Early versions of the Unicode standard actually included "Unicode
character codes have a uniform width of 16 bits." as one of "The Ten
Unicode Design Principles" (quotes from my hardcopy Unicode 2.0
standard - circa 1996).  While also defining the surrogate pair
mechanism.  The ten principles have changed since then (and the 16-bit
character thing is no longer one of them).

There are some disadvantages to variable length codes (anyone remember
DBCS and stuff for eastern languages?), and I think several groups saw
a fixed 16-bit character set as a reasonably compromise between
capability, ease of use and space requirements, especially since UTF-8
was not one of the original formats.  I suspect that if the choice had
been UCS-4, there would have been considerably more reluctance to
adopt a fixed size code.

Of course the folk who chose UCS-2 are now either stuck with the basic
plane or implementing variable length UTF-16.

[toc] | [prev] | [next] | [standalone]

#126305

From	supercat@casperkitty.com
Date	2018-02-05 13:52 -0800
Message-ID	<8e4fd1cd-cbc4-4fa9-8e2d-f4a51648b5f2@googlegroups.com>
In reply to	#126296

On Monday, February 5, 2018 at 10:37:23 AM UTC-6, robert...@yahoo.com wrote:
> There are some disadvantages to variable length codes (anyone remember
> DBCS and stuff for eastern languages?), and I think several groups saw
> a fixed 16-bit character set as a reasonably compromise between
> capability, ease of use and space requirements, especially since UTF-8
> was not one of the original formats.  I suspect that if the choice had
> been UCS-4, there would have been considerably more reluctance to
> adopt a fixed size code.

The authors of Unicode seem to have assumed that the same format should be
used for data processing, rendering, and bulk storage, even though the
three purposes have conflicting needs that could have been better served
by using different formats to serve them.  Data processing can be made
much more efficient if strings can be cut and spliced in context-independent
fashion.  Bulk storage doesn't benefit from that, but would benefit from
optimizing data density.  Rendering engines need to have data in a form that
can be converted into a sequence of boxes that can be rendered sequentially.
An attempt to have one form serve all purposes results in a form that is not
really very good at any of them.

[toc] | [prev] | [next] | [standalone]

Page 4 of 5 — ← Prev page 1 2 3 [4] 5 Next page →

csiph-web

Count string in string with loop increase 2, why is this not faster?

Contents

#126357

#126365

#126369

#126373

#126376

#126378

#126389

#126309

#126273

#126277

#126285

#126276

#126280

#126283

#126284

#126287

#126289

#126291

#126296

#126305