Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.c > #126205 > unrolled thread

Count string in string with loop increase 2, why is this not faster?

Started byBart Smissaert <bart.smissaert@gmail.com>
First post2018-02-03 00:00 -0800
Last post2018-02-03 22:03 +0000
Articles 20 on this page of 98 — 10 participants

Back to article view | Back to comp.lang.c


Contents

  Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-03 00:00 -0800
    Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-03 11:29 +0000
      Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-03 03:37 -0800
        Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-03 12:29 +0000
          Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-03 04:37 -0800
            Re: Count string in string with loop increase 2, why is this not faster? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-03 13:41 +0000
              Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-03 06:38 -0800
                Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-03 15:13 +0000
                  Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-03 15:19 +0000
                    Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-03 07:41 -0800
                  Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-03 11:38 -0500
            Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-03 13:50 +0000
    Re: Count string in string with loop increase 2, why is this not faster? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-03 11:34 +0000
      Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-03 11:32 -0500
        Re: Count string in string with loop increase 2, why is this not faster? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-03 16:41 +0000
        Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-03 09:02 -0800
          Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-03 09:06 -0800
          Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-03 12:39 -0500
            Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-03 11:56 -0800
              Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-03 19:54 -0500
                Re: Count string in string with loop increase 2, why is this not faster? Keith Thompson <kst-u@mib.org> - 2018-02-03 18:00 -0800
                  Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-04 01:53 -0800
                    Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-04 13:23 -0500
                      Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-04 11:11 -0800
                        Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-04 14:48 -0500
                          Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-04 12:17 -0800
                            Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-04 21:53 +0000
                              Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-04 14:35 -0800
                                Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-04 23:32 +0000
                                  Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-05 06:26 -0800
                                    Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-05 16:59 +0000
                                      Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-05 09:44 -0800
                                        Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-05 18:28 +0000
                                          Re: Count string in string with loop increase 2, why is this not faster? scott@slp53.sl.home (Scott Lurndal) - 2018-02-05 18:41 +0000
                                            Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-05 20:08 +0000
                                              Re: Count string in string with loop increase 2, why is this not faster? scott@slp53.sl.home (Scott Lurndal) - 2018-02-05 21:27 +0000
                                                Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-05 22:04 +0000
                                                  Re: Count string in string with loop increase 2, why is this not faster? scott@slp53.sl.home (Scott Lurndal) - 2018-02-05 22:38 +0000
                                                    Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-06 01:34 +0000
                                                      Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-06 06:39 -0800
                                                        Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-06 18:07 +0000
                                                          Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-06 12:34 -0800
                                                            Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-06 21:12 +0000
                                                              Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-06 13:42 -0800
                                                              Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-06 16:43 -0800
                                                                Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-07 01:10 +0000
                                                                  Re: Count string in string with loop increase 2, why is this not faster? Keith Thompson <kst-u@mib.org> - 2018-02-06 18:47 -0800
                                                                    Re: Count string in string with loop increase 2, why is this not faster? David Brown <david.brown@hesbynett.no> - 2018-02-07 15:13 +0100
                                                                      Re: Count string in string with loop increase 2, why is this not faster? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-07 14:48 +0000
                                                                        Re: Count string in string with loop increase 2, why is this not faster? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-07 17:13 +0000
                                                                        Re: Count string in string with loop increase 2, why is this not faster? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-07 17:27 +0000
                                                                        Re: Count string in string with loop increase 2, why is this not faster? David Brown <david.brown@hesbynett.no> - 2018-02-07 17:01 +0100
                                                                    Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-07 02:00 -0800
                                                                    Re: Count string in string with loop increase 2, why is this not faster? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-07 12:02 +0000
                                                                      Re: Count string in string with loop increase 2, why is this not faster? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-07 14:46 +0000
                                                                      Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-07 13:56 +0000
                                                                    Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-07 11:17 +0000
                                                                  Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-06 23:59 -0800
                                                                    Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-07 05:10 -0800
                                                                    Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-07 11:01 +0000
                                                              Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-07 14:24 -0800
                                                                Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-07 14:44 -0800
                                                                  Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-07 22:59 +0000
                                                                Re: Count string in string with loop increase 2, why is this not faster? Keith Thompson <kst-u@mib.org> - 2018-02-07 15:20 -0800
                                                                  Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-07 23:31 +0000
                                                                  Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-07 15:32 -0800
                                                                    Re: Count string in string with loop increase 2, why is this not faster? Keith Thompson <kst-u@mib.org> - 2018-02-07 16:05 -0800
                                                  Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-05 15:29 -0800
                              Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-04 17:41 -0500
                                Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-04 23:11 +0000
                                  Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-04 23:45 -0500
                              Re: Count string in string with loop increase 2, why is this not faster? Keith Thompson <kst-u@mib.org> - 2018-02-04 15:11 -0800
                                Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-04 23:37 +0000
                                  Re: Count string in string with loop increase 2, why is this not faster? Keith Thompson <kst-u@mib.org> - 2018-02-04 18:08 -0800
                                  Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-04 23:12 -0500
                                  Re: Count string in string with loop increase 2, why is this not faster? David Brown <david.brown@hesbynett.no> - 2018-02-05 13:08 +0100
                                    Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-05 08:24 -0500
                                      Re: Count string in string with loop increase 2, why is this not faster? David Brown <david.brown@hesbynett.no> - 2018-02-05 14:39 +0100
                                      Re: Count string in string with loop increase 2, why is this not faster? Robert Wessel <robertwessel2@yahoo.com> - 2018-02-05 10:37 -0600
                                        Re: Count string in string with loop increase 2, why is this not faster? supercat@casperkitty.com - 2018-02-05 13:52 -0800
                                          Re: Count string in string with loop increase 2, why is this not faster? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-05 22:29 +0000
                                    Re: Count string in string with loop increase 2, why is this not faster? Keith Thompson <kst-u@mib.org> - 2018-02-05 09:01 -0800
                                      Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-05 18:44 +0000
                                      Re: Count string in string with loop increase 2, why is this not faster? David Brown <david.brown@hesbynett.no> - 2018-02-06 10:42 +0100
                                        Re: Count string in string with loop increase 2, why is this not faster? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-06 11:17 +0000
                                          Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-06 08:40 -0500
                                            Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-06 14:08 +0000
                                            Re: Count string in string with loop increase 2, why is this not faster? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-06 14:42 +0000
                                        Re: Count string in string with loop increase 2, why is this not faster? Keith Thompson <kst-u@mib.org> - 2018-02-06 12:35 -0800
                                          Re: Count string in string with loop increase 2, why is this not faster? David Brown <david.brown@hesbynett.no> - 2018-02-06 23:04 +0100
                            Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-04 17:35 -0500
                              Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-04 14:38 -0800
                                Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-04 17:57 -0500
                                  Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-04 15:22 -0800
                          Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-04 12:19 -0800
                            Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-04 16:42 -0500
            Re: Count string in string with loop increase 2, why is this not faster? Keith Thompson <kst-u@mib.org> - 2018-02-03 13:38 -0800
    Re: Count string in string with loop increase 2, why is this not faster? Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-02-03 22:03 +0000

Page 1 of 5  [1] 2 3 4 5  Next page →


#126205 — Count string in string with loop increase 2, why is this not faster?

FromBart Smissaert <bart.smissaert@gmail.com>
Date2018-02-03 00:00 -0800
SubjectCount string in string with loop increase 2, why is this not faster?
Message-ID<ddaa3397-15fc-499c-baad-0b15ee7ae61d@googlegroups.com>
Have simple function to count the number of occurrences of strFind in strSearch:

int sqlite3_CountStringInString(unsigned char *strSearch, unsigned char *strFind, int nSearch, int nFind)
{
	int i = 0;
	int countstring = 0;

	if (nSearch == 0 || nFind == 0) {
		return 0;
	}

	for (i = 0; i <= nSearch - nFind; i++) {
		if (memcmp(strSearch + i, strFind, nFind) == 0) {
			countstring++;
			i += nFind - 1;
		}
	}

	return countstring;
}

Now I thought I could speed this up by skipping the zero bytes for ansi Unicode strings:

int sqlite3_CountStringInStringA(unsigned char *strSearch, unsigned char *strFind, int nSearch, int nFind)
{
	int i = 0;
	int countstring = 0;

	if (nSearch == 0 || nFind == 0) {
		return 0;
	}

	for (i = 0; i <= nSearch - nFind; i += 2) {
		if (memcmp(strSearch + i, strFind, nFind) == 0) {
			countstring++;
			i = i + (nFind - 2);
		}
	}

	return countstring;
}

When I time this it gives however exactly the same speed.
Any idea why this is?
Thanks for any insight.


RBS

[toc] | [next] | [standalone]


#126207

Frombartc <bc@freeuk.com>
Date2018-02-03 11:29 +0000
Message-ID<T1hdC.1500750$OX.460191@fx47.am4>
In reply to#126205
On 03/02/2018 08:00, Bart Smissaert wrote:
> Have simple function to count the number of occurrences of strFind in strSearch:
> 
> int sqlite3_CountStringInString(unsigned char *strSearch, unsigned char *strFind, int nSearch, int nFind)
> {
> 	int i = 0;
> 	int countstring = 0;
> 
> 	if (nSearch == 0 || nFind == 0) {
> 		return 0;
> 	}
> 
> 	for (i = 0; i <= nSearch - nFind; i++) {
> 		if (memcmp(strSearch + i, strFind, nFind) == 0) {
> 			countstring++;
> 			i += nFind - 1;
> 		}
> 	}
> 
> 	return countstring;
> }
> 
> Now I thought I could speed this up by skipping the zero bytes for ansi Unicode strings:
> 
> int sqlite3_CountStringInStringA(unsigned char *strSearch, unsigned char *strFind, int nSearch, int nFind)
> {
> 	int i = 0;
> 	int countstring = 0;
> 
> 	if (nSearch == 0 || nFind == 0) {
> 		return 0;
> 	}
> 
> 	for (i = 0; i <= nSearch - nFind; i += 2) {
> 		if (memcmp(strSearch + i, strFind, nFind) == 0) {
> 			countstring++;
> 			i = i + (nFind - 2);
> 		}
> 	}
> 
> 	return countstring;
> }
> 
> When I time this it gives however exactly the same speed.
> Any idea why this is?

(Do both give the same results?)

How many times is memcmp() called in the two versions?

How big an overhead is this routine compared with the rest of your code? 
(Try calling it twice and seeing what difference it makes.)

(I don't get the bit about the zero bytes, unless the char* strings are 
actually 16-bit wide character sequences where the even bytes will be 
zero for ASCII text.)


-- 
bartc

[toc] | [prev] | [next] | [standalone]


#126209

FromBart Smissaert <bart.smissaert@gmail.com>
Date2018-02-03 03:37 -0800
Message-ID<99f23855-f67a-47b2-a313-e52f3787f921@googlegroups.com>
In reply to#126207
Yes, they give the same results.

> unless the char* strings are 
actually 16-bit wide character sequences where the even bytes will be 
zero for ASCII text.

Yes, that is exactly it.

There is no other code involved, so it measures just the performance of this function.

I think it makes very little difference because memcmp will return directly after the first byte if that byte is zero. This is unless I understand memcmp
wrongly.

RBS

[toc] | [prev] | [next] | [standalone]


#126210

Frombartc <bc@freeuk.com>
Date2018-02-03 12:29 +0000
Message-ID<eWhdC.310374$Og3.127736@fx11.am4>
In reply to#126209
On 03/02/2018 11:37, Bart Smissaert wrote:
> Yes, they give the same results.
> 
>> unless the char* strings are
> actually 16-bit wide character sequences where the even bytes will be
> zero for ASCII text.
> 
> Yes, that is exactly it.
> 
> There is no other code involved, so it measures just the performance of this function.
> 
> I think it makes very little difference because memcmp will return directly after the first byte if that byte is zero.

I don't think it specifically checks for zero; it will do the same when 
the first character of the find string doesn't match the next character 
of the search string.

Which I think it will do most of the time especially if the first 
character of the find string doesn't match any in the find string.

What does typical data look like? Are find strings often as long as or 
longer than searchstrings? (Then memcmp is only called once anyway.)

Can you test one particular combination of find and search strings, and 
do that repeatedly millions of times? Then see if skipping the zero 
bytes makes a difference.

-- 
bartc

[toc] | [prev] | [next] | [standalone]


#126211

FromBart Smissaert <bart.smissaert@gmail.com>
Date2018-02-03 04:37 -0800
Message-ID<323b5966-2f70-43b0-82af-48ae15f00ed3@googlegroups.com>
In reply to#126210
> I don't think it specifically checks for zero

I know, but if the first byte is zero, then it won't match the first byte of the string to find, so it will exit.

> What does typical data look like?

A long search string and a shortish find string

> Can you test one particular combination of find and search strings, and 
do that repeatedly millions of times?

That is how I tested.

RBS

[toc] | [prev] | [next] | [standalone]


#126212

FromBen Bacarisse <ben.usenet@bsb.me.uk>
Date2018-02-03 13:41 +0000
Message-ID<87po5m1983.fsf@bsb.me.uk>
In reply to#126211
Bart Smissaert <bart.smissaert@gmail.com> writes:

>> I don't think it specifically checks for zero
>
> I know, but if the first byte is zero, then it won't match the first
> byte of the string to find, so it will exit.

Are you sure that both your strings are UTF-16 encoded and with the same
byte order?  If they are, what you describe won't happen.  If they are
not, you have a whole other ste of problems!

-- 
Ben.

[toc] | [prev] | [next] | [standalone]


#126216

FromBart Smissaert <bart.smissaert@gmail.com>
Date2018-02-03 06:38 -0800
Message-ID<c8f04906-878b-466d-9be4-dfbe1afd7eb2@googlegroups.com>
In reply to#126212
> Are you sure that both your strings are UTF-16 encoded and with the same 
byte order?

100% sure as they are normal string variables in VBA.

> I tested with string like this:

    strSearch = "abababababababababababababababababababababababababababababababababababababababab"


strFind = "abab" or strFind = "baba"


RBS

[toc] | [prev] | [next] | [standalone]


#126220

Frombartc <bc@freeuk.com>
Date2018-02-03 15:13 +0000
Message-ID<dkkdC.405473$NO7.218043@fx43.am4>
In reply to#126216
On 03/02/2018 14:38, Bart Smissaert wrote:
>> Are you sure that both your strings are UTF-16 encoded and with the same
> byte order?
> 
> 100% sure as they are normal string variables in VBA.
> 
>> I tested with string like this:
> 
>      strSearch = "abababababababababababababababababababababababababababababababababababababababab"
> 
> 
> strFind = "abab" or strFind = "baba"

OK, I think I've found the issue. If you take out the memcmp part of the 
two main loops of each function, then one is executed 77 times, the 
other 39 times, which is what you might have expected.

But as it is, each is executed 20 times.

On the first, i has values 0,4,8, etc at the start of each iteration 
(because after a match, it will step i to the next search point, 4 chars 
on, to do more checks).

On the second, i will have values 0,4,8 etc for the same reason.

So they are both doing the same work, for this test data.

If I changed the find string to "fish", then the second was nearly 
double the speed.

You need to test with more varied data I think.

-- 
bartc

[toc] | [prev] | [next] | [standalone]


#126221

Frombartc <bc@freeuk.com>
Date2018-02-03 15:19 +0000
Message-ID<VpkdC.1521855$OX.7252@fx47.am4>
In reply to#126220
On 03/02/2018 15:13, bartc wrote:
> On 03/02/2018 14:38, Bart Smissaert wrote:
>>> Are you sure that both your strings are UTF-16 encoded and with the same
>> byte order?
>>
>> 100% sure as they are normal string variables in VBA.
>>
>>> I tested with string like this:
>>
>>      strSearch = 
>> "abababababababababababababababababababababababababababababababababababababababab" 
>>
>>
>>
>> strFind = "abab" or strFind = "baba"
> 
> OK, I think I've found the issue. If you take out the memcmp part of the 
> two main loops of each function, then one is executed 77 times, the 
> other 39 times, which is what you might have expected.
> 
> But as it is, each is executed 20 times.

Using "abab". With "baba", one was 21 times, the other 20.

The following is the test setup I used, with various bits of code that I 
commented in or out as I did various tests.

(This was done with a compiler that I don't think supports wide strings 
so they were expanded manually.)

----------------------------------------

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

char 
*searchstr8="abababababababababababababababababababababababababababababababababababababababab";
short searchstr16[500];

char *findstr8="abab";
//char *findstr8="baba";
//char *findstr8="fish";
short findstr16[10];

int nmemcmp1;
int nmemcmp2;

int test1(unsigned char *strSearch, unsigned char *strFind, int nSearch, 
int nFind)
{
     int i = 0;
     int countstring = 0;

     if (nSearch == 0 || nFind == 0) {
         return 0;
     }

     for (i = 0; i <= nSearch - nFind; i++) {
////printf("1: i=%d\n",i);
++nmemcmp1;
         if (memcmp(strSearch + i, strFind, nFind) == 0) {
             countstring++;
             i += nFind - 1;
         }
     }

     return countstring;
}

int test2(unsigned char *strSearch, unsigned char *strFind, int nSearch, 
int nFind)
{
     int i = 0;
     int countstring = 0;

     if (nSearch == 0 || nFind == 0) {
         return 0;
     }

     for (i = 0; i <= nSearch - nFind; i += 2) {
//printf("2: i=%d\n",i);
++nmemcmp2;
         if (memcmp(strSearch + i, strFind, nFind) == 0) {
             countstring++;
             i = i + (nFind - 2);
         }
     }

     return countstring;
}

int main(void) {
int nfind, nsearch, i, count;

nsearch=strlen(searchstr8);
for (i=0; i<nsearch; ++i) searchstr16[i] = searchstr8[i]; // assume ascii

nfind=strlen(findstr8);
for (i=0; i<nfind; ++i) findstr16[i] = findstr8[i];

//for (i=0; i<1000000; ++i) {
for (i=0; i<1; ++i) {
     count += test1((char*)searchstr16,(char*)findstr16, nsearch, nfind);
//  count += test2((char*)searchstr16,(char*)findstr16, nsearch, nfind);
}

printf("Nsearch = %d\n",nsearch);
printf("Nfind   = %d\n",nfind);
printf("Count   = %d\n",count);
printf("MEMCMP1 = %d\n",nmemcmp1);
printf("MEMCMP2 = %d\n",nmemcmp2);
puts("");
}

[toc] | [prev] | [next] | [standalone]


#126222

FromBart Smissaert <bart.smissaert@gmail.com>
Date2018-02-03 07:41 -0800
Message-ID<45850348-632c-4475-bc8c-e0f0cd24fcce@googlegroups.com>
In reply to#126221
OK, thanks, I understand it now.

I did some testing here for speed with more realistic (more random) strings like this:

    strSearch = "This is just a test, testing for the difference between a one-step loop and a two_step loop, counting strings."
    strFind = "test"
    lIterations = 10000000

With that (compiled C dll run from VBA via a .tlb) I got

one-step loop: 2978 milli-secs
two-step loop: 1446 milli-secs

So, this is exactly as one would expect.

Note that I can't see any way to get these speeds with VBA or even VB6, so a C dll does have an advantage here.


RBS

[toc] | [prev] | [next] | [standalone]


#126224

FromRichard Damon <Richard@Damon-Family.org>
Date2018-02-03 11:38 -0500
Message-ID<RzldC.4593$m03.2028@fx38.iad>
In reply to#126220
On 2/3/18 10:13 AM, bartc wrote:
> On 03/02/2018 14:38, Bart Smissaert wrote:
>>> Are you sure that both your strings are UTF-16 encoded and with the same
>> byte order?
>>
>> 100% sure as they are normal string variables in VBA.
>>
>>> I tested with string like this:
>>
>>      strSearch = 
>> "abababababababababababababababababababababababababababababababababababababababab" 
>>
>>
>>
>> strFind = "abab" or strFind = "baba"
> 
> OK, I think I've found the issue. If you take out the memcmp part of the 
> two main loops of each function, then one is executed 77 times, the 
> other 39 times, which is what you might have expected.
> 
> But as it is, each is executed 20 times.
> 
> On the first, i has values 0,4,8, etc at the start of each iteration 
> (because after a match, it will step i to the next search point, 4 chars 
> on, to do more checks).
> 
> On the second, i will have values 0,4,8 etc for the same reason.
> 
> So they are both doing the same work, for this test data.
> 
> If I changed the find string to "fish", then the second was nearly 
> double the speed.
> 
> You need to test with more varied data I think.
> 

One big question for this, how many times should you find the string 
abba in abbabba?

There are two separate occurrences, but they do share the same 
character. The answer really depends on the initial problem.

The method of advancing by the length of the searched for string only 
works if the intent is to break up the initial string into pieces, some 
of which are matches, and some are the leftovers, so two matches can't 
share characters.

[toc] | [prev] | [next] | [standalone]


#126213

Frombartc <bc@freeuk.com>
Date2018-02-03 13:50 +0000
Message-ID<o6jdC.548974$5Z7.238835@fx29.am4>
In reply to#126211
On 03/02/2018 12:37, Bart Smissaert wrote:
>> I don't think it specifically checks for zero
> 
> I know, but if the first byte is zero, then it won't match the first byte of the string to find, so it will exit.
> 
>> What does typical data look like?
> 
> A long search string and a shortish find string
> 
>> Can you test one particular combination of find and search strings, and
> do that repeatedly millions of times?
> 
> That is how I tested.

If you tell us what those strings are, we can run the same test.

(Unless the search string is too long to post.)

-- 
bartc

[toc] | [prev] | [next] | [standalone]


#126208

FromBen Bacarisse <ben.usenet@bsb.me.uk>
Date2018-02-03 11:34 +0000
Message-ID<87607e2tox.fsf@bsb.me.uk>
In reply to#126205
Bart Smissaert <bart.smissaert@gmail.com> writes:

> Have simple function to count the number of occurrences of strFind in strSearch:
>
> int sqlite3_CountStringInString(unsigned char *strSearch, unsigned char *strFind, int nSearch, int nFind)
> {
> 	int i = 0;
> 	int countstring = 0;
>
> 	if (nSearch == 0 || nFind == 0) {
> 		return 0;
> 	}
>
> 	for (i = 0; i <= nSearch - nFind; i++) {
> 		if (memcmp(strSearch + i, strFind, nFind) == 0) {
> 			countstring++;
> 			i += nFind - 1;
> 		}
> 	}
>
> 	return countstring;
> }
>
> Now I thought I could speed this up by skipping the zero bytes for
> ansi Unicode strings:

There's a lot to unpack in that statement.  Unicode is (at this level)
just a collection of numbers allocated to characters.  These numbers
have to be encoded into bytes in some way.  If you see a lot of zeros at
every other position, that's because the characters are encoded using
UTF-16.  However, not all Unicode code points take up two bytes so
stepping along the bytes in pairs won't always cause you to start the
comparison at the start of a character.  I don't know enough about the
detailed properties of UTF-16 to be 100% sure that this is a safe thing
to do.

None of this is will be the answer to your question but it should help
you phrase future questions in more conventional terms.  And you should
know that you are tying your search to a particular encoding, rather
than to "Unicode" itself.

> int sqlite3_CountStringInStringA(unsigned char *strSearch, unsigned char *strFind, int nSearch, int nFind)
> {
> 	int i = 0;
> 	int countstring = 0;
>
> 	if (nSearch == 0 || nFind == 0) {
> 		return 0;
> 	}
>
> 	for (i = 0; i <= nSearch - nFind; i += 2) {
> 		if (memcmp(strSearch + i, strFind, nFind) == 0) {
> 			countstring++;
> 			i = i + (nFind - 2);
> 		}
> 	}
>
> 	return countstring;
> }
>
> When I time this it gives however exactly the same speed.
> Any idea why this is?

How did you measure the speed?  What were the test strings you used?
You might have to work quite hard to see any difference at all with some
test data.  If you are just measuring the overall speed of some
application that uses this search, the effect might simply be swamped by
all the other things going on.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]


#126223

FromRichard Damon <Richard@Damon-Family.org>
Date2018-02-03 11:32 -0500
Message-ID<AuldC.4664$vP3.3700@fx15.iad>
In reply to#126208
On 2/3/18 6:34 AM, Ben Bacarisse wrote:
> Bart Smissaert <bart.smissaert@gmail.com> writes:
> 
>> Have simple function to count the number of occurrences of strFind in strSearch:
>>
>> int sqlite3_CountStringInString(unsigned char *strSearch, unsigned char *strFind, int nSearch, int nFind)
>> {
>> 	int i = 0;
>> 	int countstring = 0;
>>
>> 	if (nSearch == 0 || nFind == 0) {
>> 		return 0;
>> 	}
>>
>> 	for (i = 0; i <= nSearch - nFind; i++) {
>> 		if (memcmp(strSearch + i, strFind, nFind) == 0) {
>> 			countstring++;
>> 			i += nFind - 1;
>> 		}
>> 	}
>>
>> 	return countstring;
>> }
>>
>> Now I thought I could speed this up by skipping the zero bytes for
>> ansi Unicode strings:
> 
> There's a lot to unpack in that statement.  Unicode is (at this level)
> just a collection of numbers allocated to characters.  These numbers
> have to be encoded into bytes in some way.  If you see a lot of zeros at
> every other position, that's because the characters are encoded using
> UTF-16.  However, not all Unicode code points take up two bytes so
> stepping along the bytes in pairs won't always cause you to start the
> comparison at the start of a character.  I don't know enough about the
> detailed properties of UTF-16 to be 100% sure that this is a safe thing
> to do.
> 

UTF-16 is very similar to UTF-8 in this respect. In UTF-16, all code 
elements are two bytes long, so if strSearch and strFind are really in 
UTF-16 (or UCS-2) then you should only look at strSearch at even byte 
offsets for matches.

If you do have a character represented outside the BMP, so it takes more 
than one unit to represent the character, it will be composed of two 
special code units in a pair of 1024 value bands in a specified order, 
so if you start in the middle of that sequence, you are guaranteed that 
you can't get an accidental match, as the second unit will never match 
the first unit of strFind, just like UTF-8 is encoded so the subsequent 
byte units of a multi-byte character can never have a value that could 
be a first byte.

As was said, if the string you are looking for is actually ascii 
characters, but encoded as UTF-16 or UCS-2, then every other byte will be 0.

One question, are nSearch and nFind actually being given in bytes and 
not in code points (2 byte units)? If these are actually UTF-16 strings, 
then it is much more common to track things in code point, but if these 
are just binary blobs that happen to hold UTF-16 strings, the lengths 
would be in bytes.

[toc] | [prev] | [next] | [standalone]


#126225

FromBen Bacarisse <ben.usenet@bsb.me.uk>
Date2018-02-03 16:41 +0000
Message-ID<878tca10ww.fsf@bsb.me.uk>
In reply to#126223
Richard Damon <Richard@Damon-Family.org> writes:
<snip>
> If you do have a character represented outside the BMP, so it takes
> more than one unit to represent the character, it will be composed of
> two special code units in a pair of 1024 value bands in a specified
> order, so if you start in the middle of that sequence, you are
> guaranteed that you can't get an accidental match, as the second unit
> will never match the first unit of strFind,

Thanks.  That's the bit I was not sure about.  I thought it /probably/
worked like that, but I've never studied the details.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]


#126227

FromBart Smissaert <bart.smissaert@gmail.com>
Date2018-02-03 09:02 -0800
Message-ID<144f383b-5aa9-43c5-b844-ef266e439892@googlegroups.com>
In reply to#126223
> One question, are nSearch and nFind actually being given in bytes and 
not in code points (2 byte units)?

I have currently coded this as bytes, but as these are strings (not blobs) I suppose it makes sense to give this as code points, so in VBA/VB6 that will be
Len(strSearch) and Len(strFind).

RBS

[toc] | [prev] | [next] | [standalone]


#126228

FromBart Smissaert <bart.smissaert@gmail.com>
Date2018-02-03 09:06 -0800
Message-ID<673e3499-ffa7-4c97-a801-5211bc530890@googlegroups.com>
In reply to#126227
On Saturday, 3 February 2018 17:02:22 UTC, Bart Smissaert  wrote:
> > One question, are nSearch and nFind actually being given in bytes and 
> not in code points (2 byte units)?
> 
> I have currently coded this as bytes, but as these are strings (not blobs) I suppose it makes sense to give this as code points, so in VBA/VB6 that will be
> Len(strSearch) and Len(strFind).
> 
> RBS

Actually as memcmp works with bytes perhaps it is simpler to keep it as bytes.

RBS

[toc] | [prev] | [next] | [standalone]


#126230

FromRichard Damon <Richard@Damon-Family.org>
Date2018-02-03 12:39 -0500
Message-ID<bumdC.6317$d03.5530@fx37.iad>
In reply to#126227
On 2/3/18 12:02 PM, Bart Smissaert wrote:
>> One question, are nSearch and nFind actually being given in bytes and
> not in code points (2 byte units)?
> 
> I have currently coded this as bytes, but as these are strings (not blobs) I suppose it makes sense to give this as code points, so in VBA/VB6 that will be
> Len(strSearch) and Len(strFind).
> 
> RBS
> 

My question was based on the name, (beginning with sqlite3_) I am not 
sure of the source of the data and its format. Generally, I though, that 
sqlite normally stored strings in UTF-8 format (I think it can use 
UTF-16 though).

I personally would generally convert a string storing UTF-16/UCS-2 data 
into an array of uint16_t if at all possible, just because it is closer 
to what is really going on (or maybe wchar if I knew it was the right 
size). This isn't always possible for data coming from outside source.

If these are coming from VB, then they may have actually started like that.

[toc] | [prev] | [next] | [standalone]


#126232

FromBart Smissaert <bart.smissaert@gmail.com>
Date2018-02-03 11:56 -0800
Message-ID<a2fef07c-3140-4cb3-a064-88bca05e8b6d@googlegroups.com>
In reply to#126230
I deal with UTF-8 strings in SQLite as well, but this thread has to do with strings from VBA or VB6 so mainly ANSI strings. That is where the option to skip the zero bytes comes in handy.

> I personally would generally convert a string storing UTF-16/UCS-2 data 
into an array of uint16_t if at all possible

I am not familiar with C coding at all. How would I do that?
So what would I use instead of unsigned char *strSearch?
This is the current input argument of the countstring function.

Also, I am not sure if instead of unsigned char *strSearch I should do:
const unsigned char *strSearch

RBS

[toc] | [prev] | [next] | [standalone]


#126247

FromRichard Damon <Richard@Damon-Family.org>
Date2018-02-03 19:54 -0500
Message-ID<UQsdC.7853$9%2.6576@fx36.iad>
In reply to#126232
On 2/3/18 2:56 PM, Bart Smissaert wrote:
> I deal with UTF-8 strings in SQLite as well, but this thread has to do with strings from VBA or VB6 so mainly ANSI strings. That is where the option to skip the zero bytes comes in handy.
> 
>> I personally would generally convert a string storing UTF-16/UCS-2 data
> into an array of uint16_t if at all possible
> 
> I am not familiar with C coding at all. How would I do that?
> So what would I use instead of unsigned char *strSearch?
> This is the current input argument of the countstring function.
> 
> Also, I am not sure if instead of unsigned char *strSearch I should do:
> const unsigned char *strSearch
> 
> RBS
> 

Since the routine doesn't change the characters, it could use a pointer 
to const as the parameters, but then technically you should also change 
the prototype used at the calling site (I am not sure how much of the 
code is under your control and how much is 'system' provided).

My comment on using uint16_t (or as was also suggested char16_t) is 
dependent on changes upwards in the program, that storing a UTF-16 or 
UCS-2 string in an array of char is just not the way to do it, you 
REALLY want to store strings as a sequence of the type that their 
code-points are based on. To properly make that sort of change you would 
need to work back up the code chain to where the data started.

You keep on describing these as ANSI strings, and that you see all these 
zeros (which implies the UTF-16/UCS-2 encoding), but true ANSI strings 
are stored a consecutive 8 bit chars (with somewhere a defined code page 
for the string). It seems that somehow you have what you is maybe data 
that all would fit in the Latin-1 code page (what you think of as ANSI), 
but it is expressed as a 'wide string', but then it should have been 
using pointers to WCHAR / wchar_t / uint16_t / char16_t, not to unsigned 
char.

Looking up VBA, it looks like you are dealing with a String, also called 
at time a BSTR, which are passed as pointers to WCHAR which will be a 
type like uint16_t etc. I don't know where the unsigned char* came from.

[toc] | [prev] | [next] | [standalone]


Page 1 of 5  [1] 2 3 4 5  Next page →

Back to top | Article view | comp.lang.c


csiph-web