Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.c > #126205 > unrolled thread
| Started by | Bart Smissaert <bart.smissaert@gmail.com> |
|---|---|
| First post | 2018-02-03 00:00 -0800 |
| Last post | 2018-02-03 22:03 +0000 |
| Articles | 20 on this page of 98 — 10 participants |
Back to article view | Back to comp.lang.c
Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-03 00:00 -0800
Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-03 11:29 +0000
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-03 03:37 -0800
Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-03 12:29 +0000
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-03 04:37 -0800
Re: Count string in string with loop increase 2, why is this not faster? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-03 13:41 +0000
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-03 06:38 -0800
Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-03 15:13 +0000
Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-03 15:19 +0000
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-03 07:41 -0800
Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-03 11:38 -0500
Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-03 13:50 +0000
Re: Count string in string with loop increase 2, why is this not faster? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-03 11:34 +0000
Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-03 11:32 -0500
Re: Count string in string with loop increase 2, why is this not faster? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-03 16:41 +0000
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-03 09:02 -0800
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-03 09:06 -0800
Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-03 12:39 -0500
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-03 11:56 -0800
Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-03 19:54 -0500
Re: Count string in string with loop increase 2, why is this not faster? Keith Thompson <kst-u@mib.org> - 2018-02-03 18:00 -0800
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-04 01:53 -0800
Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-04 13:23 -0500
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-04 11:11 -0800
Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-04 14:48 -0500
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-04 12:17 -0800
Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-04 21:53 +0000
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-04 14:35 -0800
Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-04 23:32 +0000
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-05 06:26 -0800
Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-05 16:59 +0000
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-05 09:44 -0800
Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-05 18:28 +0000
Re: Count string in string with loop increase 2, why is this not faster? scott@slp53.sl.home (Scott Lurndal) - 2018-02-05 18:41 +0000
Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-05 20:08 +0000
Re: Count string in string with loop increase 2, why is this not faster? scott@slp53.sl.home (Scott Lurndal) - 2018-02-05 21:27 +0000
Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-05 22:04 +0000
Re: Count string in string with loop increase 2, why is this not faster? scott@slp53.sl.home (Scott Lurndal) - 2018-02-05 22:38 +0000
Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-06 01:34 +0000
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-06 06:39 -0800
Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-06 18:07 +0000
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-06 12:34 -0800
Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-06 21:12 +0000
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-06 13:42 -0800
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-06 16:43 -0800
Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-07 01:10 +0000
Re: Count string in string with loop increase 2, why is this not faster? Keith Thompson <kst-u@mib.org> - 2018-02-06 18:47 -0800
Re: Count string in string with loop increase 2, why is this not faster? David Brown <david.brown@hesbynett.no> - 2018-02-07 15:13 +0100
Re: Count string in string with loop increase 2, why is this not faster? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-07 14:48 +0000
Re: Count string in string with loop increase 2, why is this not faster? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-07 17:13 +0000
Re: Count string in string with loop increase 2, why is this not faster? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-07 17:27 +0000
Re: Count string in string with loop increase 2, why is this not faster? David Brown <david.brown@hesbynett.no> - 2018-02-07 17:01 +0100
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-07 02:00 -0800
Re: Count string in string with loop increase 2, why is this not faster? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-07 12:02 +0000
Re: Count string in string with loop increase 2, why is this not faster? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-07 14:46 +0000
Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-07 13:56 +0000
Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-07 11:17 +0000
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-06 23:59 -0800
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-07 05:10 -0800
Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-07 11:01 +0000
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-07 14:24 -0800
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-07 14:44 -0800
Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-07 22:59 +0000
Re: Count string in string with loop increase 2, why is this not faster? Keith Thompson <kst-u@mib.org> - 2018-02-07 15:20 -0800
Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-07 23:31 +0000
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-07 15:32 -0800
Re: Count string in string with loop increase 2, why is this not faster? Keith Thompson <kst-u@mib.org> - 2018-02-07 16:05 -0800
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-05 15:29 -0800
Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-04 17:41 -0500
Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-04 23:11 +0000
Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-04 23:45 -0500
Re: Count string in string with loop increase 2, why is this not faster? Keith Thompson <kst-u@mib.org> - 2018-02-04 15:11 -0800
Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-04 23:37 +0000
Re: Count string in string with loop increase 2, why is this not faster? Keith Thompson <kst-u@mib.org> - 2018-02-04 18:08 -0800
Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-04 23:12 -0500
Re: Count string in string with loop increase 2, why is this not faster? David Brown <david.brown@hesbynett.no> - 2018-02-05 13:08 +0100
Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-05 08:24 -0500
Re: Count string in string with loop increase 2, why is this not faster? David Brown <david.brown@hesbynett.no> - 2018-02-05 14:39 +0100
Re: Count string in string with loop increase 2, why is this not faster? Robert Wessel <robertwessel2@yahoo.com> - 2018-02-05 10:37 -0600
Re: Count string in string with loop increase 2, why is this not faster? supercat@casperkitty.com - 2018-02-05 13:52 -0800
Re: Count string in string with loop increase 2, why is this not faster? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-05 22:29 +0000
Re: Count string in string with loop increase 2, why is this not faster? Keith Thompson <kst-u@mib.org> - 2018-02-05 09:01 -0800
Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-05 18:44 +0000
Re: Count string in string with loop increase 2, why is this not faster? David Brown <david.brown@hesbynett.no> - 2018-02-06 10:42 +0100
Re: Count string in string with loop increase 2, why is this not faster? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-06 11:17 +0000
Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-06 08:40 -0500
Re: Count string in string with loop increase 2, why is this not faster? bartc <bc@freeuk.com> - 2018-02-06 14:08 +0000
Re: Count string in string with loop increase 2, why is this not faster? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-06 14:42 +0000
Re: Count string in string with loop increase 2, why is this not faster? Keith Thompson <kst-u@mib.org> - 2018-02-06 12:35 -0800
Re: Count string in string with loop increase 2, why is this not faster? David Brown <david.brown@hesbynett.no> - 2018-02-06 23:04 +0100
Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-04 17:35 -0500
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-04 14:38 -0800
Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-04 17:57 -0500
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-04 15:22 -0800
Re: Count string in string with loop increase 2, why is this not faster? Bart Smissaert <bart.smissaert@gmail.com> - 2018-02-04 12:19 -0800
Re: Count string in string with loop increase 2, why is this not faster? Richard Damon <Richard@Damon-Family.org> - 2018-02-04 16:42 -0500
Re: Count string in string with loop increase 2, why is this not faster? Keith Thompson <kst-u@mib.org> - 2018-02-03 13:38 -0800
Re: Count string in string with loop increase 2, why is this not faster? Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-02-03 22:03 +0000
Page 1 of 5 [1] 2 3 4 5 Next page →
| From | Bart Smissaert <bart.smissaert@gmail.com> |
|---|---|
| Date | 2018-02-03 00:00 -0800 |
| Subject | Count string in string with loop increase 2, why is this not faster? |
| Message-ID | <ddaa3397-15fc-499c-baad-0b15ee7ae61d@googlegroups.com> |
Have simple function to count the number of occurrences of strFind in strSearch:
int sqlite3_CountStringInString(unsigned char *strSearch, unsigned char *strFind, int nSearch, int nFind)
{
int i = 0;
int countstring = 0;
if (nSearch == 0 || nFind == 0) {
return 0;
}
for (i = 0; i <= nSearch - nFind; i++) {
if (memcmp(strSearch + i, strFind, nFind) == 0) {
countstring++;
i += nFind - 1;
}
}
return countstring;
}
Now I thought I could speed this up by skipping the zero bytes for ansi Unicode strings:
int sqlite3_CountStringInStringA(unsigned char *strSearch, unsigned char *strFind, int nSearch, int nFind)
{
int i = 0;
int countstring = 0;
if (nSearch == 0 || nFind == 0) {
return 0;
}
for (i = 0; i <= nSearch - nFind; i += 2) {
if (memcmp(strSearch + i, strFind, nFind) == 0) {
countstring++;
i = i + (nFind - 2);
}
}
return countstring;
}
When I time this it gives however exactly the same speed.
Any idea why this is?
Thanks for any insight.
RBS
[toc] | [next] | [standalone]
| From | bartc <bc@freeuk.com> |
|---|---|
| Date | 2018-02-03 11:29 +0000 |
| Message-ID | <T1hdC.1500750$OX.460191@fx47.am4> |
| In reply to | #126205 |
On 03/02/2018 08:00, Bart Smissaert wrote:
> Have simple function to count the number of occurrences of strFind in strSearch:
>
> int sqlite3_CountStringInString(unsigned char *strSearch, unsigned char *strFind, int nSearch, int nFind)
> {
> int i = 0;
> int countstring = 0;
>
> if (nSearch == 0 || nFind == 0) {
> return 0;
> }
>
> for (i = 0; i <= nSearch - nFind; i++) {
> if (memcmp(strSearch + i, strFind, nFind) == 0) {
> countstring++;
> i += nFind - 1;
> }
> }
>
> return countstring;
> }
>
> Now I thought I could speed this up by skipping the zero bytes for ansi Unicode strings:
>
> int sqlite3_CountStringInStringA(unsigned char *strSearch, unsigned char *strFind, int nSearch, int nFind)
> {
> int i = 0;
> int countstring = 0;
>
> if (nSearch == 0 || nFind == 0) {
> return 0;
> }
>
> for (i = 0; i <= nSearch - nFind; i += 2) {
> if (memcmp(strSearch + i, strFind, nFind) == 0) {
> countstring++;
> i = i + (nFind - 2);
> }
> }
>
> return countstring;
> }
>
> When I time this it gives however exactly the same speed.
> Any idea why this is?
(Do both give the same results?)
How many times is memcmp() called in the two versions?
How big an overhead is this routine compared with the rest of your code?
(Try calling it twice and seeing what difference it makes.)
(I don't get the bit about the zero bytes, unless the char* strings are
actually 16-bit wide character sequences where the even bytes will be
zero for ASCII text.)
--
bartc
[toc] | [prev] | [next] | [standalone]
| From | Bart Smissaert <bart.smissaert@gmail.com> |
|---|---|
| Date | 2018-02-03 03:37 -0800 |
| Message-ID | <99f23855-f67a-47b2-a313-e52f3787f921@googlegroups.com> |
| In reply to | #126207 |
Yes, they give the same results. > unless the char* strings are actually 16-bit wide character sequences where the even bytes will be zero for ASCII text. Yes, that is exactly it. There is no other code involved, so it measures just the performance of this function. I think it makes very little difference because memcmp will return directly after the first byte if that byte is zero. This is unless I understand memcmp wrongly. RBS
[toc] | [prev] | [next] | [standalone]
| From | bartc <bc@freeuk.com> |
|---|---|
| Date | 2018-02-03 12:29 +0000 |
| Message-ID | <eWhdC.310374$Og3.127736@fx11.am4> |
| In reply to | #126209 |
On 03/02/2018 11:37, Bart Smissaert wrote: > Yes, they give the same results. > >> unless the char* strings are > actually 16-bit wide character sequences where the even bytes will be > zero for ASCII text. > > Yes, that is exactly it. > > There is no other code involved, so it measures just the performance of this function. > > I think it makes very little difference because memcmp will return directly after the first byte if that byte is zero. I don't think it specifically checks for zero; it will do the same when the first character of the find string doesn't match the next character of the search string. Which I think it will do most of the time especially if the first character of the find string doesn't match any in the find string. What does typical data look like? Are find strings often as long as or longer than searchstrings? (Then memcmp is only called once anyway.) Can you test one particular combination of find and search strings, and do that repeatedly millions of times? Then see if skipping the zero bytes makes a difference. -- bartc
[toc] | [prev] | [next] | [standalone]
| From | Bart Smissaert <bart.smissaert@gmail.com> |
|---|---|
| Date | 2018-02-03 04:37 -0800 |
| Message-ID | <323b5966-2f70-43b0-82af-48ae15f00ed3@googlegroups.com> |
| In reply to | #126210 |
> I don't think it specifically checks for zero I know, but if the first byte is zero, then it won't match the first byte of the string to find, so it will exit. > What does typical data look like? A long search string and a shortish find string > Can you test one particular combination of find and search strings, and do that repeatedly millions of times? That is how I tested. RBS
[toc] | [prev] | [next] | [standalone]
| From | Ben Bacarisse <ben.usenet@bsb.me.uk> |
|---|---|
| Date | 2018-02-03 13:41 +0000 |
| Message-ID | <87po5m1983.fsf@bsb.me.uk> |
| In reply to | #126211 |
Bart Smissaert <bart.smissaert@gmail.com> writes: >> I don't think it specifically checks for zero > > I know, but if the first byte is zero, then it won't match the first > byte of the string to find, so it will exit. Are you sure that both your strings are UTF-16 encoded and with the same byte order? If they are, what you describe won't happen. If they are not, you have a whole other ste of problems! -- Ben.
[toc] | [prev] | [next] | [standalone]
| From | Bart Smissaert <bart.smissaert@gmail.com> |
|---|---|
| Date | 2018-02-03 06:38 -0800 |
| Message-ID | <c8f04906-878b-466d-9be4-dfbe1afd7eb2@googlegroups.com> |
| In reply to | #126212 |
> Are you sure that both your strings are UTF-16 encoded and with the same
byte order?
100% sure as they are normal string variables in VBA.
> I tested with string like this:
strSearch = "abababababababababababababababababababababababababababababababababababababababab"
strFind = "abab" or strFind = "baba"
RBS
[toc] | [prev] | [next] | [standalone]
| From | bartc <bc@freeuk.com> |
|---|---|
| Date | 2018-02-03 15:13 +0000 |
| Message-ID | <dkkdC.405473$NO7.218043@fx43.am4> |
| In reply to | #126216 |
On 03/02/2018 14:38, Bart Smissaert wrote: >> Are you sure that both your strings are UTF-16 encoded and with the same > byte order? > > 100% sure as they are normal string variables in VBA. > >> I tested with string like this: > > strSearch = "abababababababababababababababababababababababababababababababababababababababab" > > > strFind = "abab" or strFind = "baba" OK, I think I've found the issue. If you take out the memcmp part of the two main loops of each function, then one is executed 77 times, the other 39 times, which is what you might have expected. But as it is, each is executed 20 times. On the first, i has values 0,4,8, etc at the start of each iteration (because after a match, it will step i to the next search point, 4 chars on, to do more checks). On the second, i will have values 0,4,8 etc for the same reason. So they are both doing the same work, for this test data. If I changed the find string to "fish", then the second was nearly double the speed. You need to test with more varied data I think. -- bartc
[toc] | [prev] | [next] | [standalone]
| From | bartc <bc@freeuk.com> |
|---|---|
| Date | 2018-02-03 15:19 +0000 |
| Message-ID | <VpkdC.1521855$OX.7252@fx47.am4> |
| In reply to | #126220 |
On 03/02/2018 15:13, bartc wrote:
> On 03/02/2018 14:38, Bart Smissaert wrote:
>>> Are you sure that both your strings are UTF-16 encoded and with the same
>> byte order?
>>
>> 100% sure as they are normal string variables in VBA.
>>
>>> I tested with string like this:
>>
>> strSearch =
>> "abababababababababababababababababababababababababababababababababababababababab"
>>
>>
>>
>> strFind = "abab" or strFind = "baba"
>
> OK, I think I've found the issue. If you take out the memcmp part of the
> two main loops of each function, then one is executed 77 times, the
> other 39 times, which is what you might have expected.
>
> But as it is, each is executed 20 times.
Using "abab". With "baba", one was 21 times, the other 20.
The following is the test setup I used, with various bits of code that I
commented in or out as I did various tests.
(This was done with a compiler that I don't think supports wide strings
so they were expanded manually.)
----------------------------------------
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
char
*searchstr8="abababababababababababababababababababababababababababababababababababababababab";
short searchstr16[500];
char *findstr8="abab";
//char *findstr8="baba";
//char *findstr8="fish";
short findstr16[10];
int nmemcmp1;
int nmemcmp2;
int test1(unsigned char *strSearch, unsigned char *strFind, int nSearch,
int nFind)
{
int i = 0;
int countstring = 0;
if (nSearch == 0 || nFind == 0) {
return 0;
}
for (i = 0; i <= nSearch - nFind; i++) {
////printf("1: i=%d\n",i);
++nmemcmp1;
if (memcmp(strSearch + i, strFind, nFind) == 0) {
countstring++;
i += nFind - 1;
}
}
return countstring;
}
int test2(unsigned char *strSearch, unsigned char *strFind, int nSearch,
int nFind)
{
int i = 0;
int countstring = 0;
if (nSearch == 0 || nFind == 0) {
return 0;
}
for (i = 0; i <= nSearch - nFind; i += 2) {
//printf("2: i=%d\n",i);
++nmemcmp2;
if (memcmp(strSearch + i, strFind, nFind) == 0) {
countstring++;
i = i + (nFind - 2);
}
}
return countstring;
}
int main(void) {
int nfind, nsearch, i, count;
nsearch=strlen(searchstr8);
for (i=0; i<nsearch; ++i) searchstr16[i] = searchstr8[i]; // assume ascii
nfind=strlen(findstr8);
for (i=0; i<nfind; ++i) findstr16[i] = findstr8[i];
//for (i=0; i<1000000; ++i) {
for (i=0; i<1; ++i) {
count += test1((char*)searchstr16,(char*)findstr16, nsearch, nfind);
// count += test2((char*)searchstr16,(char*)findstr16, nsearch, nfind);
}
printf("Nsearch = %d\n",nsearch);
printf("Nfind = %d\n",nfind);
printf("Count = %d\n",count);
printf("MEMCMP1 = %d\n",nmemcmp1);
printf("MEMCMP2 = %d\n",nmemcmp2);
puts("");
}
[toc] | [prev] | [next] | [standalone]
| From | Bart Smissaert <bart.smissaert@gmail.com> |
|---|---|
| Date | 2018-02-03 07:41 -0800 |
| Message-ID | <45850348-632c-4475-bc8c-e0f0cd24fcce@googlegroups.com> |
| In reply to | #126221 |
OK, thanks, I understand it now.
I did some testing here for speed with more realistic (more random) strings like this:
strSearch = "This is just a test, testing for the difference between a one-step loop and a two_step loop, counting strings."
strFind = "test"
lIterations = 10000000
With that (compiled C dll run from VBA via a .tlb) I got
one-step loop: 2978 milli-secs
two-step loop: 1446 milli-secs
So, this is exactly as one would expect.
Note that I can't see any way to get these speeds with VBA or even VB6, so a C dll does have an advantage here.
RBS
[toc] | [prev] | [next] | [standalone]
| From | Richard Damon <Richard@Damon-Family.org> |
|---|---|
| Date | 2018-02-03 11:38 -0500 |
| Message-ID | <RzldC.4593$m03.2028@fx38.iad> |
| In reply to | #126220 |
On 2/3/18 10:13 AM, bartc wrote: > On 03/02/2018 14:38, Bart Smissaert wrote: >>> Are you sure that both your strings are UTF-16 encoded and with the same >> byte order? >> >> 100% sure as they are normal string variables in VBA. >> >>> I tested with string like this: >> >> strSearch = >> "abababababababababababababababababababababababababababababababababababababababab" >> >> >> >> strFind = "abab" or strFind = "baba" > > OK, I think I've found the issue. If you take out the memcmp part of the > two main loops of each function, then one is executed 77 times, the > other 39 times, which is what you might have expected. > > But as it is, each is executed 20 times. > > On the first, i has values 0,4,8, etc at the start of each iteration > (because after a match, it will step i to the next search point, 4 chars > on, to do more checks). > > On the second, i will have values 0,4,8 etc for the same reason. > > So they are both doing the same work, for this test data. > > If I changed the find string to "fish", then the second was nearly > double the speed. > > You need to test with more varied data I think. > One big question for this, how many times should you find the string abba in abbabba? There are two separate occurrences, but they do share the same character. The answer really depends on the initial problem. The method of advancing by the length of the searched for string only works if the intent is to break up the initial string into pieces, some of which are matches, and some are the leftovers, so two matches can't share characters.
[toc] | [prev] | [next] | [standalone]
| From | bartc <bc@freeuk.com> |
|---|---|
| Date | 2018-02-03 13:50 +0000 |
| Message-ID | <o6jdC.548974$5Z7.238835@fx29.am4> |
| In reply to | #126211 |
On 03/02/2018 12:37, Bart Smissaert wrote: >> I don't think it specifically checks for zero > > I know, but if the first byte is zero, then it won't match the first byte of the string to find, so it will exit. > >> What does typical data look like? > > A long search string and a shortish find string > >> Can you test one particular combination of find and search strings, and > do that repeatedly millions of times? > > That is how I tested. If you tell us what those strings are, we can run the same test. (Unless the search string is too long to post.) -- bartc
[toc] | [prev] | [next] | [standalone]
| From | Ben Bacarisse <ben.usenet@bsb.me.uk> |
|---|---|
| Date | 2018-02-03 11:34 +0000 |
| Message-ID | <87607e2tox.fsf@bsb.me.uk> |
| In reply to | #126205 |
Bart Smissaert <bart.smissaert@gmail.com> writes:
> Have simple function to count the number of occurrences of strFind in strSearch:
>
> int sqlite3_CountStringInString(unsigned char *strSearch, unsigned char *strFind, int nSearch, int nFind)
> {
> int i = 0;
> int countstring = 0;
>
> if (nSearch == 0 || nFind == 0) {
> return 0;
> }
>
> for (i = 0; i <= nSearch - nFind; i++) {
> if (memcmp(strSearch + i, strFind, nFind) == 0) {
> countstring++;
> i += nFind - 1;
> }
> }
>
> return countstring;
> }
>
> Now I thought I could speed this up by skipping the zero bytes for
> ansi Unicode strings:
There's a lot to unpack in that statement. Unicode is (at this level)
just a collection of numbers allocated to characters. These numbers
have to be encoded into bytes in some way. If you see a lot of zeros at
every other position, that's because the characters are encoded using
UTF-16. However, not all Unicode code points take up two bytes so
stepping along the bytes in pairs won't always cause you to start the
comparison at the start of a character. I don't know enough about the
detailed properties of UTF-16 to be 100% sure that this is a safe thing
to do.
None of this is will be the answer to your question but it should help
you phrase future questions in more conventional terms. And you should
know that you are tying your search to a particular encoding, rather
than to "Unicode" itself.
> int sqlite3_CountStringInStringA(unsigned char *strSearch, unsigned char *strFind, int nSearch, int nFind)
> {
> int i = 0;
> int countstring = 0;
>
> if (nSearch == 0 || nFind == 0) {
> return 0;
> }
>
> for (i = 0; i <= nSearch - nFind; i += 2) {
> if (memcmp(strSearch + i, strFind, nFind) == 0) {
> countstring++;
> i = i + (nFind - 2);
> }
> }
>
> return countstring;
> }
>
> When I time this it gives however exactly the same speed.
> Any idea why this is?
How did you measure the speed? What were the test strings you used?
You might have to work quite hard to see any difference at all with some
test data. If you are just measuring the overall speed of some
application that uses this search, the effect might simply be swamped by
all the other things going on.
--
Ben.
[toc] | [prev] | [next] | [standalone]
| From | Richard Damon <Richard@Damon-Family.org> |
|---|---|
| Date | 2018-02-03 11:32 -0500 |
| Message-ID | <AuldC.4664$vP3.3700@fx15.iad> |
| In reply to | #126208 |
On 2/3/18 6:34 AM, Ben Bacarisse wrote:
> Bart Smissaert <bart.smissaert@gmail.com> writes:
>
>> Have simple function to count the number of occurrences of strFind in strSearch:
>>
>> int sqlite3_CountStringInString(unsigned char *strSearch, unsigned char *strFind, int nSearch, int nFind)
>> {
>> int i = 0;
>> int countstring = 0;
>>
>> if (nSearch == 0 || nFind == 0) {
>> return 0;
>> }
>>
>> for (i = 0; i <= nSearch - nFind; i++) {
>> if (memcmp(strSearch + i, strFind, nFind) == 0) {
>> countstring++;
>> i += nFind - 1;
>> }
>> }
>>
>> return countstring;
>> }
>>
>> Now I thought I could speed this up by skipping the zero bytes for
>> ansi Unicode strings:
>
> There's a lot to unpack in that statement. Unicode is (at this level)
> just a collection of numbers allocated to characters. These numbers
> have to be encoded into bytes in some way. If you see a lot of zeros at
> every other position, that's because the characters are encoded using
> UTF-16. However, not all Unicode code points take up two bytes so
> stepping along the bytes in pairs won't always cause you to start the
> comparison at the start of a character. I don't know enough about the
> detailed properties of UTF-16 to be 100% sure that this is a safe thing
> to do.
>
UTF-16 is very similar to UTF-8 in this respect. In UTF-16, all code
elements are two bytes long, so if strSearch and strFind are really in
UTF-16 (or UCS-2) then you should only look at strSearch at even byte
offsets for matches.
If you do have a character represented outside the BMP, so it takes more
than one unit to represent the character, it will be composed of two
special code units in a pair of 1024 value bands in a specified order,
so if you start in the middle of that sequence, you are guaranteed that
you can't get an accidental match, as the second unit will never match
the first unit of strFind, just like UTF-8 is encoded so the subsequent
byte units of a multi-byte character can never have a value that could
be a first byte.
As was said, if the string you are looking for is actually ascii
characters, but encoded as UTF-16 or UCS-2, then every other byte will be 0.
One question, are nSearch and nFind actually being given in bytes and
not in code points (2 byte units)? If these are actually UTF-16 strings,
then it is much more common to track things in code point, but if these
are just binary blobs that happen to hold UTF-16 strings, the lengths
would be in bytes.
[toc] | [prev] | [next] | [standalone]
| From | Ben Bacarisse <ben.usenet@bsb.me.uk> |
|---|---|
| Date | 2018-02-03 16:41 +0000 |
| Message-ID | <878tca10ww.fsf@bsb.me.uk> |
| In reply to | #126223 |
Richard Damon <Richard@Damon-Family.org> writes: <snip> > If you do have a character represented outside the BMP, so it takes > more than one unit to represent the character, it will be composed of > two special code units in a pair of 1024 value bands in a specified > order, so if you start in the middle of that sequence, you are > guaranteed that you can't get an accidental match, as the second unit > will never match the first unit of strFind, Thanks. That's the bit I was not sure about. I thought it /probably/ worked like that, but I've never studied the details. -- Ben.
[toc] | [prev] | [next] | [standalone]
| From | Bart Smissaert <bart.smissaert@gmail.com> |
|---|---|
| Date | 2018-02-03 09:02 -0800 |
| Message-ID | <144f383b-5aa9-43c5-b844-ef266e439892@googlegroups.com> |
| In reply to | #126223 |
> One question, are nSearch and nFind actually being given in bytes and not in code points (2 byte units)? I have currently coded this as bytes, but as these are strings (not blobs) I suppose it makes sense to give this as code points, so in VBA/VB6 that will be Len(strSearch) and Len(strFind). RBS
[toc] | [prev] | [next] | [standalone]
| From | Bart Smissaert <bart.smissaert@gmail.com> |
|---|---|
| Date | 2018-02-03 09:06 -0800 |
| Message-ID | <673e3499-ffa7-4c97-a801-5211bc530890@googlegroups.com> |
| In reply to | #126227 |
On Saturday, 3 February 2018 17:02:22 UTC, Bart Smissaert wrote: > > One question, are nSearch and nFind actually being given in bytes and > not in code points (2 byte units)? > > I have currently coded this as bytes, but as these are strings (not blobs) I suppose it makes sense to give this as code points, so in VBA/VB6 that will be > Len(strSearch) and Len(strFind). > > RBS Actually as memcmp works with bytes perhaps it is simpler to keep it as bytes. RBS
[toc] | [prev] | [next] | [standalone]
| From | Richard Damon <Richard@Damon-Family.org> |
|---|---|
| Date | 2018-02-03 12:39 -0500 |
| Message-ID | <bumdC.6317$d03.5530@fx37.iad> |
| In reply to | #126227 |
On 2/3/18 12:02 PM, Bart Smissaert wrote: >> One question, are nSearch and nFind actually being given in bytes and > not in code points (2 byte units)? > > I have currently coded this as bytes, but as these are strings (not blobs) I suppose it makes sense to give this as code points, so in VBA/VB6 that will be > Len(strSearch) and Len(strFind). > > RBS > My question was based on the name, (beginning with sqlite3_) I am not sure of the source of the data and its format. Generally, I though, that sqlite normally stored strings in UTF-8 format (I think it can use UTF-16 though). I personally would generally convert a string storing UTF-16/UCS-2 data into an array of uint16_t if at all possible, just because it is closer to what is really going on (or maybe wchar if I knew it was the right size). This isn't always possible for data coming from outside source. If these are coming from VB, then they may have actually started like that.
[toc] | [prev] | [next] | [standalone]
| From | Bart Smissaert <bart.smissaert@gmail.com> |
|---|---|
| Date | 2018-02-03 11:56 -0800 |
| Message-ID | <a2fef07c-3140-4cb3-a064-88bca05e8b6d@googlegroups.com> |
| In reply to | #126230 |
I deal with UTF-8 strings in SQLite as well, but this thread has to do with strings from VBA or VB6 so mainly ANSI strings. That is where the option to skip the zero bytes comes in handy. > I personally would generally convert a string storing UTF-16/UCS-2 data into an array of uint16_t if at all possible I am not familiar with C coding at all. How would I do that? So what would I use instead of unsigned char *strSearch? This is the current input argument of the countstring function. Also, I am not sure if instead of unsigned char *strSearch I should do: const unsigned char *strSearch RBS
[toc] | [prev] | [next] | [standalone]
| From | Richard Damon <Richard@Damon-Family.org> |
|---|---|
| Date | 2018-02-03 19:54 -0500 |
| Message-ID | <UQsdC.7853$9%2.6576@fx36.iad> |
| In reply to | #126232 |
On 2/3/18 2:56 PM, Bart Smissaert wrote: > I deal with UTF-8 strings in SQLite as well, but this thread has to do with strings from VBA or VB6 so mainly ANSI strings. That is where the option to skip the zero bytes comes in handy. > >> I personally would generally convert a string storing UTF-16/UCS-2 data > into an array of uint16_t if at all possible > > I am not familiar with C coding at all. How would I do that? > So what would I use instead of unsigned char *strSearch? > This is the current input argument of the countstring function. > > Also, I am not sure if instead of unsigned char *strSearch I should do: > const unsigned char *strSearch > > RBS > Since the routine doesn't change the characters, it could use a pointer to const as the parameters, but then technically you should also change the prototype used at the calling site (I am not sure how much of the code is under your control and how much is 'system' provided). My comment on using uint16_t (or as was also suggested char16_t) is dependent on changes upwards in the program, that storing a UTF-16 or UCS-2 string in an array of char is just not the way to do it, you REALLY want to store strings as a sequence of the type that their code-points are based on. To properly make that sort of change you would need to work back up the code chain to where the data started. You keep on describing these as ANSI strings, and that you see all these zeros (which implies the UTF-16/UCS-2 encoding), but true ANSI strings are stored a consecutive 8 bit chars (with somewhere a defined code page for the string). It seems that somehow you have what you is maybe data that all would fit in the Latin-1 code page (what you think of as ANSI), but it is expressed as a 'wide string', but then it should have been using pointers to WCHAR / wchar_t / uint16_t / char16_t, not to unsigned char. Looking up VBA, it looks like you are dealing with a String, also called at time a BSTR, which are passed as pointers to WCHAR which will be a type like uint16_t etc. I don't know where the unsigned char* came from.
[toc] | [prev] | [next] | [standalone]
Page 1 of 5 [1] 2 3 4 5 Next page →
Back to top | Article view | comp.lang.c
csiph-web