Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #147504 > unrolled thread

So here's a fun challenge

Started byDFS <nospam@dfs.com>
First post2019-11-02 10:53 -0400
Last post2019-11-04 15:33 -0500
Articles 7 on this page of 27 — 8 participants

Back to article view | Back to comp.lang.c

Contents

  So here's a fun challenge DFS <nospam@dfs.com> - 2019-11-02 10:53 -0400
    Re: So here's a fun challenge Bart <bc@freeuk.com> - 2019-11-02 17:08 +0000
      Re: So here's a fun challenge Richard Damon <Richard@Damon-Family.org> - 2019-11-02 15:01 -0400
        Re: So here's a fun challenge Bart <bc@freeuk.com> - 2019-11-02 19:38 +0000
          Re: So here's a fun challenge Richard Damon <Richard@Damon-Family.org> - 2019-11-02 16:07 -0400
          Re: So here's a fun challenge DFS <nospam@dfs.com> - 2019-11-03 09:29 -0500
    Re: So here's a fun challenge Bart <bc@freeuk.com> - 2019-11-02 20:36 +0000
      Re: So here's a fun challenge DFS <nospam@dfs.com> - 2019-11-03 09:33 -0500
        Re: So here's a fun challenge Paul <nospam@needed.invalid> - 2019-11-03 20:53 -0500
    Re: So here's a fun challenge Ian Collins <ian-news@hotmail.com> - 2019-11-03 14:33 +1300
      Re: So here's a fun challenge Ian Collins <ian-news@hotmail.com> - 2019-11-03 14:35 +1300
        Re: So here's a fun challenge DFS <nospam@dfs.com> - 2019-11-03 00:18 -0400
          Re: So here's a fun challenge Ian Collins <ian-news@hotmail.com> - 2019-11-03 18:02 +1300
            Re: So here's a fun challenge Bart <bc@freeuk.com> - 2019-11-03 11:19 +0000
              Re: So here's a fun challenge Ian Collins <ian-news@hotmail.com> - 2019-11-04 00:33 +1300
            Re: So here's a fun challenge DFS <nospam@dfs.com> - 2019-11-03 09:39 -0500
      Re: So here's a fun challenge Autist <autist69@gmail.com> - 2019-11-03 11:16 +0100
        Re: So here's a fun challenge Autist <autist69@gmail.com> - 2019-11-03 11:29 +0100
        Re: So here's a fun challenge Ian Collins <ian-news@hotmail.com> - 2019-11-03 23:34 +1300
        Re: So here's a fun challenge Melzzzzz <Melzzzzz@zzzzz.com> - 2019-11-03 11:04 +0000
          Re: So here's a fun challenge Autist <autist69@gmail.com> - 2019-11-03 14:56 +0100
            Re: So here's a fun challenge Autist <autist69@gmail.com> - 2019-11-03 15:36 +0100
    Re: So here's a fun challenge nobody@example.org (Scott) - 2019-11-03 17:35 +0000
      Re: So here's a fun challenge Ian Collins <ian-news@hotmail.com> - 2019-11-04 08:03 +1300
      Re: So here's a fun challenge DFS <nospam@dfs.com> - 2019-11-04 10:07 -0500
        Re: So here's a fun challenge nobody@example.org (Scott) - 2019-11-04 16:06 +0000
          Re: So here's a fun challenge DFS <nospam@dfs.com> - 2019-11-04 15:33 -0500

Page 2 of 2 — ← Prev page 1 [2]

#147537

FromAutist <autist69@gmail.com>
Date2019-11-03 14:56 +0100
Message-ID<qpmmat$b8e$1@news.albasani.net>
In reply to#147531
>>       auto pStringIgnoreCaseCmp = []( string *a, string *b ) -> int
>>       {
>> #if defined(_MSC_VER)
>>           // _stricmp seems to be better optimized
>>           return _stricmp( a->c_str(), b->c_str() );
>> #else
>>           unsigned char *ua = (unsigned char *)a->c_str(),
>>                         *ub = (unsigned char *)b->c_str();
>>           int            result;
>>           for( size_t i = 0; (result = (int)tolower( ua[i] ) -
>> (int)tolower( ub[i] )) == 0; ++i )
>>               if( ua[i] == 0 )
>>                   return 0;
>>           return result;
>> #endif
>>       };
> 
> There is also strcasecmp for POSIX.

Ok:

#include ...
...
#if defined(__unix__)
#include <strings.h>
#endif
...
     auto pStringIgnoreCaseCmp = []( string *a, string *b ) -> int
     {
#if defined(_MSC_VER) && defined(blabla)
         // _stricmp seems to be better optimized
         return _stricmp( a->c_str(), b->c_str() );
#elif defined(__unix__)
         return strcasecmp( a->c_str(), b->c_str() );
#else
         unsigned char *ua = (unsigned char *)a->c_str(),
                       *ub = (unsigned char *)b->c_str();
         int            result;
         for( size_t i = 0; (result = (int)tolower( ua[i] ) - 
(int)tolower( ub[i] )) == 0; ++i )
             if( ua[i] == 0 )
                 return 0;
         return result;
#endif
     };

[toc] | [prev] | [next] | [standalone]

#147540

FromAutist <autist69@gmail.com>
Date2019-11-03 15:36 +0100
Message-ID<qpmom8$509$1@news.albasani.net>
In reply to#147537
      auto pStringIgnoreCaseCmp = []( string *a, string *b ) -int
      {
#if defined(_MSC_VER)
          // _stricmp seems to be better optimized
          return _stricmp( a->c_str(), b->c_str() );
#elif defined(__unix__)
          return strcasecmp( a->c_str(), b->c_str() );
#else
          unsigned char *ua = (unsigned char *)a->c_str(),
                        *ub = (unsigned char *)b->c_str();
          int            result;
          for( size_t i = 0; (result = (int)tolower( ua[i] ) -
(int)tolower( ub[i] )) == 0; ++i )
              if( ua[i] == 0 )
                  return 0;
          return result;
#endif
      };

Just removed && defined(blablabla).
This was to temporarily disable _stricmp().

[toc] | [prev] | [next] | [standalone]

#147547

Fromnobody@example.org (Scott)
Date2019-11-03 17:35 +0000
Message-ID<5dbf0295.12592219@core>
In reply to#147504
On Sat, 2 Nov 2019 10:53:01 -0400, DFS <nospam@dfs.com> wrote:

>- create 2 (equal or unequal sized) arrays of random strings of random
>    lengths from low to high, ie 2 to 4, or 8 to 16, etc
>- manually add a few of the same values to each array to ensure dupes
>- print the top and bottom 10 to show initial data set
>- combine the arrays
>- case-insensitive sort the arrays (I use qsort but any method is fine)
>- dedupe the combined array
>- add the unique vals to a new array that's sized appropriately
>- print the top and bottom 10 to show final sorted data

Data points for you - Win10, Intel i5-8600 @3.6GHz. I just copied your
code (I may attempt my own later):

randclc 1000000 8 16
= 1.30 s
python randclc.py 1000000 8 16
= 8.61 s (6.62x)

randclc 10000000 8 16
= 18.95 s
python randclc.py 10000000 8 16
= 95.10 s (5.02x)

randclc 50000000 8 16
=  124.19 s
python randclc.py 50000000 8 16
=  did not finish:
Traceback (most recent call last):
  File "randclc.py", line 9, in <module>
    randoms1.append(''.join([random.choice(string.ascii_letters +
string.digits) for x in xrange(random.randint(lo,hi))]))
MemoryError

randclc 100000000 8 16
= 279.29 s
python randclc.py 100000000 8 16
= did not attempt

I don't know enough about Python to say why it errored out. There
should be plenty of RAM, unless it's using a lot more than I'd guess.

[toc] | [prev] | [next] | [standalone]

#147550

FromIan Collins <ian-news@hotmail.com>
Date2019-11-04 08:03 +1300
Message-ID<h28mjmFa2cqU10@mid.individual.net>
In reply to#147547
On 04/11/2019 06:35, Scott wrote:
> On Sat, 2 Nov 2019 10:53:01 -0400, DFS <nospam@dfs.com> wrote:
> 
>> - create 2 (equal or unequal sized) arrays of random strings of random
>>     lengths from low to high, ie 2 to 4, or 8 to 16, etc
>> - manually add a few of the same values to each array to ensure dupes
>> - print the top and bottom 10 to show initial data set
>> - combine the arrays
>> - case-insensitive sort the arrays (I use qsort but any method is fine)
>> - dedupe the combined array
>> - add the unique vals to a new array that's sized appropriately
>> - print the top and bottom 10 to show final sorted data
> 
> Data points for you - Win10, Intel i5-8600 @3.6GHz. I just copied your
> code (I may attempt my own later):

<snip>

> randclc 100000000 8 16
> = 279.29 s

I wonder if the platform is playing a part?  On my Ryzen 2700 
(reasonably but not very fast) with Ubuntu 18.04, the C version:

14.9726s to create 100000000 random string elements (+ add dupes)
qsorting... 76.1025s
program ran in 103.0170s seconds
real	1m43.187s

C++:
12.8076S to create 100000000 random string elements (+ add dupes)
40.4154S to sort
1.10336S to dedup
real	0m56.337s

Which also highlights the difference between sorting in C and C++.

-- 
Ian.

[toc] | [prev] | [next] | [standalone]

#147578

FromDFS <nospam@dfs.com>
Date2019-11-04 10:07 -0500
Message-ID<s9XvF.7692$522.4454@fx05.iad>
In reply to#147547
On 11/3/2019 12:35 PM, Scott wrote:
> On Sat, 2 Nov 2019 10:53:01 -0400, DFS <nospam@dfs.com> wrote:
> 
>> - create 2 (equal or unequal sized) arrays of random strings of random
>>     lengths from low to high, ie 2 to 4, or 8 to 16, etc
>> - manually add a few of the same values to each array to ensure dupes
>> - print the top and bottom 10 to show initial data set
>> - combine the arrays
>> - case-insensitive sort the arrays (I use qsort but any method is fine)
>> - dedupe the combined array
>> - add the unique vals to a new array that's sized appropriately
>> - print the top and bottom 10 to show final sorted data
> 
> Data points for you - Win10, Intel i5-8600 @3.6GHz. I just copied your
> code (I may attempt my own later):
> 
> randclc 1000000 8 16
> = 1.30 s
> python randclc.py 1000000 8 16
> = 8.61 s (6.62x)
> 
> randclc 10000000 8 16
> = 18.95 s
> python randclc.py 10000000 8 16
> = 95.10 s (5.02x)
> 
> randclc 50000000 8 16
> =  124.19 s
> python randclc.py 50000000 8 16
> =  did not finish:
> Traceback (most recent call last):
>    File "randclc.py", line 9, in <module>
>      randoms1.append(''.join([random.choice(string.ascii_letters +
> string.digits) for x in xrange(random.randint(lo,hi))]))
> MemoryError


I got it to run for 50M elements (I have 8GB memory):

$ python random_strings_clc.py 50000000 8 16
  1. 00001WBS
  2. 0000v7HYK
  3. 0001jWQ9
  4. 0001q2delxg6iUZ
  5. 0002jFMXjtC
  6. 0002lLMqj
  7. 0002LWUk7b
  8. 0002r0jp
  9. 00037hk3a1GD
10. 0003fWNinSq
49999999. ZzzzTXKg9MyrS7
50000000. ZZzZu4ReG4SXdo
50000001. ZZZZuCBAD4CmAh
50000002. zzzzUkuDUaMfl
50000003. ZzZZUlMW7DFE7
50000004. zzZzUSb65
50000005. ZZzzWf7r
50000006. zzzzxQyrk0fIu
50000007. ZZZZy0o2kEtA
50000008. zZZZYsU42
program ran in: 2.26e+03 seconds (37.7 minutes)



> randclc 100000000 8 16
> = 279.29 s
> python randclc.py 100000000 8 16
> = did not attempt
> 
> I don't know enough about Python to say why it errored out. There
> should be plenty of RAM, unless it's using a lot more than I'd guess.


Thanks for the info.  5x to 7x slower than C isn't bad python performance.

[toc] | [prev] | [next] | [standalone]

#147582

Fromnobody@example.org (Scott)
Date2019-11-04 16:06 +0000
Message-ID<5dc04b39.97037391@core>
In reply to#147578
On Mon, 4 Nov 2019 10:07:33 -0500, DFS <nospam@dfs.com> wrote:

>I got it to run for 50M elements (I have 8GB memory):

I have 32Gb, and I'd expect virtual memory to kick in anyway. Actual
memory exhaustion is practically unknown today. Which version of
Python did you use? Mine is slightly out of date (2.6.6).

[toc] | [prev] | [next] | [standalone]

#147588

FromDFS <nospam@dfs.com>
Date2019-11-04 15:33 -0500
Message-ID<zW%vF.7267$0X7.2307@fx09.iad>
In reply to#147582
On 11/4/2019 11:06 AM, Scott wrote:
> On Mon, 4 Nov 2019 10:07:33 -0500, DFS <nospam@dfs.com> wrote:
> 
>> I got it to run for 50M elements (I have 8GB memory):
> 
> I have 32Gb, and I'd expect virtual memory to kick in anyway. Actual
> memory exhaustion is practically unknown today. Which version of
> Python did you use? Mine is slightly out of date (2.6.6).


2.7.15  (May 2018)

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.lang.c

csiph-web