Groups > comp.lang.c > #161972

Re: Allocate length of word vs fixed length

From	David Brown <david.brown@hesbynett.no>
Newsgroups	comp.lang.c
Subject	Re: Allocate length of word vs fixed length
Date	2021-07-19 21:35 +0200
Organization	A noiseless patient Spider
Message-ID	<sd4k6f$e8k$1@dont-email.me> (permalink)
References	<U3iJI.61694$VU3.5811@fx46.iad>

Show all headers | View raw

On 19/07/2021 18:50, dfs wrote:
> Loaded a list of words into an array.
> 
> The 370103 words came from https://github.com/dwyl/english-words
> file = words_alpha.txt
> 
> Tested a couple memory allocation 'strategies' during loading:
> 
> 1. allocate the strlen() of each word
> 2. allocate a fixed length (len of longest word = 32)
> 
> I figured getting strlen(word) each time would be slower than allocating
> a fixed amt of memory, but that wasn't the case.
> 
> Strategy 2 is significantly slower, and uses nearly 3x the memory.
> 
> Also, does anyone have any 'tricks' to make such a file load routine
> faster/more efficient?  Thanks
> 

#define max_word_count 500000
#define max_word_len 32


// Load your word file with:
char * word_file = malloc(max_word_count * max_word_len);
fgets(word_file, max_word_count * max_word_len, fwords);

// Put your words here:
static char words[max_word_count][max_word_len];


Adjust all that to suit - but the point is, don't mess around with
thousands of mallocs and char pointers.  Your PC has 16 MB to spare.  If
you want speed, use static arrays (or a single big malloc's), not vast
numbers of small mallocs and extra pointers.  Similarly, use a single
large read, not lots of tiny reads.

Thread

Allocate length of word vs fixed length dfs <nospam@dfs.com> - 2021-07-19 12:50 -0400
  Re: Allocate length of word vs fixed length Bart <bc@freeuk.com> - 2021-07-19 19:01 +0100
    Re: Allocate length of word vs fixed length dfs <nospam@dfs.com> - 2021-07-19 14:29 -0400
      Re: Allocate length of word vs fixed length Bart <bc@freeuk.com> - 2021-07-19 19:49 +0100
        Re: Allocate length of word vs fixed length dfs <nospam@dfs.com> - 2021-07-19 18:01 -0400
          Re: Allocate length of word vs fixed length Bart <bc@freeuk.com> - 2021-07-19 23:25 +0100
      Re: Allocate length of word vs fixed length scott@slp53.sl.home (Scott Lurndal) - 2021-07-19 20:00 +0000
        Re: Allocate length of word vs fixed length Branimir Maksimovic <branimir.maksimovic@gmail.com> - 2021-07-19 23:29 +0000
      Re: Allocate length of word vs fixed length Real Troll <real.troll@trolls.com> - 2021-07-20 00:30 +0100
        Re: Allocate length of word vs fixed length DFS <nospam@dfs.com> - 2021-07-19 20:40 -0400
          Re: Allocate length of word vs fixed length Real Troll <real.troll@trolls.com> - 2021-07-20 17:00 +0000
      Re: Allocate length of word vs fixed length Kaz Kylheku <563-365-8930@kylheku.com> - 2021-07-24 01:07 +0000
  Re: Allocate length of word vs fixed length David Brown <david.brown@hesbynett.no> - 2021-07-19 21:35 +0200
    Re: Allocate length of word vs fixed length dfs <nospam@dfs.com> - 2021-07-19 18:02 -0400
  Re: Allocate length of word vs fixed length Ike Naar <ike@rie.sdf.org> - 2021-07-19 21:11 +0000
    Re: Allocate length of word vs fixed length dfs <nospam@dfs.com> - 2021-07-19 18:03 -0400
      Re: Allocate length of word vs fixed length Ike Naar <ike@sdf.org> - 2021-07-20 05:28 +0000
  Re: Allocate length of word vs fixed length dfs <nospam@dfs.com> - 2021-07-19 18:21 -0400
    Re: Allocate length of word vs fixed length Bart <bc@freeuk.com> - 2021-07-19 23:29 +0100
      Re: Allocate length of word vs fixed length dfs <nospam@dfs.com> - 2021-07-19 18:40 -0400
  Re: Allocate length of word vs fixed length Peter 'Shaggy' Haywood <phaywood@alphalink.com.au> - 2021-07-20 11:44 +1000
  Re: Allocate length of word vs fixed length Real Troll <real.troll@trolls.com> - 2021-07-20 22:15 +0000
    Re: Allocate length of word vs fixed length dfs <nospam@dfs.com> - 2021-07-20 20:22 -0400
      Re: Allocate length of word vs fixed length dfs <nospam@dfs.com> - 2021-07-21 23:23 -0400

csiph-web