Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #112435 > unrolled thread

Program crashing on declaring large char array

Back to article view | Back to comp.lang.c

Contents

  Program crashing on declaring large char array DFS <nospam@dfs.com> - 2017-06-18 00:11 -0400
    Re: Program crashing on declaring large char array Ian Collins <ian-news@hotmail.com> - 2017-06-18 16:34 +1200
      Re: Program crashing on declaring large char array Ian Collins <ian-news@hotmail.com> - 2017-06-18 16:36 +1200
    Re: Program crashing on declaring large char array Barry Schwarz <schwarzb@dqel.com> - 2017-06-17 21:39 -0700
    Re: Program crashing on declaring large char array GOTHIER Nathan <nathan.gothier@gmail.com> - 2017-06-18 09:51 +0200
    Re: Program crashing on declaring large char array already5chosen@yahoo.com - 2017-06-18 03:49 -0700
      Re: Program crashing on declaring large char array already5chosen@yahoo.com - 2017-06-18 03:55 -0700
      Re: Program crashing on declaring large char array supercat@casperkitty.com - 2017-06-19 09:25 -0700
    Re: Program crashing on declaring large char array James Kuyper <jameskuyper@verizon.net> - 2017-06-18 14:10 -0400

#112435 — Program crashing on declaring large char array

I'm reading in a .csv file, counting the lines, and allocating arrays 
based on the line count.

I want a multidimensional array of N rows (lines) by 5 columns.  I need 
each address in the array to store up to 25 characters.

char data[lines][5][25];

If the .csv file is small (few dozen lines) it works, but if the .csv 
file has 10,000 lines, it crashes.

I assume it has something to do with memory allocation.

char data[lines][5][25] = malloc(lines * 5 * 25);  says 'Variable length 
array cannot be initialized'

Ideas?
Thanks

[toc] | [next] | [standalone]

#112437

On 06/18/17 04:11 PM, DFS wrote:
> I'm reading in a .csv file, counting the lines, and allocating arrays
> based on the line count.
>
> I want a multidimensional array of N rows (lines) by 5 columns.  I need
> each address in the array to store up to 25 characters.
>
> char data[lines][5][25];
>
> If the .csv file is small (few dozen lines) it works, but if the .csv
> file has 10,000 lines, it crashes.
>
> I assume it has something to do with memory allocation.

It doesn't have anything to do with memory allocation because you don't 
allocate any memory!

What is does show is you have exceed the stack size (let's assume there 
is a stack...) limit for your implementation.

> char data[lines][5][25] = malloc(lines * 5 * 25);  says 'Variable length
> array cannot be initialized'

Assuming "lines" isn't a compile time constant, you are creating a 
variable length array (VLA) which, as your compiler is happy to report, 
can not be initialised.

You need to allocate a block of rows, something like:

typedef char Row[5][25];

Row* data = malloc( lines * sizeof *stuff );

Then you can access the data by line.

-- 
Ian

[toc] | [prev] | [next] | [standalone]

#112438

On 06/18/17 04:34 PM, Ian Collins wrote:
> On 06/18/17 04:11 PM, DFS wrote:
>> I'm reading in a .csv file, counting the lines, and allocating arrays
>> based on the line count.
>>
>> I want a multidimensional array of N rows (lines) by 5 columns.  I need
>> each address in the array to store up to 25 characters.
>>
>> char data[lines][5][25];
>>
>> If the .csv file is small (few dozen lines) it works, but if the .csv
>> file has 10,000 lines, it crashes.
>>
>> I assume it has something to do with memory allocation.
>
> It doesn't have anything to do with memory allocation because you don't
> allocate any memory!
>
> What is does show is you have exceed the stack size (let's assume there
> is a stack...) limit for your implementation.
>
>> char data[lines][5][25] = malloc(lines * 5 * 25);  says 'Variable length
>> array cannot be initialized'
>
> Assuming "lines" isn't a compile time constant, you are creating a
> variable length array (VLA) which, as your compiler is happy to report,
> can not be initialised.
>
> You need to allocate a block of rows, something like:
>
> typedef char Row[5][25];
>
> Row* data = malloc( lines * sizeof *stuff );

Rubbish cut-n-paste:

Row* data = malloc( lines * sizeof *data );

-- 
Ian

[toc] | [prev] | [next] | [standalone]

#112440

On Sun, 18 Jun 2017 00:11:33 -0400, DFS <nospam@dfs.com> wrote:

>I'm reading in a .csv file, counting the lines, and allocating arrays 
>based on the line count.
>
>I want a multidimensional array of N rows (lines) by 5 columns.  I need 
>each address in the array to store up to 25 characters.
>
>char data[lines][5][25];
>
>If the .csv file is small (few dozen lines) it works, but if the .csv 
>file has 10,000 lines, it crashes.

It appears you are attempting to define an array that requires more
data than is available for automatic objects.  You could try declaring
the object as static.

>I assume it has something to do with memory allocation.

Allocation for automatic variables, not to be confused with dynamic
allocation.

>char data[lines][5][25] = malloc(lines * 5 * 25);  says 'Variable length 
>array cannot be initialized'

The value returned by malloc is a pointer.  Therefore, it must be
assigned to a pointer, not to an array.  Furthermore, you can never
assign a value to an array, only to an element of the array if that
element has a suitable type.

You want the pointer value returned by malloc to point to the first
line in the allocated area.  This line consists of 5 columns, each
consisting of 25 characters.  Therefore, the type of this pointer is
char(*)[5][25].  So your assignment statement should look like
    char (*data)[5][25] = ...;

The first line in the array is denoted by data[0].  This object is an
array of 5 array of 25 char.  The first column is in line j is denoted
by data[j][0].  This object is an array of 25 char.  The fist
character in line j row k is denoted by data[j][k][0].  This object
has type char.

Naturally, you will verify that the allocation was successful before
attempting to store data in the allocated memory.

-- 
Remove del for email

[toc] | [prev] | [next] | [standalone]

#112446

On Sun, 18 Jun 2017 00:11:33 -0400
DFS <nospam@dfs.com> wrote:

> Ideas?

Define a structure to process each row by field.

[toc] | [prev] | [next] | [standalone]

#112451

On Sunday, June 18, 2017 at 7:10:29 AM UTC+3, DFS wrote:
> I'm reading in a .csv file, counting the lines, and allocating arrays 
> based on the line count.
> 
> I want a multidimensional array of N rows (lines) by 5 columns.  I need 
> each address in the array to store up to 25 characters.
> 
> char data[lines][5][25];
> 
> If the .csv file is small (few dozen lines) it works, but if the .csv 
> file has 10,000 lines, it crashes.
> 
> I assume it has something to do with memory allocation.

Sounds like your implementation allocates storage for automatic variable-length arrays (VLA) on stack.
My personal opinion is that quality implementation shell choose between allocation on stack and allocation on heap dynamically (in later case with automatic generation of free()), but such smart strategy certainly not mandated by the standard.

The simplest (but not necessarily the best) solution, assuming that you don't want to handle REALLY BIG .csv files, is to increase the size of the stack by system-specific mean.

On Linux it would be setrlimit().

On Windows the easiest would be to force bigger stack via linker option.
According to my understanding, it's not possible on Windows to increase the size of stack of running thread.
Or, may be SetThreadStackGuarantee() does it? I can't figure it out from MS docs and right now have no desire for experiments.

> 
> char data[lines][5][25] = malloc(lines * 5 * 25);  says 'Variable length 
> array cannot be initialized'
> 
> Ideas?
> Thanks

char *data[5][25] = ...

It works and is perfectly practical, but less interesting than VLA.

[toc] | [prev] | [next] | [standalone]

#112453

On Sunday, June 18, 2017 at 1:49:41 PM UTC+3, already...@yahoo.com wrote:
> On Sunday, June 18, 2017 at 7:10:29 AM UTC+3, DFS wrote:
> > 
> > char data[lines][5][25] = malloc(lines * 5 * 25);  says 'Variable length 
> > array cannot be initialized'
> > 
> > Ideas?
> > Thanks
> 
> char *data[5][25] = ...
> 
> It works and is perfectly practical, but less interesting than VLA.

Sorry, of course Barry Schwarz is correct, the proper syntax is 
char (*data)[5][25] = ...

However, if even very experienced C programmer like myself got it wrong, it demonstrates that Ian Collins is "more correct" than Barry Schwarz.

[toc] | [prev] | [next] | [standalone]

#112587

On Sunday, June 18, 2017 at 5:49:41 AM UTC-5, already...@yahoo.com wrote:
> My personal opinion is that quality implementation shell choose between allocation on stack and allocation on heap dynamically (in later case with automatic generation of free()), but such smart strategy certainly not mandated by the standard.

I don't think that's really practical, since an implementation will generally
have no way of knowing how much stack a function can afford to use without
breaking nested functions.  If an implementation will ever need to allocate
variable-length arrays somewhere other than the stack, having it do so always
would generally be faster than having it do so sometimes, especially if the
implementation has a separate heap it can use for such purposes.  A set of
temporary-heap functions with LIFO semantics would be more generally useful
than VLAs, would allow controlled error recovery, and would be supportable
with portable [though not necessarily optimal] on any system (in all cases
where code with lifo_alloc/lifo_release would have defined behavior, it
would have the same defined behavior if those were replaced with
malloc/free; on the other hand, if a program repeatedly does something like:

    tempbuff = lifo_alloc(worstcase_size);
    long_term_ptr1 = malloc(something_small);
    ...
    long_term_ptr2 = malloc(actual_size);
    memcpy(*real_ptr, tempbuff, actual_size);
    lifo_release(tempbuff);
    long_term_ptr3 = malloc(something_small);

Allocations on the real heap could be placed sequentially; had the program
used malloc/free or malloc/realloc instead of the lifo functions, the
placement of small objects relative to the temporary storage would have
resulted in memory fragmentation.

[BTW, I'd like to see a standard means via which implementations could
request that failed allocations trap rather than returning null.  Having
an allocation function indicate a recoverable failure is useful when an
application is prepared to recover from such failure, but imposes a
useless burden on client code that would treat all such failures as fatal
errors.]

[toc] | [prev] | [next] | [standalone]

#112482

On 06/18/2017 12:11 AM, DFS wrote:
> I'm reading in a .csv file, counting the lines, and allocating arrays 
> based on the line count.
> 
> I want a multidimensional array of N rows (lines) by 5 columns.  I need 
> each address in the array to store up to 25 characters.
> 
> char data[lines][5][25];
> 
> If the .csv file is small (few dozen lines) it works, but if the .csv 
> file has 10,000 lines, it crashes.
> 
> I assume it has something to do with memory allocation.
> 
> char data[lines][5][25] = malloc(lines * 5 * 25);  says 'Variable length 
> array cannot be initialized'

That line does three thrings:

It creates a VLA.
It calls malloc() to attempt to dynamically allocate a block of memory
that happens to be the same size as that array.
It attempts to use the pointer returned by the call to initialize the
entire array.

There's advantages and disadvantages to be considered when deciding
between a VLA and dynamically allocated memory, but it seldom makes
sense to do both to store the same data. However, trying to initialize
an array of char using a pointer is meaningless, and not permitted.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.c

csiph-web