Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #11661 > unrolled thread

malloc and maximum size

Back to article view | Back to comp.lang.c

Contents

  malloc and maximum size "V.Subramanian, India" <subramanian100in@yahoo.com> - 2011-10-14 06:31 -0700
    Re: malloc and maximum size gordonb.pey1d@burditt.org (Gordon Burditt) - 2011-10-14 09:45 -0500
      Re: malloc and maximum size Lowell Gilbert <lgusenet@be-well.ilk.org> - 2011-10-14 11:10 -0400
        Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-14 11:40 -0400
          Re: malloc and maximum size Lowell Gilbert <lgusenet@be-well.ilk.org> - 2011-10-14 15:00 -0400
            Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-14 15:17 -0400
              Re: malloc and maximum size Lowell Gilbert <lgusenet@be-well.ilk.org> - 2011-10-14 15:46 -0400
                Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-14 16:38 -0400
                  Re: malloc and maximum size Ike Naar <ike@iceland.freeshell.org> - 2011-10-14 21:59 +0000
                    Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-14 18:26 -0400
                  Re: malloc and maximum size Lowell Gilbert <lgusenet@be-well.ilk.org> - 2011-10-14 18:42 -0400
                    Re: malloc and maximum size Keith Thompson <kst-u@mib.org> - 2011-10-14 16:27 -0700
                      Re: malloc and maximum size Richard Damon <news.x.richarddamon@xoxy.net> - 2011-10-14 20:41 -0400
                        Re: malloc and maximum size Keith Thompson <kst-u@mib.org> - 2011-10-14 18:31 -0700
                          Re: malloc and maximum size Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-10-15 02:58 +0100
                            Re: malloc and maximum size Keith Thompson <kst-u@mib.org> - 2011-10-14 22:11 -0700
                        Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-14 22:59 -0400
                          Re: malloc and maximum size Richard Damon <news.x.richarddamon@xoxy.net> - 2011-10-14 23:44 -0400
                            Re: malloc and maximum size Keith Thompson <kst-u@mib.org> - 2011-10-14 22:56 -0700
                            Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-15 07:52 -0400
                            Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 16:54 -0800
                          Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 16:51 -0800
                      Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-14 22:51 -0400
                        Re: malloc and maximum size Keith Thompson <kst-u@mib.org> - 2011-10-14 22:48 -0700
                          Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-15 07:54 -0400
                            Re: malloc and maximum size Keith Thompson <kst-u@mib.org> - 2011-10-15 11:25 -0700
                              Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-15 21:21 -0400
                                Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 17:41 -0800
                                Re: malloc and maximum size Kaz Kylheku <kaz@kylheku.com> - 2012-01-25 02:11 +0000
                        Re: malloc and maximum size Richard Damon <news.x.richarddamon@xoxy.net> - 2011-10-15 11:50 -0400
                          Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-15 21:38 -0400
                            Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 17:33 -0800
                          Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 17:06 -0800
                        Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 16:58 -0800
                          Re: malloc and maximum size Kaz Kylheku <kaz@kylheku.com> - 2012-01-25 01:23 +0000
                            Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-31 16:31 -0800
                    Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-14 22:20 -0400
                  Re: malloc and maximum size Keith Thompson <kst-u@mib.org> - 2011-10-14 16:03 -0700
                    Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-14 23:01 -0400
                    Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 17:35 -0800
                      Re: malloc and maximum size Kaz Kylheku <kaz@kylheku.com> - 2012-01-25 02:33 +0000
                        Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-31 17:14 -0800
                Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 16:45 -0800
              Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 16:32 -0800
                Re: malloc and maximum size Keith Thompson <kst-u@mib.org> - 2012-01-25 02:40 -0800
                  Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-31 17:27 -0800
            Re: malloc and maximum size Richard Damon <news.x.richarddamon@xoxy.net> - 2011-10-14 16:17 -0400
              Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-14 16:52 -0400
                Re: malloc and maximum size Keith Thompson <kst-u@mib.org> - 2011-10-14 16:12 -0700
                  Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-14 23:04 -0400
            Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 16:36 -0800
      Re: malloc and maximum size Ike Naar <ike@iceland.freeshell.org> - 2011-10-14 16:49 +0000
    Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-14 11:05 -0400
    Re: malloc and maximum size Nobody <nobody@nowhere.com> - 2011-10-14 16:11 +0100
    Re: malloc and maximum size Kenneth Brody <kenbrody@spamcop.net> - 2011-10-14 12:24 -0400
    Re: malloc and maximum size jt@toerring.de (Jens Thoms Toerring) - 2011-10-14 17:10 +0000
    Re: malloc and maximum size Stephen Sprunk <stephen@sprunk.org> - 2011-10-14 13:43 -0500
      Re: malloc and maximum size Keith Thompson <kst-u@mib.org> - 2011-10-14 12:03 -0700
        Re: malloc and maximum size jt@toerring.de (Jens Thoms Toerring) - 2011-10-14 20:36 +0000
      Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 16:26 -0800
    Re: malloc and maximum size Seebs <usenet-nospam@seebs.net> - 2011-10-17 17:21 +0000
    Re: malloc and maximum size Shao Miller <sha0.miller@gmail.com> - 2012-01-24 23:49 -0500
      Re: malloc and maximum size Shao Miller <sha0.miller@gmail.com> - 2012-01-24 23:55 -0500

Page 1 of 4  [1] 2 3 4  Next page →

#11661 — malloc and maximum size

This question is only for understanding purpose.

I am using RedHat Linux kenel 2.6.9 on Intel Pentium D dual core
processor.

The malloc function takes size_t type argument. I am using gcc 3.4.3
implementation. Under this implementation, in stdint.h, SIZE_MAX is
defined as follows:
#  define SIZE_MAX              (4294967295U)

SIZE_MAX, defined in stdint.h, is the largest value that can be
represented in type size_t. However when I pass SIZE_MAX as the
argument to malloc, it is unable to allocate SIZE_MAX bytes of memory.
In fact, malloc allocates much much less memory.

Here is the complete program max_size_to_malloc.c

#include <stdlib.h>
#include <stdio.h>
#include <stdint.h>
#include <stddef.h>

int main()
{
        size_t count = 0;

        do
        {
                const size_t size = SIZE_MAX - count;

                char* p = malloc(size);

                if (p)
                {
                        fprintf(stderr,
                                  "%u - %u : ",
                                  SIZE_MAX,
                                  count);

                        fprintf(stderr,
                                 "Allocated %u bytes of memory\n",
                                 size);
                        free(p);
                        p = NULL;
                        return EXIT_SUCCESS;
                }

        } while (++count);
}

This program compiles fine with gcc 3.4.3 as
gcc -std=c99 -pedantic -Wall -Wextra max_size_to_malloc.c

and produces the following output(after quite some time):
4294967295 - 1365266451 : Allocated 2929700844 bytes of memory

What is the reason for allocating this much less memory while the
parameter type of malloc is size_t ?

Normally what factor drives the maximum size of a single chunk of
memory allocated by malloc ?

Please explain.

Thanks
V.Subramanian

[toc] | [next] | [standalone]

#11665

> I am using RedHat Linux kenel 2.6.9 on Intel Pentium D dual core
> processor.
> 
> The malloc function takes size_t type argument. I am using gcc 3.4.3
> implementation. Under this implementation, in stdint.h, SIZE_MAX is
> defined as follows:
> #  define SIZE_MAX              (4294967295U)
> 
> SIZE_MAX, defined in stdint.h, is the largest value that can be
> represented in type size_t. However when I pass SIZE_MAX as the
> argument to malloc, it is unable to allocate SIZE_MAX bytes of memory.

Does your system *HAVE* SIZE_MAX bytes of memory not in use for
other purposes?  (Depending on your OS, swap/page space may or may
not count).

In order to allocate memory, your system needs to have the memory
available, and your process has to have the address space available
to address it, plus your program's code, data, the C library, and
perhaps address space for the kernel.  If you're using 32-bit
pointers, this may be more of a constraint than available memory.

Also, some systems have rules against one process hogging large
amounts of memory (among other reasons, this may be for deadlock
prevention).

> In fact, malloc allocates much much less memory.

Just because a function takes an argument of a particular type does
not mean that all possible values of that type will be valid.  For
example, providing memcpy() with a third argument of 0 invokes
undefined behavior.  Using memcpy() with a size argument of SIZE_MAX
may be a problem since you need, at a minimum, two non-overlapping
objects of size SIZE_MAX, plus room for your code and other data.
There is a good chance your system will not provide this.

On a 64-bit system, size_t may be a 64-bit unsigned integer, but
it is unlikely that you will be able to afford a computer with
SIZE_MAX (typically 2**64 - 1 on such a system) bytes of memory
installed.

[toc] | [prev] | [next] | [standalone]

#11667

gordonb.pey1d@burditt.org (Gordon Burditt) writes:

> On a 64-bit system, size_t may be a 64-bit unsigned integer, but
> it is unlikely that you will be able to afford a computer with
> SIZE_MAX (typically 2**64 - 1 on such a system) bytes of memory
> installed.

Does SIZE_MAX have to be the largest value representable in a size_t?  
The standard just says that it's the "maximum value" for the type, 
so it seems a bit ambiguous.  [Yes, I noticed Gordon's use of the 
word "typically"; I was just wondering.]

-- 
Lowell Gilbert, embedded/networking software engineer
	 http://be-well.ilk.org/~lowell/

[toc] | [prev] | [next] | [standalone]

#11669

On 10/14/2011 11:10 AM, Lowell Gilbert wrote:
...
> Does SIZE_MAX have to be the largest value representable in a size_t?  
> The standard just says that it's the "maximum value" for the type,

What distinction do you see between "largest value representable" if a
type and "maximum value" for a type? The phrases seem synonymous to me.

[toc] | [prev] | [next] | [standalone]

#11686

James Kuyper <jameskuyper@verizon.net> writes:

> On 10/14/2011 11:10 AM, Lowell Gilbert wrote:
> ...
>> Does SIZE_MAX have to be the largest value representable in a size_t?  
>> The standard just says that it's the "maximum value" for the type,
>
> What distinction do you see between "largest value representable" if a
> type and "maximum value" for a type? The phrases seem synonymous to me.

Certainly there's no difference in *most* cases.

For size_t, the type is just defined as the output type for sizeof,
which is defined in turn as yielding the size of its operand.  So I can
see an argument that the "maximum value" of a size_t could be viewed as
the largest value that sizeof could possibly yield on the particular
implementation.  Of course, that can be (and, indeed, often is) smaller
than a size_t could represent.

-- 
Lowell Gilbert, embedded/networking software engineer
	 http://be-well.ilk.org/~lowell/

[toc] | [prev] | [next] | [standalone]

#11688

On 10/14/2011 03:00 PM, Lowell Gilbert wrote:
> James Kuyper <jameskuyper@verizon.net> writes:
> 
>> On 10/14/2011 11:10 AM, Lowell Gilbert wrote:
>> ...
>>> Does SIZE_MAX have to be the largest value representable in a size_t?  
>>> The standard just says that it's the "maximum value" for the type,
>>
>> What distinction do you see between "largest value representable" if a
>> type and "maximum value" for a type? The phrases seem synonymous to me.
> 
> Certainly there's no difference in *most* cases.
> 
> For size_t, the type is just defined as the output type for sizeof,
> which is defined in turn as yielding the size of its operand.  So I can
> see an argument that the "maximum value" of a size_t could be viewed as
> the largest value that sizeof could possibly yield on the particular
> implementation.

Well, the maximum value returnable by sizeof has to be the same as the
maximum value representable by size_t:

	sizeof(char[(size_t)-1]) == (size_t)(-1)

It has been argued that, since it's not possible for the standard's
specifications for the value and type of sizeof(char[SIZE_MAX][2]) to be
simultaneously satisfied, an implementation is free, among other
possibilities, to treat it as having a value that is larger than
SIZE_MAX. Personally, I think such code should be rejected, but it's not
clear that the standard requires, or even allows, it to be rejected.

However, there's no possible basis that I'm aware of for rejecting
sizeof(char[(size_t)-1]).

> ...  Of course, that can be (and, indeed, often is) smaller
> than a size_t could represent.

With respect to your claim "often is" - can you cite any implementation
where the equality expression given above is NOT true?

[toc] | [prev] | [next] | [standalone]

#11689

James Kuyper <jameskuyper@verizon.net> writes:

> On 10/14/2011 03:00 PM, Lowell Gilbert wrote:
>> James Kuyper <jameskuyper@verizon.net> writes:
>> 
>>> On 10/14/2011 11:10 AM, Lowell Gilbert wrote:
>>> ...
>>>> Does SIZE_MAX have to be the largest value representable in a size_t?  
>>>> The standard just says that it's the "maximum value" for the type,
>>>
>>> What distinction do you see between "largest value representable" if a
>>> type and "maximum value" for a type? The phrases seem synonymous to me.
>> 
>> Certainly there's no difference in *most* cases.
>> 
>> For size_t, the type is just defined as the output type for sizeof,
>> which is defined in turn as yielding the size of its operand.  So I can
>> see an argument that the "maximum value" of a size_t could be viewed as
>> the largest value that sizeof could possibly yield on the particular
>> implementation.
>
> Well, the maximum value returnable by sizeof has to be the same as the
> maximum value representable by size_t:
>
> 	sizeof(char[(size_t)-1]) == (size_t)(-1)

Assuming I understand what you're getting at, you're begging the
question here.  [But the expression doesn't make sense to me, so 
the assumption may well be wrong.]

> It has been argued that, since it's not possible for the standard's
> specifications for the value and type of sizeof(char[SIZE_MAX][2]) to be
> simultaneously satisfied, an implementation is free, among other
> possibilities, to treat it as having a value that is larger than
> SIZE_MAX. Personally, I think such code should be rejected, but it's not
> clear that the standard requires, or even allows, it to be rejected.

Right; this argument has gone around a few times, and I don't think
there's a definitive answer either.

> However, there's no possible basis that I'm aware of for rejecting
> sizeof(char[(size_t)-1]).

I assume you mean SIZE_MAX rather than (size_t), in which case I agree.  
But that doesn't mean SIZE_MAX has to be the largest value a size_t can
hold, if there are other limits on how big an object can be.

>> ...  Of course, that can be (and, indeed, often is) smaller
>> than a size_t could represent.
>
> With respect to your claim "often is" - can you cite any implementation
> where the equality expression given above is NOT true?

I think you're misreading my claim.  There certainly are systems with
architectural limitations on how big an object can be, well short of
what fits in a size_t.

For example, the machine I'm posting from has 32-bit words, from which
it gets a 4-gigabyte range for its size_t.  However, the operating
system reserves some of the address space, so no program can ever have
more than 3 gigabytes.  As a result, sizeof can *never* return more than
3 gigabytes.  This prompted me to wonder whether it would be legitimate
for the system's SIZE_MAX to be 3 gigabytes.

Am I making more sense now?
-- 
Lowell Gilbert, embedded/networking software engineer
	 http://be-well.ilk.org/~lowell/

[toc] | [prev] | [next] | [standalone]

#11692

On 10/14/2011 03:46 PM, Lowell Gilbert wrote:
> James Kuyper <jameskuyper@verizon.net> writes:
> 
>> On 10/14/2011 03:00 PM, Lowell Gilbert wrote:
>>> James Kuyper <jameskuyper@verizon.net> writes:
>>>
>>>> On 10/14/2011 11:10 AM, Lowell Gilbert wrote:
>>>> ...
>>>>> Does SIZE_MAX have to be the largest value representable in a size_t?  
>>>>> The standard just says that it's the "maximum value" for the type,
>>>>
>>>> What distinction do you see between "largest value representable" if a
>>>> type and "maximum value" for a type? The phrases seem synonymous to me.
>>>
>>> Certainly there's no difference in *most* cases.
>>>
>>> For size_t, the type is just defined as the output type for sizeof,
>>> which is defined in turn as yielding the size of its operand.  So I can
>>> see an argument that the "maximum value" of a size_t could be viewed as
>>> the largest value that sizeof could possibly yield on the particular
>>> implementation.
>>
>> Well, the maximum value returnable by sizeof has to be the same as the
>> maximum value representable by size_t:
>>
>> 	sizeof(char[(size_t)-1]) == (size_t)(-1)
> 
> Assuming I understand what you're getting at, you're begging the
> question here.  [But the expression doesn't make sense to me, so 
> the assumption may well be wrong.]

I had not intended to beg the question. Per 6.3.1.3p2, (size_t)(-1) is
precisely "the maximum value that can be represented" as a size_t.

char[(size_t)(-1)] is an array type whose size in bytes is (size_t)(-1).
sizeof(char[(size_t)(-1)]) must therefore yield a value which is
(size_t)(-1). Therefore, the largest value that sizeof could possibly
yield must be (size_t)(-1), the same as the maximum representable value.

...
>> However, there's no possible basis that I'm aware of for rejecting
>> sizeof(char[(size_t)-1]).
> 
> I assume you mean SIZE_MAX rather than (size_t),

No, in don't mean SIZE_MAX, since I was trying to address your claim
that it can be less than maximum representable value. I didn't mean
(size_t), either, which is no more than a parenthesized type - I presume
that was an editing error? I meant (size_t)(-1), which is guaranteed by
the standard to be the maximum representable value.

> ... in which case I agree.  
> But that doesn't mean SIZE_MAX has to be the largest value a size_t can
> hold, if there are other limits on how big an object can be.

The key point is that sizeof acts on types, not objects. Even when you
write "sizeof expression", and expression is an lvalue referring to an
object, it doesn't look into the expression to determine which object it
refers to; it just looks at the type of the expression. It doesn't care
whether or not there's any actual object of that type. sizeof(int) ==
sizeof *(int*)0, even though the standard prohibits (int*)0 from
pointing at any actual object.

>>> ...  Of course, that can be (and, indeed, often is) smaller
>>> than a size_t could represent.
>>
>> With respect to your claim "often is" - can you cite any implementation
>> where the equality expression given above is NOT true?
> 
> I think you're misreading my claim.  There certainly are systems with
> architectural limitations on how big an object can be, well short of
> what fits in a size_t.

And sizeof(char[(size_t)(-1)]) exceeds none of those limitations,
because it doesn't cause the creation of any object of the specified type.

> For example, the machine I'm posting from has 32-bit words, from which
> it gets a 4-gigabyte range for its size_t.  However, the operating
> system reserves some of the address space, so no program can ever have
> more than 3 gigabytes.  As a result, sizeof can *never* return more than
> 3 gigabytes.

Not even sizeof(char[3500000000])? For an implementation where
(size_t)(-1) is 4294967295, what gives that implementation permission to
do anything with that expression other than yield a value of
(size_t)3500000000? The implementation is free to issue a diagnostic, of
course, but not to reject the code.

> ... This prompted me to wonder whether it would be legitimate
> for the system's SIZE_MAX to be 3 gigabytes.
> 
> Am I making more sense now?

Not yet. You seem to have missed the point I was making, while I think I
understand yours perfectly. Is my point clearer now?

[toc] | [prev] | [next] | [standalone]

#11696

On 2011-10-14, James Kuyper <jameskuyper@verizon.net> wrote:
> char[(size_t)(-1)] is an array type whose size in bytes is (size_t)(-1).
> sizeof(char[(size_t)(-1)]) must therefore yield a value which is
> (size_t)(-1). Therefore, the largest value that sizeof could possibly
> yield must be (size_t)(-1), the same as the maximum representable value.

Then the behaviour of gcc 4.1.3 on NetBSD 5.1.0 comes as a surprise:
apparently it does not consider char[(size_t)(-1)] a valid array type.

  $ cat a.c
  #include <stddef.h>
  size_t f(void)
  {
    return sizeof(char[(size_t)(-1)]);
  }
  $ gcc -c -ansi -pedantic a.c
  a.c: In function 'f':
  a.c:4: error: size of array 'type name' is too large

Interestingly, it compiles without errors if the array is halved,
i.e. when the type is changed to char[(size_t)(-1) / 2] .

[toc] | [prev] | [next] | [standalone]

#11697

On 10/14/2011 05:59 PM, Ike Naar wrote:
> On 2011-10-14, James Kuyper <jameskuyper@verizon.net> wrote:
>> char[(size_t)(-1)] is an array type whose size in bytes is (size_t)(-1).
>> sizeof(char[(size_t)(-1)]) must therefore yield a value which is
>> (size_t)(-1). Therefore, the largest value that sizeof could possibly
>> yield must be (size_t)(-1), the same as the maximum representable value.
> 
> Then the behaviour of gcc 4.1.3 on NetBSD 5.1.0 comes as a surprise:
> apparently it does not consider char[(size_t)(-1)] a valid array type.
> 
>   $ cat a.c
>   #include <stddef.h>
>   size_t f(void)
>   {
>     return sizeof(char[(size_t)(-1)]);
>   }
>   $ gcc -c -ansi -pedantic a.c
>   a.c: In function 'f':
>   a.c:4: error: size of array 'type name' is too large

Annoying - but I'm not surprised that there's disagreement about this.
SIZE_MAX seems to be one of the more controversial issues about C, and
it really shouldn't be.

> Interestingly, it compiles without errors if the array is halved,
> i.e. when the type is changed to char[(size_t)(-1) / 2] .

[toc] | [prev] | [next] | [standalone]

#11699

James Kuyper <jameskuyper@verizon.net> writes:

> On 10/14/2011 03:46 PM, Lowell Gilbert wrote:
>> James Kuyper <jameskuyper@verizon.net> writes:
>> 
>>> On 10/14/2011 03:00 PM, Lowell Gilbert wrote:
>>>> James Kuyper <jameskuyper@verizon.net> writes:
>>>>
>>>>> On 10/14/2011 11:10 AM, Lowell Gilbert wrote:
>>>>> ...
>>>>>> Does SIZE_MAX have to be the largest value representable in a size_t?  
>>>>>> The standard just says that it's the "maximum value" for the type,
>>>>>
>>>>> What distinction do you see between "largest value representable" if a
>>>>> type and "maximum value" for a type? The phrases seem synonymous to me.
>>>>
>>>> Certainly there's no difference in *most* cases.
>>>>
>>>> For size_t, the type is just defined as the output type for sizeof,
>>>> which is defined in turn as yielding the size of its operand.  So I can
>>>> see an argument that the "maximum value" of a size_t could be viewed as
>>>> the largest value that sizeof could possibly yield on the particular
>>>> implementation.
>>>
>>> Well, the maximum value returnable by sizeof has to be the same as the
>>> maximum value representable by size_t:
>>>
>>> 	sizeof(char[(size_t)-1]) == (size_t)(-1)
>> 
>> Assuming I understand what you're getting at, you're begging the
>> question here.  [But the expression doesn't make sense to me, so 
>> the assumption may well be wrong.]
>
> I had not intended to beg the question. Per 6.3.1.3p2, (size_t)(-1) is
> precisely "the maximum value that can be represented" as a size_t.

Then you were indeed begging the question; the equality is *exactly*
what I'm questioning.

> char[(size_t)(-1)] is an array type whose size in bytes is (size_t)(-1).
> sizeof(char[(size_t)(-1)]) must therefore yield a value which is
> (size_t)(-1). Therefore, the largest value that sizeof could possibly
> yield must be (size_t)(-1), the same as the maximum representable value.

I see; I was assuming you were giving a C expression, since it looked
like one.  Thanks for the explanation.

> The key point is that sizeof acts on types, not objects. Even when you
> write "sizeof expression", and expression is an lvalue referring to an
> object, it doesn't look into the expression to determine which object it
> refers to; it just looks at the type of the expression. It doesn't care
> whether or not there's any actual object of that type. sizeof(int) ==
> sizeof *(int*)0, even though the standard prohibits (int*)0 from
> pointing at any actual object.

Quite right; I was being sloppy there.  I'm not yet convinced that it's
the *key* point, though.

>> For example, the machine I'm posting from has 32-bit words, from which
>> it gets a 4-gigabyte range for its size_t.  However, the operating
>> system reserves some of the address space, so no program can ever have
>> more than 3 gigabytes.  As a result, sizeof can *never* return more than
>> 3 gigabytes.
>
> Not even sizeof(char[3500000000])? For an implementation where
> (size_t)(-1) is 4294967295, what gives that implementation permission to
> do anything with that expression other than yield a value of
> (size_t)3500000000? The implementation is free to issue a diagnostic, of
> course, but not to reject the code.

Okay, I see where you're going with this; the fact that the
implementation can't instantiate the type doesn't keep it from
recognizing that type.  What I'm not getting (and I have, indeed,
looked in the standard) is what *requires* the implementation to
accept the "sizeof(char[3500000000])".  I'm sure the implementation
*can* accept it, and I suspect that most will, but I haven't found
such a requirement.  Your question about what gives it permission is
backwards; the implementation doesn't need permission if it hasn't
been required to do otherwise.

Be well.
-- 
Lowell Gilbert, embedded/networking software engineer
	 http://be-well.ilk.org/~lowell/

[toc] | [prev] | [next] | [standalone]

#11702

Lowell Gilbert <lgusenet@be-well.ilk.org> writes:
> James Kuyper <jameskuyper@verizon.net> writes:
[...]
>> Not even sizeof(char[3500000000])? For an implementation where
>> (size_t)(-1) is 4294967295, what gives that implementation permission to
>> do anything with that expression other than yield a value of
>> (size_t)3500000000? The implementation is free to issue a diagnostic, of
>> course, but not to reject the code.
>
> Okay, I see where you're going with this; the fact that the
> implementation can't instantiate the type doesn't keep it from
> recognizing that type.  What I'm not getting (and I have, indeed,
> looked in the standard) is what *requires* the implementation to
> accept the "sizeof(char[3500000000])".  I'm sure the implementation
> *can* accept it, and I suspect that most will, but I haven't found
> such a requirement.  Your question about what gives it permission is
> backwards; the implementation doesn't need permission if it hasn't
> been required to do otherwise.

I don't think James is going at it backwards.  I will now argue both
sides of this.

What (arguably) requires the implementation to accept
"sizeof(char[3500000000])" is the same thing that requires it to
accept "sizeof(char[10])", namely the sections discussing the sizeof
operator, the char type, integer constants, and array types.

C99 5.2.4.1 discusses translation limits (127 nesting levels of blocks,
63 nesting levels of conditional inclusion, etc.).  It requires an
implementation to accept 65535 bytes in an object, but that doesn't
apply to the above expression, since it doesn't refer to an object.

On the other hand, 5.2.4.1 doesn't even say that an implementation
must always accept 65535 bytes in an object.  It says:

    The implementation shall be able to translate and execute at least
    one program that contains at least one instance of every one of the
    following limits:

followed by a list of limits.  By rejecting "sizeof(char[3500000000])",
gcc isn't violating any requirement in 5.2.4.1; presumably it does
translate and execute that one program (which doesn't include an
instance of "sizeof(char[3500000000])".

Presumably an implementation may legitimately reject a program violating
some limit not listed in 5.2.4.1, such as a trillion-line translation
unit.  So I *think* that rejecting any program referring to a type
bigger than 2**31 bytes is permitted.  But I'm not 100% certain.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#11704

On 10/14/11 7:27 PM, Keith Thompson wrote:
>
> What (arguably) requires the implementation to accept
> "sizeof(char[3500000000])" is the same thing that requires it to
> accept "sizeof(char[10])", namely the sections discussing the sizeof
> operator, the char type, integer constants, and array types.
>

Why should the compiler be forced to allow the definition of a type that 
defines an object bigger than is allowed??

Note that all valid object sizes must fit within a size_t, but that 
doesn't mean that all values of size_t must be valid object sizes. If 
you look at the hardware architecture I posted earlier, on such a 
machine an array of chars with more than 2^24 elements could justifiable 
be rejected as not a valid object. (I will admit that I goofed and 
forgot that SIZE_MAX specified the limit on the *type* size_t and not 
the limit of sizeof).

The compiler limits section does establish a lower limit for the size 
that the compiler could decide is flat out "too big", but that is much 
smaller than is being discussed here.

[toc] | [prev] | [next] | [standalone]

#11706

Richard Damon <news.x.richarddamon@xoxy.net> writes:
> On 10/14/11 7:27 PM, Keith Thompson wrote:
>>
>> What (arguably) requires the implementation to accept
>> "sizeof(char[3500000000])" is the same thing that requires it to
>> accept "sizeof(char[10])", namely the sections discussing the sizeof
>> operator, the char type, integer constants, and array types.
>
> Why should the compiler be forced to allow the definition of a type that 
> defines an object bigger than is allowed??

Why should the compiler be forced to allow *anythihg*?

As I tried to make clear in the part of my article that you snipped,
I'm not at all sure whether rejecting "sizeof(char[3500000000])"
makes a compiler non-conforming or not.  I think the strongest
argument is simply that the standard's treatment of translation
limits is incomplete (and it would be impractical to make it
complete).

> Note that all valid object sizes must fit within a size_t,

I'm not convinced of that.

Here's my counterargument.  "sizeof", like "+", is an operator, If
x and y are of type int, then "x + y" yields a result of type int;
that doesn't imply that int must be big enough to hold the result
of adding any two int values.  Similarly, the fact that "sizeof
(some_type)" is of type size_t doesn't imply that size_t must be
big enough to hold the result of "sizeof (some_type)" for all types.

In particular, I can argue that a conforming compiler can permit

    typedef char[2 * SIZE_MAX] way_too_big;

while rejecting (or having undefined behavior for) an attempt to
evaluate "sizeof (way_too_big)".

In practice, any *sane* implementation should make its size_t
type big enough to hold the size of any type or object that it
can support, and will reject the above declaration of way_too_big.
But implementations are not required to be sane.

Similarly, a sane implementation will always have calloc(SIZE_MAX,
2) return NULL, but a conforming implementation could actually
allocate the object.

On the other hand, if the standard were amended (or even interpreted)
to say that no object can exceed SIZE_MAX bytes, I wouldn't complain.

[snip]

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#11708

Keith Thompson <kst-u@mib.org> writes:
<snip>
> In particular, I can argue that a conforming compiler can permit
>
>     typedef char[2 * SIZE_MAX] way_too_big;

Irrelevant to your point, but you mean, I think,

     typedef char way_too_big[2 * SIZE_MAX];

<snip>
-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#11720

Ben Bacarisse <ben.usenet@bsb.me.uk> writes:
> Keith Thompson <kst-u@mib.org> writes:
> <snip>
>> In particular, I can argue that a conforming compiler can permit
>>
>>     typedef char[2 * SIZE_MAX] way_too_big;
>
> Irrelevant to your point, but you mean, I think,
>
>      typedef char way_too_big[2 * SIZE_MAX];
>
> <snip>

Yes, thanks.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#11711

On 10/14/2011 08:41 PM, Richard Damon wrote:
> On 10/14/11 7:27 PM, Keith Thompson wrote:
>>
>> What (arguably) requires the implementation to accept
>> "sizeof(char[3500000000])" is the same thing that requires it to
>> accept "sizeof(char[10])", namely the sections discussing the sizeof
>> operator, the char type, integer constants, and array types.
>>
> 
> Why should the compiler be forced to allow the definition of a type that 
> defines an object bigger than is allowed??

No type was defined by that expression.

> Note that all valid object sizes must fit within a size_t,

More accurately, all sizeof expressions must have a result that fits
within a size_t. That's true whether they have a type argument or an
expression argument. It's true whether their expression argument is an
lvalue or an rvalue. It's true whether or not there's any actual object
referred to by the lvalue expression that is their argument.

It's also, apparently, true whether or not it's actually possible for
the size to be represented by a size_t, because the standard specifies
no exemption from that requirement just because it can't be met.  By my
standards that constitutes a defect in the standard.
-- 
James Kuyper

[toc] | [prev] | [next] | [standalone]

#11714

On 10/14/11 10:59 PM, James Kuyper wrote:
> On 10/14/2011 08:41 PM, Richard Damon wrote:
>> On 10/14/11 7:27 PM, Keith Thompson wrote:
>>>
>>> What (arguably) requires the implementation to accept
>>> "sizeof(char[3500000000])" is the same thing that requires it to
>>> accept "sizeof(char[10])", namely the sections discussing the sizeof
>>> operator, the char type, integer constants, and array types.
>>>
>>
>> Why should the compiler be forced to allow the definition of a type that
>> defines an object bigger than is allowed??
>
> No type was defined by that expression.
>
>> Note that all valid object sizes must fit within a size_t,
>
> More accurately, all sizeof expressions must have a result that fits
> within a size_t. That's true whether they have a type argument or an
> expression argument. It's true whether their expression argument is an
> lvalue or an rvalue. It's true whether or not there's any actual object
> referred to by the lvalue expression that is their argument.
>
> It's also, apparently, true whether or not it's actually possible for
> the size to be represented by a size_t, because the standard specifies
> no exemption from that requirement just because it can't be met.  By my
> standards that constitutes a defect in the standard.

If there exists a type/object whose size can not be expressed by sizeof, 
then the compiler is going to be obligated to generate a diagnostic and 
fail to compile the program. If it doesn't, it can't meet the 
requirements of the standard. If it allowed the type/object to exist, 
then it is going to need to rely on the ability to fail for exceeding an 
implementation limit.

Due to the various problems that arise by having objects bigger than can 
be expressed by size_t, it is better for the compiler to generate the 
diagnostic on attempting to create the enormous type rather than wait 
for the user to apply sizeof to it.

[toc] | [prev] | [next] | [standalone]

#11722

Richard Damon <news.x.richarddamon@xoxy.net> writes:
> On 10/14/11 10:59 PM, James Kuyper wrote:
>> On 10/14/2011 08:41 PM, Richard Damon wrote:
[...]
>>> Note that all valid object sizes must fit within a size_t,
>>
>> More accurately, all sizeof expressions must have a result that fits
>> within a size_t. That's true whether they have a type argument or an
>> expression argument. It's true whether their expression argument is an
>> lvalue or an rvalue. It's true whether or not there's any actual object
>> referred to by the lvalue expression that is their argument.
>>
>> It's also, apparently, true whether or not it's actually possible for
>> the size to be represented by a size_t, because the standard specifies
>> no exemption from that requirement just because it can't be met.  By my
>> standards that constitutes a defect in the standard.
>
> If there exists a type/object whose size can not be expressed by sizeof, 
> then the compiler is going to be obligated to generate a diagnostic and 
> fail to compile the program. If it doesn't, it can't meet the 
> requirements of the standard. If it allowed the type/object to exist, 
> then it is going to need to rely on the ability to fail for exceeding an 
> implementation limit.

An implementation is *never* required to fail to compile a program
unless it contains a "#error" directive.  Violations of syntax rules and
constraints require a diagnostic; once the diagnostic is issued, the
behavior is undefined.

But do you mean that a diagnostic is required even if the program
doesn't invoke sizeof?  What requirements can't it meet?

> Due to the various problems that arise by having objects bigger than can 
> be expressed by size_t, it is better for the compiler to generate the 
> diagnostic on attempting to create the enormous type rather than wait 
> for the user to apply sizeof to it.

C99 6.5p5:

    If an _exceptional condition_ occurs during the evaluation of
    an expression (that is, if the result is not mathematically
    defined or not in the range of representable values for its
    type), the behavior is undefined.

I think that applies to "sizeof" as much as it applies to "+".
Which means that a program that evaluates "sizeof (char[SIZE_MAX+1])"
has undefined behavior; no diagnostic is required.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#11728

On 10/14/2011 11:44 PM, Richard Damon wrote:
> On 10/14/11 10:59 PM, James Kuyper wrote:
...
>> More accurately, all sizeof expressions must have a result that fits
>> within a size_t. That's true whether they have a type argument or an
>> expression argument. It's true whether their expression argument is an
>> lvalue or an rvalue. It's true whether or not there's any actual object
>> referred to by the lvalue expression that is their argument.
>>
>> It's also, apparently, true whether or not it's actually possible for
>> the size to be represented by a size_t, because the standard specifies
>> no exemption from that requirement just because it can't be met.  By my
>> standards that constitutes a defect in the standard.
> 
> If there exists a type/object whose size can not be expressed by sizeof, 
> then the compiler is going to be obligated to generate a diagnostic and 
> fail to compile the program.

Citation, please? Diagnostics are mandatory only for syntax errors,
constraint violations, and #error directives. Which of those applies to
a program that uses the type char[2][SIZE_MAX]?

If that type is used to define an object, an implementation limit may be
exceeded, but no diagnostic is required when that happens. If the type
is used as an argument for sizeof(), what the standard specifies for the
behavior is an impossibility - but again, no diagnostic is required. If
the type is used in any other way, I don't see any special problems due
to the size of the type.

> ... If it doesn't, it can't meet the 
> requirements of the standard.

If such a type is used neither to define an object nor as the argument
of a sizeof expression, what are the requirements that you're referring
to which cannot be met?
-- 
James Kuyper

[toc] | [prev] | [next] | [standalone]

Page 1 of 4  [1] 2 3 4  Next page →

Back to top | Article view | comp.lang.c

csiph-web