Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #11661 > unrolled thread

malloc and maximum size

Back to article view | Back to comp.lang.c

Contents

  malloc and maximum size "V.Subramanian, India" <subramanian100in@yahoo.com> - 2011-10-14 06:31 -0700
    Re: malloc and maximum size gordonb.pey1d@burditt.org (Gordon Burditt) - 2011-10-14 09:45 -0500
      Re: malloc and maximum size Lowell Gilbert <lgusenet@be-well.ilk.org> - 2011-10-14 11:10 -0400
        Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-14 11:40 -0400
          Re: malloc and maximum size Lowell Gilbert <lgusenet@be-well.ilk.org> - 2011-10-14 15:00 -0400
            Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-14 15:17 -0400
              Re: malloc and maximum size Lowell Gilbert <lgusenet@be-well.ilk.org> - 2011-10-14 15:46 -0400
                Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-14 16:38 -0400
                  Re: malloc and maximum size Ike Naar <ike@iceland.freeshell.org> - 2011-10-14 21:59 +0000
                    Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-14 18:26 -0400
                  Re: malloc and maximum size Lowell Gilbert <lgusenet@be-well.ilk.org> - 2011-10-14 18:42 -0400
                    Re: malloc and maximum size Keith Thompson <kst-u@mib.org> - 2011-10-14 16:27 -0700
                      Re: malloc and maximum size Richard Damon <news.x.richarddamon@xoxy.net> - 2011-10-14 20:41 -0400
                        Re: malloc and maximum size Keith Thompson <kst-u@mib.org> - 2011-10-14 18:31 -0700
                          Re: malloc and maximum size Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-10-15 02:58 +0100
                            Re: malloc and maximum size Keith Thompson <kst-u@mib.org> - 2011-10-14 22:11 -0700
                        Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-14 22:59 -0400
                          Re: malloc and maximum size Richard Damon <news.x.richarddamon@xoxy.net> - 2011-10-14 23:44 -0400
                            Re: malloc and maximum size Keith Thompson <kst-u@mib.org> - 2011-10-14 22:56 -0700
                            Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-15 07:52 -0400
                            Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 16:54 -0800
                          Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 16:51 -0800
                      Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-14 22:51 -0400
                        Re: malloc and maximum size Keith Thompson <kst-u@mib.org> - 2011-10-14 22:48 -0700
                          Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-15 07:54 -0400
                            Re: malloc and maximum size Keith Thompson <kst-u@mib.org> - 2011-10-15 11:25 -0700
                              Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-15 21:21 -0400
                                Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 17:41 -0800
                                Re: malloc and maximum size Kaz Kylheku <kaz@kylheku.com> - 2012-01-25 02:11 +0000
                        Re: malloc and maximum size Richard Damon <news.x.richarddamon@xoxy.net> - 2011-10-15 11:50 -0400
                          Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-15 21:38 -0400
                            Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 17:33 -0800
                          Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 17:06 -0800
                        Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 16:58 -0800
                          Re: malloc and maximum size Kaz Kylheku <kaz@kylheku.com> - 2012-01-25 01:23 +0000
                            Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-31 16:31 -0800
                    Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-14 22:20 -0400
                  Re: malloc and maximum size Keith Thompson <kst-u@mib.org> - 2011-10-14 16:03 -0700
                    Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-14 23:01 -0400
                    Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 17:35 -0800
                      Re: malloc and maximum size Kaz Kylheku <kaz@kylheku.com> - 2012-01-25 02:33 +0000
                        Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-31 17:14 -0800
                Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 16:45 -0800
              Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 16:32 -0800
                Re: malloc and maximum size Keith Thompson <kst-u@mib.org> - 2012-01-25 02:40 -0800
                  Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-31 17:27 -0800
            Re: malloc and maximum size Richard Damon <news.x.richarddamon@xoxy.net> - 2011-10-14 16:17 -0400
              Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-14 16:52 -0400
                Re: malloc and maximum size Keith Thompson <kst-u@mib.org> - 2011-10-14 16:12 -0700
                  Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-14 23:04 -0400
            Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 16:36 -0800
      Re: malloc and maximum size Ike Naar <ike@iceland.freeshell.org> - 2011-10-14 16:49 +0000
    Re: malloc and maximum size James Kuyper <jameskuyper@verizon.net> - 2011-10-14 11:05 -0400
    Re: malloc and maximum size Nobody <nobody@nowhere.com> - 2011-10-14 16:11 +0100
    Re: malloc and maximum size Kenneth Brody <kenbrody@spamcop.net> - 2011-10-14 12:24 -0400
    Re: malloc and maximum size jt@toerring.de (Jens Thoms Toerring) - 2011-10-14 17:10 +0000
    Re: malloc and maximum size Stephen Sprunk <stephen@sprunk.org> - 2011-10-14 13:43 -0500
      Re: malloc and maximum size Keith Thompson <kst-u@mib.org> - 2011-10-14 12:03 -0700
        Re: malloc and maximum size jt@toerring.de (Jens Thoms Toerring) - 2011-10-14 20:36 +0000
      Re: malloc and maximum size Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 16:26 -0800
    Re: malloc and maximum size Seebs <usenet-nospam@seebs.net> - 2011-10-17 17:21 +0000
    Re: malloc and maximum size Shao Miller <sha0.miller@gmail.com> - 2012-01-24 23:49 -0500
      Re: malloc and maximum size Shao Miller <sha0.miller@gmail.com> - 2012-01-24 23:55 -0500

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

#16168

On 2012-01-25, Tim Rentsch <txr@alumni.caltech.edu> wrote:
> Keith Thompson <kst-u@mib.org> writes:
>
>> James Kuyper <jameskuyper@verizon.net> writes:
>> [...]
>>> Not even sizeof(char[3500000000])? For an implementation where
>>> (size_t)(-1) is 4294967295, what gives that implementation permission to
>>> do anything with that expression other than yield a value of
>>> (size_t)3500000000? The implementation is free to issue a diagnostic, of
>>> course, but not to reject the code.
>>
>> Interesting.  For the following program:
>>
>> #include <stdio.h>
>> int main(void) {
>>     printf("%zu\n", sizeof (char[(size_t)-1]));
>>     return 0;
>> }
>>
>> "gcc -std=c99" rejects it with an error message:
>>
>> c.c: In function 'main':
>> c.c:3:5: error: size of unnamed array is too large
>>
>> Change the array size to (size_t)-1/2 and it compiles and prints
>> "2147483647".  Change it to (size_t)-1/2+1 and it rejects it with the
>> same error message.
>>
>> Does this make "gcc -std=c99" non-conforming?
>
> No, because there is no requirement that this program
> be accepted.

If for every feature of the above program, we can deduce what the behavior
should be (and thus in doing so we must find that it does not violate any
limit, or invoke any UB) then we can deduce that this program must be accepted,
or else it represents a test case that shows the implementation to be
nonconforming.

I don't believe that it violates any limits or invokes any UB, which leads me
to conclude that it does show the above gcc installation to be nonconforming.

This all follows from the software engineering sense of what it means to have a
test case, what requirements are, and what it means for a test case to fail.

It might be a test case that is considered "degenerate", which is a code word
for "something that the users probably won't run into or care about".
Or that it's not a "showstopper bug".

But that sort of negotation or classification of defects is outside of the
standard.

Now if you make a program which stresses each of the limists and gcc chokes on
it, then you have not proven a nonconformity. This is because 5.4.2.1
has the effect of excusing the implementors from handling /all/ such programs.
They have to handle only one, and it can be any program from that set;
not restricted to one that is chosen by the users of the implementation.

If you change your program so that it does not exercise all of the limits
(maybe by scaling back just one of them) and if you can still reproduce the
failure, then you have found a nonconformance.

[toc] | [prev] | [next] | [standalone]

#16541

Kaz Kylheku <kaz@kylheku.com> writes:

> On 2012-01-25, Tim Rentsch <txr@alumni.caltech.edu> wrote:
>> Keith Thompson <kst-u@mib.org> writes:
>>
>>> James Kuyper <jameskuyper@verizon.net> writes:
>>> [...]
>>>> Not even sizeof(char[3500000000])? For an implementation where
>>>> (size_t)(-1) is 4294967295, what gives that implementation permission to
>>>> do anything with that expression other than yield a value of
>>>> (size_t)3500000000? The implementation is free to issue a diagnostic, of
>>>> course, but not to reject the code.
>>>
>>> Interesting.  For the following program:
>>>
>>> #include <stdio.h>
>>> int main(void) {
>>>     printf("%zu\n", sizeof (char[(size_t)-1]));
>>>     return 0;
>>> }
>>>
>>> "gcc -std=c99" rejects it with an error message:
>>>
>>> c.c: In function 'main':
>>> c.c:3:5: error: size of unnamed array is too large
>>>
>>> Change the array size to (size_t)-1/2 and it compiles and prints
>>> "2147483647".  Change it to (size_t)-1/2+1 and it rejects it with the
>>> same error message.
>>>
>>> Does this make "gcc -std=c99" non-conforming?
>>
>> No, because there is no requirement that this program
>> be accepted.
>
> If for every feature of the above program, we can deduce what the behavior
> should be (and thus in doing so we must find that it does not violate any
> limit, or invoke any UB) then we can deduce that this program must be accepted,
> or else it represents a test case that shows the implementation to be
> nonconforming.  [snip elaboration]

I don't find any support for this conclusion in the Standard.  The
Standard _does_ require that implementations accept any strictly
conforming program, but the program given above is not strictly
conforming.  I'm not aware of any requirement in the Standard
that any program other than strictly conforming ones be accepted.
There is the one distinguished program (of each implementation's
choosing) that must accepted and executed successfully, but the
program above cannot be that.

In the absence of a requirement to accept a particular program, I
believe implementations are not required to accept it, especially
since for some programs there is such a requirement.

[toc] | [prev] | [next] | [standalone]

#16152

Lowell Gilbert <lgusenet@be-well.ilk.org> writes:

> James Kuyper <jameskuyper@verizon.net> writes:
>
>> On 10/14/2011 03:00 PM, Lowell Gilbert wrote:
>>> James Kuyper <jameskuyper@verizon.net> writes:
>>> 
>>>> On 10/14/2011 11:10 AM, Lowell Gilbert wrote:
>>>> ...
>>>>> Does SIZE_MAX have to be the largest value representable in a size_t?  
>>>>> The standard just says that it's the "maximum value" for the type,
>>>>
>>>> What distinction do you see between "largest value representable" if a
>>>> type and "maximum value" for a type? The phrases seem synonymous to me.
>>> 
>>> Certainly there's no difference in *most* cases.
>>> 
>>> For size_t, the type is just defined as the output type for sizeof,
>>> which is defined in turn as yielding the size of its operand.  So I can
>>> see an argument that the "maximum value" of a size_t could be viewed as
>>> the largest value that sizeof could possibly yield on the particular
>>> implementation.
>>
>> Well, the maximum value returnable by sizeof has to be the same as the
>> maximum value representable by size_t:
>>
>> 	sizeof(char[(size_t)-1]) == (size_t)(-1)
>
> Assuming I understand what you're getting at, you're begging the
> question here.  [But the expression doesn't make sense to me, so 
> the assumption may well be wrong.]
>
>> It has been argued that, since it's not possible for the standard's
>> specifications for the value and type of sizeof(char[SIZE_MAX][2]) to be
>> simultaneously satisfied, an implementation is free, among other
>> possibilities, to treat it as having a value that is larger than
>> SIZE_MAX. Personally, I think such code should be rejected, but it's not
>> clear that the standard requires, or even allows, it to be rejected.
>
> Right; this argument has gone around a few times, and I don't think
> there's a definitive answer either.
>
>> However, there's no possible basis that I'm aware of for rejecting
>> sizeof(char[(size_t)-1]).
>
> I assume you mean SIZE_MAX rather than (size_t), in which case I agree.  
> But that doesn't mean SIZE_MAX has to be the largest value a size_t can
> hold, if there are other limits on how big an object can be.
>
>>> ...  Of course, that can be (and, indeed, often is) smaller
>>> than a size_t could represent.
>>
>> With respect to your claim "often is" - can you cite any implementation
>> where the equality expression given above is NOT true?
>
> I think you're misreading my claim.  There certainly are systems with
> architectural limitations on how big an object can be, well short of
> what fits in a size_t.
>
> For example, the machine I'm posting from has 32-bit words, from which
> it gets a 4-gigabyte range for its size_t.  However, the operating
> system reserves some of the address space, so no program can ever have
> more than 3 gigabytes.  As a result, sizeof can *never* return more than
> 3 gigabytes.  This prompted me to wonder whether it would be legitimate
> for the system's SIZE_MAX to be 3 gigabytes.

The type size_t is an unsigned integer type.  The range of unsigned
integer types is specified in 6.2.6.1 p1;  3 gigabytes isn't one of
the possibilities.  The range of unsigned types depends only on how
many value bits are present in their representation;  nothing else.
That applies to size_t just as much as it does any other unsigned
integer type.

Does that help?

[toc] | [prev] | [next] | [standalone]

#16149

James Kuyper <jameskuyper@verizon.net> writes:

> On 10/14/2011 03:00 PM, Lowell Gilbert wrote:
>> James Kuyper <jameskuyper@verizon.net> writes:
>> 
>>> On 10/14/2011 11:10 AM, Lowell Gilbert wrote:
>>> ...
>>>> Does SIZE_MAX have to be the largest value representable in a size_t?  
>>>> The standard just says that it's the "maximum value" for the type,
>>>
>>> What distinction do you see between "largest value representable" if a
>>> type and "maximum value" for a type? The phrases seem synonymous to me.
>> 
>> Certainly there's no difference in *most* cases.
>> 
>> For size_t, the type is just defined as the output type for sizeof,
>> which is defined in turn as yielding the size of its operand.  So I can
>> see an argument that the "maximum value" of a size_t could be viewed as
>> the largest value that sizeof could possibly yield on the particular
>> implementation.
>
> Well, the maximum value returnable by sizeof has to be the same as the
> maximum value representable by size_t:
>
> 	sizeof(char[(size_t)-1]) == (size_t)(-1)
>
> It has been argued that, since it's not possible for the standard's
> specifications for the value and type of sizeof(char[SIZE_MAX][2]) to be
> simultaneously satisfied, an implementation is free, among other
> possibilities, to treat it as having a value that is larger than
> SIZE_MAX. Personally, I think such code should be rejected, but it's not
> clear that the standard requires, or even allows, it to be rejected.
>
> However, there's no possible basis that I'm aware of for rejecting
> sizeof(char[(size_t)-1]).

Only strictly conforming programs are Standard-ly required to be
accepted.  Any other programs (which this example presuably was on
the implementation in question) don't have to be.

[toc] | [prev] | [next] | [standalone]

#16185

Tim Rentsch <txr@alumni.caltech.edu> writes:
[...]
> Only strictly conforming programs are Standard-ly required to be
> accepted.  Any other programs (which this example presuably was on
> the implementation in question) don't have to be.

I think C99 4p3 contradicts that:

    A program that is correct in all other aspects, operating on
    correct data, containing unspecified behavior shall be a correct
    program and act in accordance with 5.1.2.3.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
    Will write code for food.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#16542

Keith Thompson <kst-u@mib.org> writes:

> Tim Rentsch <txr@alumni.caltech.edu> writes:
> [...]
>> Only strictly conforming programs are Standard-ly required to be
>> accepted.  Any other programs (which this example presuably was on
>> the implementation in question) don't have to be.
>
> I think C99 4p3 contradicts that:
>
>     A program that is correct in all other aspects, operating on
>     correct data, containing unspecified behavior shall be a correct
>     program and act in accordance with 5.1.2.3.

You bring up a good point.  I suppose one could argue that a
non-strictly conforming program that an implemention chooses
not to accept is not "correct in all other aspects", but
certainly the issue is open to debate.

However, even if we grant this exception, it is very narrow.
It speaks only to the possibility of unspecified behavior,
and not (for example) a program exceeding some minimum level
of required implementation limit, so 4p3 doesn't apply to
the program (not shown) under discussion.

[toc] | [prev] | [next] | [standalone]

#11690

On 10/14/11 3:00 PM, Lowell Gilbert wrote:
> James Kuyper<jameskuyper@verizon.net>  writes:
>
>> On 10/14/2011 11:10 AM, Lowell Gilbert wrote:
>> ...
>>> Does SIZE_MAX have to be the largest value representable in a size_t?
>>> The standard just says that it's the "maximum value" for the type,
>>
>> What distinction do you see between "largest value representable" if a
>> type and "maximum value" for a type? The phrases seem synonymous to me.
>
> Certainly there's no difference in *most* cases.
>
> For size_t, the type is just defined as the output type for sizeof,
> which is defined in turn as yielding the size of its operand.  So I can
> see an argument that the "maximum value" of a size_t could be viewed as
> the largest value that sizeof could possibly yield on the particular
> implementation.  Of course, that can be (and, indeed, often is) smaller
> than a size_t could represent.
>

It is quite possible for an architecture to have a smaller value of 
SIZE_MAX than the maximum value that could be placed inside of a size_t.

Take a machine that represents addresses as an 8 bit "Segment" and a 24 
bit "Offset" which via some mapping registers get mapped into memory. 24 
bits is a big enough offset, that the implementation may decide not to 
implement "huge" pointers for objects, thus the largest an object could 
be would be 2^24 bytes, but size_t may well be a 32 bit integer.

[toc] | [prev] | [next] | [standalone]

#11693

On 10/14/2011 04:17 PM, Richard Damon wrote:
...
> It is quite possible for an architecture to have a smaller value of 
> SIZE_MAX than the maximum value that could be placed inside of a size_t.

How does that satisfy the requirement that
"The following object-like macros227) specify the minimum and maximum
limits of integer types corresponding to types defined in other standard
headers." (7.18.3p1)?

If there were a value larger than SIZE_MAX that could be stored in
size_t, then SIZE_MAX would not be the "maximum limit" for that type.

> Take a machine that represents addresses as an 8 bit "Segment" and a 24 
> bit "Offset" which via some mapping registers get mapped into memory. 24 
> bits is a big enough offset, that the implementation may decide not to 
> implement "huge" pointers for objects, thus the largest an object could 
> be would be 2^24 bytes, but size_t may well be a 32 bit integer.

In which case, the limit on size_t is 2^32-1, not 2^24-1.
sizeof(lvalue_referring_to_actual_object) might not be able to exceed
2^24-1, but sizeof(type) could, and so could sizeof(*(type*)0).

If you're talking about an implementation where size_t is a 24-bit
unsigned integer implemented using 24 value bits and 8 padding bits,
then SIZE_MAX could be be 0xFFFFFFFFFFFFF. However, that requires that
emulation of a 24-bit type be performed correctly: (size_t)(-1) has to
be SIZE_MAX, and size_t_expression & 0xFFFF000000000000 must always be 0
for any expression of type size_t. Those 8 padding bits must be
accessible only by type punning with unsigned char, or by expressions
with undefined behavior.

[toc] | [prev] | [next] | [standalone]

#11701

James Kuyper <jameskuyper@verizon.net> writes:
> On 10/14/2011 04:17 PM, Richard Damon wrote:
> ...
>> It is quite possible for an architecture to have a smaller value of 
>> SIZE_MAX than the maximum value that could be placed inside of a size_t.
>
> How does that satisfy the requirement that
> "The following object-like macros227) specify the minimum and maximum
> limits of integer types corresponding to types defined in other standard
> headers." (7.18.3p1)?
>
> If there were a value larger than SIZE_MAX that could be stored in
> size_t, then SIZE_MAX would not be the "maximum limit" for that type.

Agreed.  SIZE_MAX is the maximum limit of the type, not of the values
that the sizeof operator can yield.

[...]
> If you're talking about an implementation where size_t is a 24-bit
> unsigned integer implemented using 24 value bits and 8 padding bits,
> then SIZE_MAX could be be 0xFFFFFFFFFFFFF.

Do you mean 0xFFFFFF?

>                                            However, that requires that
> emulation of a 24-bit type be performed correctly: (size_t)(-1) has to
> be SIZE_MAX, and size_t_expression & 0xFFFF000000000000

0xFF000000?

>                                                         must always be 0
> for any expression of type size_t. Those 8 padding bits must be
> accessible only by type punning with unsigned char, or by expressions
> with undefined behavior.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#11713

On 10/14/2011 07:12 PM, Keith Thompson wrote:
> James Kuyper <jameskuyper@verizon.net> writes:
...
>> If you're talking about an implementation where size_t is a 24-bit
>> unsigned integer implemented using 24 value bits and 8 padding bits,
>> then SIZE_MAX could be be 0xFFFFFFFFFFFFF.
> 
> Do you mean 0xFFFFFF?
> 
>>                                            However, that requires that
>> emulation of a 24-bit type be performed correctly: (size_t)(-1) has to
>> be SIZE_MAX, and size_t_expression & 0xFFFF000000000000
> 
> 0xFF000000?

I spent a lot of time recently writing code with hex masks for 64 bit
types. The shorter, correct versions you've give above didn't look right
- they were too short. :-(
-- 
James Kuyper

[toc] | [prev] | [next] | [standalone]

#16151

Lowell Gilbert <lgusenet@be-well.ilk.org> writes:

> James Kuyper <jameskuyper@verizon.net> writes:
>
>> On 10/14/2011 11:10 AM, Lowell Gilbert wrote:
>> ...
>>> Does SIZE_MAX have to be the largest value representable in a size_t?  
>>> The standard just says that it's the "maximum value" for the type,
>>
>> What distinction do you see between "largest value representable" if a
>> type and "maximum value" for a type? The phrases seem synonymous to me.
>
> Certainly there's no difference in *most* cases.
>
> For size_t, the type is just defined as the output type for
> sizeof, [snip]

That's backwards.  The sizeof operator is defined as
returning a value of type size_t, not the other way
around.  SIZE_MAX is the largest value the type can
hold, not the largest possible value of doing a sizeof.

[toc] | [prev] | [next] | [standalone]

#11676

On 2011-10-14, Gordon Burditt <gordonb.pey1d@burditt.org> wrote:
> Just because a function takes an argument of a particular type does
> not mean that all possible values of that type will be valid.  For
> example, providing memcpy() with a third argument of 0 invokes
> undefined behavior.

7.21.1 seems to explicitly allow memcpy's third argument to be zero:

  7.21.1 String function conventions

  2  Where an argument declared as size_t n specifies the length of
     the array for a function, n can have the value zero on a call
     to that function.

Where is it stated the behaviour is undefined?

[toc] | [prev] | [next] | [standalone]

#11666

On 10/14/2011 09:31 AM, V.Subramanian, India wrote:
> This question is only for understanding purpose.
> 
> I am using RedHat Linux kenel 2.6.9 on Intel Pentium D dual core
> processor.
> 
> The malloc function takes size_t type argument. I am using gcc 3.4.3
> implementation. Under this implementation, in stdint.h, SIZE_MAX is
> defined as follows:
> #  define SIZE_MAX              (4294967295U)
> 
> SIZE_MAX, defined in stdint.h, is the largest value that can be
> represented in type size_t. However when I pass SIZE_MAX as the
> argument to malloc, it is unable to allocate SIZE_MAX bytes of memory.
> In fact, malloc allocates much much less memory.

This is entirely normal. The implementation chose size_t to be big
enough for all of the purposes it needs to be useful for - primarily the
type of sizeof expressions, and for dynamic memory allocations, but
there's many other ways the standard library uses it.

The type chosen for size_t must be an unsigned type big enough to hold
the largest number that could reasonably be used for any of those
purposes; but keep in mind that most platforms nowadays provide hardware
support only for integers of 8, 16, 32, or 64 bytes (and not necessarily
all of those on any given machine); types of other sizes could be
implemented by software emulation, but would be very inefficient. The
maximum amount of memory allocatable by malloc() could be as small as
65536 bytes, and still be large enough to require size_t to be a 32-bit
type.
Moral: don't assume that you can allocate as much as SIZE_MAX bytes; the
actual maximum could be a LOT less.

> Here is the complete program max_size_to_malloc.c
> 
> #include <stdlib.h>
> #include <stdio.h>
> #include <stdint.h>
> #include <stddef.h>
> 
> int main()
> {
>         size_t count = 0;
> 
>         do
>         {
>                 const size_t size = SIZE_MAX - count;
> 
>                 char* p = malloc(size);
> 
>                 if (p)
>                 {
>                         fprintf(stderr,
>                                   "%u - %u : ",
>                                   SIZE_MAX,
>                                   count);
> 
>                         fprintf(stderr,
>                                  "Allocated %u bytes of memory\n",
>                                  size);
>                         free(p);
>                         p = NULL;
>                         return EXIT_SUCCESS;
>                 }
> 
>         } while (++count);

Keeping separate track of count and size serves little purpose. I'd
recommend dropping count, dropping the first fprintf() call, and
replacing the do-while() loop with

	for(size_t size = SIZE_MAX; size; size--)

Also, setting p=NULL is pretty pointless when the lifetime of 'p' will
end almost immediately after you do so. For precisely that reason, a
decently optimizing compiler will simply ignore that statement, but it's
better to simply not write it in the first place. In fact, I normally
try to arrange that pointer variables are free()d immediately before
their lifetime ends, so I won't have to worry about them being misused.

> }
> 
> This program compiles fine with gcc 3.4.3 as
> gcc -std=c99 -pedantic -Wall -Wextra max_size_to_malloc.c
> 
> and produces the following output(after quite some time):
> 4294967295 - 1365266451 : Allocated 2929700844 bytes of memory
> 
> What is the reason for allocating this much less memory while the
> parameter type of malloc is size_t ?
> 
> Normally what factor drives the maximum size of a single chunk of
> memory allocated by malloc ?

Memory fragmentation can make the maximum size of a single allocatable
chunk of memory be MUCH smaller than the total amount of memory
available for allocation.

[toc] | [prev] | [next] | [standalone]

#11668

On Fri, 14 Oct 2011 06:31:28 -0700, V.Subramanian, India wrote:

> What is the reason for allocating this much less memory while the
> parameter type of malloc is size_t ?
> 
> Normally what factor drives the maximum size of a single chunk of
> memory allocated by malloc ?

Availability.

In order to allocate a chunk of N bytes, there must be an unused N-byte
chunk of contiguous address space available, and sufficient storage
(physical RAM or swap) available to "back" it (i.e. to remember whatever
is stored in it).

[toc] | [prev] | [next] | [standalone]

#11674

On 10/14/2011 9:31 AM, V.Subramanian, India wrote:
> This question is only for understanding purpose.
>
> I am using RedHat Linux kenel 2.6.9 on Intel Pentium D dual core
> processor.
>
> The malloc function takes size_t type argument. I am using gcc 3.4.3
> implementation. Under this implementation, in stdint.h, SIZE_MAX is
> defined as follows:
> #  define SIZE_MAX              (4294967295U)
[...]
> and produces the following output(after quite some time):
> 4294967295 - 1365266451 : Allocated 2929700844 bytes of memory
>
> What is the reason for allocating this much less memory while the
> parameter type of malloc is size_t ?
>
> Normally what factor drives the maximum size of a single chunk of
> memory allocated by malloc ?
[...]

There is a difference between "the maximum value you can pass to malloc()" 
and "the largest contiguous chuck of memory currently available which can be 
allocated by malloc".

There are numerous factors (amount of physical and virtual memory on the 
computer, the amount of memory currently being used by all processes, the 
maximum process size allowed with the current system configuration, the 
fragmentation of your process' memory space, and so on) which affect the 
largest chunk of memory that malloc() can return at the moment.

It is quite possible that the same program run on a different Linux box 
would succeed with a nearly 4GB malloc().  It is also quite possible to get 
different numbers with multiple runs on the same system.

-- 
Kenneth Brody

[toc] | [prev] | [next] | [standalone]

#11678

V.Subramanian, India <subramanian100in@yahoo.com> wrote:
> This question is only for understanding purpose.

> I am using RedHat Linux kenel 2.6.9 on Intel Pentium D dual core
> processor.

> The malloc function takes size_t type argument. I am using gcc 3.4.3
> implementation. Under this implementation, in stdint.h, SIZE_MAX is
> defined as follows:
> #  define SIZE_MAX              (4294967295U)

> SIZE_MAX, defined in stdint.h, is the largest value that can be
> represented in type size_t. However when I pass SIZE_MAX as the
> argument to malloc, it is unable to allocate SIZE_MAX bytes of memory.
> In fact, malloc allocates much much less memory.

As other's already have pointed out the simple reason for
the result you obtained.

Thus I just want to comment on the lines like

>                         fprintf(stderr,
>                                   "%u - %u : ",
>                                   SIZE_MAX,
>                                   count);

You're using the '%u' format specifier for a size_t type.
'%u' is meant for unsigned ints. You might get away with
this on your machine when 'size_t' is actually typedef'ed
(or '#define'-ed) to 'unsigned int' but that's not some-
thing you should get used  to rely on. E.g. on my machine
SIZE_MAX is 18446744073709551615 and that's much too large
for an unsigned int (UINT_MAX is 4294967295 here and I thus
get compiler warnings). You should, if available, always
use the '%zu' format specifier (introduced in C99) or, if
you are not sure it's available, cast size_t values to the
largest available unsigned int type (at least 'unsigned
long int') and use the appropriate format specifier to be
reasonably sure that what you get printed out is correct.

                           Regards, Jens
-- 
  \   Jens Thoms Toerring  ___      jt@toerring.de
   \__________________________      http://toerring.de

[toc] | [prev] | [next] | [standalone]

#11685

On 14-Oct-11 08:31, V.Subramanian, India wrote:
> SIZE_MAX, defined in stdint.h, is the largest value that can be
> represented in type size_t. However when I pass SIZE_MAX as the
> argument to malloc, it is unable to allocate SIZE_MAX bytes of memory.

size_t must be capable of representing the size of the largest object
the implementation can create.

That does _not_ mean the implementation must be capable of creating an
object the size of the largest value a size_t can represent.

Those are two very different things.

> In fact, malloc allocates much much less memory.
>
> Here is the complete program max_size_to_malloc.c
> ...
> This program compiles fine with gcc 3.4.3 as
> gcc -std=c99 -pedantic -Wall -Wextra max_size_to_malloc.c
> 
> and produces the following output(after quite some time):
> 4294967295 - 1365266451 : Allocated 2929700844 bytes of memory
> 
> What is the reason for allocating this much less memory while the
> parameter type of malloc is size_t ?

Maybe your system didn't have any contiguous blocks of memory larger
than 2,929,700,844 bytes available at the time of your test.

> Normally what factor drives the maximum size of a single chunk of
> memory allocated by malloc ?

There are other things in memory (eg. your program, the stack, any
objects in the heap, OS services, libraries your program is linked to,
etc.), so at least some of it will not be available for you to allocate
for your own purposes; how much will vary.

Available memory is often fragmented (because those things above are
located in different places in memory), so the largest object you can
allocate will be limited to the largest _contiguous_ block.

Also, there may be administrative policy limiting the amount of memory
any program can allocate to below what is technically possible.

S

-- 
Stephen Sprunk         "God does not play dice."  --Albert Einstein
CCIE #3723         "God is an inveterate gambler, and He throws the
K5SSS        dice at every possible opportunity." --Stephen Hawking

[toc] | [prev] | [next] | [standalone]

#11687

Stephen Sprunk <stephen@sprunk.org> writes:
> On 14-Oct-11 08:31, V.Subramanian, India wrote:
[...]
>> Here is the complete program max_size_to_malloc.c
>> ...
>> This program compiles fine with gcc 3.4.3 as
>> gcc -std=c99 -pedantic -Wall -Wextra max_size_to_malloc.c
>> 
>> and produces the following output(after quite some time):
>> 4294967295 - 1365266451 : Allocated 2929700844 bytes of memory
>> 
>> What is the reason for allocating this much less memory while the
>> parameter type of malloc is size_t ?
>
> Maybe your system didn't have any contiguous blocks of memory larger
> than 2,929,700,844 bytes available at the time of your test.

And quite possibly it didn't have that much.  On Linux systems,
a malloc() call can appear to succeed, but not actually allocate
memory, just address space.  Things can go bad later when you try
to access the apparently allocated memory.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#11691

Keith Thompson <kst-u@mib.org> wrote:
> Stephen Sprunk <stephen@sprunk.org> writes:
> > On 14-Oct-11 08:31, V.Subramanian, India wrote:
> [...]
> >> Here is the complete program max_size_to_malloc.c
> >> ...
> >> This program compiles fine with gcc 3.4.3 as
> >> gcc -std=c99 -pedantic -Wall -Wextra max_size_to_malloc.c
> >> 
> >> and produces the following output(after quite some time):
> >> 4294967295 - 1365266451 : Allocated 2929700844 bytes of memory
> >> 
> >> What is the reason for allocating this much less memory while the
> >> parameter type of malloc is size_t ?
> >
> > Maybe your system didn't have any contiguous blocks of memory larger
> > than 2,929,700,844 bytes available at the time of your test.

> And quite possibly it didn't have that much.  On Linux systems,
> a malloc() call can appear to succeed, but not actually allocate
> memory, just address space.  Things can go bad later when you try
> to access the apparently allocated memory.

Depending on the settings for memory overcommitment...

                                   Regards, Jens
-- 
  \   Jens Thoms Toerring  ___      jt@toerring.de
   \__________________________      http://toerring.de

[toc] | [prev] | [next] | [standalone]

#16147

Stephen Sprunk <stephen@sprunk.org> writes:

> On 14-Oct-11 08:31, V.Subramanian, India wrote:
>> SIZE_MAX, defined in stdint.h, is the largest value that can be
>> represented in type size_t. However when I pass SIZE_MAX as the
>> argument to malloc, it is unable to allocate SIZE_MAX bytes of memory.
>
> size_t must be capable of representing the size of the largest object
> the implementation can create.  [snip]

The Standard does not impose this requirement.

[toc] | [prev] | [next] | [standalone]

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

Back to top | Article view | comp.lang.c

csiph-web