Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #146600 > unrolled thread

Indexing one past the end of array?

Back to article view | Back to comp.lang.c

Contents

  Indexing one past the end of array? Johann 'Myrkraverk' Oskarsson <johann@myrkraverk.invalid> - 2019-10-06 04:02 +0800
    Re: Indexing one past the end of array? Keith Thompson <kst-u@mib.org> - 2019-10-05 13:13 -0700
      Re: Indexing one past the end of array? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-10-05 22:01 +0100
        Re: Indexing one past the end of array? Keith Thompson <kst-u@mib.org> - 2019-10-05 14:35 -0700
        Re: Indexing one past the end of array? Richard Damon <Richard@Damon-Family.org> - 2019-10-05 19:00 -0400
          Re: Indexing one past the end of array? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2019-10-06 08:05 -0700
        Re: Indexing one past the end of array? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2019-10-06 06:36 -0700
          Re: Indexing one past the end of array? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-10-06 16:31 +0100
      Re: Indexing one past the end of array? Richard Damon <Richard@Damon-Family.org> - 2019-10-05 17:10 -0400
        Re: Indexing one past the end of array? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2019-10-06 07:40 -0700
    Re: Indexing one past the end of array? Mateusz Viste <mateusz@wont.tell> - 2019-10-05 20:26 +0000
    Re: Indexing one past the end of array? chad <cdalten@gmail.com> - 2019-10-05 16:55 -0700
    Re: Indexing one past the end of array? "Oliver S." <Follow.Me@gmx.net> - 2019-10-06 15:50 +0200
    Re: Indexing one past the end of array? "Oliver S." <Follow.Me@gmx.net> - 2019-10-06 17:04 +0200
      Re: Indexing one past the end of array? scott@slp53.sl.home (Scott Lurndal) - 2019-10-07 13:32 +0000
        Re: Indexing one past the end of array? "Oliver S." <Follow.Me@gmx.net> - 2019-10-07 17:23 +0200

#146600 — Indexing one past the end of array?

Dear comp.lang.c,

I've been reading a thread on LWN[1], notably this quote:

 > Just some pedantry, but I believe you're allowed to point to the first
 > element after the end of an array, as long as you don't dereference
 > that element. Pointer comparisons with such a pointer are supposed to
 > work, too.

Elsewhere, I've read that on embedded systems memory can be allocated 
right up to the end of memory, and therefore such indexes are invalid 
(but I cannot remember where).

So, is it always legal to calculate one past the end of an array or 
allocated memory?

[1] https://lwn.net/Articles/278137/

-- 
Johann | email: invalid -> com | www.myrkraverk.com/blog/
I'm not from the Internet, I just work there. | twitter: @myrkraverk

[toc] | [next] | [standalone]

#146602

Johann 'Myrkraverk' Oskarsson <johann@myrkraverk.invalid> writes:
> Dear comp.lang.c,
>
> I've been reading a thread on LWN[1], notably this quote:
>
>  > Just some pedantry, but I believe you're allowed to point to the first
>  > element after the end of an array, as long as you don't dereference
>  > that element. Pointer comparisons with such a pointer are supposed to
>  > work, too.
>
> Elsewhere, I've read that on embedded systems memory can be allocated 
> right up to the end of memory, and therefore such indexes are invalid 
> (but I cannot remember where).
>
> So, is it always legal to calculate one past the end of an array or 
> allocated memory?
>
> [1] https://lwn.net/Articles/278137/

Yes, it's always legal.  The language guarantees it.

N1570 6.5.6 "Additive operators" paragraph 8:

    When an expression that has integer type is added to or subtracted
    from a pointer, the result has the type of the pointer operand. 
    [...]
    Moreover, if the expression P points to the last element of an
    array object, the expression (P)+1 points one past the last element
    of the array object, and if the expression Q points one past the
    last element of an array object, the expression (Q)-1 points to
    the last element of the array object. If both the pointer operand
    and the result point to elements of the same array object, or one
    past the last element of the array object, the evaluation shall
    not produce an overflow; otherwise, the behavior is undefined. If
    the result points one past the last element of the array object,
    it shall not be used as the operand of a unary * operator that
    is evaluated.  
    [...]

A compiler needs to do whatever it has to to make this work as
specified.  Allocating an object at the very end of memory doesn't
necessarily break this, as long as pointer addition and subtraction
yield the expected results.  There doesn't have to be any memory at
the pointed-to location, since the pointer cannot be dereferenced.
If the system requires actual memory at the location to construct the
address, then the compiler needs to avoid allocating an object at
the very end of addressable memory -- unless it can prove that the
program never computes the address one past the last element of the
object.  (That only requires reserving one byte in the worst case.)

For example, assume a system with 32-bit addresses and exactly 2**32
bytes of addressable memory.  Assume an array object
    char buf[100];
is allocated starting at address 2**32-100.  Then (buf+100) points just
past the end of the array, and also just past the end of addressable
memory.  As long as evaluating (buf+100)-1 yields a valid pointer to
buf[99], that's ok.  (This is just an example.  The standard doesn't
assume this kind of mapping between addresses and integer values.)

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Will write code for food.
void Void(void) { Void(); } /* The recursive call of the void */

[toc] | [prev] | [next] | [standalone]

#146607

Keith Thompson <kst-u@mib.org> writes:

> Johann 'Myrkraverk' Oskarsson <johann@myrkraverk.invalid> writes:
>> Dear comp.lang.c,
>>
>> I've been reading a thread on LWN[1], notably this quote:
>>
>>  > Just some pedantry, but I believe you're allowed to point to the first
>>  > element after the end of an array, as long as you don't dereference
>>  > that element. Pointer comparisons with such a pointer are supposed to
>>  > work, too.
>>
>> Elsewhere, I've read that on embedded systems memory can be allocated 
>> right up to the end of memory, and therefore such indexes are invalid 
>> (but I cannot remember where).
>>
>> So, is it always legal to calculate one past the end of an array or 
>> allocated memory?
>>
>> [1] https://lwn.net/Articles/278137/
>
> Yes, it's always legal.  The language guarantees it.
<cut>
> A compiler needs to do whatever it has to to make this work as
> specified.  Allocating an object at the very end of memory doesn't
> necessarily break this, as long as pointer addition and subtraction
> yield the expected results.
<cut>
> For example, assume a system with 32-bit addresses and exactly 2**32
> bytes of addressable memory.  Assume an array object
>     char buf[100];
> is allocated starting at address 2**32-100.  Then (buf+100) points just
> past the end of the array, and also just past the end of addressable
> memory.  As long as evaluating (buf+100)-1 yields a valid pointer to
> buf[99], that's ok.

A nit: the relational operators (<, >, <=, >=) have to work on the
resulting pointers as well.  For example (buf+100)-1 < (buf+100) must be
true, and that's not automatically the case where pointers are
interpreted as unsigned bit patterns.  If, however, pointers are signed
2's complement values, it will all come out in the wash.

> (This is just an example.  The standard doesn't
> assume this kind of mapping between addresses and integer values.)

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#146612

Ben Bacarisse <ben.usenet@bsb.me.uk> writes:
> Keith Thompson <kst-u@mib.org> writes:
>> Johann 'Myrkraverk' Oskarsson <johann@myrkraverk.invalid> writes:
>>> Dear comp.lang.c,
>>>
>>> I've been reading a thread on LWN[1], notably this quote:
>>>
>>>  > Just some pedantry, but I believe you're allowed to point to the first
>>>  > element after the end of an array, as long as you don't dereference
>>>  > that element. Pointer comparisons with such a pointer are supposed to
>>>  > work, too.
>>>
>>> Elsewhere, I've read that on embedded systems memory can be allocated 
>>> right up to the end of memory, and therefore such indexes are invalid 
>>> (but I cannot remember where).
>>>
>>> So, is it always legal to calculate one past the end of an array or 
>>> allocated memory?
>>>
>>> [1] https://lwn.net/Articles/278137/
>>
>> Yes, it's always legal.  The language guarantees it.
> <cut>
>> A compiler needs to do whatever it has to to make this work as
>> specified.  Allocating an object at the very end of memory doesn't
>> necessarily break this, as long as pointer addition and subtraction
>> yield the expected results.
> <cut>
>> For example, assume a system with 32-bit addresses and exactly 2**32
>> bytes of addressable memory.  Assume an array object
>>     char buf[100];
>> is allocated starting at address 2**32-100.  Then (buf+100) points just
>> past the end of the array, and also just past the end of addressable
>> memory.  As long as evaluating (buf+100)-1 yields a valid pointer to
>> buf[99], that's ok.
>
> A nit: the relational operators (<, >, <=, >=) have to work on the
> resulting pointers as well.  For example (buf+100)-1 < (buf+100) must be
> true, and that's not automatically the case where pointers are
> interpreted as unsigned bit patterns.  If, however, pointers are signed
> 2's complement values, it will all come out in the wash.

Good nit, thanks.

A compiler *could* generate code that checks for that special case, and
optimize away the extra code when it can prove that the pointer didn't
wrap around.  But I'd be surprised if any actual compiler did that.

>> (This is just an example.  The standard doesn't
>> assume this kind of mapping between addresses and integer values.)

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Will write code for food.
void Void(void) { Void(); } /* The recursive call of the void */

[toc] | [prev] | [next] | [standalone]

#146613

On 10/5/19 5:01 PM, Ben Bacarisse wrote:
> Keith Thompson <kst-u@mib.org> writes:
> 
>> Johann 'Myrkraverk' Oskarsson <johann@myrkraverk.invalid> writes:
>>> Dear comp.lang.c,
>>>
>>> I've been reading a thread on LWN[1], notably this quote:
>>>
>>>  > Just some pedantry, but I believe you're allowed to point to the first
>>>  > element after the end of an array, as long as you don't dereference
>>>  > that element. Pointer comparisons with such a pointer are supposed to
>>>  > work, too.
>>>
>>> Elsewhere, I've read that on embedded systems memory can be allocated 
>>> right up to the end of memory, and therefore such indexes are invalid 
>>> (but I cannot remember where).
>>>
>>> So, is it always legal to calculate one past the end of an array or 
>>> allocated memory?
>>>
>>> [1] https://lwn.net/Articles/278137/
>>
>> Yes, it's always legal.  The language guarantees it.
> <cut>
>> A compiler needs to do whatever it has to to make this work as
>> specified.  Allocating an object at the very end of memory doesn't
>> necessarily break this, as long as pointer addition and subtraction
>> yield the expected results.
> <cut>
>> For example, assume a system with 32-bit addresses and exactly 2**32
>> bytes of addressable memory.  Assume an array object
>>     char buf[100];
>> is allocated starting at address 2**32-100.  Then (buf+100) points just
>> past the end of the array, and also just past the end of addressable
>> memory.  As long as evaluating (buf+100)-1 yields a valid pointer to
>> buf[99], that's ok.
> 
> A nit: the relational operators (<, >, <=, >=) have to work on the
> resulting pointers as well.  For example (buf+100)-1 < (buf+100) must be
> true, and that's not automatically the case where pointers are
> interpreted as unsigned bit patterns.  If, however, pointers are signed
> 2's complement values, it will all come out in the wash.
> 
>> (This is just an example.  The standard doesn't
>> assume this kind of mapping between addresses and integer values.)
> 

If pointers act like signed values, then the problem occurs at the
midpoint (so sooner).

Pointer differences are generally considered signed, and if pointer
values are compared by this signed difference, then the issue only
happens if a given array is over half the address space (which might
well not be well supported).

If ptrdiff_t is the same size and size_t, than you get issues with
arrays with more than SIZE_T_MAX/2 elements giving you overflow, but it
is rare enough that I don't think many implementations make ptrdiff_t
bigger than size_t to handle it.

[toc] | [prev] | [next] | [standalone]

#146644

Richard Damon <Richard@Damon-Family.org> writes:

> On 10/5/19 5:01 PM, Ben Bacarisse wrote:
>
>> Keith Thompson <kst-u@mib.org> writes:
>>
>>> Johann 'Myrkraverk' Oskarsson <johann@myrkraverk.invalid> writes:
>>>
>>>> Dear comp.lang.c,
>>>>
>>>> I've been reading a thread on LWN[1], notably this quote:
>>>>
>>>>> Just some pedantry, but I believe you're allowed to point to the first
>>>>> element after the end of an array, as long as you don't dereference
>>>>> that element.  Pointer comparisons with such a pointer are supposed to
>>>>> work, too.
>>>>
>>>> Elsewhere, I've read that on embedded systems memory can be allocated
>>>> right up to the end of memory, and therefore such indexes are invalid
>>>> (but I cannot remember where).
>>>>
>>>> So, is it always legal to calculate one past the end of an array or
>>>> allocated memory?
>>>>
>>>> [1] https://lwn.net/Articles/278137/
>>>
>>> Yes, it's always legal.  The language guarantees it.
>>
>> <cut>
>>
>>> A compiler needs to do whatever it has to to make this work as
>>> specified.  Allocating an object at the very end of memory doesn't
>>> necessarily break this, as long as pointer addition and subtraction
>>> yield the expected results.
>>
>> <cut>
>>
>>> For example, assume a system with 32-bit addresses and exactly 2**32
>>> bytes of addressable memory.  Assume an array object
>>>     char buf[100];
>>> is allocated starting at address 2**32-100.  Then (buf+100) points just
>>> past the end of the array, and also just past the end of addressable
>>> memory.  As long as evaluating (buf+100)-1 yields a valid pointer to
>>> buf[99], that's ok.
>>
>> A nit:  the relational operators (<, >, <=, >=) have to work on the
>> resulting pointers as well.  For example (buf+100)-1 < (buf+100) must be
>> true, and that's not automatically the case where pointers are
>> interpreted as unsigned bit patterns.  If, however, pointers are signed
>> 2's complement values, it will all come out in the wash.
>>
>>> (This is just an example.  The standard doesn't
>>> assume this kind of mapping between addresses and integer values.)
>>
>
> If pointers act like signed values, then the problem occurs at the
> midpoint (so sooner).

If pointers are taken as being signed, presumably the address space
starts at the most pointer value.  The whole space is still usable.

> Pointer differences are generally considered signed, and if pointer
> values are compared by this signed difference, then the issue only
> happens if a given array is over half the address space (which might
> well not be well supported).

The type of subtracting one pointer from another has been a signed
type (ptrdiff_t) since C90.  As you point out in the next paragraph
there is a problem only when the number of elements being spanned
is greater than PTRDIFF_MAX or less than PTRDIFF_MIN (or the
analogous values in C90).  This range may be larger than half the
address space, but is very likely not in most implementations.
However, when size_t is 16 bits, the whole address space will be
usable, as PTRDIFF_MAX must be at least 65535 (C99 onwards).

> If ptrdiff_t is the same size and size_t, than you get issues with
> arrays with more than SIZE_T_MAX/2 elements giving you overflow, but it
> is rare enough that I don't think many implementations make ptrdiff_t
> bigger than size_t to handle it.

The Standard gives an out here, by saying any difference of
pointers that gives a value not representable in type ptrdiff_t
gives undefined behavior.

[toc] | [prev] | [next] | [standalone]

#146638

Ben Bacarisse <ben.usenet@bsb.me.uk> writes:

> Keith Thompson <kst-u@mib.org> writes:
>
>> Johann 'Myrkraverk' Oskarsson <johann@myrkraverk.invalid> writes:
>>
>>> Dear comp.lang.c,
>>>
>>> I've been reading a thread on LWN[1], notably this quote:
>>>
>>>> Just some pedantry, but I believe you're allowed to point to the first
>>>> element after the end of an array, as long as you don't dereference
>>>> that element.  Pointer comparisons with such a pointer are supposed to
>>>> work, too.
>>>
>>> Elsewhere, I've read that on embedded systems memory can be allocated
>>> right up to the end of memory, and therefore such indexes are invalid
>>> (but I cannot remember where).
>>>
>>> So, is it always legal to calculate one past the end of an array or
>>> allocated memory?
>>>
>>> [1] https://lwn.net/Articles/278137/
>>
>> Yes, it's always legal.  The language guarantees it.
>
> <cut>
>
>> A compiler needs to do whatever it has to to make this work as
>> specified.  Allocating an object at the very end of memory doesn't
>> necessarily break this, as long as pointer addition and subtraction
>> yield the expected results.
>
> <cut>
>
>> For example, assume a system with 32-bit addresses and exactly 2**32
>> bytes of addressable memory.  Assume an array object
>>     char buf[100];
>> is allocated starting at address 2**32-100.  Then (buf+100) points just
>> past the end of the array, and also just past the end of addressable
>> memory.  As long as evaluating (buf+100)-1 yields a valid pointer to
>> buf[99], that's ok.
>
> A nit:  the relational operators (<, >, <=, >=) have to work on the
> resulting pointers as well.  For example (buf+100)-1 < (buf+100) must be
> true, and that's not automatically the case where pointers are
> interpreted as unsigned bit patterns.  If, however, pointers are signed
> 2's complement values, it will all come out in the wash.

It took me a while to figure out what you meant by the statement
about signed pointers.  What I think you mean is that if the
signed address of 'buf' is -100, then the comparisons (and
arithmetic) all work as required.  But if pointers are signed
there is still a singularity; it just occurs at a different point
in the address space.  A 100-byte array starting at 2**31-100
will still have the problem:  a pointer past the last element of
the array will be less than -- instead of greater than -- a
pointer to any element of the array.

[toc] | [prev] | [next] | [standalone]

#146647

Tim Rentsch <tr.17687@z991.linuxsc.com> writes:

> Ben Bacarisse <ben.usenet@bsb.me.uk> writes:
>
>> Keith Thompson <kst-u@mib.org> writes:
>>
>>> Johann 'Myrkraverk' Oskarsson <johann@myrkraverk.invalid> writes:
>>>
>>>> Dear comp.lang.c,
>>>>
>>>> I've been reading a thread on LWN[1], notably this quote:
>>>>
>>>>> Just some pedantry, but I believe you're allowed to point to the first
>>>>> element after the end of an array, as long as you don't dereference
>>>>> that element.  Pointer comparisons with such a pointer are supposed to
>>>>> work, too.
>>>>
>>>> Elsewhere, I've read that on embedded systems memory can be allocated
>>>> right up to the end of memory, and therefore such indexes are invalid
>>>> (but I cannot remember where).
>>>>
>>>> So, is it always legal to calculate one past the end of an array or
>>>> allocated memory?
>>>>
>>>> [1] https://lwn.net/Articles/278137/
>>>
>>> Yes, it's always legal.  The language guarantees it.
>>
>> <cut>
>>
>>> A compiler needs to do whatever it has to to make this work as
>>> specified.  Allocating an object at the very end of memory doesn't
>>> necessarily break this, as long as pointer addition and subtraction
>>> yield the expected results.
>>
>> <cut>
>>
>>> For example, assume a system with 32-bit addresses and exactly 2**32
>>> bytes of addressable memory.  Assume an array object
>>>     char buf[100];
>>> is allocated starting at address 2**32-100.  Then (buf+100) points just
>>> past the end of the array, and also just past the end of addressable
>>> memory.  As long as evaluating (buf+100)-1 yields a valid pointer to
>>> buf[99], that's ok.
>>
>> A nit:  the relational operators (<, >, <=, >=) have to work on the
>> resulting pointers as well.  For example (buf+100)-1 < (buf+100) must be
>> true, and that's not automatically the case where pointers are
>> interpreted as unsigned bit patterns.  If, however, pointers are signed
>> 2's complement values, it will all come out in the wash.
>
> It took me a while to figure out what you meant by the statement
> about signed pointers.  What I think you mean is that if the
> signed address of 'buf' is -100, then the comparisons (and
> arithmetic) all work as required.

Kind of.  It was an almost entirely unhelpful remark because I was
referring only to the 'all-bits set' to 'no bits set' wrap around case.
That becomes -1 to 0 rather than 2**32-1 to 0.  I should not have said
"it will all" because that suggests too much generality.

> But if pointers are signed
> there is still a singularity; it just occurs at a different point
> in the address space.

Yes, indeed.  And in some sense, the singularity is always in the same
place: at the (arithmetic) top of the address range.

> A 100-byte array starting at 2**31-100
> will still have the problem:  a pointer past the last element of
> the array will be less than -- instead of greater than -- a
> pointer to any element of the array.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#146609

On 10/5/19 4:13 PM, Keith Thompson wrote:
> Johann 'Myrkraverk' Oskarsson <johann@myrkraverk.invalid> writes:
>> Dear comp.lang.c,
>>
>> I've been reading a thread on LWN[1], notably this quote:
>>
>>  > Just some pedantry, but I believe you're allowed to point to the first
>>  > element after the end of an array, as long as you don't dereference
>>  > that element. Pointer comparisons with such a pointer are supposed to
>>  > work, too.
>>
>> Elsewhere, I've read that on embedded systems memory can be allocated 
>> right up to the end of memory, and therefore such indexes are invalid 
>> (but I cannot remember where).
>>
>> So, is it always legal to calculate one past the end of an array or 
>> allocated memory?
>>
>> [1] https://lwn.net/Articles/278137/
> 
> Yes, it's always legal.  The language guarantees it.
> 
> N1570 6.5.6 "Additive operators" paragraph 8:
> 
>     When an expression that has integer type is added to or subtracted
>     from a pointer, the result has the type of the pointer operand. 
>     [...]
>     Moreover, if the expression P points to the last element of an
>     array object, the expression (P)+1 points one past the last element
>     of the array object, and if the expression Q points one past the
>     last element of an array object, the expression (Q)-1 points to
>     the last element of the array object. If both the pointer operand
>     and the result point to elements of the same array object, or one
>     past the last element of the array object, the evaluation shall
>     not produce an overflow; otherwise, the behavior is undefined. If
>     the result points one past the last element of the array object,
>     it shall not be used as the operand of a unary * operator that
>     is evaluated.  
>     [...]
> 
> A compiler needs to do whatever it has to to make this work as
> specified.  Allocating an object at the very end of memory doesn't
> necessarily break this, as long as pointer addition and subtraction
> yield the expected results.  There doesn't have to be any memory at
> the pointed-to location, since the pointer cannot be dereferenced.
> If the system requires actual memory at the location to construct the
> address, then the compiler needs to avoid allocating an object at
> the very end of addressable memory -- unless it can prove that the
> program never computes the address one past the last element of the
> object.  (That only requires reserving one byte in the worst case.)
> 
> For example, assume a system with 32-bit addresses and exactly 2**32
> bytes of addressable memory.  Assume an array object
>     char buf[100];
> is allocated starting at address 2**32-100.  Then (buf+100) points just
> past the end of the array, and also just past the end of addressable
> memory.  As long as evaluating (buf+100)-1 yields a valid pointer to
> buf[99], that's ok.  (This is just an example.  The standard doesn't
> assume this kind of mapping between addresses and integer values.)
> 

One possible issue you (or maybe it is the compier) can run into is that
buf + 100 must be > buf + 99. In you case above, buf + 100 would be
2**32, which if addresses are treated as 32 bit numbers, wraps to zero
and thus the relationship doesn't hold.

The one out for the compiler is that on most machines, the very top of
memory has reserved functionality, so that the compiler might be able to
get away with just define the problem away and just say that (normal)
object can't be at the end of memory, and thus the one past end of array
pointer can't come into issue there.

It is more of an issue with segmented memory/pointers, with a segment
size of 2**16. In such an environment, you can't have both 2**16 byte
arrays and 16 bit pointers and full conforming, and practically, often
the 2**16 byte object is just defined to be not supported in a fully
conforming manner (I don't think it is actually required to be)

[toc] | [prev] | [next] | [standalone]

#146641

Richard Damon <Richard@Damon-Family.org> writes:

> On 10/5/19 4:13 PM, Keith Thompson wrote:
>
>> [..addressing past the end of an array..]
>>
>> For example, assume a system with 32-bit addresses and exactly 2**32
>> bytes of addressable memory.  Assume an array object
>>     char buf[100];
>> is allocated starting at address 2**32-100.  Then (buf+100) points just
>> past the end of the array, and also just past the end of addressable
>> memory.  As long as evaluating (buf+100)-1 yields a valid pointer to
>> buf[99], that's ok.  (This is just an example.  The standard doesn't
>> assume this kind of mapping between addresses and integer values.)
>
> One possible issue you (or maybe it is the compier) can run into is that
> buf + 100 must be > buf + 99.  In you case above, buf + 100 would be
> 2**32, which if addresses are treated as 32 bit numbers, wraps to zero
> and thus the relationship doesn't hold.

The Rationale mentions this issue and points out that there need
be only one never-allocated byte at the end of memory (or of
course a few more than one if alignment causes the second-to-last
byte not to be usable).

> an issue with segmented memory/pointers, with a segment
> size of 2**16.  In such an environment, you can't have both 2**16 byte
> arrays and 16 bit pointers and full conforming, and practically, often
> the 2**16 byte object is just defined to be not supported in a fully
> conforming manner (I don't think it is actually required to be)

In C90 hosted implementations were required to support objects of
at least 32767 bytes.

In C99 onwards (assuming it hasn't changed) hosted implementations
are required to support objects of at least 65535 bytes (and the
minimum value for SIZE_MAX is 65535).  So segments of 2**16 bytes
can still be conforming.

[toc] | [prev] | [next] | [standalone]

#146604

On Sun, 06 Oct 2019 04:02:17 +0800, Johann 'Myrkraverk' Oskarsson wrote:

> I've been reading a thread on LWN[1], notably this quote:
> 
>  > Just some pedantry, but I believe you're allowed to point to the
>  > first element after the end of an array, as long as you don't
>  > dereference that element. Pointer comparisons with such a pointer are
>  > supposed to work, too.
> 
> Elsewhere, I've read that on embedded systems memory can be allocated
> right up to the end of memory, and therefore such indexes are invalid
> (but I cannot remember where).
> 
> So, is it always legal to calculate one past the end of an array or
> allocated memory?

Have you read the bible lately?

"(...) Moreover, if the expression P points to the last element of an 
array object, the expression (P)+1 points one past the last element of 
the array object, and if the expression Q points one past the last 
element of an array object, the expression (Q)-1 points to the last 
element of the array object.
If both the pointer operand and the result point to elements of the same 
array object, or one past the last element of the array object, the 
evaluation shall not produce an overflow; otherwise, the behavior is 
undefined."

ANSI 9899-1990, page 47.


Mateusz

[toc] | [prev] | [next] | [standalone]

#146615

On Saturday, October 5, 2019 at 1:02:32 PM UTC-7, Johann 'Myrkraverk' Oskarsson wrote:
> Dear comp.lang.c,
> 
> I've been reading a thread on LWN[1], notably this quote:
> 
>  > Just some pedantry, but I believe you're allowed to point to the first
>  > element after the end of an array, as long as you don't dereference
>  > that element. Pointer comparisons with such a pointer are supposed to
>  > work, too.
> 
> Elsewhere, I've read that on embedded systems memory can be allocated 
> right up to the end of memory, and therefore such indexes are invalid 
> (but I cannot remember where).
> 
I've done enough embedded system programming to tell you that is wrong.

[toc] | [prev] | [next] | [standalone]

#146639

>> Just some pedantry, but I believe you're allowed to point to the first
>> element after the end of an array, as long as you don't dereference
>> that element. Pointer comparisons with such a pointer are supposed to
>> work, too.

> Elsewhere, I've read that on embedded systems memory can be allocated 
> right up to the end of memory, and therefore such indexes are invalid 
> (but I cannot remember where).
> So, is it always legal to calculate one past the end of an array or 
> allocated memory?
> [1] https://lwn.net/Articles/278137/

In Theory assigning a value to a pointer to which is not accessible
might result in some type of protection-fault. But even with Windows
3.1 the compilers didn't the segment-registers until you really ac-
cessed any memory they referred to. Further doing any arithmetics on
hat (f.e. incrementing a hughe-pointer and thereby breaking a segment
-boundary) also doesn't result in loading the changed segment to a
segment-register until you access any memory through it. So 
_practi-cally_ you can simply assume you can assign anything to a pointer
and nothing happens until you access the memory.

[toc] | [prev] | [next] | [standalone]

#146643

> Elsewhere, I've read that on embedded systems memory can be allocated 
> right up to the end of memory, and therefore such indexes are invalid 
> (but I cannot remember where).
> So, is it always legal to calculate one past the end of an array or 
> allocated memory?
> [1] https://lwn.net/Articles/278137/

Userspace has its addresses ussually in the lower part of the logical
address-space, so addresses calculated to be above an array would never
wrap-around.
In kernel-space addresses could wrap-around but I think a proper ker-
nel would reserve a reasonable amount of memory at top of the logical
address-space so that a past-the-end-pointer would maximally point
into that area and not wrap around.

[toc] | [prev] | [next] | [standalone]

#146685

"Oliver S." <Follow.Me@gmx.net> writes:
>> Elsewhere, I've read that on embedded systems memory can be allocated 
>> right up to the end of memory, and therefore such indexes are invalid 
>> (but I cannot remember where).
>> So, is it always legal to calculate one past the end of an array or 
>> allocated memory?
>> [1] https://lwn.net/Articles/278137/
>
>Userspace has its addresses ussually in the lower part of the logical
>address-space, so addresses calculated to be above an array would never
>wrap-around.
>In kernel-space addresses could wrap-around but I think a proper ker-
>nel would reserve a reasonable amount of memory at top of the logical
>address-space so that a past-the-end-pointer would maximally point
>into that area and not wrap around.

Most of them mark the first page of the virtual address space as
not-present to catch inadvertent wrapping behavior.

On the physical address side, both ARM and Intel will use unsigned modulo
arithmetic, so addresses will simply wrap.

[toc] | [prev] | [next] | [standalone]

#146692

> Most of them mark the first page of the virtual address space as
> not-present to catch inadvertent wrapping behavior.

That's mostly not relevant for upper array bound behaviour.

> On the physical address side, both ARM and Intel will use unsigned modulo
> arithmetic, so addresses will simply wrap.

No, they use 128 bit decimal floating point to address memory!!!

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.c

csiph-web