Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #109727 > unrolled thread

Help on this typedef

Back to article view | Back to comp.lang.c

Contents

  Help on this typedef fl <rxjwg98@gmail.com> - 2017-05-10 19:12 -0700
    Re: Help on this typedef Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2017-05-10 21:06 -0600
    Re: Help on this typedef Keith Thompson <kst-u@mib.org> - 2017-05-10 20:11 -0700
      Re: Help on this typedef David Brown <david.brown@hesbynett.no> - 2017-05-11 10:03 +0200
        Re: Help on this typedef supercat@casperkitty.com - 2017-05-11 08:35 -0700
          Re: Help on this typedef David Brown <david.brown@hesbynett.no> - 2017-05-12 10:34 +0200
            Re: Help on this typedef joel.rees@gmail.com - 2017-05-12 06:01 -0700
            Re: Help on this typedef Keith Thompson <kst-u@mib.org> - 2017-05-12 09:38 -0700
          Re: Help on this typedef joel.rees@gmail.com - 2017-05-12 05:08 -0700
            Re: Help on this typedef Tim Rentsch <txr@alumni.caltech.edu> - 2017-05-13 14:25 -0700
              Re: Help on this typedef joel.rees@gmail.com - 2017-05-14 23:36 -0700
                Re: Help on this typedef Tim Rentsch <txr@alumni.caltech.edu> - 2017-05-15 07:23 -0700
                  Re: Help on this typedef joel.rees@gmail.com - 2017-05-18 07:40 -0700
                    Re: Help on this typedef Keith Thompson <kst-u@mib.org> - 2017-05-18 09:35 -0700
                      Re: Help on this typedef supercat@casperkitty.com - 2017-05-18 10:59 -0700
                      Re: Help on this typedef joel.rees@gmail.com - 2017-05-18 23:40 -0700
                        Re: Help on this typedef jadill33@gmail.com - 2017-05-19 08:12 -0700
                          Re: Help on this typedef supercat@casperkitty.com - 2017-05-19 08:41 -0700
                            Re: Help on this typedef jadill33@gmail.com - 2017-05-19 09:41 -0700
                        Re: Help on this typedef Keith Thompson <kst-u@mib.org> - 2017-05-19 09:34 -0700
                          Re: Help on this typedef supercat@casperkitty.com - 2017-05-19 10:18 -0700
                          Re: Help on this typedef Ben Bacarisse <ben.usenet@bsb.me.uk> - 2017-05-19 21:35 +0100
                            Re: Help on this typedef supercat@casperkitty.com - 2017-05-19 14:16 -0700
                            Re: Help on this typedef Ike Naar <ike@iceland.freeshell.org> - 2017-05-19 21:15 +0000
                              Re: Help on this typedef Ben Bacarisse <ben.usenet@bsb.me.uk> - 2017-05-19 23:54 +0100
                            Re: Help on this typedef Keith Thompson <kst-u@mib.org> - 2017-05-19 14:25 -0700
                              Re: Help on this typedef supercat@casperkitty.com - 2017-05-19 15:18 -0700
                              Re: Help on this typedef Ben Bacarisse <ben.usenet@bsb.me.uk> - 2017-05-19 23:52 +0100
                                Re: Help on this typedef Keith Thompson <kst-u@mib.org> - 2017-05-19 16:42 -0700
                                  Re: Help on this typedef Ben Bacarisse <ben.usenet@bsb.me.uk> - 2017-05-20 01:16 +0100
                            Re: Help on this typedef Tim Rentsch <txr@alumni.caltech.edu> - 2017-05-20 01:58 -0700
                              Re: Help on this typedef joel.rees@gmail.com - 2017-05-20 02:25 -0700
                          Re: Help on this typedef joel.rees@gmail.com - 2017-05-19 22:09 -0700
                            Re: Help on this typedef supercat@casperkitty.com - 2017-05-19 23:23 -0700
                        Re: Help on this typedef "James R. Kuyper" <jameskuyper@verizon.net> - 2017-05-19 13:33 -0400
                          Re: Help on this typedef supercat@casperkitty.com - 2017-05-19 13:44 -0700
                            Re: Help on this typedef jameskuyper@verizon.net - 2017-05-19 19:20 -0700
                              Re: Help on this typedef supercat@casperkitty.com - 2017-05-19 23:30 -0700
                                Re: Help on this typedef jameskuyper@verizon.net - 2017-05-20 07:20 -0700
                                Re: Help on this typedef jameskuyper@verizon.net - 2017-05-20 07:46 -0700
                                  Re: Help on this typedef supercat@casperkitty.com - 2017-05-22 10:50 -0700
                    Re: Help on this typedef "James R. Kuyper" <jameskuyper@verizon.net> - 2017-05-18 12:37 -0400
                    Re: Help on this typedef Ben Bacarisse <ben.usenet@bsb.me.uk> - 2017-05-18 20:02 +0100
                      Re: Help on this typedef joel.rees@gmail.com - 2017-05-19 00:20 -0700
                        Re: Help on this typedef Keith Thompson <kst-u@mib.org> - 2017-05-19 09:51 -0700
                          Re: Help on this typedef supercat@casperkitty.com - 2017-05-19 10:31 -0700
                    Re: Help on this typedef Ike Naar <ike@iceland.freeshell.org> - 2017-05-19 19:48 +0000
                    Re: Help on this typedef Tim Rentsch <txr@alumni.caltech.edu> - 2017-05-20 08:47 -0700
          Re: Help on this typedef "Patrick.Schluter" <Patrick.Schluter@free.fr> - 2017-05-27 18:50 +0200
    Re: Help on this typedef joel.rees@gmail.com - 2017-05-12 06:17 -0700
      Re: Help on this typedef supercat@casperkitty.com - 2017-05-12 08:46 -0700
        Re: Help on this typedef joel.rees@gmail.com - 2017-05-12 18:11 -0700
          Re: Help on this typedef Keith Thompson <kst-u@mib.org> - 2017-05-12 18:37 -0700
            Re: Help on this typedef Tim Rentsch <txr@alumni.caltech.edu> - 2017-05-13 09:44 -0700
              Re: Help on this typedef Keith Thompson <kst-u@mib.org> - 2017-05-13 13:47 -0700
                Re: Help on this typedef Robert Wessel <robertwessel2@yahoo.com> - 2017-05-14 00:28 -0500
                Re: Help on this typedef Tim Rentsch <txr@alumni.caltech.edu> - 2017-05-14 13:39 -0700
                  Re: Help on this typedef raltbos@xs4all.nl (Richard Bos) - 2017-05-15 13:02 +0000
                    Re: Help on this typedef Tim Rentsch <txr@alumni.caltech.edu> - 2017-05-15 07:30 -0700
          Re: Help on this typedef supercat@casperkitty.com - 2017-05-12 20:45 -0700
          Re: Help on this typedef Tim Rentsch <txr@alumni.caltech.edu> - 2017-05-13 09:57 -0700
          Re: Help on this typedef Philip Lantz <prl@canterey.us> - 2017-05-13 12:07 -0700
          Re: Help on this typedef David Brown <david.brown@hesbynett.no> - 2017-05-14 15:34 +0200
        Re: Help on this typedef Tim Rentsch <txr@alumni.caltech.edu> - 2017-05-14 12:48 -0700
          Re: Help on this typedef supercat@casperkitty.com - 2017-05-15 07:39 -0700
            Re: Help on this typedef Tim Rentsch <txr@alumni.caltech.edu> - 2017-05-24 08:23 -0700
              Re: Help on this typedef supercat@casperkitty.com - 2017-05-24 09:17 -0700
                Re: Help on this typedef Tim Rentsch <txr@alumni.caltech.edu> - 2017-05-27 06:15 -0700

Page 2 of 4 — ← Prev page 1 [2] 3 4  Next page →

#110393

On Friday, May 19, 2017 at 11:34:11 AM UTC-5, Keith Thompson wrote:
> p->text[3] = 'd'
> 
> But both the strcpy() call and the reference to p->text[3] have
> undefined behavior.  Concentrating on the latter, the indexing
> operator [] is defined in terms of pointer addition, in this case
> (p->text + 3).  The description of pointer addition, in N1570
> 6.5.6p8, says:
> 
>     If both the pointer operand and the result point to elements
>     of the same array object, or one past the last element of the
>     array object, the evaluation shall not produce an overflow;
>     otherwise, the behavior is undefined.

When objects are nested, the C Standard is often unclear about whether
the term "object" refers to an inner object or the containing object.
Given, e.g.

    int foo[4][3];
    int *p = foo+2;

the C89 Standard could be read in such fashion as to suggest that p
could only be used to access integers within foo[2], but:

   1. There are times when it would be useful or vital to be able to take
      such a pointer and be able to access other elements of "foo".

   2. If "p" were only usable to access integers within foo[2], the
      Standard would define no means of producing a pointer that could
      access all of "foo".

   3. Compiler practice had, so far as I can tell, unanimously supported
      the ability to use "p" to access all elements within "foo", and there
      is no evidence that the authors of C89 intended to change that.

Under C89, a pointer received via malloc essentially behaves as a
"char[size]" which--for purposes of aliasing--also behaves as a union of
all types that would fit in that amount of space.

Thus, given:

    struct s {int x; char z[1];} *p;
    p=malloc(1000);

if offsetof(struct s,z) is 4, then p->z will be a pointer to the fifth byte
of a char[1000].

[toc] | [prev] | [next] | [standalone]

#110416

Keith Thompson <kst-u@mib.org> writes:
<snip>
> Here's a demo program, based on your declaration above with the syntax
> error corrected:
>
> #include <stdio.h>
> #include <stdlib.h>
> #include <string.h>
> int main(void) {
>     typedef struct string_header_s {
>         short length; /* limited length */
>         char text[ 1 ];
>     } string_header_t;
>
>     string_header_t *p = malloc(sizeof *p + 10); /* more than we need */
>     if (p == NULL) exit(EXIT_FAILURE);
>     p->length = 4;
>     strcpy(p->text, "abcd");
>     printf("p->text[3] = '%c'\n", p->text[3]);
> }
>
> When I compile and run this on my system, I get no compile-time
> diagnostics and the output, as expected, is:
>
> p->text[3] = 'd'
>
> But both the strcpy() call and the reference to p->text[3] have
> undefined behavior.  Concentrating on the latter, the indexing
> operator [] is defined in terms of pointer addition, in this case
> (p->text + 3).  The description of pointer addition, in N1570
> 6.5.6p8, says:
>
>     If both the pointer operand and the result point to elements
>     of the same array object, or one past the last element of the
>     array object, the evaluation shall not produce an overflow;
>     otherwise, the behavior is undefined.
>
> The array object in question is p->text, which is of type char[1].
> We've used malloc to allocate additional memory past the end of
> that array object, but it's not part of the array object.

I think something might be missing from the standard in the area.  I
think no one disputes that this is defined behaviour:

  int *ip = malloc(10);
  if (ip) ip[3] = 42;

but what is the array in question here?  The storage allocated by malloc
has no effective type (yet) so how can ip and ip+3 point to elements of
the same array?  (I've used and int * because there are special
dispensations for accessing any objects byte as a sequence of chars.)

And does that mean we can make the struct hack valid like this:

  p->text[3] = 'x'; // invalid
  char *cp = (void *)p;
  cp += offsetof(sting_header_t, text);
  cp[3] = 'x'; // this one OK?

I'd say yes and I think my answer would stay the same if the 'text'
member and the pointer access where to int types rather than char.
(You'd need a char * to do the first offsetof addition, of course.)

<snip>
-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#110421

On Friday, May 19, 2017 at 3:35:11 PM UTC-5, Ben Bacarisse wrote:
> I think something might be missing from the standard in the area.  I
> think no one disputes that this is defined behaviour:
> 
>   int *ip = malloc(10);
>   if (ip) ip[3] = 42;
> 
> but what is the array in question here?  The storage allocated by malloc
> has no effective type (yet) so how can ip and ip+3 point to elements of
> the same array?  (I've used and int * because there are special
> dispensations for accessing any objects byte as a sequence of chars.)

Under C89, In the absence of aliasing rules, there would be no problem
with describing malloc(N) as returning a pointer to a "char[N]" which
will happen to be aligned suitably for any type that might be contained
therein.  Any pointers derived from that pointer could be indexed in
any fashion that would continue to point within that original array
object.

[toc] | [prev] | [next] | [standalone]

#110422

On 2017-05-19, Ben Bacarisse <ben.usenet@bsb.me.uk> wrote:
> I think something might be missing from the standard in the area.  I
> think no one disputes that this is defined behaviour:
>
>   int *ip = malloc(10);
>   if (ip) ip[3] = 42;

If sizeof (int) > 2 the assignment to ip[3] writes beyond the
allocated space.

[toc] | [prev] | [next] | [standalone]

#110431

Ike Naar <ike@iceland.freeshell.org> writes:

> On 2017-05-19, Ben Bacarisse <ben.usenet@bsb.me.uk> wrote:
>> I think something might be missing from the standard in the area.  I
>> think no one disputes that this is defined behaviour:
>>
>>   int *ip = malloc(10);
>>   if (ip) ip[3] = 42;
>
> If sizeof (int) > 2 the assignment to ip[3] writes beyond the
> allocated space.

Rats!  That was not, of course, intended to be the point!  I copied the
10 from the earlier example and forgot to check it.  Assume small ints
or imagine I wrote malloc(100).

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#110424

Ben Bacarisse <ben.usenet@bsb.me.uk> writes:
[...]
> I think something might be missing from the standard in the area.  I
> think no one disputes that this is defined behaviour:
>
>   int *ip = malloc(10);
>   if (ip) ip[3] = 42;
>
> but what is the array in question here?  The storage allocated by malloc
> has no effective type (yet) so how can ip and ip+3 point to elements of
> the same array?  (I've used and int * because there are special
> dispensations for accessing any objects byte as a sequence of chars.)

N1570 7.22.3p1:

    The pointer returned if the allocation succeeds is suitably
    aligned so that it may be assigned to a pointer to any type of
    object with a fundamental alignment requirement and then used
    to access such an object or an array of such objects in the
    space allocated (until the space is explicitly deallocated).

The wording does seem to imply that the pointer may be used
to access "such an object or an array of such objects" merely
*because* it's suitably aligned, which I'm not comfortable with,
but the intent seems to be that the allocation functions can and
do create accessible objects.

So a pointer to malloc()ed memory can be treated as a pointer to an
object or array of arbitrary types, but I don't think that implies that
an array member within a structure can be treated as if it were a bigger
array.

> And does that mean we can make the struct hack valid like this:
>
>   p->text[3] = 'x'; // invalid
>   char *cp = (void *)p;
>   cp += offsetof(sting_header_t, text);
>   cp[3] = 'x'; // this one OK?
>
> I'd say yes and I think my answer would stay the same if the 'text'
> member and the pointer access where to int types rather than char.
> (You'd need a char * to do the first offsetof addition, of course.)
>
> <snip>

I'd say that doesn't take advantage of the explicit permission granted
for malloc() and friends, so the behavior is undefined.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#110428

On Friday, May 19, 2017 at 4:26:02 PM UTC-5, Keith Thompson wrote:
> N1570 7.22.3p1:
> 
>     The pointer returned if the allocation succeeds is suitably
>     aligned so that it may be assigned to a pointer to any type of
>     object with a fundamental alignment requirement and then used
>     to access such an object or an array of such objects in the
>     space allocated (until the space is explicitly deallocated).
> 
> The wording does seem to imply that the pointer may be used
> to access "such an object or an array of such objects" merely
> *because* it's suitably aligned, which I'm not comfortable with,
> but the intent seems to be that the allocation functions can and
> do create accessible objects.

Compilers had certainly treated such constructs that way prior to C89,
and I see no reason to believe the C89 authors intended--or even imagined--
that any compiler writers would interpret the lack of an explicit mandate
for such behavior as justification to do otherwise.

> So a pointer to malloc()ed memory can be treated as a pointer to an
> object or array of arbitrary types, but I don't think that implies that
> an array member within a structure can be treated as if it were a bigger
> array.

Consider the declaration:

    union { char small[2]; char big[1000];} *up;

Is there anything in C89 that would suggest that (char*)up->small,
(char*)up->big, and (char*)up would not all be equivalent expressions
of type "char*"?  Bear in mind that a pointer to a union is equivalent
to a pointer to any of its members when suitably converted.

Now consider:

    struct small_s { char x[2]; };
    struct big_s { char x[2000]; };
    union uu { struct small_s s; struct big_s b; } *up2;

Is there anything in C89 that would suggest that (char*)up2->s.x,
(char*)up2->b.x, and (char*)up2 would not all be equivalent expressions
of type "char*"?  Bear in mind that a pointer to a struct is equivalent
to a pointer to its first member when suitably converted.

If one adds to the above declarations:

    struct small_s *ssp;

would anything in C89 suggest that (char*)(ssp->x) and
(char*)((union uu*)ssp)->b.x would not be equivalent expressions of
type "char*"?

If a compiler can't see how pointer *ssp was created, would it have any
basis under C89 for assuming that (char*)(ssp->x) could not be a pointer
to char[2000]?

[toc] | [prev] | [next] | [standalone]

#110430

Keith Thompson <kst-u@mib.org> writes:

> Ben Bacarisse <ben.usenet@bsb.me.uk> writes:
> [...]
>> I think something might be missing from the standard in the area.  I
>> think no one disputes that this is defined behaviour:
>>
>>   int *ip = malloc(10);
>>   if (ip) ip[3] = 42;
>>
>> but what is the array in question here?  The storage allocated by malloc
>> has no effective type (yet) so how can ip and ip+3 point to elements of
>> the same array?  (I've used and int * because there are special
>> dispensations for accessing any objects byte as a sequence of chars.)
>
> N1570 7.22.3p1:
>
>     The pointer returned if the allocation succeeds is suitably
>     aligned so that it may be assigned to a pointer to any type of
>     object with a fundamental alignment requirement and then used
>     to access such an object or an array of such objects in the
>     space allocated (until the space is explicitly deallocated).
>
> The wording does seem to imply that the pointer may be used
> to access "such an object or an array of such objects" merely
> *because* it's suitably aligned, which I'm not comfortable with,
> but the intent seems to be that the allocation functions can and
> do create accessible objects.

It falls just short of saying that it is an array.  I'm sure we are
supposed to assume the the space returned by malloc is an array as far
as the explanation of pointer arithmetic is concerned, but I think it
could be more explicit.

> So a pointer to malloc()ed memory can be treated as a pointer to an
> object or array of arbitrary types, but I don't think that implies that
> an array member within a structure can be treated as if it were a bigger
> array.

Absolutely.  I just wondered where the array is that ip and ip+3 point
into in my example.

>> And does that mean we can make the struct hack valid like this:
>>
>>   p->text[3] = 'x'; // invalid
>>   char *cp = (void *)p;
>>   cp += offsetof(sting_header_t, text);
>>   cp[3] = 'x'; // this one OK?
>>
>> I'd say yes and I think my answer would stay the same if the 'text'
>> member and the pointer access where to int types rather than char.
>> (You'd need a char * to do the first offsetof addition, of course.)
>>
>> <snip>
>
> I'd say that doesn't take advantage of the explicit permission granted
> for malloc() and friends, so the behavior is undefined.

I'm not entirely sure of your objection.  Is it that p, converted to a
char *, does not point to the while object that malloc returned but only
to the object that p points too?

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#110433

Ben Bacarisse <ben.usenet@bsb.me.uk> writes:
> Keith Thompson <kst-u@mib.org> writes:
>> Ben Bacarisse <ben.usenet@bsb.me.uk> writes:
>> [...]
>>> I think something might be missing from the standard in the area.  I
>>> think no one disputes that this is defined behaviour:
>>>
>>>   int *ip = malloc(10);
>>>   if (ip) ip[3] = 42;
>>>
>>> but what is the array in question here?  The storage allocated by malloc
>>> has no effective type (yet) so how can ip and ip+3 point to elements of
>>> the same array?  (I've used and int * because there are special
>>> dispensations for accessing any objects byte as a sequence of chars.)
>>
>> N1570 7.22.3p1:
>>
>>     The pointer returned if the allocation succeeds is suitably
>>     aligned so that it may be assigned to a pointer to any type of
>>     object with a fundamental alignment requirement and then used
>>     to access such an object or an array of such objects in the
>>     space allocated (until the space is explicitly deallocated).
>>
>> The wording does seem to imply that the pointer may be used
>> to access "such an object or an array of such objects" merely
>> *because* it's suitably aligned, which I'm not comfortable with,
>> but the intent seems to be that the allocation functions can and
>> do create accessible objects.
>
> It falls just short of saying that it is an array.  I'm sure we are
> supposed to assume the the space returned by malloc is an array as far
> as the explanation of pointer arithmetic is concerned, but I think it
> could be more explicit.

It seems clear to me that it really is an array.  Otherwise, "then used
to access such an object or an array of such objects in the space
allocated" wouldn't make any sense.

>> So a pointer to malloc()ed memory can be treated as a pointer to an
>> object or array of arbitrary types, but I don't think that implies that
>> an array member within a structure can be treated as if it were a bigger
>> array.
>
> Absolutely.  I just wondered where the array is that ip and ip+3 point
> into in my example.
>
>>> And does that mean we can make the struct hack valid like this:
>>>
>>>   p->text[3] = 'x'; // invalid
>>>   char *cp = (void *)p;
>>>   cp += offsetof(sting_header_t, text);
>>>   cp[3] = 'x'; // this one OK?
>>>
>>> I'd say yes and I think my answer would stay the same if the 'text'
>>> member and the pointer access where to int types rather than char.
>>> (You'd need a char * to do the first offsetof addition, of course.)
>>>
>>> <snip>
>>
>> I'd say that doesn't take advantage of the explicit permission granted
>> for malloc() and friends, so the behavior is undefined.
>
> I'm not entirely sure of your objection.  Is it that p, converted to a
> char *, does not point to the while object that malloc returned but only
                               (whole object)
> to the object that p points too?

Yes.  You can treat the space allocated by malloc() as an object or as
an array of objects.  That doesn't, I think, imply anything about
subsets of that memory.

(I'm not at all confident that I understand this correctly.)

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#110434

Keith Thompson <kst-u@mib.org> writes:

> Ben Bacarisse <ben.usenet@bsb.me.uk> writes:
>> Keith Thompson <kst-u@mib.org> writes:
>>> Ben Bacarisse <ben.usenet@bsb.me.uk> writes:
>>> [...]
>>>> I think something might be missing from the standard in the area.  I
>>>> think no one disputes that this is defined behaviour:
>>>>
>>>>   int *ip = malloc(10);
>>>>   if (ip) ip[3] = 42;
>>>>
>>>> but what is the array in question here?  The storage allocated by malloc
>>>> has no effective type (yet) so how can ip and ip+3 point to elements of
>>>> the same array?  (I've used and int * because there are special
>>>> dispensations for accessing any objects byte as a sequence of chars.)
>>>
>>> N1570 7.22.3p1:
>>>
>>>     The pointer returned if the allocation succeeds is suitably
>>>     aligned so that it may be assigned to a pointer to any type of
>>>     object with a fundamental alignment requirement and then used
>>>     to access such an object or an array of such objects in the
>>>     space allocated (until the space is explicitly deallocated).
>>>
>>> The wording does seem to imply that the pointer may be used
>>> to access "such an object or an array of such objects" merely
>>> *because* it's suitably aligned, which I'm not comfortable with,
>>> but the intent seems to be that the allocation functions can and
>>> do create accessible objects.
>>
>> It falls just short of saying that it is an array.  I'm sure we are
>> supposed to assume the the space returned by malloc is an array as far
>> as the explanation of pointer arithmetic is concerned, but I think it
>> could be more explicit.
>
> It seems clear to me that it really is an array.  Otherwise, "then used
> to access such an object or an array of such objects in the space
> allocated" wouldn't make any sense.

I could argue that I accept that it can become an array -- after all the
effective type rules say the type the type of the object *becomes* that of
the effective type of the expression use to store data into.  That would
give some reason to talk about accessing an array in the space
allocated.  In my example, no type had yet been determined.

But this point seems trivial.  I certainly agree on what is intended.  I
just expected it to be simpler to find words that explain what array ip
and ip+4 point into.

>>> So a pointer to malloc()ed memory can be treated as a pointer to an
>>> object or array of arbitrary types, but I don't think that implies that
>>> an array member within a structure can be treated as if it were a bigger
>>> array.
>>
>> Absolutely.  I just wondered where the array is that ip and ip+3 point
>> into in my example.
>>
>>>> And does that mean we can make the struct hack valid like this:
>>>>
>>>>   p->text[3] = 'x'; // invalid
>>>>   char *cp = (void *)p;
>>>>   cp += offsetof(sting_header_t, text);
>>>>   cp[3] = 'x'; // this one OK?
>>>>
>>>> I'd say yes and I think my answer would stay the same if the 'text'
>>>> member and the pointer access where to int types rather than char.
>>>> (You'd need a char * to do the first offsetof addition, of course.)
>>>>
>>>> <snip>
>>>
>>> I'd say that doesn't take advantage of the explicit permission granted
>>> for malloc() and friends, so the behavior is undefined.
>>
>> I'm not entirely sure of your objection.  Is it that p, converted to a
>> char *, does not point to the while object that malloc returned but only
>                                (whole object)
>> to the object that p points too?
>
> Yes.  You can treat the space allocated by malloc() as an object or as
> an array of objects.  That doesn't, I think, imply anything about
> subsets of that memory.

So after

   int *p = malloc(100);
   char *cp = (void *)p;

cp now only points to one int and cp[sizeof (int) + 1] constructs an
invalid pointer?

> (I'm not at all confident that I understand this correctly.)

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#110444

Ben Bacarisse <ben.usenet@bsb.me.uk> writes:

> Keith Thompson <kst-u@mib.org> writes:
> <snip>
>> Here's a demo program, based on your declaration above with the syntax
>> error corrected:
>>
>> #include <stdio.h>
>> #include <stdlib.h>
>> #include <string.h>
>> int main(void) {
>>     typedef struct string_header_s {
>>         short length; /* limited length */
>>         char text[ 1 ];
>>     } string_header_t;
>>
>>     string_header_t *p = malloc(sizeof *p + 10); /* more than we need */
>>     if (p == NULL) exit(EXIT_FAILURE);
>>     p->length = 4;
>>     strcpy(p->text, "abcd");
>>     printf("p->text[3] = '%c'\n", p->text[3]);
>> }
>>
>> When I compile and run this on my system, I get no compile-time
>> diagnostics and the output, as expected, is:
>>
>> p->text[3] = 'd'
>>
>> But both the strcpy() call and the reference to p->text[3] have
>> undefined behavior.  Concentrating on the latter, the indexing
>> operator [] is defined in terms of pointer addition, in this case
>> (p->text + 3).  The description of pointer addition, in N1570
>> 6.5.6p8, says:
>>
>>     If both the pointer operand and the result point to elements
>>     of the same array object, or one past the last element of the
>>     array object, the evaluation shall not produce an overflow;
>>     otherwise, the behavior is undefined.
>>
>> The array object in question is p->text, which is of type char[1].
>> We've used malloc to allocate additional memory past the end of
>> that array object, but it's not part of the array object.

I wasn't sure where to jump into this thread so here seems
as good a place as any.

> I think something might be missing from the standard in the area.  I
> think no one disputes that this is defined behaviour:
>
>   int *ip = malloc(40);  // [earlier size bug corrected]
>   if (ip) ip[3] = 42;
>
> but what is the array in question here?  The storage allocated by malloc
> has no effective type (yet) so how can ip and ip+3 point to elements of
> the same array?  (I've used and int * because there are special
> dispensations for accessing any objects byte as a sequence of chars.)

Assuming 'sizeof (int) == 4', this malloc call returns a pointer
to a space that can be used as an array of 10 ints.  Given that,
the index operation, and subsequent assignment operation, have
defined behavior.  Asking "what is the array" is not important,
because the relevant passage for malloc(), etc, gives the right
to use the returned pointer value in the same way that a pointer
to the start of an array could be used.

> And does that mean we can make the struct hack valid like this:
>
>   /*[previously]*/  string_header_t *p = malloc(sizeof *p + 10);
>   p->text[3] = 'x'; // invalid
>   char *cp = (void *)p;
>   cp += offsetof(sting_header_t, text);
>   cp[3] = 'x'; // this one OK?
>
> I'd say yes

In my view there is no question that the behavior involving 'cp'
is defined, actually for two different reasons.  The more subtle
reason I will discuss below.  The more obvious reason is that for
any object, including the implied array object pointed to by 'p',
it's always allowed to convert a pointer to the start of the
object to a pointer-to-character type, and access the object as
an array of bytes.  (Some people might quibble about having to
increment pointers one-by-one to access the characters, but I'm
going to ignore that since it doesn't pass the laugh test.)
Surely the key provision in the first paragraph regarding memory
management functions, ie, the one that says the returned pointer
from malloc() and friends may be

    used to access such an object or an array of such objects in
    the space allocated

is meant to include the ability to access the same space as a
character array.  The operations on 'cp' are allowed under that
umbrella.

> and I think my answer would stay the same if the 'text'
> member and the pointer access where to int types rather than char.
> (You'd need a char * to do the first offsetof addition, of course.)

Here we get to the heart of the matter, and the more subtle
reasoning alluded to above.  Again I believe the behavior
is defined (with a very small asterisk covering a DS9000
type implementation, as explained further below).

Suppose first, as will most commonly be true, that the offset of
the int member in question is a multiple of sizeof (int).  We are
allowed to take the return value of malloc() and convert it to
an 'int *', and use that to access an array of ints.  Casting the
pointer 'p' to (void *) gives that same value back (please no
quibbles about "same" versus "equal"), which lets us use that
value to access an array of ints.  So something like this

   int *ip = (void *)p;
   ip += offsetof(string_header_t, ints) / sizeof (int);
   ip[3] = 42; 

has to work.  (Using a char * initially and using a byte offset
to increment that pointer is not an important difference.)

Let me say this again more directly, to convey the impact of what
the Standard says about malloc() values.  We may do this:

    void *v = malloc( 10000 );
    int  *ip = v;
    long *lp = v;
    float *fp = v;
    string_header_t *shp = v;

after which /all/ of these pointers may be used to access arrays
of their respective types.  As long as we don't run afoul of
effective type rules, pointers to space returned by malloc() may
be freely mixed and matched, intermingled, converted to char *
and adjusted by byte lengths, etc, and everything works.  In
effect, the type of space returned by malloc() is a union over
/all possible types/ that will fit in the space, including array
types.  Treating part of the space one way and another part a
different way is allowed in all cases (again, assuming no
interference from effective type rules or struct/union type
overlap).  It is this broad guarantee that gives workaround
code like that shown above defined behavior.

Now for the asterisk.  Suppose the byte offset for the "struct
hack" member (of type int[1]) is not a multiple of sizeof(int).
This means an int array starting at the beginning of the malloc()
space doesn't do the job for us.  Normally this won't be a
problem, since in addition to

    struct { short s; int i1[1]; }

there will be types like

    struct { short s; int i10[10]; }
    struct { short s; int i100[100]; }
    struct { short s; int i1000[1000]; }

with the same offset for the int array members as that of the
first "struct hack" type.  However, a perverse implementation
could choose a different offset for the extent 1 case and all
cases with extent greater than 1, which means in principle the
workaround access scheme would give undefined behavior on such
implementations.  Not counting such far-fetched scenarios however
the behavior is defined.

(It occurs to me now I should mention one other thing, namely,
storing into a member of a struct gives the freedom to put
indeterminate values into bytes of the struct that don't
correspond to other members.  So if there is padding after the
"struct hack" member, and a struct member is stored into after
the SH member array has been set (in those locations), then that
is also a potential conflict.  Probably not likely to make a
difference, but I try to be thorough.)

[toc] | [prev] | [next] | [standalone]

#110445

On Saturday, May 20, 2017 at 5:58:52 PM UTC+9, Tim Rentsch wrote:
> Ben Bacarisse <ben.usenet@somewhere> writes:
> 
> > Keith Thompson <kst-u@somewhere> writes:
> > <snip>
> >> Here's a demo program, based on your declaration above with the syntax
> >> error corrected:
> >>
> >> #include <stdio.h>
> >> #include <stdlib.h>
> >> #include <string.h>
> >> int main(void) {
> >>     typedef struct string_header_s {
> >>         short length; /* limited length */
> >>         char text[ 1 ];
> >>     } string_header_t;
> >>
> >>     string_header_t *p = malloc(sizeof *p + 10); /* more than we need */
> >>     if (p == NULL) exit(EXIT_FAILURE);
> >>     p->length = 4;
> >>     strcpy(p->text, "abcd");
> >>     printf("p->text[3] = '%c'\n", p->text[3]);
> >> }
> >>
> >> When I compile and run this on my system, I get no compile-time
> >> diagnostics and the output, as expected, is:
> >>
> >> p->text[3] = 'd'
> >>
> >> But both the strcpy() call and the reference to p->text[3] have
> >> undefined behavior.  Concentrating on the latter, the indexing
> >> operator [] is defined in terms of pointer addition, in this case
> >> (p->text + 3).  The description of pointer addition, in N1570
> >> 6.5.6p8, says:
> >>
> >>     If both the pointer operand and the result point to elements
> >>     of the same array object, or one past the last element of the
> >>     array object, the evaluation shall not produce an overflow;
> >>     otherwise, the behavior is undefined.
> >>
> >> The array object in question is p->text, which is of type char[1].
> >> We've used malloc to allocate additional memory past the end of
> >> that array object, but it's not part of the array object.
> 
> I wasn't sure where to jump into this thread so here seems
> as good a place as any.
> 
> > I think something might be missing from the standard in the area.  I
> > think no one disputes that this is defined behaviour:
> >
> >   int *ip = malloc(40);  // [earlier size bug corrected]
> >   if (ip) ip[3] = 42;
> >
> > but what is the array in question here?  The storage allocated by malloc
> > has no effective type (yet) so how can ip and ip+3 point to elements of
> > the same array?  (I've used and int * because there are special
> > dispensations for accessing any objects byte as a sequence of chars.)
> 
> Assuming 'sizeof (int) == 4', this malloc call returns a pointer
> to a space that can be used as an array of 10 ints.  Given that,
> the index operation, and subsequent assignment operation, have
> defined behavior.  Asking "what is the array" is not important,
> because the relevant passage for malloc(), etc, gives the right
> to use the returned pointer value in the same way that a pointer
> to the start of an array could be used.
> 
> > And does that mean we can make the struct hack valid like this:
> >
> >   /*[previously]*/  string_header_t *p = malloc(sizeof *p + 10);
> >   p->text[3] = 'x'; // invalid
> >   char *cp = (void *)p;
> >   cp += offsetof(sting_header_t, text);
> >   cp[3] = 'x'; // this one OK?
> >
> > I'd say yes
> 
> In my view there is no question that the behavior involving 'cp'
> is defined, actually for two different reasons.  The more subtle
> reason I will discuss below.  The more obvious reason is that for
> any object, including the implied array object pointed to by 'p',
> it's always allowed to convert a pointer to the start of the
> object to a pointer-to-character type, and access the object as
> an array of bytes.  (Some people might quibble about having to
> increment pointers one-by-one to access the characters, but I'm
> going to ignore that since it doesn't pass the laugh test.)
> Surely the key provision in the first paragraph regarding memory
> management functions, ie, the one that says the returned pointer
> from malloc() and friends may be
> 
>     used to access such an object or an array of such objects in
>     the space allocated
> 
> is meant to include the ability to access the same space as a
> character array.  The operations on 'cp' are allowed under that
> umbrella.
> 
> > and I think my answer would stay the same if the 'text'
> > member and the pointer access where to int types rather than char.
> > (You'd need a char * to do the first offsetof addition, of course.)
> 
> Here we get to the heart of the matter, and the more subtle
> reasoning alluded to above.  Again I believe the behavior
> is defined (with a very small asterisk covering a DS9000
> type implementation, as explained further below).
> 
> Suppose first, as will most commonly be true, that the offset of
> the int member in question is a multiple of sizeof (int).  We are
> allowed to take the return value of malloc() and convert it to
> an 'int *', and use that to access an array of ints.  Casting the
> pointer 'p' to (void *) gives that same value back (please no
> quibbles about "same" versus "equal"), which lets us use that
> value to access an array of ints.  So something like this
> 
>    int *ip = (void *)p;
>    ip += offsetof(string_header_t, ints) / sizeof (int);
>    ip[3] = 42; 
> 
> has to work.  (Using a char * initially and using a byte offset
> to increment that pointer is not an important difference.)
> 
> Let me say this again more directly, to convey the impact of what
> the Standard says about malloc() values.  We may do this:
> 
>     void *v = malloc( 10000 );
>     int  *ip = v;
>     long *lp = v;
>     float *fp = v;
>     string_header_t *shp = v;
> 
> after which /all/ of these pointers may be used to access arrays
> of their respective types.  As long as we don't run afoul of
> effective type rules, pointers to space returned by malloc() may
> be freely mixed and matched, intermingled, converted to char *
> and adjusted by byte lengths, etc, and everything works.  In
> effect, the type of space returned by malloc() is a union over
> /all possible types/ that will fit in the space, including array
> types.  Treating part of the space one way and another part a
> different way is allowed in all cases (again, assuming no
> interference from effective type rules or struct/union type
> overlap).  It is this broad guarantee that gives workaround
> code like that shown above defined behavior.
> 
> Now for the asterisk.  Suppose the byte offset for the "struct
> hack" member (of type int[1]) is not a multiple of sizeof(int).
> This means an int array starting at the beginning of the malloc()
> space doesn't do the job for us.  Normally this won't be a
> problem, since in addition to
> 
>     struct { short s; int i1[1]; }
> 
> there will be types like
> 
>     struct { short s; int i10[10]; }
>     struct { short s; int i100[100]; }
>     struct { short s; int i1000[1000]; }
> 
> with the same offset for the int array members as that of the
> first "struct hack" type.  However, a perverse implementation
> could choose a different offset for the extent 1 case and all
> cases with extent greater than 1, which means in principle the
> workaround access scheme would give undefined behavior on such
> implementations.  Not counting such far-fetched scenarios however
> the behavior is defined.
> 
> (It occurs to me now I should mention one other thing, namely,
> storing into a member of a struct gives the freedom to put
> indeterminate values into bytes of the struct that don't
> correspond to other members.  So if there is padding after the
> "struct hack" member, and a struct member is stored into after
> the SH member array has been set (in those locations), then that
> is also a potential conflict.  Probably not likely to make a
> difference, but I try to be thorough.)

Ick. Yes.

I was forgetting the struct member ordering and padding problems.

I suppose that would be the reason for wanting to use the empty size
as a flag to say that the array member should come at the end, laid out 
to allow arbitrary lengths, where, with arrays of specified sizes, the
compiler "ought to be" free to optimize the structure, and thus re-order 
the elements and add padding, etc.

Pragmas exist for some compilers, but I wonder if anyone has ever 
suggested a struct type modifier that would require strict layout rules,
under the same sort of reasoning that gave us the volatile modifier.

Seventeen years of separation puts cruft in the brain.

Apologies to all who corrected me.

--
Joel Rees

Delusions of being a novelist:
http://reiisi.blogspot.com/p/novels-i-am-writing.html

[toc] | [prev] | [next] | [standalone]

#110441

On Saturday, May 20, 2017 at 1:34:11 AM UTC+9, Keith Thompson wrote:
> joel.rees@something writes:
> > On Friday, May 19, 2017 at 1:35:33 AM UTC+9, Keith Thompson wrote:
> >> joel.rees@somewhere writes:
> >> > On Monday, May 15, 2017 at 11:23:23 PM UTC+9, Tim Rentsch wrote:
> >> [...]
> >> >> Ah.  In that case I must demur regarding part of your later
> >> >> statement.  An access through an array type of extent 1 is
> >> >> not definedly allowed to access elements after the element
> >> >> at index 0.
> >> >
> >> > typedef string_header_s struct
> >> > {  short length; /* limited length */
> >> >    char text[ 1 ];
> >> > } string_header_t;
> >> 
> >> That's called the "struct hack", and it was always of questionable
> >> validity.  You can likely get away with allocating extra memory and
> >> referring to, for example, obj.text[10], but the behavior is undefined.
> >
> > You don't believe in reading before you spout, I take it.
> 
> I do believe in it, though I don't always succeed.  I have occasionally
> made the mistake of posting without reading and understanding an entire
> article, and when I do that I appreciate having it pointed out.

I've done such things, too.

And I have to apologize for getting peeved the other day.

The conversation has moved in the direction I intended, so I won't bother 
adding empty bits here.

Well, except, JFTR.

Except that it is turning out to be more than just for the record. 
Having read ahead in the thread, I'll borrow this point to start
on another tack.

> > Go back, read the source you clipped and tell me that again.
> 
> Ok, I've re-read your previous article and I'll tell you that again.
> The behavior is undefined.
> 
> Here's a demo program, based on your declaration above with the syntax
> error corrected:

I assume you mean by eliminating the allocation test I messed up. :)

> #include <stdio.h>
> #include <stdlib.h>
> #include <string.h>
> int main(void) {
>     typedef struct string_header_s {
>         short length; /* limited length */
>         char text[ 1 ];
>     } string_header_t;
> 
>     string_header_t *p = malloc(sizeof *p + 10); /* more than we need */
>     if (p == NULL) exit(EXIT_FAILURE);
>     p->length = 4;
>     strcpy(p->text, "abcd");
>     printf("p->text[3] = '%c'\n", p->text[3]);
> }
> 
> When I compile and run this on my system, I get no compile-time
> diagnostics and the output, as expected, is:
> 
> p->text[3] = 'd'
> 
> But both the strcpy() call and the reference to p->text[3] have
> undefined behavior.  Concentrating on the latter, the indexing
> operator [] is defined in terms of pointer addition, in this case
> (p->text + 3).  The description of pointer addition, in N1570
> 6.5.6p8, says:
> 
>     If both the pointer operand and the result point to elements
>     of the same array object, or one past the last element of the
>     array object, the evaluation shall not produce an overflow;
>     otherwise, the behavior is undefined.
> 
> The array object in question is p->text, which is of type char[1].
> We've used malloc to allocate additional memory past the end of
> that array object, but it's not part of the array object.
> 
> Most, perhaps all, C compilers will accept this and generate code
> that behaves as expected, precisely because the struct hack is
> a common idiom.  Nevertheless, the behavior is not defined by
> the standard.
> 
> If you disagree, I'm paying attention.

As Ben notes, it's hard to say that malloc doesn't return an array.

But that was part of the reason I did my own allocation, to make 
it clear that what I was working on was, in fact, within one array.

Which leaves us with the question of which array we look at when, 
and I'm thinking that ...

------------ internal processing record --------------
well, in my mind, Microsoft spearheaded the aggressive (Can I call 
it that?) compile time analysis of C objects. That may not be where
I'm trying to head.

Pascal, as it was originally envisioned, kept a run-time record of 
array definitions so that limit checks could be performed at run-time.

If the implementor made that record part of the array object, 
physically (so to speak) adjacent to the array, making arrays of 
arrays became messy.

C originally kept no such run-time record, which made it much easier 
to do multi-dimensioned arrays as just arrays of arrays.

But private allocation is a bigger reason.
------------ end internal processing record --------------

If I had to worry that the compiler would be free to fight with me over 
the interpretation of the array in the header type I've shown, I'd have 
a really hard time writing allocation records when I'm doing my own 
allocation. (I'm including counted strings in this.)

Unless my allocation records, in spite of being adjacent to the 
allocated block, included a pointer to the block.

Allocation records adjacent to the allocated blocks can be argued as 
being good or bad, but the compiler should not militate against the 
technique.

Which leaves us with the empty size declaration as the standard 
approved syntactic flag (which I had missed somehow over the last 
fifteen years), and the question of whether a size of an array 
in such a declaration should be considered by the compiler as a 
minimum or maximum, or should be considered at all.

I think the original specification of the language intended to 
allow size declarations as indicating minimums as well as maximums.

In other words, the size of the struct would be the minimum 
allocatable, and the programmer would know whether he or she had 
actually allocated more.

Let's try another header:

typedef struct string_header_s {
   short actual_length;
   short allocated_max;
   char text[ 10 ];
} string_header_t;

And we'll assume code supporting the apparent meaning of the field
names.

Compare that to the form that the standard would urge on us:

typedef struct string_header_s {
   short actual_length;
   short allocated_max;
   char text[]; /* Allocation minimum MUST be 10 */
} string_header_t;

An assertion could help, but, with the empty size declaration, what 
could we assert on that would be (effectively) any more meaningful 
than the comment?

> -- 
> Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
> Working, but not speaking, for JetHead Development, Inc.
> "We must do something.  This is something.  Therefore, we must do this."
>     -- Antony Jay and Jonathan Lynn, "Yes Minister"

--
Joel Rees

Delusions of being a novelist:
http://reiisi.blogspot.com/p/novels-i-am-writing.html

[toc] | [prev] | [next] | [standalone]

#110442

On Saturday, May 20, 2017 at 12:09:55 AM UTC-5, joel...@gmail.com wrote:
> I think the original specification of the language intended to 
> allow size declarations as indicating minimums as well as maximums.

In the closest thing I know to original specification of the language--the
1974 C Reference Manual written by the inventor of the language found at

    https://www.bell-labs.com/usr/dmr/www/cman.pdf

on a machine with 16-bit int, and 8-bit char, the array dimension in the
lines marked with /*****/ serves two identifiable purposes.

    struct foo {
      int moe;
      char larry[10]; /*****/
      int curly;
      int shemp[6];   /*****/
    };

it serves to tell the compiler how far the next item (if any) or the end
of the struct should be from the start of the current item, and it allows
the compiler to compute the value it should report if code calls "sizeof"
on one of those members.  If member larry or shemp of a structure is used
in anything other than an lvalue expression, the compiler will take the
address of the structure, add 2 or 14, and process the resulting address
as either a char* or int*; once that is done, the compiler will have no
reason to know or care about the size of the array in question.  Note that
in the 1974 CRM, the meaning of structure operations is *defined* in terms
of addresses, thus unambiguously defining a number of useful constructs
which some of today's compiler writers claim were never defined.

[toc] | [prev] | [next] | [standalone]

#110397

On 05/19/2017 02:40 AM, joel.rees@gmail.com wrote:
> On Friday, May 19, 2017 at 1:35:33 AM UTC+9, Keith Thompson wrote:
>> joel.rees@somewhere writes:
...
>>> typedef string_header_s struct
>>> {  short length; /* limited length */
>>>    char text[ 1 ];
>>> } string_header_t;
>>
>> That's called the "struct hack", and it was always of questionable
>> validity.  You can likely get away with allocating extra memory and
>> referring to, for example, obj.text[10], but the behavior is undefined.
>
> You don't believe in reading before you spout, I take it.
>
> Go back, read the source you clipped and tell me that again.
>
> If you prove that you have read and understood the source that you
> say is undefined, I'll listen to you.

Here's the relevant lines of code:

> typedef struct string_header_s
> {  short length;
>    char string[ 1 ];
> } string_header_t;
...
> string_header_t * string_save( char string[] )
> {  long length = strlen( string );
>    string_header_t * headerp = string_allocate( length );
>    memcpy( headerp->string, string, length );
>    headerp->string[ length ] = '\0';
>    return headerp;
> }
...
>    thing = string_save( "hello" );

That's all that's needed to prove that the behavior is undefined - the 
details of string_allocate() are irrelevant, so long as it returns a 
correctly aligned pointer to sufficient memory - if it failed to do so, 
that would in itself be justification for the behavior to be undefined.

When string_save("hello") is called, it sets length = strlen("hello"), 
which is 5. Later on, it evaluates headerp->string[length], which is 
equivalent to *(headerp->string + length).

Here's what the standard says about that addition:

"When an expression that has integer type is added to or subtracted from 
a pointer, the result has the type of the pointer operand. If the 
pointer operand points to an element of an array object, and the array 
is large enough, the result points to an element offset from the 
original element such that the difference of the subscripts of the 
resulting and original array elements equals the integer expression. In 
other words, if the expression P points to the i-th element of an array 
object, the expressions (P)+N (equivalently, N+(P)) and (P)-N (where N 
has the value n) point to, respectively, the i+n-th and i−n-th elements 
of  the array object, provided they exist. Moreover, if the expression P 
points to the last element of an array object, the expression (P)+1 
points one past the last element of the array object, and if the  as I 
said before, expression Q points one past the last element of an array 
object,
the expression (Q)-1 points to the last element of the array object. If 
both the pointer operand and the result point to elements of the same 
array object, or one past the last element of the array object, the 
evaluation shall not produce an overflow; otherwise, the  behavior is 
undefined." (6.5.6p8)

The relevant array is headerp->string, and it has a length of exactly 1. 
headerp->string+5 does not point at any element of that array, nor does 
it point one past the end of that array. Therefore, "the behavior is 
undefined".

This is a relatively safe example of undefined behavior. Most, possibly 
all, C90 compilers dealt with such code exactly as you incorrectly think 
they were required to deal with it - that's one of the possibilities 
that's allowed when the behavior is undefined. That doesn't change the 
fact that the behavior is, in fact, undefined.

If you don't believe my argument is valid, check the official committee 
responses to DR 051 and DR 072:
<http://www.open-std.org/jtc1/sc22/wg14/docs/rr/dr_051.html>
<http://www.open-std.org/jtc1/sc22/wg14/docs/rr/dr_072.html>.

However, in C99, flexible array members were invented, which use syntax 
that is only slightly different than that of the struct hack, to give 
defined behavior to code that is otherwise quite similar to what you 
wrote. The key clause that makes the difference says: "However, when a . 
(or ->) operator has a left operand that is (a pointer to) a structure 
with a flexible array member and the right operand names that member, it 
behaves as if that member were replaced with the longest array (with the 
same element type) that would not make the structure larger than the 
object being accessed;" (6.7.2.1p18)

Since you allocated a block of memory large enough to hold "hello", if 
you had used the flexible array member syntax, that block would have 
been "the object" to which 6.7.2.1p18 refers, and "headerp->string" 
would therefore have been treated as if it were declared string[6], in 
which case headerp->string+length would have had defined behavior.

Because the standard has given it's official blessing to the use of 
flexible array members, I think there's a small but real chance that 
some compilers will no longer support the struct hack. The simplest way 
to do that would be to implement run-time array-bounds checking. 
However, run-time array-bounds checking is very expensive. A more 
plausible way for the struct hack to fail would be optimizations based 
upon the unchecked assumption that headerp->string will never have any 
integer added to it with a value other than 0 or 1, and that if the 
amount added is 1, the resulting pointer will never be dereferenced. 
Arbitrarily obscure failure modes could occur as a result of such 
optimizations if that assumption is violated.

Still, the struct hack was very widely used before flexible array 
members were invented, and I wouldn't recommend holding your breath 
waiting for a compiler to come out that breaks it.

[toc] | [prev] | [next] | [standalone]

#110417

On Friday, May 19, 2017 at 12:33:20 PM UTC-5, James R. Kuyper wrote:
> If you don't believe my argument is valid, check the official committee 
> responses to DR 051 and DR 072:
> <http://www.open-std.org/jtc1/sc22/wg14/docs/rr/dr_051.html>
> <http://www.open-std.org/jtc1/sc22/wg14/docs/rr/dr_072.html>.

How are defect reports intended to be interpreted?  What kind of wording
in a defect report would distinguish among the following scenarios:

   1. The authors of the Standard almost certainly intended to say that X
      was defined, but the wording of the Standard is defective and says
      that is is not.  The next version of the Standard should fix the wording
      so that X is unambiguously defined; until then, compilers should treat
      X as defined.

   2. The authors of the Standard intended to say that X is undefined, but
      the wording of the Standard is defective and could be misconstrued as
      saying that X is defined, but it is unlikely that any programmers would
      be relying upon that.  The next version of the Standard should improve
      the wording to make clear that X is not defined, and programmers should
      expect that compilers may interpret X as undefined unless they
      explicitly document otherwise.

   3. The authors of the Standard intended to say that X is undefined, but
      the wording of the Standard is defective and could be misconstrued as
      saying that X is defined, and programmers might reasonably be relying
      upon that.  The next version of the Standard should define a standard
      macro to say whether a particular implementation treats X as defined;
      until then, compilers should either treat X as defined or explicitly
      document that they do not.

   4. It is unclear whether the authors of the Standard intended X to be
      defined or undefined; the wording of the Standard would seem to
      suggest that it is not defined, but could also be construed as saying
      that it is defined, and programmers might reasonably be relying upon
      that.  The Standard is clearly defective in some fashion, but without
      knowing the intended meaning it's impossible to know in which way it's
      defective or what should be done to "fix" it.

A finding that X is "undefined" would be consistent with any of the above,
but the different scenarios should be handled differently.

[toc] | [prev] | [next] | [standalone]

#110438

On Friday, May 19, 2017 at 4:44:37 PM UTC-4, supe...@casperkitty.com wrote:
> On Friday, May 19, 2017 at 12:33:20 PM UTC-5, James R. Kuyper wrote:
> > If you don't believe my argument is valid, check the official committee 
> > responses to DR 051 and DR 072:
> > <http://www.open-std.org/jtc1/sc22/wg14/docs/rr/dr_051.html>
> > <http://www.open-std.org/jtc1/sc22/wg14/docs/rr/dr_072.html>.
> 
> How are defect reports intended to be interpreted?  What kind of wording
> in a defect report would distinguish among the following scenarios:
> 
>    1. The authors of the Standard almost certainly intended to say that X
>       was defined, but the wording of the Standard is defective and says
>       that is is not.  The next version of the Standard should fix the wording
>       so that X is unambiguously defined; until then, compilers should treat
>       X as defined.
> 
>    2. The authors of the Standard intended to say that X is undefined, but
>       the wording of the Standard is defective and could be misconstrued as
>       saying that X is defined, but it is unlikely that any programmers would
>       be relying upon that.  The next version of the Standard should improve
>       the wording to make clear that X is not defined, and programmers should
>       expect that compilers may interpret X as undefined unless they
>       explicitly document otherwise.
> 
>    3. The authors of the Standard intended to say that X is undefined, but
>       the wording of the Standard is defective and could be misconstrued as
>       saying that X is defined, and programmers might reasonably be relying
>       upon that.  The next version of the Standard should define a standard
>       macro to say whether a particular implementation treats X as defined;
>       until then, compilers should either treat X as defined or explicitly
>       document that they do not.
> 
>    4. It is unclear whether the authors of the Standard intended X to be
>       defined or undefined; the wording of the Standard would seem to
>       suggest that it is not defined, but could also be construed as saying
>       that it is defined, and programmers might reasonably be relying upon
>       that.  The Standard is clearly defective in some fashion, but without
>       knowing the intended meaning it's impossible to know in which way it's
>       defective or what should be done to "fix" it.
> 
> A finding that X is "undefined" would be consistent with any of the above,
> but the different scenarios should be handled differently.

No, a finding that "X is undefined" is not consistent with any of the above.
Unless modified by the addition of words saying something else, "X is undefined"
means nothing more or less than:

"The committee intended the behavior to be undefined, and in the opinion of the
committee, the standard expresses that intent clearly enough, andany one who's
been interpreting it in a contrary manner should adjust their interpretations
accordingly."

As far as I know (and I haven't done an exhaustive search, so I could be
mistaken), the committee tends not to resolve defect reports by telling people
that they should act as though the standard says something other than what it
actually says. That would constitute an official change to the standard, and
making such changes is what Technical Corrigenda (TC) are for. The resolution of
a defect report may recommend issuing a TC, but the resolution itself does not
change the meaning of the standard - that's achieved only when the TC has been
approved.

All of the other nuances you mention above can appear in a defect report, and I
believe that most of them have appeared in at least one such report. But they
are never implied by the statement "X is undefined". Such things as "the wording
does not correctly express the intent of the committee" or "the wording could be
improved to make the intent clearer" are stated explicitly, when the committee
believes them to be applicable.

[toc] | [prev] | [next] | [standalone]

#110443

On Friday, May 19, 2017 at 9:20:55 PM UTC-5, james...@verizon.net wrote:
> On Friday, May 19, 2017 at 4:44:37 PM UTC-4, supe...@casperkitty.com wrote:
> > How are defect reports intended to be interpreted?  What kind of wording
> > in a defect report would distinguish among the following scenarios:
> > 
> >  1. The authors of the Standard almost certainly intended to say that X
> >     was defined, but the wording of the Standard is defective and says
> >     that is is not.  The next version of the Standard should fix the wording
> >     so that X is unambiguously defined; until then, compilers should treat
> >     X as defined.
> > 
> >  2. The authors of the Standard intended to say that X is undefined, but
> >     the wording of the Standard is defective and could be misconstrued as
> >     saying that X is defined, but it is unlikely that any programmers would
> >     be relying upon that.  The next version of the Standard should improve
> >     the wording to make clear that X is not defined, and programmers should
> >     expect that compilers may interpret X as undefined unless they
> >     explicitly document otherwise.
> > 
> >  3. The authors of the Standard intended to say that X is undefined, but
> >     the wording of the Standard is defective and could be misconstrued as
> >     saying that X is defined, and programmers might reasonably be relying
> >     upon that.  The next version of the Standard should define a standard
> >     macro to say whether a particular implementation treats X as defined;
> >     until then, compilers should either treat X as defined or explicitly
> >     document that they do not.
> > 
> >  4. It is unclear whether the authors of the Standard intended X to be
> >     defined or undefined; the wording of the Standard would seem to
> >     suggest that it is not defined, but could also be construed as saying
> >     that it is defined, and programmers might reasonably be relying upon
> >     that.  The Standard is clearly defective in some fashion, but without
> >     knowing the intended meaning it's impossible to know in which way it's
> >     defective or what should be done to "fix" it.
> > 
> > A finding that X is "undefined" would be consistent with any of the above,
> > but the different scenarios should be handled differently.
> 
> No, a finding that "X is undefined" is not consistent with any of the above.
> Unless modified by the addition of words saying something else, "X is undefined"
> means nothing more or less than:
> 
> "The committee intended the behavior to be undefined, and in the opinion of the
> committee, the standard expresses that intent clearly enough, andany one who's
> been interpreting it in a contrary manner should adjust their interpretations
> accordingly."

My question was "what kind of wording would distinguish among the various
possibilities".  If the Standard had been adequately clear, I would think
there would be no need for a defect report.  If any significant number of
people reading the Standard aren't certain about what it means, that would
constitute prima facie evidence that at minimum the Standard is less clear
than it should be.

[toc] | [prev] | [next] | [standalone]

#110451

On Saturday, May 20, 2017 at 2:30:40 AM UTC-4, supe...@casperkitty.com wrote:
> On Friday, May 19, 2017 at 9:20:55 PM UTC-5, james...@verizon.net wrote:
...
> > No, a finding that "X is undefined" is not consistent with any of the above.
> > Unless modified by the addition of words saying something else, "X is undefined"
> > means nothing more or less than:
> > 
> > "The committee intended the behavior to be undefined, and in the opinion of the
> > committee, the standard expresses that intent clearly enough, andany one who's
> > been interpreting it in a contrary manner should adjust their interpretations
> > accordingly."
> 
> My question was "what kind of wording would distinguish among the various
> possibilities".  If the Standard had been adequately clear, I would think
> there would be no need for a defect report.  If any significant number of
> people reading the Standard aren't certain about what it means, that would
> constitute prima facie evidence that at minimum the Standard is less clear
> than it should be.

There's no description so clear that no one can misunderstand it. If the
committee thinks that the description is either wrong, or even merely unclear,
the committee's resolution to a DR will express that opinion explicitly. In the
absence of any statement to that effect, the DR resolution is intended only to
instruct people in the correct interpretation of the standard, even if it's not
the one that would have occurred to them without instruction.

[toc] | [prev] | [next] | [standalone]

#110452

On Saturday, May 20, 2017 at 2:30:40 AM UTC-4, supe...@casperkitty.com wrote:
> On Friday, May 19, 2017 at 9:20:55 PM UTC-5, james...@verizon.net wrote:
> > On Friday, May 19, 2017 at 4:44:37 PM UTC-4, supe...@casperkitty.com wrote:
> > > How are defect reports intended to be interpreted?  What kind of wording
> > > in a defect report would distinguish among the following scenarios:
> > > 
> > >  1. The authors of the Standard almost certainly intended to say that X
> > >     was defined, but the wording of the Standard is defective and says
> > >     that is is not.  The next version of the Standard should fix the wording
> > >     so that X is unambiguously defined; until then, compilers should treat
> > >     X as defined.
> > > 
> > >  2. The authors of the Standard intended to say that X is undefined, but
> > >     the wording of the Standard is defective and could be misconstrued as
> > >     saying that X is defined, but it is unlikely that any programmers would
> > >     be relying upon that.  The next version of the Standard should improve
> > >     the wording to make clear that X is not defined, and programmers should
> > >     expect that compilers may interpret X as undefined unless they
> > >     explicitly document otherwise.
> > > 
> > >  3. The authors of the Standard intended to say that X is undefined, but
> > >     the wording of the Standard is defective and could be misconstrued as
> > >     saying that X is defined, and programmers might reasonably be relying
> > >     upon that.  The next version of the Standard should define a standard
> > >     macro to say whether a particular implementation treats X as defined;
> > >     until then, compilers should either treat X as defined or explicitly
> > >     document that they do not.
> > > 
> > >  4. It is unclear whether the authors of the Standard intended X to be
> > >     defined or undefined; the wording of the Standard would seem to
> > >     suggest that it is not defined, but could also be construed as saying
> > >     that it is defined, and programmers might reasonably be relying upon
> > >     that.  The Standard is clearly defective in some fashion, but without
> > >     knowing the intended meaning it's impossible to know in which way it's
> > >     defective or what should be done to "fix" it.
> > > 
> > > A finding that X is "undefined" would be consistent with any of the above,
> > > but the different scenarios should be handled differently.
> > 
> > No, a finding that "X is undefined" is not consistent with any of the above.
> > Unless modified by the addition of words saying something else, "X is undefined"
> > means nothing more or less than:
> > 
> > "The committee intended the behavior to be undefined, and in the opinion of the
> > committee, the standard expresses that intent clearly enough, andany one who's
> > been interpreting it in a contrary manner should adjust their interpretations
> > accordingly."
> 
> My question was "what kind of wording would distinguish among the various
> possibilities".

It just occurred to me that I've only answered that question in an oblique way.
I should answer it directly. First of all, your list of the various
possibilities did not include the one possibility that actually applied.

Secondly, the kind of wording that would distinguish those possibilities is
wording that explicitly states the things you were hoping were implied. The
possibility that the committee had decided that the wording did not correctly
convey the committee's intent would be conveyed by explicit words to that
effect. The possibility that the committee had concluded that the person filing
the DR was justified in being confused, because the wording was confusing, would
be conveyed by an explicit statement that the wording was confusing, and needed
to be improved. A recommendation that a future version of the standard should be
modified to address these flaws is implicit in either of those cases - but it is
most definitely NOT implicit in a statement that "X is undefined". 

One of the possibilities that you mentioned is " The next version of the
Standard should define a standard macro to say whether a particular
implementation treats X as defined; until then, compilers should either treat X
as defined or explicitly document that they do not." That is not something that
the committee would ever say, so you don't need to wonder about what wording
they'd use to convey that meaning. First of all, if the standard's wording does
need improving, the improvement will consist of the standard either defining X,
or saying that X is undefined, or leaving X implicitly undefined by failing to
define it. They would not leave the question of whether or not it is defined to
be determined by checking a macro. The committee has created optional features
in C, but never have they done so in response to a defect report questioning
whether or not something is defined.
Secondly, committee resolutions never take the form of advice for implementors
about how to deal with the issue raised by the DR - they either express what the
committee believes is the correct interpretation of the words as they are
currently written (in which cases implementors who wish to conform to the
standard should modify their implementation, if necessary, to match that
interpretation), or they recommend that the words be changed (in which case
implementors have the option of anticipating the correction by implementing it
as an extension now, or waiting until the correction has been approved before
implementing it as a conformance requirement).

You also mentioned "The Standard is clearly defective in some fashion, but
without knowing the intended meaning it's impossible to know in which way it's
defective or what should be done to "fix" it." as a possibility. It's not
appropriate to mark a DR as resolved if there's still that much uncertainty
about the issue. The committee should know what it intended - if it doesn't,
then resolving the DR would be premature.

[toc] | [prev] | [next] | [standalone]

Page 2 of 4 — ← Prev page 1 [2] 3 4  Next page →

Back to top | Article view | comp.lang.c

csiph-web