Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.std.c > #1833 > unrolled thread

Tracking and Reporting Effective Type

Back to article view | Back to comp.std.c

Contents

  Tracking and Reporting Effective Type Shao Miller <sha0.miller@gmail.com> - 2013-01-11 07:09 -0500
    Re: Tracking and Reporting Effective Type Wojtek Lerch <wojtek_l@yahoo.ca> - 2013-01-13 23:48 -0500
      Re: Tracking and Reporting Effective Type Shao Miller <sha0.miller@gmail.com> - 2013-01-14 01:03 -0500
        Re: Tracking and Reporting Effective Type Wojtek Lerch <wojtek_l@yahoo.ca> - 2013-01-15 00:28 -0500
          Re: Tracking and Reporting Effective Type Shao Miller <sha0.miller@gmail.com> - 2013-01-15 05:39 -0500
            Re: Tracking and Reporting Effective Type Wojtek Lerch <wojtek_l@yahoo.ca> - 2013-01-17 00:32 -0500
              Re: Tracking and Reporting Effective Type Wojtek Lerch <wojtek_l@yahoo.ca> - 2013-01-17 01:23 -0500
              Re: Tracking and Reporting Effective Type Shao Miller <sha0.miller@gmail.com> - 2013-01-17 22:47 -0500
                Re: Tracking and Reporting Effective Type Wojtek Lerch <wojtek_l@yahoo.ca> - 2013-01-18 16:14 -0500
                  Re: Tracking and Reporting Effective Type Wojtek Lerch <wojtek_l@yahoo.ca> - 2013-01-18 16:21 -0500

#1833 — Tracking and Reporting Effective Type

Good day to All.

   "The effective type of an object for an access to its stored value is 
the declared type of the object, if any.87) If a value is stored into an 
object having no declared type through an lvalue having a type that is 
not a character type, then the type of the lvalue becomes the effective 
type of the object for that access and for subsequent accesses that do 
not modify the stored value. If a value is copied into an object having 
no declared type using memcpy or memmove, or is copied as an array of 
character type, then the effective type of the modified object for that 
access and for subsequent accesses that do not modify the value is the 
effective type of the object from which the value is copied, if it has 
one. For all other accesses to an object having no declared type, the 
effective type of the object is simply the type of the lvalue used for 
the access."

I'm planning on writing implementation code that tracks and reports 
effective type, whenever possible, as a "bonus" feature.  I've run into 
a challenge:

   #include <stdlib.h>

   #define obj_sz (sizeof (float))

   float test(float f) {
       void * ptr;
       unsigned char * src;
       unsigned char * trgt;
       size_t i;
       float new_float;

       ptr = malloc(obj_sz);
       if (!ptr)
         return 0.0f;

       src = (void *) &f;
       trgt = ptr;
       for (i = 0; i < obj_sz; ++i) {
           if (src[i] + 0 == src[i])
             trgt[i] = src[i] + 0;
             else
             return 0.0f;
         }

       /*
        * What is the effective type of the storage
        * pointed-to by 'ptr'?  Is the following
        * access via indirection a violation of 6.5p7?
        */
       new_float = *(float *) ptr;

       free(ptr);
       return new_float;
     }

   int main(void) {
       test(3.14f);
       return EXIT_SUCCESS;
     }

Intuitively, I'd like to answer "float" and "no" to the two questions in 
the comment, but I'm not 100% sure.  Does the population of the 
allocated storage qualify for "copied" in the Standard paragraph, above?

-- 
- Shao Miller
--
"Thank you for the kind words; those are the kind of words I like to hear.

Cheerily,"  -- Richard Harter

[toc] | [next] | [standalone]

#1834

On 11/01/2013 7:09 AM, Shao Miller wrote:
> I'm planning on writing implementation code that tracks and reports
> effective type, whenever possible, as a "bonus" feature.  I've run into
> a challenge:

I have another challenge for you:

union u {
   struct sl { int16_t s; int32_t l; } sl;
   struct ls { int32_t l; int16_t s; } ls;
   uint32_t u;
   float f;
};

int main(void) {
   void *p;
   if ( sizeof(union u) == 6 // no padding
     && ( m = malloc(6) ) != NULL ) {
     union u u;
     memset( u, 0, 6 );
     memcpy( p, (void*)&u.sl, 6 );
     ...
   }
}

At the "...", does the 6-byte object that p points to have an effective 
type?  Is it union u, struct sl, or perhaps struct ls?  Would the answer 
change if the second argument of the memcpy were (void*)&u instead of 
(void*)&u.sl?  Or (void*)&u.f?

Does the four-byte sub-object at the beginning of "u" have an effective 
type?  Is it int32_t, uint32_t, or perhaps (if sizeof(float)==4) float? 
  Or somehow all of the above?  What about the four-byte object starting 
at the byte that p points to?  And the two-byte object starting at the 
same byte?

[toc] | [prev] | [next] | [standalone]

#1835

On 1/13/2013 23:48, Wojtek Lerch wrote:
> On 11/01/2013 7:09 AM, Shao Miller wrote:
>> I'm planning on writing implementation code that tracks and reports
>> effective type, whenever possible, as a "bonus" feature.  I've run into
>> a challenge:
>
> I have another challenge for you:
>
> union u {
>    struct sl { int16_t s; int32_t l; } sl;
>    struct ls { int32_t l; int16_t s; } ls;
>    uint32_t u;
>    float f;
> };
>
> int main(void) {
>    void *p;
>    if ( sizeof(union u) == 6 // no padding
>      && ( m = malloc(6) ) != NULL ) {
>      union u u;
>      memset( u, 0, 6 );
>      memcpy( p, (void*)&u.sl, 6 );
>      ...
>    }
> }
>
> At the "...", does the 6-byte object that p points to have an effective
> type?  Is it union u, struct sl, or perhaps struct ls?  Would the answer
> change if the second argument of the memcpy were (void*)&u instead of
> (void*)&u.sl?  Or (void*)&u.f?
>

Assuming you meant 'memset( &u, 0, 6 );', I think I saw something like 
this in Defect Report #219.

I think that the effective type is indeed 'struct sl' due to the 
declared type of the only lvalue in sight in the 'memcpy': 'u.sl'.  I do 
think the answer would change.

> Does the four-byte sub-object at the beginning of "u" have an effective
> type?

I'd say each of the 4 bytes would be 'unsigned char', due to the 
'memset' copying as an array of 'unsigned char', but the effective type 
of the whole would be 'union u', due to the lvalue '&u' in the 'memset', 
and the fact that all bytes were populated.

> Is it int32_t, uint32_t, or perhaps (if sizeof(float)==4) float?

If I understand a point from DR #219, the bytes "can be treated" as 
these others, but could be a trap representation for a 'float'.

> Or somehow all of the above?

I don't think so, for tracking and reporting purposes.

> What about the four-byte object starting
> at the byte that p points to?

I'd say no.  The 4 bytes so happen to be correctly aligned and contain a 
correct representation for some of the other members, though.

But if "provenance" was tracked, I'd track it to 'u.sl'; there'd be an 
'int16_t' and an 'int32_t'.  So I might trap '((union u *) p)->ls.l', 
for example, but not '((union u *) p)->sl.s' or '((struct sl *) p)->l'.

> And the two-byte object starting at the
> same byte?
>

I'd say yes, as that'd have the effective type of 'u.sl.s'; 'int16_t'. 
What do you think?

-- 
- Shao Miller
--
"Thank you for the kind words; those are the kind of words I like to hear.

Cheerily,"  -- Richard Harter

[toc] | [prev] | [next] | [standalone]

#1836

On 14/01/2013 1:03 AM, Shao Miller wrote:
> On 1/13/2013 23:48, Wojtek Lerch wrote:
>>
>> union u {
>>    struct sl { int16_t s; int32_t l; } sl;
>>    struct ls { int32_t l; int16_t s; } ls;
>>    uint32_t u;
>>    float f;
>> };
>>
>> int main(void) {
>>    void *p;
>>    if ( sizeof(union u) == 6 // no padding
>>      && ( p = malloc(6) ) != NULL ) {
>>      union u u;
>>      memset( u, 0, 6 );
>>      memcpy( p, (void*)&u.sl, 6 );
>>      ...
>>    }
>> }
>>
>> At the "...", does the 6-byte object that p points to have an effective
>> type?  Is it union u, struct sl, or perhaps struct ls?  Would the answer
>> change if the second argument of the memcpy were (void*)&u instead of
>> (void*)&u.sl?  Or (void*)&u.f?
>
> Assuming you meant 'memset( &u, 0, 6 );',

Indeed I did, thank you.

>   I think I saw something like this in Defect Report #219.

That's a good find, thanks.  But there seems to be an inconsistency 
between the "committee Discussion" (which I believe is non-normative) 
and the "Committee Response" (which is normative) about the treatment of 
character types.

The DR clarifies one important thing though: it's not only objects named 
in declarations that have a "declared type", but also their subobjects, 
such as members or elements.  But that's the case, then it's still not 
clear to me what happens when a compound object and its element or 
member, or two members of a union, are exactly the same object.  It 
would make sense to me that the object might have several declared types 
at the same time, but the standard's discussion of effective types 
doesn't create the impression that that was considered a possibility, 
does it?

> I think that the effective type is indeed 'struct sl' due to the
> declared type of the only lvalue in sight in the 'memcpy': 'u.sl'.  I do
> think the answer would change.

But that lvalue is never used to access the object -- its only role is 
to produce a pointer to the first byte of the object.  The pointers 
(char*)&u and (char*)&u.sl point to the same byte, right?

Imagine that u is a union of two different structures x and y, and it 
happens that u.x.r and u.y.s have the same offsets and sizes.  Would you 
claim that

   memcpy( p, (char*)&u + offsetof(struct x, r), sizeof(u.x.r) );

and

   memcpy( p, (char*)&u + offsetof(struct y, s), sizeof(u.y.s) );

copy different effective types into p, even if they pass exactly the 
same values to memcpy?

Remember that an object is just a region of storage; when we talk about 
its type, that's just a short way of talking about types of lvalues, or 
declared types, or effective types.

Back to my original example, what if I used &u.sl.s in the memcpy?  Do 
you think the effective type of my allocated object would be int16_t? 
Or an array of three int16_ts?  Something else?  Or do you think such a 
memcpy() would be invalid?

>> Does the four-byte sub-object at the beginning of "u" have an effective
>> type?
>
> I'd say each of the 4 bytes would be 'unsigned char', due to the
> 'memset' copying as an array of 'unsigned char', but the effective type
> of the whole would be 'union u', due to the lvalue '&u' in the 'memset',
> and the fact that all bytes were populated.

You explained four one-byte objects and one six-byte object.  But what 
about my four-byte object that can (I hope) be accessed using such 
lvalues as u.ls.l, u.u, and u.f?

>> Is it int32_t, uint32_t, or perhaps (if sizeof(float)==4) float?
>
> If I understand a point from DR #219, the bytes "can be treated" as
> these others,

Is the "can be treated" supposed to mean that an object can have more 
than one effective type at the same time?  I wouldn't mind that, but I 
would prefer some normative text to say it a little more clearly.

>  but could be a trap representation for a 'float'.

My concern (for now) is not about the contents of the object but about 
its effective type.  In particular, the "sticky" kind of effective type 
that can conflict with read accesses using an lvalue with the wrong 
type, and gets propagated by memcpying to other typeless objects.  In 
other words, whether one of the standard's statements about the 
effective type "for subsequent accesses that do not modify the value" 
apply to it or not.

>> Or somehow all of the above?
>
> I don't think so, for tracking and reporting purposes.
>
>> What about the four-byte object starting
>> at the byte that p points to?
>
> I'd say no.  The 4 bytes so happen to be correctly aligned and contain a
> correct representation for some of the other members, though.

Sure, but do they have an effective type?  The definition of the 
effective type does not refer to alignment or contents -- it seems to be 
based only on the types of lvalues used for accesses, and effective 
types of objects that contents are copied from.

Hey, here's another one:

   #define N sizeof(float)
   float x = 1.0;
   char y[ N ];
   float *p = memcpy( malloc(N), memcpy( y, &x, N ), N );
   // Does *p have a "sticky" effective type now?  Is it char or float?
   float z = *p; // Is this valid?

> But if "provenance" was tracked, I'd track it to 'u.sl'; there'd be an
> 'int16_t' and an 'int32_t'.  So I might trap '((union u *) p)->ls.l',
> for example, but not '((union u *) p)->sl.s' or '((struct sl *) p)->l'.

What about the examples using offsetof and sizeof?  Are we tracking the 
provenance of integers now?

>> And the two-byte object starting at the
>> same byte?
>
> I'd say yes, as that'd have the effective type of 'u.sl.s'; 'int16_t'.
> What do you think?

I don't know.  I still find the standard's explanation of effective 
types hopelessly unclear.

[toc] | [prev] | [next] | [standalone]

#1837

On 1/15/2013 00:28, Wojtek Lerch wrote:
> On 14/01/2013 1:03 AM, Shao Miller wrote:
>> On 1/13/2013 23:48, Wojtek Lerch wrote:
>>>
>>> union u {
>>>    struct sl { int16_t s; int32_t l; } sl;
>>>    struct ls { int32_t l; int16_t s; } ls;
>>>    uint32_t u;
>>>    float f;
>>> };
>>>
>>> int main(void) {
>>>    void *p;
>>>    if ( sizeof(union u) == 6 // no padding
>>>      && ( p = malloc(6) ) != NULL ) {
>>>      union u u;
>>>      memset( u, 0, 6 );
>>>      memcpy( p, (void*)&u.sl, 6 );
>>>      ...
>>>    }
>>> }
>>>
>>> At the "...", does the 6-byte object that p points to have an effective
>>> type?  Is it union u, struct sl, or perhaps struct ls?  Would the answer
>>> change if the second argument of the memcpy were (void*)&u instead of
>>> (void*)&u.sl?  Or (void*)&u.f?
>>
>> Assuming you meant 'memset( &u, 0, 6 );',
>
> Indeed I did, thank you.
>
>>   I think I saw something like this in Defect Report #219.
>
> That's a good find, thanks.  But there seems to be an inconsistency
> between the "committee Discussion" (which I believe is non-normative)
> and the "Committee Response" (which is normative) about the treatment of
> character types.
>

Which part/parts is/are inconsistent?

> The DR clarifies one important thing though: it's not only objects named
> in declarations that have a "declared type", but also their subobjects,
> such as members or elements.

Yes, I also thought that was a note-worthy point of clarification.

> But that's the case, then it's still not
> clear to me what happens when a compound object

(I assume you mean "aggregate")

> and its element or
> member, or two members of a union, are exactly the same object.  It
> would make sense to me that the object might have several declared types
> at the same time, but the standard's discussion of effective types
> doesn't create the impression that that was considered a possibility,
> does it?
>

I don't think I agree.  When there's a declared type, I think the 
object's effective type can only be that declared type.  I certainly 
hope that this is the case, else tracking and reporting could be more 
complicated!

If you are talking about how 'int ia[1];' declares an object with 
(sizeof (int) * CHAR_BIT) bits and its declared type seems to be both 
'int[1]' and 'int', I don't think that's the case.  I think the 
effective type here must match the declared type, 'int[1]'.  The pointer 
expression 'ia' (after conversion from array type) no longer refers to 
the same object, but refers to its first element.  The first element has 
some properties in common with with its containing object, but not all 
of them, including its effective type.  (I would say.)

By using the declared type as the effective type, I don't think there 
can be multiple effective types for the object.  I think that we are 
able to discuss sub-objects separately from a containing object, even if 
there is exact overlap.

If we copied from '&ia' ('int(*)[1]') to allocated storage, I'd say the 
effective type for the modified storage becomes 'int[1]'.  If we have an 
'int *' and point to this storage with it, we can argue about whether we 
are pointing to the array or to its first element.  The obvious answer 
is that we are pointing to its first element, because our pointer's 
referenced type matches the element type.

>> I think that the effective type is indeed 'struct sl' due to the
>> declared type of the only lvalue in sight in the 'memcpy': 'u.sl'.  I do
>> think the answer would change.
>
> But that lvalue is never used to access the object -- its only role is
> to produce a pointer to the first byte of the object.  The pointers
> (char*)&u and (char*)&u.sl point to the same byte, right?
>

I agree that the lvalue 'u.sl' is not used to access the object, but it 
still designates that sub-object.  The value of '&u.sl' thus addresses 
that sub-object and 'memcpy' is thus working with that sub-object, in my 
view.

I agree that those two 'char *' pointers address the same byte, but only 
as a consequence of the two objects occupying the same location...  So I 
don't quite agree that its only role is to produce a pointer to the 
first byte; I think the pointer has more properties than which byte it 
addresses.

> Imagine that u is a union of two different structures x and y, and it
> happens that u.x.r and u.y.s have the same offsets and sizes.  Would you
> claim that
>
>    memcpy( p, (char*)&u + offsetof(struct x, r), sizeof(u.x.r) );
>
> and
>
>    memcpy( p, (char*)&u + offsetof(struct y, s), sizeof(u.y.s) );
>
> copy different effective types into p, even if they pass exactly the
> same values to memcpy?
>

This case depends on the offset and number of bytes copied, to me.

'(char *) &u' addresses the first byte of 'u'.  If the offset is 0 and 
the number of bytes equals 'sizeof u', then I'd say the effective type 
of the modified storage is the union type.

For any other case of offset or number of bytes, I'd ask "which object 
is being copied?"  The answer might depend on the last-stored value, for 
union involvement.  Regardless of what the answer depends on, if there 
are multiple answers, then all we know for certain is that a 
'char[num_of_bytes]' is being copied.  I think that answer could be a 
default, in the case where there is no single better answer.

> Remember that an object is just a region of storage; when we talk about
> its type, that's just a short way of talking about types of lvalues, or
> declared types, or effective types.
>

"is just": I think this is consistent with the initial definition of 
'object' under section 3, but I think the actual definition accumulates 
some more from other parts of the Standard.  The discussion of pointer 
arithmetic, for example, distinguishes between an "array object" and its 
"array elements", but these different objects can exactly overlap.  But 
in general, I agree with this.

> Back to my original example, what if I used &u.sl.s in the memcpy?

That pointer addresses 'u.sl.s'.  The number of bytes to copy is more 
than that, however.  I think this yields undefined behaviour, just as:

   *( ((char *) ((void *) &u.sl.s)) + sizeof u.sl.s )

would.  (Bounds-checking.)

> Do
> you think the effective type of my allocated object would be int16_t? Or
> an array of three int16_ts?  Something else?  Or do you think such a
> memcpy() would be invalid?
>

The last one. :)

>>> Does the four-byte sub-object at the beginning of "u" have an effective
>>> type?
>>
>> I'd say each of the 4 bytes would be 'unsigned char', due to the
>> 'memset' copying as an array of 'unsigned char', but the effective type
>> of the whole would be 'union u', due to the lvalue '&u' in the 'memset',
>> and the fact that all bytes were populated.
>
> You explained four one-byte objects and one six-byte object.  But what
> about my four-byte object that can (I hope) be accessed using such
> lvalues as u.ls.l, u.u, and u.f?
>

In order to determine the effective type, I'd ask "which object(s) do 
the first four bytes refer to?"  For a union, I think the answer would 
first depend on the last-stored value.  The last-stored value of the 
union was not set via any of the members, so that does not resolve the 
answer.  After that, there are multiple answers, but none of those 
scores as highly as the type of the value that was stored, making the 
answer "4 bytes of an 'unsigned char[6]'".

However, the 'memset' wasn't working with only the first 4 bytes.  The 
'memset' did more than that.  It involved '&u' and involved all of the 
bytes, so it produced a union value.  I believe that union value could 
be used with another union of the same type with 'union u u2 = u;' or 
'union u u2; u2 = u;', for example.

'u.ls.l', 'u.u' and 'u.f' designate members which do not have a 
last-stored value, and no union member was used for the last-stored 
value, so I'd say that evaluating any of them yields undefined 
behaviour.  (!!)

Because there is no padding for meta-data in your example, you could 
circumvent the undefined behaviour with any kind of code that makes it 
impossible to prove any claim about such meta-data (such as whether or 
not a member was used to store a value), yielding defined behaviour.  In 
practice, I'd guess that for most implementations, the call to 'memset' 
itself would be enough that the implementation drops any special 
knowledge about 'u' and its stored value.  In that case, type-punning 
would happen without a 6.5p7 violation, but could still be a trap if 
there's a scalar trap representation.

>>> Is it int32_t, uint32_t, or perhaps (if sizeof(float)==4) float?
>>
>> If I understand a point from DR #219, the bytes "can be treated" as
>> these others,
>
> Is the "can be treated" supposed to mean that an object can have more
> than one effective type at the same time?  I wouldn't mind that, but I
> would prefer some normative text to say it a little more clearly.
>

Strongly agreed; a little more precision could help, there.

>>  but could be a trap representation for a 'float'.
>
> My concern (for now) is not about the contents of the object but about
> its effective type.  In particular, the "sticky" kind of effective type
> that can conflict with read accesses using an lvalue with the wrong
> type, and gets propagated by memcpying to other typeless objects.  In
> other words, whether one of the standard's statements about the
> effective type "for subsequent accesses that do not modify the value"
> apply to it or not.
>

We share the same concern.  Some of the considerations of the Standard 
involve subjects which invoke undefined behaviour, but some of those 
subjects appear to be hard to prove, under the right circumstances. 
"Effective type" strikes me as one such subject.  My interest is in an 
implementation which attempts to trap _all_ code which yields undefined 
behaviour and report it.

>>> Or somehow all of the above?
>>
>> I don't think so, for tracking and reporting purposes.
>>
>>> What about the four-byte object starting
>>> at the byte that p points to?
>>
>> I'd say no.  The 4 bytes so happen to be correctly aligned and contain a
>> correct representation for some of the other members, though.
>
> Sure, but do they have an effective type?  The definition of the
> effective type does not refer to alignment or contents -- it seems to be
> based only on the types of lvalues used for accesses, and effective
> types of objects that contents are copied from.
>

It sure does seem that way.

> Hey, here's another one:
>
>    #define N sizeof(float)
>    float x = 1.0;
>    char y[ N ];
>    float *p = memcpy( malloc(N), memcpy( y, &x, N ), N );
>    // Does *p have a "sticky" effective type now?  Is it char or float?
>    float z = *p; // Is this valid?
>

I would indeed track and report the effective type of the modified 
storage pointed to by 'p' as having effective type 'float'.  An array of 
character type is always the case, so it's an uninteresting case (to 
me), if there's more that can be proven about the provenance of the 
"stored value".

>> But if "provenance" was tracked, I'd track it to 'u.sl'; there'd be an
>> 'int16_t' and an 'int32_t'.  So I might trap '((union u *) p)->ls.l',
>> for example, but not '((union u *) p)->sl.s' or '((struct sl *) p)->l'.
>
> What about the examples using offsetof and sizeof?  Are we tracking the
> provenance of integers now?
>

I definitely wouldn't. :)  I did mention how I might use an offset as 
part of consideration, above.

>>> And the two-byte object starting at the
>>> same byte?
>>
>> I'd say yes, as that'd have the effective type of 'u.sl.s'; 'int16_t'.
>> What do you think?
>
> I don't know.  I still find the standard's explanation of effective
> types hopelessly unclear.
>

   "The intent of this list is to specify those circumstances in which 
an object may or may not be aliased."

(Not offered as an explanatory response to your last line, but just as 
associated, additional information, which you're likely to have already 
read.)

-- 
- Shao Miller
--
"Thank you for the kind words; those are the kind of words I like to hear.

Cheerily,"  -- Richard Harter

[toc] | [prev] | [next] | [standalone]

#1847

On 15/01/2013 5:39 AM, Shao Miller wrote:
> On 1/15/2013 00:28, Wojtek Lerch wrote:
>> On 14/01/2013 1:03 AM, Shao Miller wrote:
>>> On 1/13/2013 23:48, Wojtek Lerch wrote:
>>>>
>>>> union u {
>>>>    struct sl { int16_t s; int32_t l; } sl;
>>>>    struct ls { int32_t l; int16_t s; } ls;
>>>>    uint32_t u;
>>>>    float f;
>>>> };
>>>>
>>>> int main(void) {
>>>>    void *p;
>>>>    if ( sizeof(union u) == 6 // no padding
>>>>      && ( p = malloc(6) ) != NULL ) {
>>>>      union u u;
>>>>      memset( u, 0, 6 );
>>>>      memcpy( p, (void*)&u.sl, 6 );
>>>>      ...
>>>>    }
>>>> }

...

>>>   I think I saw something like this in Defect Report #219.
>>
>> That's a good find, thanks.  But there seems to be an inconsistency
>> between the "committee Discussion" (which I believe is non-normative)
>> and the "Committee Response" (which is normative) about the treatment of
>> character types.
>
> Which part/parts is/are inconsistent?

Discussion:

   6. memcpy (p1, (char *)&s + offsetof (s, i), len1);

   In lines 6 and 7, the effective type of the source arguments
   to memcpy is an array of char

   The effective type for p1 and p2 in lines 6 and 7 is: array of char.

Response:

   memcpy (p1, (char *)&s + offsetof (s, i), len1); copies all
   of the bytes of s.i to an alignment suitable for an object
   of type int. The effective type of the resulting copy can
   be treated as having effective type int.

(Also, I have to say that the "can be treated" part feels out of place 
in what is supposed to be a normative response.  Is the Committee really 
telling us that accessing the object as an int "can be treated" as not 
having undefined behaviour?)

>> The DR clarifies one important thing though: it's not only objects named
>> in declarations that have a "declared type", but also their subobjects,
>> such as members or elements.  But if that's the case, then it's still not
>> clear to me what happens when a compound object
>
> (I assume you mean "aggregate")

Yes, thank you again.

>> and its element or
>> member, or two members of a union, are exactly the same object.  It
>> would make sense to me that the object might have several declared types
>> at the same time, but the standard's discussion of effective types
>> doesn't create the impression that that was considered a possibility,
>> does it?
>
> I don't think I agree.  When there's a declared type, I think the
> object's effective type can only be that declared type.

That's what the standard says; but when an object has three declared 
types at the same time, how many effective types does it have?  In my 
example, the expressions "u", "u.sl", and "u.ls" designate the same 
6-byte object.  Does that object have just one declared type, or does it 
have three?  As far as I can tell we don't have a clear definition of 
"declared type" that would cover members and elements of objects 
declared with a union or aggregate type.

>    I certainly
> hope that this is the case, else tracking and reporting could be more
> complicated!

I think the priority should be to have a clear definition of what you 
want to track, before worrying about how complicated it is to track it.

> If you are talking about how 'int ia[1];' declares an object with
> (sizeof (int) * CHAR_BIT) bits and its declared type seems to be both
> 'int[1]' and 'int', I don't think that's the case.  I think the
> effective type here must match the declared type, 'int[1]'.  The pointer
> expression 'ia' (after conversion from array type) no longer refers to
> the same object, but refers to its first element.

The array and the element are the same object.

3.15 "object: region of data storage in the execution environment, the 
contents of which can represent values.  NOTE When referenced, an object 
may be interpreted as having a particular type; see 6.3.2.1."

6.3.2.1#1 "When an object is said to have a particular type, the type is 
specified by the lvalue used to designate the object."

An object can have a declared type, an effective type, and can be 
accessed using lvalues that have various types; but when the same region 
of memory is designated by two lvalues that have different types, it's 
still one object, not two.

Otherwise, the whole concept of effective types falls apart.  If two 
accesses to the same region of memory count as accesses to two distinct 
objects, then the effective type of one object doesn't affect how the 
other object can be accessed, does it?  Unless there are clear rules 
about when a region of memory counts as the same object for the purpose 
of aliasing, the rules of effective types say nothing meaningful about 
aliasing.

I would prefer to go one step in the opposite direction, and insist that 
the standard should say something about objects that partially overlap. 
  When an assignment to a subobject of an allocated object gives it a 
new effective type, shouldn't it also wipe out any effective types that 
overlapping objects may have?

...
> I agree that those two 'char *' pointers address the same byte, but only
> as a consequence of the two objects occupying the same location...  So I
> don't quite agree that its only role is to produce a pointer to the
> first byte; I think the pointer has more properties than which byte it
> addresses.

It seems that it does; trouble is, the standard doesn't say much about 
what those extra properties are and what happens to them when pointers 
are converted.  And the few clues that it does offer don't seem very 
consistent.

For instance, the description of pointer addition and subtraction 
strongly implies that every pointer to an object has a range of integers 
associated with it that can be added to it without causing undefined 
behaviour.  On the other hand, the discussion of pointer conversions 
seems to think it's sufficient to guararantee that a resulting pointer 
compares equal to some other pointer, without saying anything about the 
associated range of integers.  But without knowing that range, there 
isn't much we can safely do with a pointer -- in particular, 
dereferencing, incrementing, or decrementing a pointer may invoke 
undefined behaviour, even if we know that the pointer compares equal to 
some other pointer that can be safely dereferenced, incremented, or 
decremented.

The concept of pointer provenance, insinuated by DR260, is hopelessly 
underspecified too: we're supposed to rely on it to somehow figure out 
what operations have a defined behaviour and which ones don't, from a 
few vague sentences that talk about what implementations are permitted 
to track, without much of an explanation of how those permissions affect 
the guarantees given to programs.

>> Imagine that u is a union of two different structures x and y, and it
>> happens that u.x.r and u.y.s have the same offsets and sizes.  Would you
>> claim that
>>
>>    memcpy( p, (char*)&u + offsetof(struct x, r), sizeof(u.x.r) );
>>
>> and
>>
>>    memcpy( p, (char*)&u + offsetof(struct y, s), sizeof(u.y.s) );
>>
>> copy different effective types into p, even if they pass exactly the
>> same values to memcpy?
>
> This case depends on the offset and number of bytes copied, to me.
>
> '(char *) &u' addresses the first byte of 'u'.  If the offset is 0 and
> the number of bytes equals 'sizeof u', then I'd say the effective type
> of the modified storage is the union type.

OK.  The case where the offset is nonzero is more interesting.

> For any other case of offset or number of bytes, I'd ask "which object
> is being copied?"  The answer might depend on the last-stored value, for
> union involvement.

Only if the object doesn't have a declared type.  If it does have a 
declared type, then that's its effective type too, and storing values 
doesn't affect it, right?

>   Regardless of what the answer depends on, if there
> are multiple answers, then all we know for certain is that a
> 'char[num_of_bytes]' is being copied.  I think that answer could be a
> default, in the case where there is no single better answer.

Well I think it's the standard's job to give us a better answer.

>> Remember that an object is just a region of storage; when we talk about
>> its type, that's just a short way of talking about types of lvalues, or
>> declared types, or effective types.
>
> "is just": I think this is consistent with the initial definition of
> 'object' under section 3, but I think the actual definition accumulates
> some more from other parts of the Standard.  The discussion of pointer
> arithmetic, for example, distinguishes between an "array object" and its
> "array elements", but these different objects can exactly overlap.

Sure; but we're never told whether this "arrayness" is based on some 
properties of the object, such as its declared or effective type, or on 
the lvalue expression that the pointer originated from.  I tend to think 
it's the latter, not only because that seems consistent with the need to 
talk about the provenance of pointers, but also because it applies to 
math on pointers to memory that has just been allocated and has not been 
accessed yet and therefore doesn't have any declared or effective type.

>> Back to my original example, what if I used &u.sl.s in the memcpy?
>
> That pointer addresses 'u.sl.s'.  The number of bytes to copy is more
> than that, however.  I think this yields undefined behaviour, just as:
>
>    *( ((char *) ((void *) &u.sl.s)) + sizeof u.sl.s )
>
> would.  (Bounds-checking.)

OK, I can buy that.  :)


>>>> Does the four-byte sub-object at the beginning of "u" have an effective
>>>> type?
>>  my four-byte object that can (I hope) be accessed using such
>> lvalues as u.ls.l, u.u, and u.f?
>
> In order to determine the effective type, I'd ask "which object(s) do
> the first four bytes refer to?"  For a union, I think the answer would
> first depend on the last-stored value.

Only if there's no declared type.  Is there?

>   The last-stored value of the
> union was not set via any of the members, so that does not resolve the
> answer.  After that, there are multiple answers, but none of those
> scores as highly as the type of the value that was stored, making the
> answer "4 bytes of an 'unsigned char[6]'".

Your scoring of multiple answers is not based on anything that the 
standard actually says about effective types, is it?

> However, the 'memset' wasn't working with only the first 4 bytes.  The
> 'memset' did more than that.  It involved '&u' and involved all of the
> bytes, so it produced a union value.  I believe that union value could
> be used with another union of the same type with 'union u u2 = u;' or
> 'union u u2; u2 = u;', for example.
>
> 'u.ls.l', 'u.u' and 'u.f' designate members which do not have a
> last-stored value, and no union member was used for the last-stored
> value, so I'd say that evaluating any of them yields undefined
> behaviour.  (!!)

My first guess would be that accessing any if of the integers using its 
declared type should be safe, because the memset has put a valid 
representation of zero in it.

> Because there is no padding for meta-data in your example, you could
> circumvent the undefined behaviour with any kind of code that makes it
> impossible to prove any claim about such meta-data (such as whether or
> not a member was used to store a value), yielding defined behaviour.

I'm not sure if I understand what you're saying here, but just because 
there's no padding to store extra information at runtime doesn't mean 
that the implementation can't track that information at compile time, or 
maybe sometimes outside of the object.  Even if you make it hard for 
implementations to figure out that you're breaking the rules, that does 
not make the behaviour defined.

...
>> My concern (for now) is not about the contents of the object but about
>> its effective type.  In particular, the "sticky" kind of effective type
>> that can conflict with read accesses using an lvalue with the wrong
>> type, and gets propagated by memcpying to other typeless objects.  In
>> other words, whether one of the standard's statements about the
>> effective type "for subsequent accesses that do not modify the value"
>> apply to it or not.
>
> We share the same concern.  Some of the considerations of the Standard
> involve subjects which invoke undefined behaviour, but some of those
> subjects appear to be hard to prove, under the right circumstances.
> "Effective type" strikes me as one such subject.  My interest is in an
> implementation which attempts to trap _all_ code which yields undefined
> behaviour and report it.

Well good luck with that.  I'm afraid the best you can possibly do is 
detect and report code that has undefined behaviour *according to your 
interpretation of the standard*.

...
>> Hey, here's another one:
>>
>>    #define N sizeof(float)
>>    float x = 1.0;
>>    char y[ N ];
>>    float *p = memcpy( malloc(N), memcpy( y, &x, N ), N );
>>    // Does *p have a "sticky" effective type now?  Is it char or float?
>>    float z = *p; // Is this valid?
>>
>
> I would indeed track and report the effective type of the modified
> storage pointed to by 'p' as having effective type 'float'.  An array of
> character type is always the case, so it's an uninteresting case (to
> me), if there's more that can be proven about the provenance of the
> "stored value".

Actually, since y has a declared type, shouldn't it make any traces of 
floatness disappear?

I think what I really had in mind back then was an unaligned object 
without a declared type:

   char *y = 1 + malloc(N+1);

[toc] | [prev] | [next] | [standalone]

#1848

On 17/01/2013 12:32 AM, Wojtek Lerch wrote:
> I think what I really had in mind back then was an unaligned object
> without a declared type:
>
>    char *y = 1 + malloc(N+1);

Oops!  Add a cast to char* in front of the malloc.

[toc] | [prev] | [next] | [standalone]

#1851

On 1/17/2013 00:32, Wojtek Lerch wrote:
> On 15/01/2013 5:39 AM, Shao Miller wrote:
>> On 1/15/2013 00:28, Wojtek Lerch wrote:
>>> On 14/01/2013 1:03 AM, Shao Miller wrote:
>>>> On 1/13/2013 23:48, Wojtek Lerch wrote:
>>>>>
>>>>> union u {
>>>>>    struct sl { int16_t s; int32_t l; } sl;
>>>>>    struct ls { int32_t l; int16_t s; } ls;
>>>>>    uint32_t u;
>>>>>    float f;
>>>>> };
>>>>>
>>>>> int main(void) {
>>>>>    void *p;
>>>>>    if ( sizeof(union u) == 6 // no padding
>>>>>      && ( p = malloc(6) ) != NULL ) {
>>>>>      union u u;
>>>>>      memset( u, 0, 6 );
>>>>>      memcpy( p, (void*)&u.sl, 6 );
>>>>>      ...
>>>>>    }
>>>>> }
>
> ...
>
>>>>   I think I saw something like this in Defect Report #219.
>>>
>>> That's a good find, thanks.  But there seems to be an inconsistency
>>> between the "committee Discussion" (which I believe is non-normative)
>>> and the "Committee Response" (which is normative) about the treatment of
>>> character types.
>>
>> Which part/parts is/are inconsistent?
>
> Discussion:
>
>    6. memcpy (p1, (char *)&s + offsetof (s, i), len1);
>
>    In lines 6 and 7, the effective type of the source arguments
>    to memcpy is an array of char
>
>    The effective type for p1 and p2 in lines 6 and 7 is: array of char.
>
> Response:
>
>    memcpy (p1, (char *)&s + offsetof (s, i), len1); copies all
>    of the bytes of s.i to an alignment suitable for an object
>    of type int. The effective type of the resulting copy can
>    be treated as having effective type int.
>
> (Also, I have to say that the "can be treated" part feels out of place
> in what is supposed to be a normative response.  Is the Committee really
> telling us that accessing the object as an int "can be treated" as not
> having undefined behaviour?)
>

Ah, ok.  Thank you for clarifying.

>>> The DR clarifies one important thing though: it's not only objects named
>>> in declarations that have a "declared type", but also their subobjects,
>>> such as members or elements.  But if that's the case, then it's still
>>> not
>>> clear to me what happens when a[n aggregate] object
>>> and its element or
>>> member, or two members of a union, are exactly the same object.  It
>>> would make sense to me that the object might have several declared types
>>> at the same time, but the standard's discussion of effective types
>>> doesn't create the impression that that was considered a possibility,
>>> does it?
>>
>> I don't think I agree.  When there's a declared type, I think the
>> object's effective type can only be that declared type.
>
> That's what the standard says; but when an object has three declared
> types at the same time, how many effective types does it have?  In my
> example, the expressions "u", "u.sl", and "u.ls" designate the same
> 6-byte object.  Does that object have just one declared type, or does it
> have three?  As far as I can tell we don't have a clear definition of
> "declared type" that would cover members and elements of objects
> declared with a union or aggregate type.
>

I see your point, if the object is only considered to be the region of 
data storage and if its entirety is accessible via different lvalues 
with different declared types.

>>    I certainly
>> hope that this is the case, else tracking and reporting could be more
>> complicated!
>
> I think the priority should be to have a clear definition of what you
> want to track, before worrying about how complicated it is to track it.
>

You're probably right about that.

>> If you are talking about how 'int ia[1];' declares an object with
>> (sizeof (int) * CHAR_BIT) bits and its declared type seems to be both
>> 'int[1]' and 'int', I don't think that's the case.  I think the
>> effective type here must match the declared type, 'int[1]'.  The pointer
>> expression 'ia' (after conversion from array type) no longer refers to
>> the same object, but refers to its first element.
>
> The array and the element are the same object.
>
> 3.15 "object: region of data storage in the execution environment, the
> contents of which can represent values.  NOTE When referenced, an object
> may be interpreted as having a particular type; see 6.3.2.1."
>
> 6.3.2.1#1 "When an object is said to have a particular type, the type is
> specified by the lvalue used to designate the object."
>

I suggested that 3.15 was not the whole story in a previous post, below. 
  But it is certainly possible to work with an interpretation where it 
is. :)

> An object can have a declared type, an effective type, and can be
> accessed using lvalues that have various types; but when the same region
> of memory is designated by two lvalues that have different types, it's
> still one object, not two.
>

Ok.  Working with that model, an object (as a region of data storage) 
can be declared with multiple types, so can have multiple effective types.

> Otherwise, the whole concept of effective types falls apart.  If two
> accesses to the same region of memory count as accesses to two distinct
> objects, then the effective type of one object doesn't affect how the
> other object can be accessed, does it?  Unless there are clear rules
> about when a region of memory counts as the same object for the purpose
> of aliasing, the rules of effective types say nothing meaningful about
> aliasing.
>

But multiple effective types for one object has the same problem as a 
single effective type for each sub-object, right?  It's just a shuffle?

> I would prefer to go one step in the opposite direction, and insist that
> the standard should say something about objects that partially overlap.
>   When an assignment to a subobject of an allocated object gives it a
> new effective type, shouldn't it also wipe out any effective types that
> overlapping objects may have?
>

With that strategy, if you had [S|O|M|E| |T|Y|P|E], would a partial 
erasure yield [?|?[U|I|N|T]?|?|?], or could we continue to believe that 
the effective type of the whole was unaffected?

> ...
>> I agree that those two 'char *' pointers address the same byte, but only
>> as a consequence of the two objects occupying the same location...  So I
>> don't quite agree that its only role is to produce a pointer to the
>> first byte; I think the pointer has more properties than which byte it
>> addresses.
>
> It seems that it does; trouble is, the standard doesn't say much about
> what those extra properties are and what happens to them when pointers
> are converted.  And the few clues that it does offer don't seem very
> consistent.
>
> For instance, the description of pointer addition and subtraction
> strongly implies that every pointer to an object has a range of integers
> associated with it that can be added to it without causing undefined
> behaviour.  On the other hand, the discussion of pointer conversions
> seems to think it's sufficient to guararantee that a resulting pointer
> compares equal to some other pointer, without saying anything about the
> associated range of integers.  But without knowing that range, there
> isn't much we can safely do with a pointer -- in particular,
> dereferencing, incrementing, or decrementing a pointer may invoke
> undefined behaviour, even if we know that the pointer compares equal to
> some other pointer that can be safely dereferenced, incremented, or
> decremented.
>

Heheh.

> The concept of pointer provenance, insinuated by DR260, is hopelessly
> underspecified too: we're supposed to rely on it to somehow figure out
> what operations have a defined behaviour and which ones don't, from a
> few vague sentences that talk about what implementations are permitted
> to track, without much of an explanation of how those permissions affect
> the guarantees given to programs.
>

I just assume DR #260 generally means that the implementation can track 
as much as possible, such as uninitialized objects.  Later on, DR #338 
wound up saying so, for register-candidate objects.  Handy for enforcing 
safety!

>> For any other case of offset or number of bytes, I'd ask "which object
>> is being copied?"  The answer might depend on the last-stored value, for
>> union involvement.
>
> Only if the object doesn't have a declared type.  If it does have a
> declared type, then that's its effective type too, and storing values
> doesn't affect it, right?
>

Right.  If the outermost containing object was declared in such a way 
that some particular sub-object (from offset and number of bytes) could 
only correspond to one sub-object of the outermost type, then I'd say 
that sub-object had a declared type.  If there were multiple 
possibilities, I'd ask, "well which one is consistent with the 
programmer's intention, given the last-stored value?"

But that's assuming the "outer-most" effective type business (which I 
think is easiest to track. :) )

>>   Regardless of what the answer depends on, if there
>> are multiple answers, then all we know for certain is that a
>> 'char[num_of_bytes]' is being copied.  I think that answer could be a
>> default, in the case where there is no single better answer.
>
> Well I think it's the standard's job to give us a better answer.
>

I figure a goal of "effective type" is so that an implementation can 
make assumptions that yield benefit much of the time, but when the 
programmer is bitten, a rule can be shown as having been violated.

>>> Remember that an object is just a region of storage; when we talk about
>>> its type, that's just a short way of talking about types of lvalues, or
>>> declared types, or effective types.
>>
>> "is just": I think this is consistent with the initial definition of
>> 'object' under section 3, but I think the actual definition accumulates
>> some more from other parts of the Standard.  The discussion of pointer
>> arithmetic, for example, distinguishes between an "array object" and its
>> "array elements", but these different objects can exactly overlap.
>
> Sure; but we're never told whether this "arrayness" is based on some
> properties of the object, such as its declared or effective type, or on
> the lvalue expression that the pointer originated from.  I tend to think
> it's the latter, not only because that seems consistent with the need to
> talk about the provenance of pointers, but also because it applies to
> math on pointers to memory that has just been allocated and has not been
> accessed yet and therefore doesn't have any declared or effective type.
>

An object "can be treated" as an array, in some cases, as you say. :)

>>>>> Does the four-byte sub-object at the beginning of "u" have an
>>>>> effective
>>>>> type?
>>>  my four-byte object that can (I hope) be accessed using such
>>> lvalues as u.ls.l, u.u, and u.f?
>>
>> In order to determine the effective type, I'd ask "which object(s) do
>> the first four bytes refer to?"  For a union, I think the answer would
>> first depend on the last-stored value.
>
> Only if there's no declared type.  Is there?
>

There is an outer-most declared type for the containing object, but 
there isn't a one-to-one correspondence for these four bytes with a 
sub-object.

>>   The last-stored value of the
>> union was not set via any of the members, so that does not resolve the
>> answer.  After that, there are multiple answers, but none of those
>> scores as highly as the type of the value that was stored, making the
>> answer "4 bytes of an 'unsigned char[6]'".
>
> Your scoring of multiple answers is not based on anything that the
> standard actually says about effective types, is it?
>

Actually, yes (I had hoped).  It is only clear to me that a declaration 
declares the type of a whole object having no containing object.  Then 
if we have an lvalue that designates some included region of data 
storage, we can ask, "does that region of data storage have a declared 
type?"  The answer might be obvious from an inspection of the type of 
the container.  If multiple answers are possible, we can answer "no 
declared type."  Then the effective type heuristics for "no declared 
type" are invoked, involving the last-stored value.  That was my rationale.

>> However, the 'memset' wasn't working with only the first 4 bytes.  The
>> 'memset' did more than that.  It involved '&u' and involved all of the
>> bytes, so it produced a union value.  I believe that union value could
>> be used with another union of the same type with 'union u u2 = u;' or
>> 'union u u2; u2 = u;', for example.
>>
>> 'u.ls.l', 'u.u' and 'u.f' designate members which do not have a
>> last-stored value, and no union member was used for the last-stored
>> value, so I'd say that evaluating any of them yields undefined
>> behaviour.  (!!)
>
> My first guess would be that accessing any if of the integers using its
> declared type should be safe, because the memset has put a valid
> representation of zero in it.
>

Which takes precedence?  The type-punning foot-note, or the "shall" in 
6.5p7?

>> Because there is no padding for meta-data in your example, you could
>> circumvent the undefined behaviour with any kind of code that makes it
>> impossible to prove any claim about such meta-data (such as whether or
>> not a member was used to store a value), yielding defined behaviour.
>
> I'm not sure if I understand what you're saying here, but just because
> there's no padding to store extra information at runtime doesn't mean
> that the implementation can't track that information at compile time, or
> maybe sometimes outside of the object.  Even if you make it hard for
> implementations to figure out that you're breaking the rules, that does
> not make the behaviour defined.
>

I didn't say "hard," I said "impossible." :)  When there isn't any room 
in the object representation for meta-data, you can send the object to a 
friend, not know if or how they tampered with it, then read it back. 
Some beliefs about anything other than the object representation would 
seem unfounded, after that.

I suppose the implementation could call your friend...

> ...
>>> My concern (for now) is not about the contents of the object but about
>>> its effective type.  In particular, the "sticky" kind of effective type
>>> that can conflict with read accesses using an lvalue with the wrong
>>> type, and gets propagated by memcpying to other typeless objects.  In
>>> other words, whether one of the standard's statements about the
>>> effective type "for subsequent accesses that do not modify the value"
>>> apply to it or not.
>>
>> We share the same concern.  Some of the considerations of the Standard
>> involve subjects which invoke undefined behaviour, but some of those
>> subjects appear to be hard to prove, under the right circumstances.
>> "Effective type" strikes me as one such subject.  My interest is in an
>> implementation which attempts to trap _all_ code which yields undefined
>> behaviour and report it.
>
> Well good luck with that.  I'm afraid the best you can possibly do is
> detect and report code that has undefined behaviour *according to your
> interpretation of the standard*.
>

And so the reason for the original post is revealed: To find out if 
there's common ground.

> ...
>>> Hey, here's another one:
>>>
>>>    #define N sizeof(float)
>>>    float x = 1.0;
>>>    char y[ N ];
>>>    float *p = memcpy( malloc(N), memcpy( y, &x, N ), N );
>>>    // Does *p have a "sticky" effective type now?  Is it char or float?
>>>    float z = *p; // Is this valid?
>>>
>>
>> I would indeed track and report the effective type of the modified
>> storage pointed to by 'p' as having effective type 'float'.  An array of
>> character type is always the case, so it's an uninteresting case (to
>> me), if there's more that can be proven about the provenance of the
>> "stored value".
>
> Actually, since y has a declared type, shouldn't it make any traces of
> floatness disappear?
>

The effective type of 'y' might be 'char[N]', but the effective type of 
the object that was copied through it needn't be, since the original is 
"copied as an array of character type".

> I think what I really had in mind back then was an unaligned object
> without a declared type:
>
>    char *y = 1 + (char*)malloc(N+1);
>

Right, I think that goes to show that the character-type intermediary is 
unaffected.

-- 
- Shao Miller
--
"Thank you for the kind words; those are the kind of words I like to hear.

Cheerily,"  -- Richard Harter

[toc] | [prev] | [next] | [standalone]

#1859

On 17-Jan-13 10:47 PM, Shao Miller wrote:
> On 1/17/2013 00:32, Wojtek Lerch wrote:
>> On 15/01/2013 5:39 AM, Shao Miller wrote:
>>> On 1/15/2013 00:28, Wojtek Lerch wrote:
>>>> On 14/01/2013 1:03 AM, Shao Miller wrote:
>>>>> On 1/13/2013 23:48, Wojtek Lerch wrote:
>>>>>>
>>>>>> union u {
>>>>>>    struct sl { int16_t s; int32_t l; } sl;
>>>>>>    struct ls { int32_t l; int16_t s; } ls;
>>>>>>    uint32_t u;
>>>>>>    float f;
>>>>>> };
>>>>>>
>>>>>> int main(void) {
>>>>>>    void *p;
>>>>>>    if ( sizeof(union u) == 6 // no padding
>>>>>>      && ( p = malloc(6) ) != NULL ) {
>>>>>>      union u u;
>>>>>>      memset( u, 0, 6 );
>>>>>>      memcpy( p, (void*)&u.sl, 6 );
>>>>>>      ...
>>>>>>    }
>>>>>> }
>>
>> ...
>>
>>>>>   I think I saw something like this in Defect Report #219.
>>>>
>>>> That's a good find, thanks.  But there seems to be an inconsistency
>>>> between the "committee Discussion" (which I believe is non-normative)
>>>> and the "Committee Response" (which is normative) about the
>>>> treatment of
>>>> character types.
>>>
>>> Which part/parts is/are inconsistent?
>>
>> Discussion:
>>
>>    6. memcpy (p1, (char *)&s + offsetof (s, i), len1);
>>
>>    In lines 6 and 7, the effective type of the source arguments
>>    to memcpy is an array of char
>>
>>    The effective type for p1 and p2 in lines 6 and 7 is: array of char.
>>
>> Response:
>>
>>    memcpy (p1, (char *)&s + offsetof (s, i), len1); copies all
>>    of the bytes of s.i to an alignment suitable for an object
>>    of type int. The effective type of the resulting copy can
>>    be treated as having effective type int.
>>
>> (Also, I have to say that the "can be treated" part feels out of place
>> in what is supposed to be a normative response.  Is the Committee really
>> telling us that accessing the object as an int "can be treated" as not
>> having undefined behaviour?)
>>
>
> Ah, ok.  Thank you for clarifying.
>
>>>> The DR clarifies one important thing though: it's not only objects
>>>> named
>>>> in declarations that have a "declared type", but also their subobjects,
>>>> such as members or elements.  But if that's the case, then it's still
>>>> not
>>>> clear to me what happens when a[n aggregate] object
>>>> and its element or
>>>> member, or two members of a union, are exactly the same object.  It
>>>> would make sense to me that the object might have several declared
>>>> types
>>>> at the same time, but the standard's discussion of effective types
>>>> doesn't create the impression that that was considered a possibility,
>>>> does it?
>>>
>>> I don't think I agree.  When there's a declared type, I think the
>>> object's effective type can only be that declared type.
>>
>> That's what the standard says; but when an object has three declared
>> types at the same time, how many effective types does it have?  In my
>> example, the expressions "u", "u.sl", and "u.ls" designate the same
>> 6-byte object.  Does that object have just one declared type, or does it
>> have three?  As far as I can tell we don't have a clear definition of
>> "declared type" that would cover members and elements of objects
>> declared with a union or aggregate type.
>>
>
> I see your point, if the object is only considered to be the region of
> data storage and if its entirety is accessible via different lvalues
> with different declared types.
>
>>>    I certainly
>>> hope that this is the case, else tracking and reporting could be more
>>> complicated!
>>
>> I think the priority should be to have a clear definition of what you
>> want to track, before worrying about how complicated it is to track it.
>>
>
> You're probably right about that.
>
>>> If you are talking about how 'int ia[1];' declares an object with
>>> (sizeof (int) * CHAR_BIT) bits and its declared type seems to be both
>>> 'int[1]' and 'int', I don't think that's the case.  I think the
>>> effective type here must match the declared type, 'int[1]'.  The pointer
>>> expression 'ia' (after conversion from array type) no longer refers to
>>> the same object, but refers to its first element.
>>
>> The array and the element are the same object.
>>
>> 3.15 "object: region of data storage in the execution environment, the
>> contents of which can represent values.  NOTE When referenced, an object
>> may be interpreted as having a particular type; see 6.3.2.1."
>>
>> 6.3.2.1#1 "When an object is said to have a particular type, the type is
>> specified by the lvalue used to designate the object."
>>
>
> I suggested that 3.15 was not the whole story in a previous post, below.
>   But it is certainly possible to work with an interpretation where it
> is. :)
>
>> An object can have a declared type, an effective type, and can be
>> accessed using lvalues that have various types; but when the same region
>> of memory is designated by two lvalues that have different types, it's
>> still one object, not two.
>>
>
> Ok.  Working with that model, an object (as a region of data storage)
> can be declared with multiple types, so can have multiple effective types.
>
>> Otherwise, the whole concept of effective types falls apart.  If two
>> accesses to the same region of memory count as accesses to two distinct
>> objects, then the effective type of one object doesn't affect how the
>> other object can be accessed, does it?  Unless there are clear rules
>> about when a region of memory counts as the same object for the purpose
>> of aliasing, the rules of effective types say nothing meaningful about
>> aliasing.
>>
>
> But multiple effective types for one object has the same problem as a
> single effective type for each sub-object, right?  It's just a shuffle?
>
>> I would prefer to go one step in the opposite direction, and insist that
>> the standard should say something about objects that partially overlap.
>>   When an assignment to a subobject of an allocated object gives it a
>> new effective type, shouldn't it also wipe out any effective types that
>> overlapping objects may have?
>>
>
> With that strategy, if you had [S|O|M|E| |T|Y|P|E], would a partial
> erasure yield [?|?[U|I|N|T]?|?|?], or could we continue to believe that
> the effective type of the whole was unaffected?
>
>> ...
>>> I agree that those two 'char *' pointers address the same byte, but only
>>> as a consequence of the two objects occupying the same location...  So I
>>> don't quite agree that its only role is to produce a pointer to the
>>> first byte; I think the pointer has more properties than which byte it
>>> addresses.
>>
>> It seems that it does; trouble is, the standard doesn't say much about
>> what those extra properties are and what happens to them when pointers
>> are converted.  And the few clues that it does offer don't seem very
>> consistent.
>>
>> For instance, the description of pointer addition and subtraction
>> strongly implies that every pointer to an object has a range of integers
>> associated with it that can be added to it without causing undefined
>> behaviour.  On the other hand, the discussion of pointer conversions
>> seems to think it's sufficient to guararantee that a resulting pointer
>> compares equal to some other pointer, without saying anything about the
>> associated range of integers.  But without knowing that range, there
>> isn't much we can safely do with a pointer -- in particular,
>> dereferencing, incrementing, or decrementing a pointer may invoke
>> undefined behaviour, even if we know that the pointer compares equal to
>> some other pointer that can be safely dereferenced, incremented, or
>> decremented.
>>
>
> Heheh.
>
>> The concept of pointer provenance, insinuated by DR260, is hopelessly
>> underspecified too: we're supposed to rely on it to somehow figure out
>> what operations have a defined behaviour and which ones don't, from a
>> few vague sentences that talk about what implementations are permitted
>> to track, without much of an explanation of how those permissions affect
>> the guarantees given to programs.
>>
>
> I just assume DR #260 generally means that the implementation can track
> as much as possible, such as uninitialized objects.

I think it goes without saying that the implementation can track 
whatever it wants to (in addition to tracking what it's required to 
track).  I think it's pretty obvious that it's nor forbidden to detect 
scenarios leading to of undefined behaviour, and report them or perform 
optimizations based on the assumption that they don't happen.  Stating 
those simple facts does not count as an explanation of which programs or 
scenarios do in fact have undefined behaviour.

>   Later on, DR #338
> wound up saying so, for register-candidate objects.  Handy for enforcing
> safety!

Well no, DR338 says that the behaviour in a particular type of scenarios 
is undefined.  Implementations were already permitted to detect those 
scenarios before they were undefined.  Many do, for instance for the 
purpose of printing out a warning.

...
> Right.  If the outermost containing object was declared in such a way
> that some particular sub-object (from offset and number of bytes) could
> only correspond to one sub-object of the outermost type, then I'd say
> that sub-object had a declared type.  If there were multiple
> possibilities, I'd ask, "well which one is consistent with the
> programmer's intention, given the last-stored value?"

Why?  There's no wording in the standard that would make the answer to 
that question relevant to the declared type of any object.  If there are 
multiple possibilities, that just means that either the standard either 
forgot to explain what happens when there are multiple possibilities, or 
our assumption that subobjects have declared types was wrong.  You and I 
can try to come up with additional rules that fill these blanks, but 
that's not very useful unless we convince the Committee to add them to 
the standard; I don't know about you, but I don't really have the time 
or determination to try to do that.

> But that's assuming the "outer-most" effective type business (which I
> think is easiest to track. :) )
>
>>>   Regardless of what the answer depends on, if there
>>> are multiple answers, then all we know for certain is that a
>>> 'char[num_of_bytes]' is being copied.  I think that answer could be a
>>> default, in the case where there is no single better answer.
>>
>> Well I think it's the standard's job to give us a better answer.
>
> I figure a goal of "effective type" is so that an implementation can
> make assumptions that yield benefit much of the time, but when the
> programmer is bitten, a rule can be shown as having been violated.

Right.  That's why we need clear and unambiguous rules.  In the 
standard.  So that we don't have to wonder what answer to choose when 
the standard doesn't give you a single answer to a question that it's 
the standard's job to answer.

>>>> Remember that an object is just a region of storage; when we talk about
>>>> its type, that's just a short way of talking about types of lvalues, or
>>>> declared types, or effective types.
>>>
>>> "is just": I think this is consistent with the initial definition of
>>> 'object' under section 3, but I think the actual definition accumulates
>>> some more from other parts of the Standard.  The discussion of pointer
>>> arithmetic, for example, distinguishes between an "array object" and its
>>> "array elements", but these different objects can exactly overlap.
>>
>> Sure; but we're never told whether this "arrayness" is based on some
>> properties of the object, such as its declared or effective type, or on
>> the lvalue expression that the pointer originated from.  I tend to think
>> it's the latter, not only because that seems consistent with the need to
>> talk about the provenance of pointers, but also because it applies to
>> math on pointers to memory that has just been allocated and has not been
>> accessed yet and therefore doesn't have any declared or effective type.
>>
>
> An object "can be treated" as an array, in some cases, as you say. :)

>>>>>> Does the four-byte sub-object at the beginning of "u" have an
>>>>>> effective
>>>>>> type?
>>>>  my four-byte object that can (I hope) be accessed using such
>>>> lvalues as u.ls.l, u.u, and u.f?
>>>
>>> In order to determine the effective type, I'd ask "which object(s) do
>>> the first four bytes refer to?"  For a union, I think the answer would
>>> first depend on the last-stored value.
>>
>> Only if there's no declared type.  Is there?
>
> There is an outer-most declared type for the containing object, but
> there isn't a one-to-one correspondence for these four bytes with a
> sub-object.

So what does that tell us about the intended meaning of "declared type" 
of a subobject?  It seems that the Committee didn't think that the 
concept is tricky enough to warrant a written definition; do you think 
they though it was obvious that a union member may or may not a declared 
type depending on whether it precisely overlaps with another member of 
the whole union?  Or do you think it's more likely that they simply 
forgot about unions?

Another possibility is that "declared type" only applies to the whole 
object named in a declaration; but DR319 seems inconsistent with that 
interpretation.

...
> Actually, yes (I had hoped).  It is only clear to me that a declaration
> declares the type of a whole object having no containing object.  Then
> if we have an lvalue that designates some included region of data
> storage, we can ask, "does that region of data storage have a declared
> type?"  The answer might be obvious from an inspection of the type of
> the container.  If multiple answers are possible, we can answer "no
> declared type."  Then the effective type heuristics for "no declared
> type" are invoked, involving the last-stored value.  That was my rationale.

I don't think so.  If multiple answers are possible, then either they're 
all correct (i.e. there are multiple declared types), or we 
misunderstood the question.  There's nothing in the standard that would 
imply that a union member may or may not have a declared type depending 
on whether its overlap with another member is exact or not.

>>> However, the 'memset' wasn't working with only the first 4 bytes.  The
>>> 'memset' did more than that.  It involved '&u' and involved all of the
>>> bytes, so it produced a union value.  I believe that union value could
>>> be used with another union of the same type with 'union u u2 = u;' or
>>> 'union u u2; u2 = u;', for example.
>>>
>>> 'u.ls.l', 'u.u' and 'u.f' designate members which do not have a
>>> last-stored value, and no union member was used for the last-stored
>>> value, so I'd say that evaluating any of them yields undefined
>>> behaviour.  (!!)
>>
>> My first guess would be that accessing any if of the integers using its
>> declared type should be safe, because the memset has put a valid
>> representation of zero in it.
>
> Which takes precedence?  The type-punning foot-note, or the "shall" in
> 6.5p7?

Footnotes are not normative.

>>> Because there is no padding for meta-data in your example, you could
>>> circumvent the undefined behaviour with any kind of code that makes it
>>> impossible to prove any claim about such meta-data (such as whether or
>>> not a member was used to store a value), yielding defined behaviour.
>>
>> I'm not sure if I understand what you're saying here, but just because
>> there's no padding to store extra information at runtime doesn't mean
>> that the implementation can't track that information at compile time, or
>> maybe sometimes outside of the object.  Even if you make it hard for
>> implementations to figure out that you're breaking the rules, that does
>> not make the behaviour defined.
>>
>
> I didn't say "hard," I said "impossible." :)  When there isn't any room
> in the object representation for meta-data, you can send the object to a
> friend, not know if or how they tampered with it, then read it back.
> Some beliefs about anything other than the object representation would
> seem unfounded, after that.
>
> I suppose the implementation could call your friend...

The implementation has seen your source code.  If a person can prove 
that its behaviour is undefined, then an implementation, in principle, 
can too.  There's no rule that undefined behaviour must be provable 
based on object representations only.

>> ...
>>>> My concern (for now) is not about the contents of the object but about
>>>> its effective type.  In particular, the "sticky" kind of effective type
>>>> that can conflict with read accesses using an lvalue with the wrong
>>>> type, and gets propagated by memcpying to other typeless objects.  In
>>>> other words, whether one of the standard's statements about the
>>>> effective type "for subsequent accesses that do not modify the value"
>>>> apply to it or not.
>>>
>>> We share the same concern.  Some of the considerations of the Standard
>>> involve subjects which invoke undefined behaviour, but some of those
>>> subjects appear to be hard to prove, under the right circumstances.
>>> "Effective type" strikes me as one such subject.  My interest is in an
>>> implementation which attempts to trap _all_ code which yields undefined
>>> behaviour and report it.
>>
>> Well good luck with that.  I'm afraid the best you can possibly do is
>> detect and report code that has undefined behaviour *according to your
>> interpretation of the standard*.
>
> And so the reason for the original post is revealed: To find out if
> there's common ground.
>
>> ...
>>>> Hey, here's another one:
>>>>
>>>>    #define N sizeof(float)
>>>>    float x = 1.0;
>>>>    char y[ N ];
>>>>    float *p = memcpy( malloc(N), memcpy( y, &x, N ), N );
>>>>    // Does *p have a "sticky" effective type now?  Is it char or float?
>>>>    float z = *p; // Is this valid?
>>>
>>> I would indeed track and report the effective type of the modified
>>> storage pointed to by 'p' as having effective type 'float'.  An array of
>>> character type is always the case, so it's an uninteresting case (to
>>> me), if there's more that can be proven about the provenance of the
>>> "stored value".
>>
>> Actually, since y has a declared type, shouldn't it make any traces of
>> floatness disappear?
>
> The effective type of 'y' might be 'char[N]', but the effective type of
> the object that was copied through it needn't be, since the original is
> "copied as an array of character type".

There's no such thing as a copy "through" y; there's a copy into y and 
then a separate copy from y.  In the second copy, the source object has 
a declared type, and it's an array of char.  Not float.

>> I think what I really had in mind back then was an unaligned object
>> without a declared type:
>>
>>    char *y = 1 + (char*)malloc(N+1);
>>
>
> Right, I think that goes to show that the character-type intermediary is
> unaffected.

My goal was to show that alignment is irrelevant.  It's fairly clear in 
this case that the object consisting of the bytes y[1] through y[N] has 
an effective type of float, even if it can't be accessed as a float 
because of alignment requirements.

Why then is alignment mentioned at all in DR219?

[toc] | [prev] | [next] | [standalone]

#1860

Apologies for my lousy editing -- I meant to save a draft but pushed the 
send button instead...

On 18-Jan-13 4:14 PM, Wojtek Lerch wrote:
> On 17-Jan-13 10:47 PM, Shao Miller wrote:
>> On 1/17/2013 00:32, Wojtek Lerch wrote:
>>> On 15/01/2013 5:39 AM, Shao Miller wrote:
>>>> On 1/15/2013 00:28, Wojtek Lerch wrote:
>>>>> On 14/01/2013 1:03 AM, Shao Miller wrote:
>>>>>> On 1/13/2013 23:48, Wojtek Lerch wrote:
>>>>>>>
>>>>>>> union u {
>>>>>>>    struct sl { int16_t s; int32_t l; } sl;
>>>>>>>    struct ls { int32_t l; int16_t s; } ls;
>>>>>>>    uint32_t u;
>>>>>>>    float f;
>>>>>>> };
>>>>>>>
>>>>>>> int main(void) {
>>>>>>>    void *p;
>>>>>>>    if ( sizeof(union u) == 6 // no padding
>>>>>>>      && ( p = malloc(6) ) != NULL ) {
>>>>>>>      union u u;
>>>>>>>      memset( u, 0, 6 );
>>>>>>>      memcpy( p, (void*)&u.sl, 6 );
>>>>>>>      ...
>>>>>>>    }
>>>>>>> }
>>>
>>> ...
>>>
>>>>>>   I think I saw something like this in Defect Report #219.
>>>>>
>>>>> That's a good find, thanks.  But there seems to be an inconsistency
>>>>> between the "committee Discussion" (which I believe is non-normative)
>>>>> and the "Committee Response" (which is normative) about the
>>>>> treatment of
>>>>> character types.
>>>>
>>>> Which part/parts is/are inconsistent?
>>>
>>> Discussion:
>>>
>>>    6. memcpy (p1, (char *)&s + offsetof (s, i), len1);
>>>
>>>    In lines 6 and 7, the effective type of the source arguments
>>>    to memcpy is an array of char
>>>
>>>    The effective type for p1 and p2 in lines 6 and 7 is: array of char.
>>>
>>> Response:
>>>
>>>    memcpy (p1, (char *)&s + offsetof (s, i), len1); copies all
>>>    of the bytes of s.i to an alignment suitable for an object
>>>    of type int. The effective type of the resulting copy can
>>>    be treated as having effective type int.
>>>
>>> (Also, I have to say that the "can be treated" part feels out of place
>>> in what is supposed to be a normative response.  Is the Committee really
>>> telling us that accessing the object as an int "can be treated" as not
>>> having undefined behaviour?)
>>>
>>
>> Ah, ok.  Thank you for clarifying.
>>
>>>>> The DR clarifies one important thing though: it's not only objects
>>>>> named
>>>>> in declarations that have a "declared type", but also their
>>>>> subobjects,
>>>>> such as members or elements.  But if that's the case, then it's still
>>>>> not
>>>>> clear to me what happens when a[n aggregate] object
>>>>> and its element or
>>>>> member, or two members of a union, are exactly the same object.  It
>>>>> would make sense to me that the object might have several declared
>>>>> types
>>>>> at the same time, but the standard's discussion of effective types
>>>>> doesn't create the impression that that was considered a possibility,
>>>>> does it?
>>>>
>>>> I don't think I agree.  When there's a declared type, I think the
>>>> object's effective type can only be that declared type.
>>>
>>> That's what the standard says; but when an object has three declared
>>> types at the same time, how many effective types does it have?  In my
>>> example, the expressions "u", "u.sl", and "u.ls" designate the same
>>> 6-byte object.  Does that object have just one declared type, or does it
>>> have three?  As far as I can tell we don't have a clear definition of
>>> "declared type" that would cover members and elements of objects
>>> declared with a union or aggregate type.
>>>
>>
>> I see your point, if the object is only considered to be the region of
>> data storage and if its entirety is accessible via different lvalues
>> with different declared types.
>>
>>>>    I certainly
>>>> hope that this is the case, else tracking and reporting could be more
>>>> complicated!
>>>
>>> I think the priority should be to have a clear definition of what you
>>> want to track, before worrying about how complicated it is to track it.
>>>
>>
>> You're probably right about that.
>>
>>>> If you are talking about how 'int ia[1];' declares an object with
>>>> (sizeof (int) * CHAR_BIT) bits and its declared type seems to be both
>>>> 'int[1]' and 'int', I don't think that's the case.  I think the
>>>> effective type here must match the declared type, 'int[1]'.  The
>>>> pointer
>>>> expression 'ia' (after conversion from array type) no longer refers to
>>>> the same object, but refers to its first element.
>>>
>>> The array and the element are the same object.
>>>
>>> 3.15 "object: region of data storage in the execution environment, the
>>> contents of which can represent values.  NOTE When referenced, an object
>>> may be interpreted as having a particular type; see 6.3.2.1."
>>>
>>> 6.3.2.1#1 "When an object is said to have a particular type, the type is
>>> specified by the lvalue used to designate the object."
>>>
>>
>> I suggested that 3.15 was not the whole story in a previous post, below.
>>   But it is certainly possible to work with an interpretation where it
>> is. :)
>>
>>> An object can have a declared type, an effective type, and can be
>>> accessed using lvalues that have various types; but when the same region
>>> of memory is designated by two lvalues that have different types, it's
>>> still one object, not two.
>>>
>>
>> Ok.  Working with that model, an object (as a region of data storage)
>> can be declared with multiple types, so can have multiple effective
>> types.
>>
>>> Otherwise, the whole concept of effective types falls apart.  If two
>>> accesses to the same region of memory count as accesses to two distinct
>>> objects, then the effective type of one object doesn't affect how the
>>> other object can be accessed, does it?  Unless there are clear rules
>>> about when a region of memory counts as the same object for the purpose
>>> of aliasing, the rules of effective types say nothing meaningful about
>>> aliasing.
>>>
>>
>> But multiple effective types for one object has the same problem as a
>> single effective type for each sub-object, right?  It's just a shuffle?
>>
>>> I would prefer to go one step in the opposite direction, and insist that
>>> the standard should say something about objects that partially overlap.
>>>   When an assignment to a subobject of an allocated object gives it a
>>> new effective type, shouldn't it also wipe out any effective types that
>>> overlapping objects may have?
>>>
>>
>> With that strategy, if you had [S|O|M|E| |T|Y|P|E], would a partial
>> erasure yield [?|?[U|I|N|T]?|?|?], or could we continue to believe that
>> the effective type of the whole was unaffected?
>>
>>> ...
>>>> I agree that those two 'char *' pointers address the same byte, but
>>>> only
>>>> as a consequence of the two objects occupying the same location...
>>>> So I
>>>> don't quite agree that its only role is to produce a pointer to the
>>>> first byte; I think the pointer has more properties than which byte it
>>>> addresses.
>>>
>>> It seems that it does; trouble is, the standard doesn't say much about
>>> what those extra properties are and what happens to them when pointers
>>> are converted.  And the few clues that it does offer don't seem very
>>> consistent.
>>>
>>> For instance, the description of pointer addition and subtraction
>>> strongly implies that every pointer to an object has a range of integers
>>> associated with it that can be added to it without causing undefined
>>> behaviour.  On the other hand, the discussion of pointer conversions
>>> seems to think it's sufficient to guararantee that a resulting pointer
>>> compares equal to some other pointer, without saying anything about the
>>> associated range of integers.  But without knowing that range, there
>>> isn't much we can safely do with a pointer -- in particular,
>>> dereferencing, incrementing, or decrementing a pointer may invoke
>>> undefined behaviour, even if we know that the pointer compares equal to
>>> some other pointer that can be safely dereferenced, incremented, or
>>> decremented.
>>>
>>
>> Heheh.
>>
>>> The concept of pointer provenance, insinuated by DR260, is hopelessly
>>> underspecified too: we're supposed to rely on it to somehow figure out
>>> what operations have a defined behaviour and which ones don't, from a
>>> few vague sentences that talk about what implementations are permitted
>>> to track, without much of an explanation of how those permissions affect
>>> the guarantees given to programs.
>>>
>>
>> I just assume DR #260 generally means that the implementation can track
>> as much as possible, such as uninitialized objects.
>
> I think it goes without saying that the implementation can track
> whatever it wants to (in addition to tracking what it's required to
> track).  I think it's pretty obvious that it's nor forbidden to detect
> scenarios leading to of undefined behaviour, and report them or perform
> optimizations based on the assumption that they don't happen.  Stating
> those simple facts does not count as an explanation of which programs or
> scenarios do in fact have undefined behaviour.
>
>>   Later on, DR #338
>> wound up saying so, for register-candidate objects.  Handy for enforcing
>> safety!
>
> Well no, DR338 says that the behaviour in a particular type of scenarios
> is undefined.  Implementations were already permitted to detect those
> scenarios before they were undefined.  Many do, for instance for the
> purpose of printing out a warning.
>
> ...
>> Right.  If the outermost containing object was declared in such a way
>> that some particular sub-object (from offset and number of bytes) could
>> only correspond to one sub-object of the outermost type, then I'd say
>> that sub-object had a declared type.  If there were multiple
>> possibilities, I'd ask, "well which one is consistent with the
>> programmer's intention, given the last-stored value?"
>
> Why?  There's no wording in the standard that would make the answer to
> that question relevant to the declared type of any object.  If there are
> multiple possibilities, that just means that either the standard either
> forgot to explain what happens when there are multiple possibilities, or
> our assumption that subobjects have declared types was wrong.  You and I
> can try to come up with additional rules that fill these blanks, but
> that's not very useful unless we convince the Committee to add them to
> the standard; I don't know about you, but I don't really have the time
> or determination to try to do that.
>
>> But that's assuming the "outer-most" effective type business (which I
>> think is easiest to track. :) )
>>
>>>>   Regardless of what the answer depends on, if there
>>>> are multiple answers, then all we know for certain is that a
>>>> 'char[num_of_bytes]' is being copied.  I think that answer could be a
>>>> default, in the case where there is no single better answer.
>>>
>>> Well I think it's the standard's job to give us a better answer.
>>
>> I figure a goal of "effective type" is so that an implementation can
>> make assumptions that yield benefit much of the time, but when the
>> programmer is bitten, a rule can be shown as having been violated.
>
> Right.  That's why we need clear and unambiguous rules.  In the
> standard.  So that we don't have to wonder what answer to choose when
> the standard doesn't give you a single answer to a question that it's
> the standard's job to answer.
>
>>>>> Remember that an object is just a region of storage; when we talk
>>>>> about
>>>>> its type, that's just a short way of talking about types of
>>>>> lvalues, or
>>>>> declared types, or effective types.
>>>>
>>>> "is just": I think this is consistent with the initial definition of
>>>> 'object' under section 3, but I think the actual definition accumulates
>>>> some more from other parts of the Standard.  The discussion of pointer
>>>> arithmetic, for example, distinguishes between an "array object" and
>>>> its
>>>> "array elements", but these different objects can exactly overlap.
>>>
>>> Sure; but we're never told whether this "arrayness" is based on some
>>> properties of the object, such as its declared or effective type, or on
>>> the lvalue expression that the pointer originated from.  I tend to think
>>> it's the latter, not only because that seems consistent with the need to
>>> talk about the provenance of pointers, but also because it applies to
>>> math on pointers to memory that has just been allocated and has not been
>>> accessed yet and therefore doesn't have any declared or effective type.
>>>
>>
>> An object "can be treated" as an array, in some cases, as you say. :)
>
>>>>>>> Does the four-byte sub-object at the beginning of "u" have an
>>>>>>> effective
>>>>>>> type?
>>>>>  my four-byte object that can (I hope) be accessed using such
>>>>> lvalues as u.ls.l, u.u, and u.f?
>>>>
>>>> In order to determine the effective type, I'd ask "which object(s) do
>>>> the first four bytes refer to?"  For a union, I think the answer would
>>>> first depend on the last-stored value.
>>>
>>> Only if there's no declared type.  Is there?
>>
>> There is an outer-most declared type for the containing object, but
>> there isn't a one-to-one correspondence for these four bytes with a
>> sub-object.
>
> So what does that tell us about the intended meaning of "declared type"
> of a subobject?  It seems that the Committee didn't think that the
> concept is tricky enough to warrant a written definition; do you think
> they though it was obvious that a union member may or may not a declared
> type depending on whether it precisely overlaps with another member of
> the whole union?  Or do you think it's more likely that they simply
> forgot about unions?
>
> Another possibility is that "declared type" only applies to the whole
> object named in a declaration; but DR319 seems inconsistent with that
> interpretation.
>
> ...
>> Actually, yes (I had hoped).  It is only clear to me that a declaration
>> declares the type of a whole object having no containing object.  Then
>> if we have an lvalue that designates some included region of data
>> storage, we can ask, "does that region of data storage have a declared
>> type?"  The answer might be obvious from an inspection of the type of
>> the container.  If multiple answers are possible, we can answer "no
>> declared type."  Then the effective type heuristics for "no declared
>> type" are invoked, involving the last-stored value.  That was my
>> rationale.
>
> I don't think so.  If multiple answers are possible, then either they're
> all correct (i.e. there are multiple declared types), or we
> misunderstood the question.  There's nothing in the standard that would
> imply that a union member may or may not have a declared type depending
> on whether its overlap with another member is exact or not.
>
>>>> However, the 'memset' wasn't working with only the first 4 bytes.  The
>>>> 'memset' did more than that.  It involved '&u' and involved all of the
>>>> bytes, so it produced a union value.  I believe that union value could
>>>> be used with another union of the same type with 'union u u2 = u;' or
>>>> 'union u u2; u2 = u;', for example.
>>>>
>>>> 'u.ls.l', 'u.u' and 'u.f' designate members which do not have a
>>>> last-stored value, and no union member was used for the last-stored
>>>> value, so I'd say that evaluating any of them yields undefined
>>>> behaviour.  (!!)
>>>
>>> My first guess would be that accessing any if of the integers using its
>>> declared type should be safe, because the memset has put a valid
>>> representation of zero in it.
>>
>> Which takes precedence?  The type-punning foot-note, or the "shall" in
>> 6.5p7?
>
> Footnotes are not normative.
>
>>>> Because there is no padding for meta-data in your example, you could
>>>> circumvent the undefined behaviour with any kind of code that makes it
>>>> impossible to prove any claim about such meta-data (such as whether or
>>>> not a member was used to store a value), yielding defined behaviour.
>>>
>>> I'm not sure if I understand what you're saying here, but just because
>>> there's no padding to store extra information at runtime doesn't mean
>>> that the implementation can't track that information at compile time, or
>>> maybe sometimes outside of the object.  Even if you make it hard for
>>> implementations to figure out that you're breaking the rules, that does
>>> not make the behaviour defined.
>>>
>>
>> I didn't say "hard," I said "impossible." :)  When there isn't any room
>> in the object representation for meta-data, you can send the object to a
>> friend, not know if or how they tampered with it, then read it back.
>> Some beliefs about anything other than the object representation would
>> seem unfounded, after that.
>>
>> I suppose the implementation could call your friend...
>
> The implementation has seen your source code.  If a person can prove
> that its behaviour is undefined, then an implementation, in principle,
> can too.  There's no rule that undefined behaviour must be provable
> based on object representations only.
>
>>> ...
>>>>> My concern (for now) is not about the contents of the object but about
>>>>> its effective type.  In particular, the "sticky" kind of effective
>>>>> type
>>>>> that can conflict with read accesses using an lvalue with the wrong
>>>>> type, and gets propagated by memcpying to other typeless objects.  In
>>>>> other words, whether one of the standard's statements about the
>>>>> effective type "for subsequent accesses that do not modify the value"
>>>>> apply to it or not.
>>>>
>>>> We share the same concern.  Some of the considerations of the Standard
>>>> involve subjects which invoke undefined behaviour, but some of those
>>>> subjects appear to be hard to prove, under the right circumstances.
>>>> "Effective type" strikes me as one such subject.  My interest is in an
>>>> implementation which attempts to trap _all_ code which yields undefined
>>>> behaviour and report it.
>>>
>>> Well good luck with that.  I'm afraid the best you can possibly do is
>>> detect and report code that has undefined behaviour *according to your
>>> interpretation of the standard*.
>>
>> And so the reason for the original post is revealed: To find out if
>> there's common ground.
>>
>>> ...
>>>>> Hey, here's another one:
>>>>>
>>>>>    #define N sizeof(float)
>>>>>    float x = 1.0;
>>>>>    char y[ N ];
>>>>>    float *p = memcpy( malloc(N), memcpy( y, &x, N ), N );
>>>>>    // Does *p have a "sticky" effective type now?  Is it char or
>>>>> float?
>>>>>    float z = *p; // Is this valid?
>>>>
>>>> I would indeed track and report the effective type of the modified
>>>> storage pointed to by 'p' as having effective type 'float'.  An
>>>> array of
>>>> character type is always the case, so it's an uninteresting case (to
>>>> me), if there's more that can be proven about the provenance of the
>>>> "stored value".
>>>
>>> Actually, since y has a declared type, shouldn't it make any traces of
>>> floatness disappear?
>>
>> The effective type of 'y' might be 'char[N]', but the effective type of
>> the object that was copied through it needn't be, since the original is
>> "copied as an array of character type".
>
> There's no such thing as a copy "through" y; there's a copy into y and
> then a separate copy from y.  In the second copy, the source object has
> a declared type, and it's an array of char.  Not float.
>
>>> I think what I really had in mind back then was an unaligned object
>>> without a declared type:
>>>
>>>    char *y = 1 + (char*)malloc(N+1);
>>>
>>
>> Right, I think that goes to show that the character-type intermediary is
>> unaffected.
>
> My goal was to show that alignment is irrelevant.  It's fairly clear in
> this case that the object consisting of the bytes y[1] through y[N] has
> an effective type of float, even if it can't be accessed as a float
> because of alignment requirements.
>
> Why then is alignment mentioned at all in DR219?
>

[toc] | [prev] | [standalone]

Back to top | Article view | comp.std.c

csiph-web