Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.std.c > #1860

Re: Tracking and Reporting Effective Type

Show all headers | View raw

Apologies for my lousy editing -- I meant to save a draft but pushed the 
send button instead...

On 18-Jan-13 4:14 PM, Wojtek Lerch wrote:
> On 17-Jan-13 10:47 PM, Shao Miller wrote:
>> On 1/17/2013 00:32, Wojtek Lerch wrote:
>>> On 15/01/2013 5:39 AM, Shao Miller wrote:
>>>> On 1/15/2013 00:28, Wojtek Lerch wrote:
>>>>> On 14/01/2013 1:03 AM, Shao Miller wrote:
>>>>>> On 1/13/2013 23:48, Wojtek Lerch wrote:
>>>>>>>
>>>>>>> union u {
>>>>>>>    struct sl { int16_t s; int32_t l; } sl;
>>>>>>>    struct ls { int32_t l; int16_t s; } ls;
>>>>>>>    uint32_t u;
>>>>>>>    float f;
>>>>>>> };
>>>>>>>
>>>>>>> int main(void) {
>>>>>>>    void *p;
>>>>>>>    if ( sizeof(union u) == 6 // no padding
>>>>>>>      && ( p = malloc(6) ) != NULL ) {
>>>>>>>      union u u;
>>>>>>>      memset( u, 0, 6 );
>>>>>>>      memcpy( p, (void*)&u.sl, 6 );
>>>>>>>      ...
>>>>>>>    }
>>>>>>> }
>>>
>>> ...
>>>
>>>>>>   I think I saw something like this in Defect Report #219.
>>>>>
>>>>> That's a good find, thanks.  But there seems to be an inconsistency
>>>>> between the "committee Discussion" (which I believe is non-normative)
>>>>> and the "Committee Response" (which is normative) about the
>>>>> treatment of
>>>>> character types.
>>>>
>>>> Which part/parts is/are inconsistent?
>>>
>>> Discussion:
>>>
>>>    6. memcpy (p1, (char *)&s + offsetof (s, i), len1);
>>>
>>>    In lines 6 and 7, the effective type of the source arguments
>>>    to memcpy is an array of char
>>>
>>>    The effective type for p1 and p2 in lines 6 and 7 is: array of char.
>>>
>>> Response:
>>>
>>>    memcpy (p1, (char *)&s + offsetof (s, i), len1); copies all
>>>    of the bytes of s.i to an alignment suitable for an object
>>>    of type int. The effective type of the resulting copy can
>>>    be treated as having effective type int.
>>>
>>> (Also, I have to say that the "can be treated" part feels out of place
>>> in what is supposed to be a normative response.  Is the Committee really
>>> telling us that accessing the object as an int "can be treated" as not
>>> having undefined behaviour?)
>>>
>>
>> Ah, ok.  Thank you for clarifying.
>>
>>>>> The DR clarifies one important thing though: it's not only objects
>>>>> named
>>>>> in declarations that have a "declared type", but also their
>>>>> subobjects,
>>>>> such as members or elements.  But if that's the case, then it's still
>>>>> not
>>>>> clear to me what happens when a[n aggregate] object
>>>>> and its element or
>>>>> member, or two members of a union, are exactly the same object.  It
>>>>> would make sense to me that the object might have several declared
>>>>> types
>>>>> at the same time, but the standard's discussion of effective types
>>>>> doesn't create the impression that that was considered a possibility,
>>>>> does it?
>>>>
>>>> I don't think I agree.  When there's a declared type, I think the
>>>> object's effective type can only be that declared type.
>>>
>>> That's what the standard says; but when an object has three declared
>>> types at the same time, how many effective types does it have?  In my
>>> example, the expressions "u", "u.sl", and "u.ls" designate the same
>>> 6-byte object.  Does that object have just one declared type, or does it
>>> have three?  As far as I can tell we don't have a clear definition of
>>> "declared type" that would cover members and elements of objects
>>> declared with a union or aggregate type.
>>>
>>
>> I see your point, if the object is only considered to be the region of
>> data storage and if its entirety is accessible via different lvalues
>> with different declared types.
>>
>>>>    I certainly
>>>> hope that this is the case, else tracking and reporting could be more
>>>> complicated!
>>>
>>> I think the priority should be to have a clear definition of what you
>>> want to track, before worrying about how complicated it is to track it.
>>>
>>
>> You're probably right about that.
>>
>>>> If you are talking about how 'int ia[1];' declares an object with
>>>> (sizeof (int) * CHAR_BIT) bits and its declared type seems to be both
>>>> 'int[1]' and 'int', I don't think that's the case.  I think the
>>>> effective type here must match the declared type, 'int[1]'.  The
>>>> pointer
>>>> expression 'ia' (after conversion from array type) no longer refers to
>>>> the same object, but refers to its first element.
>>>
>>> The array and the element are the same object.
>>>
>>> 3.15 "object: region of data storage in the execution environment, the
>>> contents of which can represent values.  NOTE When referenced, an object
>>> may be interpreted as having a particular type; see 6.3.2.1."
>>>
>>> 6.3.2.1#1 "When an object is said to have a particular type, the type is
>>> specified by the lvalue used to designate the object."
>>>
>>
>> I suggested that 3.15 was not the whole story in a previous post, below.
>>   But it is certainly possible to work with an interpretation where it
>> is. :)
>>
>>> An object can have a declared type, an effective type, and can be
>>> accessed using lvalues that have various types; but when the same region
>>> of memory is designated by two lvalues that have different types, it's
>>> still one object, not two.
>>>
>>
>> Ok.  Working with that model, an object (as a region of data storage)
>> can be declared with multiple types, so can have multiple effective
>> types.
>>
>>> Otherwise, the whole concept of effective types falls apart.  If two
>>> accesses to the same region of memory count as accesses to two distinct
>>> objects, then the effective type of one object doesn't affect how the
>>> other object can be accessed, does it?  Unless there are clear rules
>>> about when a region of memory counts as the same object for the purpose
>>> of aliasing, the rules of effective types say nothing meaningful about
>>> aliasing.
>>>
>>
>> But multiple effective types for one object has the same problem as a
>> single effective type for each sub-object, right?  It's just a shuffle?
>>
>>> I would prefer to go one step in the opposite direction, and insist that
>>> the standard should say something about objects that partially overlap.
>>>   When an assignment to a subobject of an allocated object gives it a
>>> new effective type, shouldn't it also wipe out any effective types that
>>> overlapping objects may have?
>>>
>>
>> With that strategy, if you had [S|O|M|E| |T|Y|P|E], would a partial
>> erasure yield [?|?[U|I|N|T]?|?|?], or could we continue to believe that
>> the effective type of the whole was unaffected?
>>
>>> ...
>>>> I agree that those two 'char *' pointers address the same byte, but
>>>> only
>>>> as a consequence of the two objects occupying the same location...
>>>> So I
>>>> don't quite agree that its only role is to produce a pointer to the
>>>> first byte; I think the pointer has more properties than which byte it
>>>> addresses.
>>>
>>> It seems that it does; trouble is, the standard doesn't say much about
>>> what those extra properties are and what happens to them when pointers
>>> are converted.  And the few clues that it does offer don't seem very
>>> consistent.
>>>
>>> For instance, the description of pointer addition and subtraction
>>> strongly implies that every pointer to an object has a range of integers
>>> associated with it that can be added to it without causing undefined
>>> behaviour.  On the other hand, the discussion of pointer conversions
>>> seems to think it's sufficient to guararantee that a resulting pointer
>>> compares equal to some other pointer, without saying anything about the
>>> associated range of integers.  But without knowing that range, there
>>> isn't much we can safely do with a pointer -- in particular,
>>> dereferencing, incrementing, or decrementing a pointer may invoke
>>> undefined behaviour, even if we know that the pointer compares equal to
>>> some other pointer that can be safely dereferenced, incremented, or
>>> decremented.
>>>
>>
>> Heheh.
>>
>>> The concept of pointer provenance, insinuated by DR260, is hopelessly
>>> underspecified too: we're supposed to rely on it to somehow figure out
>>> what operations have a defined behaviour and which ones don't, from a
>>> few vague sentences that talk about what implementations are permitted
>>> to track, without much of an explanation of how those permissions affect
>>> the guarantees given to programs.
>>>
>>
>> I just assume DR #260 generally means that the implementation can track
>> as much as possible, such as uninitialized objects.
>
> I think it goes without saying that the implementation can track
> whatever it wants to (in addition to tracking what it's required to
> track).  I think it's pretty obvious that it's nor forbidden to detect
> scenarios leading to of undefined behaviour, and report them or perform
> optimizations based on the assumption that they don't happen.  Stating
> those simple facts does not count as an explanation of which programs or
> scenarios do in fact have undefined behaviour.
>
>>   Later on, DR #338
>> wound up saying so, for register-candidate objects.  Handy for enforcing
>> safety!
>
> Well no, DR338 says that the behaviour in a particular type of scenarios
> is undefined.  Implementations were already permitted to detect those
> scenarios before they were undefined.  Many do, for instance for the
> purpose of printing out a warning.
>
> ...
>> Right.  If the outermost containing object was declared in such a way
>> that some particular sub-object (from offset and number of bytes) could
>> only correspond to one sub-object of the outermost type, then I'd say
>> that sub-object had a declared type.  If there were multiple
>> possibilities, I'd ask, "well which one is consistent with the
>> programmer's intention, given the last-stored value?"
>
> Why?  There's no wording in the standard that would make the answer to
> that question relevant to the declared type of any object.  If there are
> multiple possibilities, that just means that either the standard either
> forgot to explain what happens when there are multiple possibilities, or
> our assumption that subobjects have declared types was wrong.  You and I
> can try to come up with additional rules that fill these blanks, but
> that's not very useful unless we convince the Committee to add them to
> the standard; I don't know about you, but I don't really have the time
> or determination to try to do that.
>
>> But that's assuming the "outer-most" effective type business (which I
>> think is easiest to track. :) )
>>
>>>>   Regardless of what the answer depends on, if there
>>>> are multiple answers, then all we know for certain is that a
>>>> 'char[num_of_bytes]' is being copied.  I think that answer could be a
>>>> default, in the case where there is no single better answer.
>>>
>>> Well I think it's the standard's job to give us a better answer.
>>
>> I figure a goal of "effective type" is so that an implementation can
>> make assumptions that yield benefit much of the time, but when the
>> programmer is bitten, a rule can be shown as having been violated.
>
> Right.  That's why we need clear and unambiguous rules.  In the
> standard.  So that we don't have to wonder what answer to choose when
> the standard doesn't give you a single answer to a question that it's
> the standard's job to answer.
>
>>>>> Remember that an object is just a region of storage; when we talk
>>>>> about
>>>>> its type, that's just a short way of talking about types of
>>>>> lvalues, or
>>>>> declared types, or effective types.
>>>>
>>>> "is just": I think this is consistent with the initial definition of
>>>> 'object' under section 3, but I think the actual definition accumulates
>>>> some more from other parts of the Standard.  The discussion of pointer
>>>> arithmetic, for example, distinguishes between an "array object" and
>>>> its
>>>> "array elements", but these different objects can exactly overlap.
>>>
>>> Sure; but we're never told whether this "arrayness" is based on some
>>> properties of the object, such as its declared or effective type, or on
>>> the lvalue expression that the pointer originated from.  I tend to think
>>> it's the latter, not only because that seems consistent with the need to
>>> talk about the provenance of pointers, but also because it applies to
>>> math on pointers to memory that has just been allocated and has not been
>>> accessed yet and therefore doesn't have any declared or effective type.
>>>
>>
>> An object "can be treated" as an array, in some cases, as you say. :)
>
>>>>>>> Does the four-byte sub-object at the beginning of "u" have an
>>>>>>> effective
>>>>>>> type?
>>>>>  my four-byte object that can (I hope) be accessed using such
>>>>> lvalues as u.ls.l, u.u, and u.f?
>>>>
>>>> In order to determine the effective type, I'd ask "which object(s) do
>>>> the first four bytes refer to?"  For a union, I think the answer would
>>>> first depend on the last-stored value.
>>>
>>> Only if there's no declared type.  Is there?
>>
>> There is an outer-most declared type for the containing object, but
>> there isn't a one-to-one correspondence for these four bytes with a
>> sub-object.
>
> So what does that tell us about the intended meaning of "declared type"
> of a subobject?  It seems that the Committee didn't think that the
> concept is tricky enough to warrant a written definition; do you think
> they though it was obvious that a union member may or may not a declared
> type depending on whether it precisely overlaps with another member of
> the whole union?  Or do you think it's more likely that they simply
> forgot about unions?
>
> Another possibility is that "declared type" only applies to the whole
> object named in a declaration; but DR319 seems inconsistent with that
> interpretation.
>
> ...
>> Actually, yes (I had hoped).  It is only clear to me that a declaration
>> declares the type of a whole object having no containing object.  Then
>> if we have an lvalue that designates some included region of data
>> storage, we can ask, "does that region of data storage have a declared
>> type?"  The answer might be obvious from an inspection of the type of
>> the container.  If multiple answers are possible, we can answer "no
>> declared type."  Then the effective type heuristics for "no declared
>> type" are invoked, involving the last-stored value.  That was my
>> rationale.
>
> I don't think so.  If multiple answers are possible, then either they're
> all correct (i.e. there are multiple declared types), or we
> misunderstood the question.  There's nothing in the standard that would
> imply that a union member may or may not have a declared type depending
> on whether its overlap with another member is exact or not.
>
>>>> However, the 'memset' wasn't working with only the first 4 bytes.  The
>>>> 'memset' did more than that.  It involved '&u' and involved all of the
>>>> bytes, so it produced a union value.  I believe that union value could
>>>> be used with another union of the same type with 'union u u2 = u;' or
>>>> 'union u u2; u2 = u;', for example.
>>>>
>>>> 'u.ls.l', 'u.u' and 'u.f' designate members which do not have a
>>>> last-stored value, and no union member was used for the last-stored
>>>> value, so I'd say that evaluating any of them yields undefined
>>>> behaviour.  (!!)
>>>
>>> My first guess would be that accessing any if of the integers using its
>>> declared type should be safe, because the memset has put a valid
>>> representation of zero in it.
>>
>> Which takes precedence?  The type-punning foot-note, or the "shall" in
>> 6.5p7?
>
> Footnotes are not normative.
>
>>>> Because there is no padding for meta-data in your example, you could
>>>> circumvent the undefined behaviour with any kind of code that makes it
>>>> impossible to prove any claim about such meta-data (such as whether or
>>>> not a member was used to store a value), yielding defined behaviour.
>>>
>>> I'm not sure if I understand what you're saying here, but just because
>>> there's no padding to store extra information at runtime doesn't mean
>>> that the implementation can't track that information at compile time, or
>>> maybe sometimes outside of the object.  Even if you make it hard for
>>> implementations to figure out that you're breaking the rules, that does
>>> not make the behaviour defined.
>>>
>>
>> I didn't say "hard," I said "impossible." :)  When there isn't any room
>> in the object representation for meta-data, you can send the object to a
>> friend, not know if or how they tampered with it, then read it back.
>> Some beliefs about anything other than the object representation would
>> seem unfounded, after that.
>>
>> I suppose the implementation could call your friend...
>
> The implementation has seen your source code.  If a person can prove
> that its behaviour is undefined, then an implementation, in principle,
> can too.  There's no rule that undefined behaviour must be provable
> based on object representations only.
>
>>> ...
>>>>> My concern (for now) is not about the contents of the object but about
>>>>> its effective type.  In particular, the "sticky" kind of effective
>>>>> type
>>>>> that can conflict with read accesses using an lvalue with the wrong
>>>>> type, and gets propagated by memcpying to other typeless objects.  In
>>>>> other words, whether one of the standard's statements about the
>>>>> effective type "for subsequent accesses that do not modify the value"
>>>>> apply to it or not.
>>>>
>>>> We share the same concern.  Some of the considerations of the Standard
>>>> involve subjects which invoke undefined behaviour, but some of those
>>>> subjects appear to be hard to prove, under the right circumstances.
>>>> "Effective type" strikes me as one such subject.  My interest is in an
>>>> implementation which attempts to trap _all_ code which yields undefined
>>>> behaviour and report it.
>>>
>>> Well good luck with that.  I'm afraid the best you can possibly do is
>>> detect and report code that has undefined behaviour *according to your
>>> interpretation of the standard*.
>>
>> And so the reason for the original post is revealed: To find out if
>> there's common ground.
>>
>>> ...
>>>>> Hey, here's another one:
>>>>>
>>>>>    #define N sizeof(float)
>>>>>    float x = 1.0;
>>>>>    char y[ N ];
>>>>>    float *p = memcpy( malloc(N), memcpy( y, &x, N ), N );
>>>>>    // Does *p have a "sticky" effective type now?  Is it char or
>>>>> float?
>>>>>    float z = *p; // Is this valid?
>>>>
>>>> I would indeed track and report the effective type of the modified
>>>> storage pointed to by 'p' as having effective type 'float'.  An
>>>> array of
>>>> character type is always the case, so it's an uninteresting case (to
>>>> me), if there's more that can be proven about the provenance of the
>>>> "stored value".
>>>
>>> Actually, since y has a declared type, shouldn't it make any traces of
>>> floatness disappear?
>>
>> The effective type of 'y' might be 'char[N]', but the effective type of
>> the object that was copied through it needn't be, since the original is
>> "copied as an array of character type".
>
> There's no such thing as a copy "through" y; there's a copy into y and
> then a separate copy from y.  In the second copy, the source object has
> a declared type, and it's an array of char.  Not float.
>
>>> I think what I really had in mind back then was an unaligned object
>>> without a declared type:
>>>
>>>    char *y = 1 + (char*)malloc(N+1);
>>>
>>
>> Right, I think that goes to show that the character-type intermediary is
>> unaffected.
>
> My goal was to show that alignment is irrelevant.  It's fairly clear in
> this case that the object consisting of the bytes y[1] through y[N] has
> an effective type of float, even if it can't be accessed as a float
> because of alignment requirements.
>
> Why then is alignment mentioned at all in DR219?
>

Back to comp.std.c | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

Tracking and Reporting Effective Type Shao Miller <sha0.miller@gmail.com> - 2013-01-11 07:09 -0500
  Re: Tracking and Reporting Effective Type Wojtek Lerch <wojtek_l@yahoo.ca> - 2013-01-13 23:48 -0500
    Re: Tracking and Reporting Effective Type Shao Miller <sha0.miller@gmail.com> - 2013-01-14 01:03 -0500
      Re: Tracking and Reporting Effective Type Wojtek Lerch <wojtek_l@yahoo.ca> - 2013-01-15 00:28 -0500
        Re: Tracking and Reporting Effective Type Shao Miller <sha0.miller@gmail.com> - 2013-01-15 05:39 -0500
          Re: Tracking and Reporting Effective Type Wojtek Lerch <wojtek_l@yahoo.ca> - 2013-01-17 00:32 -0500
            Re: Tracking and Reporting Effective Type Wojtek Lerch <wojtek_l@yahoo.ca> - 2013-01-17 01:23 -0500
            Re: Tracking and Reporting Effective Type Shao Miller <sha0.miller@gmail.com> - 2013-01-17 22:47 -0500
              Re: Tracking and Reporting Effective Type Wojtek Lerch <wojtek_l@yahoo.ca> - 2013-01-18 16:14 -0500
                Re: Tracking and Reporting Effective Type Wojtek Lerch <wojtek_l@yahoo.ca> - 2013-01-18 16:21 -0500

csiph-web