Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.std.c > #1851

Re: Tracking and Reporting Effective Type

Show all headers | View raw

On 1/17/2013 00:32, Wojtek Lerch wrote:
> On 15/01/2013 5:39 AM, Shao Miller wrote:
>> On 1/15/2013 00:28, Wojtek Lerch wrote:
>>> On 14/01/2013 1:03 AM, Shao Miller wrote:
>>>> On 1/13/2013 23:48, Wojtek Lerch wrote:
>>>>>
>>>>> union u {
>>>>>    struct sl { int16_t s; int32_t l; } sl;
>>>>>    struct ls { int32_t l; int16_t s; } ls;
>>>>>    uint32_t u;
>>>>>    float f;
>>>>> };
>>>>>
>>>>> int main(void) {
>>>>>    void *p;
>>>>>    if ( sizeof(union u) == 6 // no padding
>>>>>      && ( p = malloc(6) ) != NULL ) {
>>>>>      union u u;
>>>>>      memset( u, 0, 6 );
>>>>>      memcpy( p, (void*)&u.sl, 6 );
>>>>>      ...
>>>>>    }
>>>>> }
>
> ...
>
>>>>   I think I saw something like this in Defect Report #219.
>>>
>>> That's a good find, thanks.  But there seems to be an inconsistency
>>> between the "committee Discussion" (which I believe is non-normative)
>>> and the "Committee Response" (which is normative) about the treatment of
>>> character types.
>>
>> Which part/parts is/are inconsistent?
>
> Discussion:
>
>    6. memcpy (p1, (char *)&s + offsetof (s, i), len1);
>
>    In lines 6 and 7, the effective type of the source arguments
>    to memcpy is an array of char
>
>    The effective type for p1 and p2 in lines 6 and 7 is: array of char.
>
> Response:
>
>    memcpy (p1, (char *)&s + offsetof (s, i), len1); copies all
>    of the bytes of s.i to an alignment suitable for an object
>    of type int. The effective type of the resulting copy can
>    be treated as having effective type int.
>
> (Also, I have to say that the "can be treated" part feels out of place
> in what is supposed to be a normative response.  Is the Committee really
> telling us that accessing the object as an int "can be treated" as not
> having undefined behaviour?)
>

Ah, ok.  Thank you for clarifying.

>>> The DR clarifies one important thing though: it's not only objects named
>>> in declarations that have a "declared type", but also their subobjects,
>>> such as members or elements.  But if that's the case, then it's still
>>> not
>>> clear to me what happens when a[n aggregate] object
>>> and its element or
>>> member, or two members of a union, are exactly the same object.  It
>>> would make sense to me that the object might have several declared types
>>> at the same time, but the standard's discussion of effective types
>>> doesn't create the impression that that was considered a possibility,
>>> does it?
>>
>> I don't think I agree.  When there's a declared type, I think the
>> object's effective type can only be that declared type.
>
> That's what the standard says; but when an object has three declared
> types at the same time, how many effective types does it have?  In my
> example, the expressions "u", "u.sl", and "u.ls" designate the same
> 6-byte object.  Does that object have just one declared type, or does it
> have three?  As far as I can tell we don't have a clear definition of
> "declared type" that would cover members and elements of objects
> declared with a union or aggregate type.
>

I see your point, if the object is only considered to be the region of 
data storage and if its entirety is accessible via different lvalues 
with different declared types.

>>    I certainly
>> hope that this is the case, else tracking and reporting could be more
>> complicated!
>
> I think the priority should be to have a clear definition of what you
> want to track, before worrying about how complicated it is to track it.
>

You're probably right about that.

>> If you are talking about how 'int ia[1];' declares an object with
>> (sizeof (int) * CHAR_BIT) bits and its declared type seems to be both
>> 'int[1]' and 'int', I don't think that's the case.  I think the
>> effective type here must match the declared type, 'int[1]'.  The pointer
>> expression 'ia' (after conversion from array type) no longer refers to
>> the same object, but refers to its first element.
>
> The array and the element are the same object.
>
> 3.15 "object: region of data storage in the execution environment, the
> contents of which can represent values.  NOTE When referenced, an object
> may be interpreted as having a particular type; see 6.3.2.1."
>
> 6.3.2.1#1 "When an object is said to have a particular type, the type is
> specified by the lvalue used to designate the object."
>

I suggested that 3.15 was not the whole story in a previous post, below. 
  But it is certainly possible to work with an interpretation where it 
is. :)

> An object can have a declared type, an effective type, and can be
> accessed using lvalues that have various types; but when the same region
> of memory is designated by two lvalues that have different types, it's
> still one object, not two.
>

Ok.  Working with that model, an object (as a region of data storage) 
can be declared with multiple types, so can have multiple effective types.

> Otherwise, the whole concept of effective types falls apart.  If two
> accesses to the same region of memory count as accesses to two distinct
> objects, then the effective type of one object doesn't affect how the
> other object can be accessed, does it?  Unless there are clear rules
> about when a region of memory counts as the same object for the purpose
> of aliasing, the rules of effective types say nothing meaningful about
> aliasing.
>

But multiple effective types for one object has the same problem as a 
single effective type for each sub-object, right?  It's just a shuffle?

> I would prefer to go one step in the opposite direction, and insist that
> the standard should say something about objects that partially overlap.
>   When an assignment to a subobject of an allocated object gives it a
> new effective type, shouldn't it also wipe out any effective types that
> overlapping objects may have?
>

With that strategy, if you had [S|O|M|E| |T|Y|P|E], would a partial 
erasure yield [?|?[U|I|N|T]?|?|?], or could we continue to believe that 
the effective type of the whole was unaffected?

> ...
>> I agree that those two 'char *' pointers address the same byte, but only
>> as a consequence of the two objects occupying the same location...  So I
>> don't quite agree that its only role is to produce a pointer to the
>> first byte; I think the pointer has more properties than which byte it
>> addresses.
>
> It seems that it does; trouble is, the standard doesn't say much about
> what those extra properties are and what happens to them when pointers
> are converted.  And the few clues that it does offer don't seem very
> consistent.
>
> For instance, the description of pointer addition and subtraction
> strongly implies that every pointer to an object has a range of integers
> associated with it that can be added to it without causing undefined
> behaviour.  On the other hand, the discussion of pointer conversions
> seems to think it's sufficient to guararantee that a resulting pointer
> compares equal to some other pointer, without saying anything about the
> associated range of integers.  But without knowing that range, there
> isn't much we can safely do with a pointer -- in particular,
> dereferencing, incrementing, or decrementing a pointer may invoke
> undefined behaviour, even if we know that the pointer compares equal to
> some other pointer that can be safely dereferenced, incremented, or
> decremented.
>

Heheh.

> The concept of pointer provenance, insinuated by DR260, is hopelessly
> underspecified too: we're supposed to rely on it to somehow figure out
> what operations have a defined behaviour and which ones don't, from a
> few vague sentences that talk about what implementations are permitted
> to track, without much of an explanation of how those permissions affect
> the guarantees given to programs.
>

I just assume DR #260 generally means that the implementation can track 
as much as possible, such as uninitialized objects.  Later on, DR #338 
wound up saying so, for register-candidate objects.  Handy for enforcing 
safety!

>> For any other case of offset or number of bytes, I'd ask "which object
>> is being copied?"  The answer might depend on the last-stored value, for
>> union involvement.
>
> Only if the object doesn't have a declared type.  If it does have a
> declared type, then that's its effective type too, and storing values
> doesn't affect it, right?
>

Right.  If the outermost containing object was declared in such a way 
that some particular sub-object (from offset and number of bytes) could 
only correspond to one sub-object of the outermost type, then I'd say 
that sub-object had a declared type.  If there were multiple 
possibilities, I'd ask, "well which one is consistent with the 
programmer's intention, given the last-stored value?"

But that's assuming the "outer-most" effective type business (which I 
think is easiest to track. :) )

>>   Regardless of what the answer depends on, if there
>> are multiple answers, then all we know for certain is that a
>> 'char[num_of_bytes]' is being copied.  I think that answer could be a
>> default, in the case where there is no single better answer.
>
> Well I think it's the standard's job to give us a better answer.
>

I figure a goal of "effective type" is so that an implementation can 
make assumptions that yield benefit much of the time, but when the 
programmer is bitten, a rule can be shown as having been violated.

>>> Remember that an object is just a region of storage; when we talk about
>>> its type, that's just a short way of talking about types of lvalues, or
>>> declared types, or effective types.
>>
>> "is just": I think this is consistent with the initial definition of
>> 'object' under section 3, but I think the actual definition accumulates
>> some more from other parts of the Standard.  The discussion of pointer
>> arithmetic, for example, distinguishes between an "array object" and its
>> "array elements", but these different objects can exactly overlap.
>
> Sure; but we're never told whether this "arrayness" is based on some
> properties of the object, such as its declared or effective type, or on
> the lvalue expression that the pointer originated from.  I tend to think
> it's the latter, not only because that seems consistent with the need to
> talk about the provenance of pointers, but also because it applies to
> math on pointers to memory that has just been allocated and has not been
> accessed yet and therefore doesn't have any declared or effective type.
>

An object "can be treated" as an array, in some cases, as you say. :)

>>>>> Does the four-byte sub-object at the beginning of "u" have an
>>>>> effective
>>>>> type?
>>>  my four-byte object that can (I hope) be accessed using such
>>> lvalues as u.ls.l, u.u, and u.f?
>>
>> In order to determine the effective type, I'd ask "which object(s) do
>> the first four bytes refer to?"  For a union, I think the answer would
>> first depend on the last-stored value.
>
> Only if there's no declared type.  Is there?
>

There is an outer-most declared type for the containing object, but 
there isn't a one-to-one correspondence for these four bytes with a 
sub-object.

>>   The last-stored value of the
>> union was not set via any of the members, so that does not resolve the
>> answer.  After that, there are multiple answers, but none of those
>> scores as highly as the type of the value that was stored, making the
>> answer "4 bytes of an 'unsigned char[6]'".
>
> Your scoring of multiple answers is not based on anything that the
> standard actually says about effective types, is it?
>

Actually, yes (I had hoped).  It is only clear to me that a declaration 
declares the type of a whole object having no containing object.  Then 
if we have an lvalue that designates some included region of data 
storage, we can ask, "does that region of data storage have a declared 
type?"  The answer might be obvious from an inspection of the type of 
the container.  If multiple answers are possible, we can answer "no 
declared type."  Then the effective type heuristics for "no declared 
type" are invoked, involving the last-stored value.  That was my rationale.

>> However, the 'memset' wasn't working with only the first 4 bytes.  The
>> 'memset' did more than that.  It involved '&u' and involved all of the
>> bytes, so it produced a union value.  I believe that union value could
>> be used with another union of the same type with 'union u u2 = u;' or
>> 'union u u2; u2 = u;', for example.
>>
>> 'u.ls.l', 'u.u' and 'u.f' designate members which do not have a
>> last-stored value, and no union member was used for the last-stored
>> value, so I'd say that evaluating any of them yields undefined
>> behaviour.  (!!)
>
> My first guess would be that accessing any if of the integers using its
> declared type should be safe, because the memset has put a valid
> representation of zero in it.
>

Which takes precedence?  The type-punning foot-note, or the "shall" in 
6.5p7?

>> Because there is no padding for meta-data in your example, you could
>> circumvent the undefined behaviour with any kind of code that makes it
>> impossible to prove any claim about such meta-data (such as whether or
>> not a member was used to store a value), yielding defined behaviour.
>
> I'm not sure if I understand what you're saying here, but just because
> there's no padding to store extra information at runtime doesn't mean
> that the implementation can't track that information at compile time, or
> maybe sometimes outside of the object.  Even if you make it hard for
> implementations to figure out that you're breaking the rules, that does
> not make the behaviour defined.
>

I didn't say "hard," I said "impossible." :)  When there isn't any room 
in the object representation for meta-data, you can send the object to a 
friend, not know if or how they tampered with it, then read it back. 
Some beliefs about anything other than the object representation would 
seem unfounded, after that.

I suppose the implementation could call your friend...

> ...
>>> My concern (for now) is not about the contents of the object but about
>>> its effective type.  In particular, the "sticky" kind of effective type
>>> that can conflict with read accesses using an lvalue with the wrong
>>> type, and gets propagated by memcpying to other typeless objects.  In
>>> other words, whether one of the standard's statements about the
>>> effective type "for subsequent accesses that do not modify the value"
>>> apply to it or not.
>>
>> We share the same concern.  Some of the considerations of the Standard
>> involve subjects which invoke undefined behaviour, but some of those
>> subjects appear to be hard to prove, under the right circumstances.
>> "Effective type" strikes me as one such subject.  My interest is in an
>> implementation which attempts to trap _all_ code which yields undefined
>> behaviour and report it.
>
> Well good luck with that.  I'm afraid the best you can possibly do is
> detect and report code that has undefined behaviour *according to your
> interpretation of the standard*.
>

And so the reason for the original post is revealed: To find out if 
there's common ground.

> ...
>>> Hey, here's another one:
>>>
>>>    #define N sizeof(float)
>>>    float x = 1.0;
>>>    char y[ N ];
>>>    float *p = memcpy( malloc(N), memcpy( y, &x, N ), N );
>>>    // Does *p have a "sticky" effective type now?  Is it char or float?
>>>    float z = *p; // Is this valid?
>>>
>>
>> I would indeed track and report the effective type of the modified
>> storage pointed to by 'p' as having effective type 'float'.  An array of
>> character type is always the case, so it's an uninteresting case (to
>> me), if there's more that can be proven about the provenance of the
>> "stored value".
>
> Actually, since y has a declared type, shouldn't it make any traces of
> floatness disappear?
>

The effective type of 'y' might be 'char[N]', but the effective type of 
the object that was copied through it needn't be, since the original is 
"copied as an array of character type".

> I think what I really had in mind back then was an unaligned object
> without a declared type:
>
>    char *y = 1 + (char*)malloc(N+1);
>

Right, I think that goes to show that the character-type intermediary is 
unaffected.

-- 
- Shao Miller
--
"Thank you for the kind words; those are the kind of words I like to hear.

Cheerily,"  -- Richard Harter

Back to comp.std.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Tracking and Reporting Effective Type Shao Miller <sha0.miller@gmail.com> - 2013-01-11 07:09 -0500
  Re: Tracking and Reporting Effective Type Wojtek Lerch <wojtek_l@yahoo.ca> - 2013-01-13 23:48 -0500
    Re: Tracking and Reporting Effective Type Shao Miller <sha0.miller@gmail.com> - 2013-01-14 01:03 -0500
      Re: Tracking and Reporting Effective Type Wojtek Lerch <wojtek_l@yahoo.ca> - 2013-01-15 00:28 -0500
        Re: Tracking and Reporting Effective Type Shao Miller <sha0.miller@gmail.com> - 2013-01-15 05:39 -0500
          Re: Tracking and Reporting Effective Type Wojtek Lerch <wojtek_l@yahoo.ca> - 2013-01-17 00:32 -0500
            Re: Tracking and Reporting Effective Type Wojtek Lerch <wojtek_l@yahoo.ca> - 2013-01-17 01:23 -0500
            Re: Tracking and Reporting Effective Type Shao Miller <sha0.miller@gmail.com> - 2013-01-17 22:47 -0500
              Re: Tracking and Reporting Effective Type Wojtek Lerch <wojtek_l@yahoo.ca> - 2013-01-18 16:14 -0500
                Re: Tracking and Reporting Effective Type Wojtek Lerch <wojtek_l@yahoo.ca> - 2013-01-18 16:21 -0500

csiph-web