Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
| From | Wojtek Lerch <wojtek_l@yahoo.ca> |
|---|---|
| Newsgroups | comp.std.c |
| Subject | Re: Tracking and Reporting Effective Type |
| Date | 2013-01-17 00:32 -0500 |
| Message-ID | <alpgk6Fh1jcU1@mid.individual.net> (permalink) |
| References | <kcovdt$1ef$1@dont-email.me> <alhgt2Fme55U1@mid.individual.net> <kd0732$q7v$1@dont-email.me> <alk7j2Fb7lqU1@mid.individual.net> <kd3bk8$l2o$1@dont-email.me> |
On 15/01/2013 5:39 AM, Shao Miller wrote:
> On 1/15/2013 00:28, Wojtek Lerch wrote:
>> On 14/01/2013 1:03 AM, Shao Miller wrote:
>>> On 1/13/2013 23:48, Wojtek Lerch wrote:
>>>>
>>>> union u {
>>>> struct sl { int16_t s; int32_t l; } sl;
>>>> struct ls { int32_t l; int16_t s; } ls;
>>>> uint32_t u;
>>>> float f;
>>>> };
>>>>
>>>> int main(void) {
>>>> void *p;
>>>> if ( sizeof(union u) == 6 // no padding
>>>> && ( p = malloc(6) ) != NULL ) {
>>>> union u u;
>>>> memset( u, 0, 6 );
>>>> memcpy( p, (void*)&u.sl, 6 );
>>>> ...
>>>> }
>>>> }
...
>>> I think I saw something like this in Defect Report #219.
>>
>> That's a good find, thanks. But there seems to be an inconsistency
>> between the "committee Discussion" (which I believe is non-normative)
>> and the "Committee Response" (which is normative) about the treatment of
>> character types.
>
> Which part/parts is/are inconsistent?
Discussion:
6. memcpy (p1, (char *)&s + offsetof (s, i), len1);
In lines 6 and 7, the effective type of the source arguments
to memcpy is an array of char
The effective type for p1 and p2 in lines 6 and 7 is: array of char.
Response:
memcpy (p1, (char *)&s + offsetof (s, i), len1); copies all
of the bytes of s.i to an alignment suitable for an object
of type int. The effective type of the resulting copy can
be treated as having effective type int.
(Also, I have to say that the "can be treated" part feels out of place
in what is supposed to be a normative response. Is the Committee really
telling us that accessing the object as an int "can be treated" as not
having undefined behaviour?)
>> The DR clarifies one important thing though: it's not only objects named
>> in declarations that have a "declared type", but also their subobjects,
>> such as members or elements. But if that's the case, then it's still not
>> clear to me what happens when a compound object
>
> (I assume you mean "aggregate")
Yes, thank you again.
>> and its element or
>> member, or two members of a union, are exactly the same object. It
>> would make sense to me that the object might have several declared types
>> at the same time, but the standard's discussion of effective types
>> doesn't create the impression that that was considered a possibility,
>> does it?
>
> I don't think I agree. When there's a declared type, I think the
> object's effective type can only be that declared type.
That's what the standard says; but when an object has three declared
types at the same time, how many effective types does it have? In my
example, the expressions "u", "u.sl", and "u.ls" designate the same
6-byte object. Does that object have just one declared type, or does it
have three? As far as I can tell we don't have a clear definition of
"declared type" that would cover members and elements of objects
declared with a union or aggregate type.
> I certainly
> hope that this is the case, else tracking and reporting could be more
> complicated!
I think the priority should be to have a clear definition of what you
want to track, before worrying about how complicated it is to track it.
> If you are talking about how 'int ia[1];' declares an object with
> (sizeof (int) * CHAR_BIT) bits and its declared type seems to be both
> 'int[1]' and 'int', I don't think that's the case. I think the
> effective type here must match the declared type, 'int[1]'. The pointer
> expression 'ia' (after conversion from array type) no longer refers to
> the same object, but refers to its first element.
The array and the element are the same object.
3.15 "object: region of data storage in the execution environment, the
contents of which can represent values. NOTE When referenced, an object
may be interpreted as having a particular type; see 6.3.2.1."
6.3.2.1#1 "When an object is said to have a particular type, the type is
specified by the lvalue used to designate the object."
An object can have a declared type, an effective type, and can be
accessed using lvalues that have various types; but when the same region
of memory is designated by two lvalues that have different types, it's
still one object, not two.
Otherwise, the whole concept of effective types falls apart. If two
accesses to the same region of memory count as accesses to two distinct
objects, then the effective type of one object doesn't affect how the
other object can be accessed, does it? Unless there are clear rules
about when a region of memory counts as the same object for the purpose
of aliasing, the rules of effective types say nothing meaningful about
aliasing.
I would prefer to go one step in the opposite direction, and insist that
the standard should say something about objects that partially overlap.
When an assignment to a subobject of an allocated object gives it a
new effective type, shouldn't it also wipe out any effective types that
overlapping objects may have?
...
> I agree that those two 'char *' pointers address the same byte, but only
> as a consequence of the two objects occupying the same location... So I
> don't quite agree that its only role is to produce a pointer to the
> first byte; I think the pointer has more properties than which byte it
> addresses.
It seems that it does; trouble is, the standard doesn't say much about
what those extra properties are and what happens to them when pointers
are converted. And the few clues that it does offer don't seem very
consistent.
For instance, the description of pointer addition and subtraction
strongly implies that every pointer to an object has a range of integers
associated with it that can be added to it without causing undefined
behaviour. On the other hand, the discussion of pointer conversions
seems to think it's sufficient to guararantee that a resulting pointer
compares equal to some other pointer, without saying anything about the
associated range of integers. But without knowing that range, there
isn't much we can safely do with a pointer -- in particular,
dereferencing, incrementing, or decrementing a pointer may invoke
undefined behaviour, even if we know that the pointer compares equal to
some other pointer that can be safely dereferenced, incremented, or
decremented.
The concept of pointer provenance, insinuated by DR260, is hopelessly
underspecified too: we're supposed to rely on it to somehow figure out
what operations have a defined behaviour and which ones don't, from a
few vague sentences that talk about what implementations are permitted
to track, without much of an explanation of how those permissions affect
the guarantees given to programs.
>> Imagine that u is a union of two different structures x and y, and it
>> happens that u.x.r and u.y.s have the same offsets and sizes. Would you
>> claim that
>>
>> memcpy( p, (char*)&u + offsetof(struct x, r), sizeof(u.x.r) );
>>
>> and
>>
>> memcpy( p, (char*)&u + offsetof(struct y, s), sizeof(u.y.s) );
>>
>> copy different effective types into p, even if they pass exactly the
>> same values to memcpy?
>
> This case depends on the offset and number of bytes copied, to me.
>
> '(char *) &u' addresses the first byte of 'u'. If the offset is 0 and
> the number of bytes equals 'sizeof u', then I'd say the effective type
> of the modified storage is the union type.
OK. The case where the offset is nonzero is more interesting.
> For any other case of offset or number of bytes, I'd ask "which object
> is being copied?" The answer might depend on the last-stored value, for
> union involvement.
Only if the object doesn't have a declared type. If it does have a
declared type, then that's its effective type too, and storing values
doesn't affect it, right?
> Regardless of what the answer depends on, if there
> are multiple answers, then all we know for certain is that a
> 'char[num_of_bytes]' is being copied. I think that answer could be a
> default, in the case where there is no single better answer.
Well I think it's the standard's job to give us a better answer.
>> Remember that an object is just a region of storage; when we talk about
>> its type, that's just a short way of talking about types of lvalues, or
>> declared types, or effective types.
>
> "is just": I think this is consistent with the initial definition of
> 'object' under section 3, but I think the actual definition accumulates
> some more from other parts of the Standard. The discussion of pointer
> arithmetic, for example, distinguishes between an "array object" and its
> "array elements", but these different objects can exactly overlap.
Sure; but we're never told whether this "arrayness" is based on some
properties of the object, such as its declared or effective type, or on
the lvalue expression that the pointer originated from. I tend to think
it's the latter, not only because that seems consistent with the need to
talk about the provenance of pointers, but also because it applies to
math on pointers to memory that has just been allocated and has not been
accessed yet and therefore doesn't have any declared or effective type.
>> Back to my original example, what if I used &u.sl.s in the memcpy?
>
> That pointer addresses 'u.sl.s'. The number of bytes to copy is more
> than that, however. I think this yields undefined behaviour, just as:
>
> *( ((char *) ((void *) &u.sl.s)) + sizeof u.sl.s )
>
> would. (Bounds-checking.)
OK, I can buy that. :)
>>>> Does the four-byte sub-object at the beginning of "u" have an effective
>>>> type?
>> my four-byte object that can (I hope) be accessed using such
>> lvalues as u.ls.l, u.u, and u.f?
>
> In order to determine the effective type, I'd ask "which object(s) do
> the first four bytes refer to?" For a union, I think the answer would
> first depend on the last-stored value.
Only if there's no declared type. Is there?
> The last-stored value of the
> union was not set via any of the members, so that does not resolve the
> answer. After that, there are multiple answers, but none of those
> scores as highly as the type of the value that was stored, making the
> answer "4 bytes of an 'unsigned char[6]'".
Your scoring of multiple answers is not based on anything that the
standard actually says about effective types, is it?
> However, the 'memset' wasn't working with only the first 4 bytes. The
> 'memset' did more than that. It involved '&u' and involved all of the
> bytes, so it produced a union value. I believe that union value could
> be used with another union of the same type with 'union u u2 = u;' or
> 'union u u2; u2 = u;', for example.
>
> 'u.ls.l', 'u.u' and 'u.f' designate members which do not have a
> last-stored value, and no union member was used for the last-stored
> value, so I'd say that evaluating any of them yields undefined
> behaviour. (!!)
My first guess would be that accessing any if of the integers using its
declared type should be safe, because the memset has put a valid
representation of zero in it.
> Because there is no padding for meta-data in your example, you could
> circumvent the undefined behaviour with any kind of code that makes it
> impossible to prove any claim about such meta-data (such as whether or
> not a member was used to store a value), yielding defined behaviour.
I'm not sure if I understand what you're saying here, but just because
there's no padding to store extra information at runtime doesn't mean
that the implementation can't track that information at compile time, or
maybe sometimes outside of the object. Even if you make it hard for
implementations to figure out that you're breaking the rules, that does
not make the behaviour defined.
...
>> My concern (for now) is not about the contents of the object but about
>> its effective type. In particular, the "sticky" kind of effective type
>> that can conflict with read accesses using an lvalue with the wrong
>> type, and gets propagated by memcpying to other typeless objects. In
>> other words, whether one of the standard's statements about the
>> effective type "for subsequent accesses that do not modify the value"
>> apply to it or not.
>
> We share the same concern. Some of the considerations of the Standard
> involve subjects which invoke undefined behaviour, but some of those
> subjects appear to be hard to prove, under the right circumstances.
> "Effective type" strikes me as one such subject. My interest is in an
> implementation which attempts to trap _all_ code which yields undefined
> behaviour and report it.
Well good luck with that. I'm afraid the best you can possibly do is
detect and report code that has undefined behaviour *according to your
interpretation of the standard*.
...
>> Hey, here's another one:
>>
>> #define N sizeof(float)
>> float x = 1.0;
>> char y[ N ];
>> float *p = memcpy( malloc(N), memcpy( y, &x, N ), N );
>> // Does *p have a "sticky" effective type now? Is it char or float?
>> float z = *p; // Is this valid?
>>
>
> I would indeed track and report the effective type of the modified
> storage pointed to by 'p' as having effective type 'float'. An array of
> character type is always the case, so it's an uninteresting case (to
> me), if there's more that can be proven about the provenance of the
> "stored value".
Actually, since y has a declared type, shouldn't it make any traces of
floatness disappear?
I think what I really had in mind back then was an unaligned object
without a declared type:
char *y = 1 + malloc(N+1);
Back to comp.std.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Tracking and Reporting Effective Type Shao Miller <sha0.miller@gmail.com> - 2013-01-11 07:09 -0500
Re: Tracking and Reporting Effective Type Wojtek Lerch <wojtek_l@yahoo.ca> - 2013-01-13 23:48 -0500
Re: Tracking and Reporting Effective Type Shao Miller <sha0.miller@gmail.com> - 2013-01-14 01:03 -0500
Re: Tracking and Reporting Effective Type Wojtek Lerch <wojtek_l@yahoo.ca> - 2013-01-15 00:28 -0500
Re: Tracking and Reporting Effective Type Shao Miller <sha0.miller@gmail.com> - 2013-01-15 05:39 -0500
Re: Tracking and Reporting Effective Type Wojtek Lerch <wojtek_l@yahoo.ca> - 2013-01-17 00:32 -0500
Re: Tracking and Reporting Effective Type Wojtek Lerch <wojtek_l@yahoo.ca> - 2013-01-17 01:23 -0500
Re: Tracking and Reporting Effective Type Shao Miller <sha0.miller@gmail.com> - 2013-01-17 22:47 -0500
Re: Tracking and Reporting Effective Type Wojtek Lerch <wojtek_l@yahoo.ca> - 2013-01-18 16:14 -0500
Re: Tracking and Reporting Effective Type Wojtek Lerch <wojtek_l@yahoo.ca> - 2013-01-18 16:21 -0500
csiph-web