Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #10872 > unrolled thread

partial struct usage

Back to article view | Back to comp.lang.c

Contents

  partial struct usage Mark Adler <madler@alumni.caltech.edu> - 2011-09-24 10:23 -0700
    Re: partial struct usage Richard Damon <news.x.richarddamon@xoxy.net> - 2011-09-24 13:44 -0400
      Re: partial struct usage Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 14:08 -0800
    Re: partial struct usage BGB <cr88192@hotmail.com> - 2011-09-24 11:08 -0700
    Re: partial struct usage Phil Carmody <thefatphil_demunged@yahoo.co.uk> - 2011-09-24 23:36 +0300
      Re: partial struct usage Mark Adler <madler@alumni.caltech.edu> - 2011-09-25 13:23 -0700
        Re: partial struct usage Gene <gene.ressler@gmail.com> - 2011-09-26 01:14 -0700
          Re: partial struct usage Richard Damon <news.x.richarddamon@xoxy.net> - 2011-09-26 12:37 -0400
            Re: partial struct usage Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 16:04 -0800
          Re: partial struct usage Mark Adler <madler@alumni.caltech.edu> - 2011-09-26 18:52 -0700
            Re: partial struct usage Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-09-27 12:14 +0100
      Re: partial struct usage Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-24 15:56 -0800
        Re: partial struct usage Shao Miller <sha0.miller@gmail.com> - 2012-01-24 23:30 -0500
          Re: partial struct usage Tim Rentsch <txr@alumni.caltech.edu> - 2012-01-31 16:11 -0800
            Re: partial struct usage Shao Miller <sha0.miller@gmail.com> - 2012-02-01 00:42 -0500
              Re: partial struct usage Tim Rentsch <txr@alumni.caltech.edu> - 2012-03-08 09:33 -0800
        Re: partial struct usage Phil Carmody <thefatphil_demunged@yahoo.co.uk> - 2012-02-12 02:10 +0200
          Re: partial struct usage Tim Rentsch <txr@alumni.caltech.edu> - 2012-03-08 09:38 -0800
            Re: partial struct usage Phil Carmody <thefatphil_demunged@yahoo.co.uk> - 2012-03-13 18:22 +0200
              Re: partial struct usage blp@cs.stanford.edu (Ben Pfaff) - 2012-03-13 09:55 -0700
    Re: partial struct usage Eric Sosman <esosman@ieee-dot-org.invalid> - 2011-09-24 17:56 -0400

Page 1 of 2  [1] 2  Next page →

#10872 — partial struct usage

I have an internal structure with lots of stuff in it for controlling a 
data input operation.  E.g.:

struct data {
    int have;
    char *next;
    ... lots more stuff to keep track of the state
};

I would like to expose to the user (in the interface header file) only 
those first two elements, so that I can provide a macro (also in the 
header file) analogous to getc() to pull data that is readily available 
in the buffer, otherwise to call a function to go get more data.  But I 
want to hide the other stuff in the structure, in order to make sure 
that users of the library don't end up depending on things that I 
change later for internal use.  And for efficiency I don't want yet 
another layer of indirection pointing to another structure from within 
the exposed structure for the other stuff.

E.g.

struct ex {
    int have;
    char *next;
};

#define getd(d) (((struct ex *)d)->have ? (((struct ex *)d)->have--, 
*(((struct ex *)d)->next)++) : getdf(d))

where d is a struct data *.

My question is: does the C standard provide an assurance that this will 
work?  I.e. that two different structures with identical sets of prefix 
elements will have those prefix elements layed out identically in 
memory so that casting one structure to the other will permit correct 
access to those prefix elements?

Mark

[toc] | [next] | [standalone]

#10875

On 9/24/11 1:23 PM, Mark Adler wrote:
> I have an internal structure with lots of stuff in it for controlling a
> data input operation. E.g.:
>
> struct data {
> int have;
> char *next;
> ... lots more stuff to keep track of the state
> };
>
> I would like to expose to the user (in the interface header file) only
> those first two elements, so that I can provide a macro (also in the
> header file) analogous to getc() to pull data that is readily available
> in the buffer, otherwise to call a function to go get more data. But I
> want to hide the other stuff in the structure, in order to make sure
> that users of the library don't end up depending on things that I change
> later for internal use. And for efficiency I don't want yet another
> layer of indirection pointing to another structure from within the
> exposed structure for the other stuff.
>
> E.g.
>
> struct ex {
> int have;
> char *next;
> };
>
> #define getd(d) (((struct ex *)d)->have ? (((struct ex *)d)->have--,
> *(((struct ex *)d)->next)++) : getdf(d))
>
> where d is a struct data *.
>
> My question is: does the C standard provide an assurance that this will
> work? I.e. that two different structures with identical sets of prefix
> elements will have those prefix elements layed out identically in memory
> so that casting one structure to the other will permit correct access to
> those prefix elements?
>
> Mark
>

If you look at the specifications for unions, you will see a requirement 
for access compatibility for structs with equivalent initial sequences. 
While it technically means it only applies in unions, to make it work 
for unions but not elsewhere is hard, so unless you are using a "debug" 
version of the compiler checking for this sort of "error", it should work.

Another method it to place the "public" members in a separate stuct 
which it the first member of the full struct, then use the fact that the 
address of the first member of a struct is the same as the address of 
the struct (which means the cast gives a usable value). No pointer 
indirection costs, (just a level of . syntax).

[toc] | [prev] | [next] | [standalone]

#16133

Richard Damon <news.x.richarddamon@xoxy.net> writes:

> On 9/24/11 1:23 PM, Mark Adler wrote:
>> I have an internal structure with lots of stuff in it for controlling a
>> data input operation. E.g.:
>>
>> struct data {
>> int have;
>> char *next;
>> ... lots more stuff to keep track of the state
>> };
>>
>> I would like to expose to the user (in the interface header file) only
>> those first two elements, so that I can provide a macro (also in the
>> header file) analogous to getc() to pull data that is readily available
>> in the buffer, otherwise to call a function to go get more data. But I
>> want to hide the other stuff in the structure, in order to make sure
>> that users of the library don't end up depending on things that I change
>> later for internal use. And for efficiency I don't want yet another
>> layer of indirection pointing to another structure from within the
>> exposed structure for the other stuff.
>>
>> E.g.
>>
>> struct ex {
>> int have;
>> char *next;
>> };
>>
>> #define getd(d) (((struct ex *)d)->have ? (((struct ex *)d)->have--,
>> *(((struct ex *)d)->next)++) : getdf(d))
>>
>> where d is a struct data *.
>>
>> My question is: does the C standard provide an assurance that this will
>> work? I.e. that two different structures with identical sets of prefix
>> elements will have those prefix elements layed out identically in memory
>> so that casting one structure to the other will permit correct access to
>> those prefix elements?
>>
>> Mark
>>
>
> If you look at the specifications for unions, you will see a
> requirement for access compatibility for structs with equivalent
> initial sequences. While it technically means it only applies in
> unions, to make it work for unions but not elsewhere is hard, so
> unless you are using a "debug" version of the compiler checking for
> this sort of "error", it should work.  [snip unrelated]

Dangerous advice.  As Eric Sosman correctly points out, identical
layout is not enough in the presence of aggressive (and allowed
under the Standard) optimization.

[toc] | [prev] | [next] | [standalone]

#10876

On 9/24/2011 10:23 AM, Mark Adler wrote:
> I have an internal structure with lots of stuff in it for controlling a
> data input operation. E.g.:
>
> struct data {
> int have;
> char *next;
> ... lots more stuff to keep track of the state
> };
>
> I would like to expose to the user (in the interface header file) only
> those first two elements, so that I can provide a macro (also in the
> header file) analogous to getc() to pull data that is readily available
> in the buffer, otherwise to call a function to go get more data. But I
> want to hide the other stuff in the structure, in order to make sure
> that users of the library don't end up depending on things that I change
> later for internal use. And for efficiency I don't want yet another
> layer of indirection pointing to another structure from within the
> exposed structure for the other stuff.
>
> E.g.
>
> struct ex {
> int have;
> char *next;
> };
>
> #define getd(d) (((struct ex *)d)->have ? (((struct ex *)d)->have--,
> *(((struct ex *)d)->next)++) : getdf(d))
>
> where d is a struct data *.
>
> My question is: does the C standard provide an assurance that this will
> work? I.e. that two different structures with identical sets of prefix
> elements will have those prefix elements layed out identically in memory
> so that casting one structure to the other will permit correct access to
> those prefix elements?
>

I think, technically, no...
but off-hand, I don't know of any implementations where it wont work.

however, if the structs are physically nested, like:
struct mylib_publicstuff_s
{
...
};

struct mylib_privatestuff_s
{
struct mylib_publicstuff_s pu;
...
};

then potentially, one could just use pointer-casts (although, I am not 
certain if this is guaranteed to work either, so alas...).


admittedly, I have not generally dealt with this case, as more commonly 
I expose/hide entire structs (externally, only a basic typedef is 
visible), and would probably just use the pointer indirection.

maybe also possible would be to hide the entire struct, and internally 
make use of getter/setter functions?...

[toc] | [prev] | [next] | [standalone]

#10883

Mark Adler <madler@alumni.caltech.edu> writes:
> I have an internal structure with lots of stuff in it for controlling
> a data input operation.  E.g.:
> 
> struct data {
>     int have;
>     char *next;
>     ... lots more stuff to keep track of the state
> };
> 
> I would like to expose to the user (in the interface header file) only
> those first two elements, so that I can provide a macro (also in the
> header file) analogous to getc() to pull data that is readily
> available in the buffer, otherwise to call a function to go get more
> data.  But I want to hide the other stuff in the structure, in order
> to make sure that users of the library don't end up depending on
> things that I change later for internal use.  And for efficiency I
> don't want yet another layer of indirection pointing to another
> structure from within the exposed structure for the other stuff.
> 
> E.g.
> 
> struct ex {
>     int have;
>     char *next;
> };
> 
> #define getd(d) (((struct ex *)d)->have ? (((struct ex *)d)->have--,
> *(((struct ex *)d)->next)++) : getdf(d))
> 
> where d is a struct data *.
> 
> My question is: does the C standard provide an assurance that this
> will work?  I.e. that two different structures with identical sets of
> prefix elements will have those prefix elements layed out identically
> in memory so that casting one structure to the other will permit
> correct access to those prefix elements?

struct private {
        struct public {
                int have;
                char *next;
        } pub;
        other stuff;
}

let the clients see and pass around pointers to pub, and then either
use something like container_of() (which is not strictly portable), or
just rely on the address of the first member being the address of the
structure and just cast, to map that back to the struct private.

I wonder whether the union of structures with common initial sequences
might be the way of absolutely ensuring the fields are in the same
place:

union private {
        struct public {
                int have;
                char *next;
        } pub;
        struct priv {
                int have;
                char *next;
                other stuff;
        } priv;
};

The address of the union is guaranteed to be the same as the address
of pub and priv, so casting is safe, but in addition, have and next
will always be at the same relative address thanks to 6.5.2.3p6:

"""
6 One special guarantee is made in order to simplify the use of unions: if a union contains
  several structures that share a common initial sequence (see below), and if the union
  object currently contains one of these structures, it is permitted to inspect the common
  initial part of any of them anywhere that a declaration of the completed type of the union
  is visible. Two structures share a common initial sequence if corresponding members
  have compatible types (and, for bit-#elds, the same widths) for a sequence of one or more
  initial members.
"""


Phil

-- 
"Religion is what keeps the poor from murdering the rich."
-- Napoleon

[toc] | [prev] | [next] | [standalone]

#10932

Thank you guys.  I got what I needed.

Mark

[toc] | [prev] | [next] | [standalone]

#10941

On Sep 25, 4:23 pm, Mark Adler <mad...@alumni.caltech.edu> wrote:
> Thank you guys.  I got what I needed.
>
> Mark
>
>

Questions about pointers to struct sharing fields seem popular.  As
others have said there are few compilers that won't allow you to cast
blithely among structs to access common initial elements.

But there is a way to do it that the Standard guarentees will work.
Someone here kindly pointed me to the key phrase.

From the publicly available draft of C99 at
http://www.open-std.org/jtc1/sc22/WG14/www/docs/n1256.pdf (but I think
it's in C90, too):

6.7.2.13 ... A pointer to a structure object, suitably converted,
points to its initial member (or if that member is a
bit-field, then to the unit in which it resides), and vice versa.

In your case you'd expose the initial field (singular, but it can be a
struct) by casting from your own full-powered pointer to that field.
So you'd end up with code like

// what the user sees
struct api_visible_data {
    int have;
    char *next;
};

// what your code can see
struct data {
    // one-element array trick for uniform syntax
    struct api_visible_data user_fields[1];

  ... lots more stuff to keep track of the state

};

struct api_visible_data *get_data( ... )
{
  struct data *the_data = malloc(sizeof(struct data));

  ...  set up the data

  return data->user_fields;
}

void process_data(struct api_visible_data *data_user_fields)
{
  struct data *data = (struct data *)data_user_fields;

  ... use all the fields including
       data->user_fields->have and ->next

}

The need to touch user fields with different syntax can be bad or good
depending on your point of view.

[toc] | [prev] | [next] | [standalone]

#10954

On 9/26/11 4:14 AM, Gene wrote:
>
> Questions about pointers to struct sharing fields seem popular.  As
> others have said there are few compilers that won't allow you to cast
> blithely among structs to access common initial elements.
>
> But there is a way to do it that the Standard guarentees will work.
> Someone here kindly pointed me to the key phrase.
>
>  From the publicly available draft of C99 at
> http://www.open-std.org/jtc1/sc22/WG14/www/docs/n1256.pdf (but I think
> it's in C90, too):
>
> 6.7.2.13 ... A pointer to a structure object, suitably converted,
> points to its initial member (or if that member is a
> bit-field, then to the unit in which it resides), and vice versa.
>

You also get a fairly good promise from 6.5.2.3 para 5

One special guarantee is made in order to simplify the use of unions: if 
a union contains several structures that share a common initial sequence 
(see below), and if the union object currently contains one of these 
structures, it is permitted to inspect the common initial part of any of 
them anywhere that a declaration of the complete type of the union is 
visible. Two structures share a common initial sequence if corresponding 
members have compatible types (and, for bit-fields, the same widths) for 
a sequence of one or more initial members.

While technically only applying to structs within a union, since a 
pointer to the structs can be passed to other code, unless the pointer 
includes information that the struct is actually part of a union, and 
modifies access rules based on that, it will hold for more general 
access. The only case were I would expect the compiler to possible go to 
that effort would be for a heavy debug build. (this is a somewhat common 
idiom).

[toc] | [prev] | [next] | [standalone]

#16141

Richard Damon <news.x.richarddamon@xoxy.net> writes:

> On 9/26/11 4:14 AM, Gene wrote:
>>
>> Questions about pointers to struct sharing fields seem popular.  As
>> others have said there are few compilers that won't allow you to cast
>> blithely among structs to access common initial elements.
>>
>> But there is a way to do it that the Standard guarentees will work.
>> Someone here kindly pointed me to the key phrase.
>>
>>  From the publicly available draft of C99 at
>> http://www.open-std.org/jtc1/sc22/WG14/www/docs/n1256.pdf (but I think
>> it's in C90, too):
>>
>> 6.7.2.13 ... A pointer to a structure object, suitably converted,
>> points to its initial member (or if that member is a
>> bit-field, then to the unit in which it resides), and vice versa.
>>
>
> You also get a fairly good promise from 6.5.2.3 para 5
>
> One special guarantee is made in order to simplify the use of unions:
> if a union contains several structures that share a common initial
> sequence (see below), and if the union object currently contains one
> of these structures, it is permitted to inspect the common initial
> part of any of them anywhere that a declaration of the complete type
> of the union is visible. Two structures share a common initial
> sequence if corresponding members have compatible types (and, for
> bit-fields, the same widths) for a sequence of one or more initial
> members.
>
> While technically only applying to structs within a union, since a
> pointer to the structs can be passed to other code, unless the pointer
> includes information that the struct is actually part of a union, and
> modifies access rules based on that, it will hold for more general
> access. [snip]

Technically wrong, because of optimization, but also
bad advice for a more practical reason:  the guarantee
of 6.5.2.3 p5 holds only if the complete type of such
a union is visible at the point of access.  No union
declaration visible means the compiler can freely
assume the two struct types don't intermingle.

[toc] | [prev] | [next] | [standalone]

#10988

Thanks!  6.7.2.13 is now my friend.

On 2011-09-26 01:14:06 -0700, Gene said:
> // what your code can see
> struct data {
>     // one-element array trick for uniform syntax
>     struct api_visible_data user_fields[1];
> 
>   ... lots more stuff to keep track of the state
> 
> };

Why does it need to be or want to be a one-element array?  If I left 
off the "[1]", wouldn't it do the same thing?

Mark

[toc] | [prev] | [next] | [standalone]

#11009

Mark Adler <madler@alumni.caltech.edu> writes:

> Thanks!  6.7.2.13 is now my friend.
>
> On 2011-09-26 01:14:06 -0700, Gene said:
>> // what your code can see
>> struct data {
>>     // one-element array trick for uniform syntax
>>     struct api_visible_data user_fields[1];
>>
>>   ... lots more stuff to keep track of the state
>>
>> };
>
> Why does it need to be or want to be a one-element array?  If I left
> off the "[1]", wouldn't it do the same thing?

It doesn't need to be.  I think the advantage is small -- maybe too
small to be worth the puzzlement that it causes.  The point is that
simply referencing user_fields we get a pointer, and since it's a
pointer to the this object that is being passed around, it may be more
convenient to have this conversion happen automatically.

It's more commonly seen where users have to allocate some types
themselves and pass these things around.  If users of some API are told
to write code like this:

   api_object a, b, c;
   api_int(a, 42);
   api_copy(b, a);
   api_do_calculation(c, a, b);

then just by defining 

  typedef struct huge { /* lots of members */ } api_object[1];

the author can not only ensure that users can declare these things but
that they will be passed using pointers rather than by copying the
structure.

The trouble is that it defeats people's expectation.  These objects
can't now be returned from a function and the fact that the API
functions can modify the object passed is not obvious from the code.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#16140

Phil Carmody <thefatphil_demunged@yahoo.co.uk> writes:

> Mark Adler <madler@alumni.caltech.edu> writes:
>> I have an internal structure with lots of stuff in it for controlling
>> a data input operation.  E.g.:
>> 
>> struct data {
>>     int have;
>>     char *next;
>>     ... lots more stuff to keep track of the state
>> };
>> 
>> I would like to expose to the user (in the interface header file) only
>> those first two elements, so that I can provide a macro (also in the
>> header file) analogous to getc() to pull data that is readily
>> available in the buffer, otherwise to call a function to go get more
>> data.  But I want to hide the other stuff in the structure, in order
>> to make sure that users of the library don't end up depending on
>> things that I change later for internal use.  And for efficiency I
>> don't want yet another layer of indirection pointing to another
>> structure from within the exposed structure for the other stuff.
>> 
>> E.g.
>> 
>> struct ex {
>>     int have;
>>     char *next;
>> };
>> 
>> #define getd(d) (((struct ex *)d)->have ? (((struct ex *)d)->have--,
>> *(((struct ex *)d)->next)++) : getdf(d))
>> 
>> where d is a struct data *.
>> 
>> My question is: does the C standard provide an assurance that this
>> will work?  I.e. that two different structures with identical sets of
>> prefix elements will have those prefix elements layed out identically
>> in memory so that casting one structure to the other will permit
>> correct access to those prefix elements?
>
> struct private {
>         struct public {
>                 int have;
>                 char *next;
>         } pub;
>         other stuff;
> }
>
> let the clients see and pass around pointers to pub, and then either
> use something like container_of() (which is not strictly portable), or

Actually container_of() should be strictly portable provided the
outer structure type being casted to matches the actual object
being referred to.  (This assuming container_of() is defined
properly.)

> just rely on the address of the first member being the address of the
> structure and just cast, to map that back to the struct private.

Using container_of() should be just as portable as this.


> I wonder whether the union of structures with common initial sequences
> might be the way of absolutely ensuring the fields are in the same
> place:
>
> union private {
>         struct public {
>                 int have;
>                 char *next;
>         } pub;
>         struct priv {
>                 int have;
>                 char *next;
>                 other stuff;
>         } priv;
> };
>
> The address of the union is guaranteed to be the same as the address
> of pub and priv, so casting is safe, but in addition, have and next
> will always be at the same relative address thanks to 6.5.2.3p6:
>
> """
> 6 One special guarantee is made in order to simplify the use of unions: if a union contains
>   several structures that share a common initial sequence (see below), and if the union
>   object currently contains one of these structures, it is permitted to inspect the common
>   initial part of any of them anywhere that a declaration of the completed type of the union
>   is visible. Two structures share a common initial sequence if corresponding members
>   have compatible types (and, for bit-#elds, the same widths) for a sequence of one or more
>   initial members.
> """

Certainly corresponding fields in the two structs are going to
have the same offsets.  The question is, are the rules for
effective types being followed?  The guarantees in 6.5.2.3 p6
hold only if the struct(s) in question are actually in a union
object, which wasn't true in the original scenario.  Strictly
speaking it isn't safe to play these kinds of tricks unless the
smaller struct is (recursively) a member of the other (or the
struct(s) in question are in an actual union object).

[toc] | [prev] | [next] | [standalone]

#16171

On 1/24/2012 18:56, Tim Rentsch wrote:
> Phil Carmody<thefatphil_demunged@yahoo.co.uk>  writes:
>
>> Mark Adler<madler@alumni.caltech.edu>  writes:
>>> I have an internal structure with lots of stuff in it for controlling
>>> a data input operation.  E.g.:
>>>
>>> struct data {
>>>      int have;
>>>      char *next;
>>>      ... lots more stuff to keep track of the state
>>> };
>>>
>>> I would like to expose to the user (in the interface header file) only
>>> those first two elements, so that I can provide a macro (also in the
>>> header file) analogous to getc() to pull data that is readily
>>> available in the buffer, otherwise to call a function to go get more
>>> data.  But I want to hide the other stuff in the structure, in order
>>> to make sure that users of the library don't end up depending on
>>> things that I change later for internal use.  And for efficiency I
>>> don't want yet another layer of indirection pointing to another
>>> structure from within the exposed structure for the other stuff.
>>>
>>> E.g.
>>>
>>> struct ex {
>>>      int have;
>>>      char *next;
>>> };
>>>
>>> #define getd(d) (((struct ex *)d)->have ? (((struct ex *)d)->have--,
>>> *(((struct ex *)d)->next)++) : getdf(d))
>>>
>>> where d is a struct data *.
>>>
>>> My question is: does the C standard provide an assurance that this
>>> will work?  I.e. that two different structures with identical sets of
>>> prefix elements will have those prefix elements layed out identically
>>> in memory so that casting one structure to the other will permit
>>> correct access to those prefix elements?
>>
>> struct private {
>>          struct public {
>>                  int have;
>>                  char *next;
>>          } pub;
>>          other stuff;
>> }
>>
>> let the clients see and pass around pointers to pub, and then either
>> use something like container_of() (which is not strictly portable), or
>
> Actually container_of() should be strictly portable provided the
> outer structure type being casted to matches the actual object
> being referred to.  (This assuming container_of() is defined
> properly.)
>
>> just rely on the address of the first member being the address of the
>> structure and just cast, to map that back to the struct private.
>
> Using container_of() should be just as portable as this.
>

For the special case where the pointer points to an object at the 
beginning of any containing sub-object...  If a pointer carries bounds 
information, subtracting from a pointer needn't be defined (apparently).

   struct s_foo {
       int i[1];
       double d[1];
     } * foo = malloc(sizeof *foo);
   double * dp;
   assert(foo);
   dp = foo->d;
   /* dp bounds might != foo bounds.  Uh oh */
   foo = container_of(dp, struct s_foo, d);

-- 
"The stationery store has moved.  Aaargh!"

[toc] | [prev] | [next] | [standalone]

#16536

Shao Miller <sha0.miller@gmail.com> writes:

> On 1/24/2012 18:56, Tim Rentsch wrote:
>> Phil Carmody<thefatphil_demunged@yahoo.co.uk>  writes:
>>
>>> Mark Adler<madler@alumni.caltech.edu>  writes:
>>>> I have an internal structure with lots of stuff in it for controlling
>>>> a data input operation.  E.g.:
>>>>
>>>> struct data {
>>>>      int have;
>>>>      char *next;
>>>>      ... lots more stuff to keep track of the state
>>>> };
>>>>
>>>> I would like to expose to the user (in the interface header file) only
>>>> those first two elements, so that I can provide a macro (also in the
>>>> header file) analogous to getc() to pull data that is readily
>>>> available in the buffer, otherwise to call a function to go get more
>>>> data.  But I want to hide the other stuff in the structure, in order
>>>> to make sure that users of the library don't end up depending on
>>>> things that I change later for internal use.  And for efficiency I
>>>> don't want yet another layer of indirection pointing to another
>>>> structure from within the exposed structure for the other stuff.
>>>>
>>>> E.g.
>>>>
>>>> struct ex {
>>>>      int have;
>>>>      char *next;
>>>> };
>>>>
>>>> #define getd(d) (((struct ex *)d)->have ? (((struct ex *)d)->have--,
>>>> *(((struct ex *)d)->next)++) : getdf(d))
>>>>
>>>> where d is a struct data *.
>>>>
>>>> My question is: does the C standard provide an assurance that this
>>>> will work?  I.e. that two different structures with identical sets of
>>>> prefix elements will have those prefix elements layed out identically
>>>> in memory so that casting one structure to the other will permit
>>>> correct access to those prefix elements?
>>>
>>> struct private {
>>>          struct public {
>>>                  int have;
>>>                  char *next;
>>>          } pub;
>>>          other stuff;
>>> }
>>>
>>> let the clients see and pass around pointers to pub, and then either
>>> use something like container_of() (which is not strictly portable), or
>>
>> Actually container_of() should be strictly portable provided the
>> outer structure type being casted to matches the actual object
>> being referred to.  (This assuming container_of() is defined
>> properly.)
>>
>>> just rely on the address of the first member being the address of the
>>> structure and just cast, to map that back to the struct private.
>>
>> Using container_of() should be just as portable as this.
>>
>
> For the special case where the pointer points to an object at the
> beginning of any containing sub-object...  If a pointer carries bounds
> information, subtracting from a pointer needn't be defined
> (apparently).
>
>   struct s_foo {
>       int i[1];
>       double d[1];
>     } * foo = malloc(sizeof *foo);
>   double * dp;
>   assert(foo);
>   dp = foo->d;
>   /* dp bounds might != foo bounds.  Uh oh */
>   foo = container_of(dp, struct s_foo, d);

The same argument applies to members that aren't arrays,
which shows the argument must be wrong, since the whole
point of offsetof() is to be able to convert back and
forth between pointers to member objects and a pointer
to the struct as a whole.

[toc] | [prev] | [next] | [standalone]

#16558

On 1/31/2012 19:11, Tim Rentsch wrote:
> Shao Miller<sha0.miller@gmail.com>  writes:
>
>> On 1/24/2012 18:56, Tim Rentsch wrote:
>>>
>>> Using container_of() should be just as portable as this.
>>>
>>
>> For the special case where the pointer points to an object at the
>> beginning of any containing sub-object...  If a pointer carries bounds
>> information, subtracting from a pointer needn't be defined
>> (apparently).
>>
>>    struct s_foo {
>>        int i[1];
>>        double d[1];
>>      } * foo = malloc(sizeof *foo);
>>    double * dp;
>>    assert(foo);
>>    dp = foo->d;
>>    /* dp bounds might != foo bounds.  Uh oh */
>>    foo = container_of(dp, struct s_foo, d);
>
> The same argument applies to members that aren't arrays,
> which shows the argument must be wrong, since

The argument _does_ apply to members that aren't arrays, but I've seen 
especial attention regarding "arrays" with "bounds-checking," so the 
example highlights that.

Why would a bounds-checking implementation _not_ be allowed to 
continually reduce the possible bounds, but never increase them?  Say if 
pointer representation went something like:

   struct s_ptr {
       unsigned char * first_byte;
       size_t size;
       size_t current_position;
     };

"internally" (C pseudo-code for the implementation of the abstract 
machine).  So imagine, in the program for consideration, we have:

   struct s_foo {
       int i;
       double d;
     };
   struct s_foo foo = { 0 };
   double * d = &foo.d;

"Internally," 'd' might have the members populated with values 
equivalent to '(unsigned char *) &foo + offsetof(struct s_foo, d)', 
'sizeof foo.d', '0', respectively.

Suppose we cast:

   unsigned char * bytes_of_d = (unsigned char *) d;

Suppose this 'bytes_of_d' now has the same "internal" member values as 
'd'.  If you try to:

   unsigned char * backwards = bytes_of_d - 1;

You've gone out-of-bounds.  If, instead, you'd been using an 'unsigned 
char *' cast from a pointer to the whole of 'foo', then the bounds would 
be sufficient to point anywhere within it.

> the whole
> point of offsetof() is to be able to convert back and
> forth between pointers to member objects and a pointer
> to the struct as a whole.

I'm not sure why you suggest this.  The whole point of 'offsetof' is to 
provide an integer constant expression giving the offset of a 'struct' 
member, as far as the standard text goes.  You might have some Committee 
experiences that grant you knowledge of motives beyond that, but those 
aren't widely accessible.

It's possible that you simply meant to type 'container_of' rather than 
'offsetof', there.

Following the absurd-but-seemingly-permitted example bounds-checking 
implementation example above, if we had:

   struct s_baz {
       unsigned char c;
       double d;
     };
   struct s_baz baz;
   unsigned char * whole = (unsigned char *) &baz;
   unsigned char * part = &baz.c;

The bounds for 'whole' could be greater than the bounds for 'part'.  Now 
if we cast:

   part = *((unsigned char (*)[sizeof baz]) part);

Then "internally," the implementation could say "they must know that 
there's more data than the current bounds" and adjust the "internal" 
'size' member value accordingly.  Then 'part' and 'whole' would be the 
same.  But there doesn't appear to be a way to say "take this pointer 
value and cast it as though its 'current_position' is part-way along; 
I'd expect that if anything, such a spiteful implementation would always 
reset 'current_position' to 0...  "Hey, you said this is what we've got 
here."

But seriously, in general, how might a bounds-checking implementation 
deal with:

   int mumble(int * i) {
       i = i - 1;
       return *i;
     }

where you might pass a pointer to the xth element of an array of 'int' 
but you might not?  What's safe to assume?  What kinds of bounds might 
be encoded or associated with the pointer value?  Would the 'beginning' 
map all the way back to some declared type somewhere?  Some effective 
type?  Or is bounds-checking a special feature of using 'a[i]' notation 
and 's.m' or 'sp->m' notation and "raw" points are immune?

-- 
"The stationery store has moved.  Aaargh!"

[toc] | [prev] | [next] | [standalone]

#18331

Shao Miller <sha0.miller@gmail.com> writes:

> On 1/31/2012 19:11, Tim Rentsch wrote:
>> Shao Miller<sha0.miller@gmail.com>  writes:
>>
>>> On 1/24/2012 18:56, Tim Rentsch wrote:
>>>>
>>>> Using container_of() should be just as portable as this.
>>>>
>>>
>>> For the special case where the pointer points to an object at the
>>> beginning of any containing sub-object...  If a pointer carries bounds
>>> information, subtracting from a pointer needn't be defined
>>> (apparently).
>>>
>>>    struct s_foo {
>>>        int i[1];
>>>        double d[1];
>>>      } * foo = malloc(sizeof *foo);
>>>    double * dp;
>>>    assert(foo);
>>>    dp = foo->d;
>>>    /* dp bounds might != foo bounds.  Uh oh */
>>>    foo = container_of(dp, struct s_foo, d);
>>
>> The same argument applies to members that aren't arrays,
>> which shows the argument must be wrong, since
>
> The argument _does_ apply to members that aren't arrays, but I've seen
> especial attention regarding "arrays" with "bounds-checking," so the
> example highlights that.
>
> Why would a bounds-checking implementation _not_ be allowed to
> continually reduce the possible bounds, but never increase them?
> [snip elaboration]

For the same reason that an implementation can't decide
that '2 == 2' is 12 - it doesn't match the specification.

Admittedly the specification for the result of '2 == 2'
is clearer and easier to understand, but that doesn't
lessen the point.  Whatever mechanism one might imagine
to explain how an implementation works, all that matters
is whether the result matches the specification.  If the
purpose is to discover what the Standard requires, arguments
based on reasoning about some putative underlying mechanism
usually serve to confuse more than illuminate.

[toc] | [prev] | [next] | [standalone]

#17158

Tim Rentsch <txr@alumni.caltech.edu> writes:
> Phil Carmody <thefatphil_demunged@yahoo.co.uk> writes:
> 
> > Mark Adler <madler@alumni.caltech.edu> writes:
> >> I have an internal structure with lots of stuff in it for controlling
> >> a data input operation.  E.g.:
> >> 
> >> struct data {
> >>     int have;
> >>     char *next;
> >>     ... lots more stuff to keep track of the state
> >> };
> >> 
> >> I would like to expose to the user (in the interface header file) only
> >> those first two elements, so that I can provide a macro (also in the
> >> header file) analogous to getc() to pull data that is readily
> >> available in the buffer, otherwise to call a function to go get more
> >> data.  But I want to hide the other stuff in the structure, in order
> >> to make sure that users of the library don't end up depending on
> >> things that I change later for internal use.  And for efficiency I
> >> don't want yet another layer of indirection pointing to another
> >> structure from within the exposed structure for the other stuff.
> >> 
> >> E.g.
> >> 
> >> struct ex {
> >>     int have;
> >>     char *next;
> >> };
> >> 
> >> #define getd(d) (((struct ex *)d)->have ? (((struct ex *)d)->have--,
> >> *(((struct ex *)d)->next)++) : getdf(d))
> >> 
> >> where d is a struct data *.
> >> 
> >> My question is: does the C standard provide an assurance that this
> >> will work?  I.e. that two different structures with identical sets of
> >> prefix elements will have those prefix elements layed out identically
> >> in memory so that casting one structure to the other will permit
> >> correct access to those prefix elements?
> >
> > struct private {
> >         struct public {
> >                 int have;
> >                 char *next;
> >         } pub;
> >         other stuff;
> > }
> >
> > let the clients see and pass around pointers to pub, and then either
> > use something like container_of() (which is not strictly portable), or
> 
> Actually container_of() should be strictly portable provided the
> outer structure type being casted to matches the actual object
> being referred to.  (This assuming container_of() is defined
> properly.)

All the implementations I've seen (which isn't a huge number) have
implemented container_of as an almost identical macro in a way that I
would consider non-portable. Looking more closely, I'm fairly sure
that this nonportability is historical accident being propagated
rather than actually being necessary. Nothing more than casting to
character pointer, subtracting offset_of, and casting again should
work portably (given the provision you mention).

Phil
-- 
Unix is simple. It just takes a genius to understand its simplicity
-- Dennis Ritchie (1941-2011), Unix Co-Creator

[toc] | [prev] | [next] | [standalone]

#18332

Phil Carmody <thefatphil_demunged@yahoo.co.uk> writes:

> Tim Rentsch <txr@alumni.caltech.edu> writes:
>> Phil Carmody <thefatphil_demunged@yahoo.co.uk> writes:
>> 
>> > Mark Adler <madler@alumni.caltech.edu> writes:
>> >> I have an internal structure with lots of stuff in it for controlling
>> >> a data input operation.  E.g.:
>> >> 
>> >> struct data {
>> >>     int have;
>> >>     char *next;
>> >>     ... lots more stuff to keep track of the state
>> >> };
>> >> 
>> >> I would like to expose to the user (in the interface header file) only
>> >> those first two elements, so that I can provide a macro (also in the
>> >> header file) analogous to getc() to pull data that is readily
>> >> available in the buffer, otherwise to call a function to go get more
>> >> data.  But I want to hide the other stuff in the structure, in order
>> >> to make sure that users of the library don't end up depending on
>> >> things that I change later for internal use.  And for efficiency I
>> >> don't want yet another layer of indirection pointing to another
>> >> structure from within the exposed structure for the other stuff.
>> >> 
>> >> E.g.
>> >> 
>> >> struct ex {
>> >>     int have;
>> >>     char *next;
>> >> };
>> >> 
>> >> #define getd(d) (((struct ex *)d)->have ? (((struct ex *)d)->have--,
>> >> *(((struct ex *)d)->next)++) : getdf(d))
>> >> 
>> >> where d is a struct data *.
>> >> 
>> >> My question is: does the C standard provide an assurance that this
>> >> will work?  I.e. that two different structures with identical sets of
>> >> prefix elements will have those prefix elements layed out identically
>> >> in memory so that casting one structure to the other will permit
>> >> correct access to those prefix elements?
>> >
>> > struct private {
>> >         struct public {
>> >                 int have;
>> >                 char *next;
>> >         } pub;
>> >         other stuff;
>> > }
>> >
>> > let the clients see and pass around pointers to pub, and then either
>> > use something like container_of() (which is not strictly portable), or
>> 
>> Actually container_of() should be strictly portable provided the
>> outer structure type being casted to matches the actual object
>> being referred to.  (This assuming container_of() is defined
>> properly.)
>
> All the implementations I've seen (which isn't a huge number) have
> implemented container_of as an almost identical macro in a way that I
> would consider non-portable. Looking more closely, I'm fairly sure
> that this nonportability is historical accident being propagated
> rather than actually being necessary. Nothing more than casting to
> character pointer, subtracting offset_of, and casting again should
> work portably (given the provision you mention).

I see, that's interesting.  Do you know what the original
motivation was for defining it that way, rather than (what
seems to me to be) the more straightfoward portable one?

[toc] | [prev] | [next] | [standalone]

#18680

Tim Rentsch <txr@alumni.caltech.edu> writes:
> Phil Carmody <thefatphil_demunged@yahoo.co.uk> writes:
> > Tim Rentsch <txr@alumni.caltech.edu> writes:
> >> Phil Carmody <thefatphil_demunged@yahoo.co.uk> writes:
> >> > Mark Adler <madler@alumni.caltech.edu> writes:
...
> >> > struct private {
> >> >         struct public {
> >> >                 int have;
> >> >                 char *next;
> >> >         } pub;
> >> >         other stuff;
> >> > }
> >> >
> >> > let the clients see and pass around pointers to pub, and then either
> >> > use something like container_of() (which is not strictly portable), or
> >> 
> >> Actually container_of() should be strictly portable provided the
> >> outer structure type being casted to matches the actual object
> >> being referred to.  (This assuming container_of() is defined
> >> properly.)
> >
> > All the implementations I've seen (which isn't a huge number) have
> > implemented container_of as an almost identical macro in a way that I
> > would consider non-portable. Looking more closely, I'm fairly sure
> > that this nonportability is historical accident being propagated
> > rather than actually being necessary. Nothing more than casting to
> > character pointer, subtracting offset_of, and casting again should
> > work portably (given the provision you mention).
> 
> I see, that's interesting.  Do you know what the original
> motivation was for defining it that way, rather than (what
> seems to me to be) the more straightfoward portable one?

I had presumed Linux's definition:

#define container_of(ptr, type, member) ({                      \
        const typeof( ((type *)0)->member ) *__mptr = (ptr);    \
        (type *)( (char *)__mptr - offsetof(type,member) );})

was lifted from an early gcc definition, as a lot of things like that were.

It seems that someone else has questioned the seemingly necessary
additional line:
http://psomas.wordpress.com/2009/07/01/weird-kernel-macros-container_of/
"""
EDIT: Apparently, the first line is there for `type checking'. It
ensures that type has a member called member(howerver this is done by
offsetof macro too, I think), and if ptr isn't a pointer to the
correct type(the type of the member), the compiler will print a
warning, which can be useful for debuging.
"""

Which implies that it wasn't a gcc-ism after all.

The gccs I have access to seem to have now pulled it out of the header
files and absorbed it into the compiler itself.

So the linux weirdness seems to be confined. (And that's all I've 
surrounded myself in for the last few years.)

Phil
-- 
> I'd argue that there is much evidence for the existence of a God.
Pics or it didn't happen.
-- Tom (/. uid 822)

[toc] | [prev] | [next] | [standalone]

#18681

Phil Carmody <thefatphil_demunged@yahoo.co.uk> writes:

> I had presumed Linux's definition:
>
> #define container_of(ptr, type, member) ({                      \
>         const typeof( ((type *)0)->member ) *__mptr = (ptr);    \
>         (type *)( (char *)__mptr - offsetof(type,member) );})
>
> was lifted from an early gcc definition, as a lot of things like that were.
>
> It seems that someone else has questioned the seemingly necessary
> additional line:
> http://psomas.wordpress.com/2009/07/01/weird-kernel-macros-container_of/
> """
> EDIT: Apparently, the first line is there for `type checking'. It
> ensures that type has a member called member(howerver this is done by
> offsetof macro too, I think), and if ptr isn't a pointer to the
> correct type(the type of the member), the compiler will print a
> warning, which can be useful for debuging.
> """
>
> Which implies that it wasn't a gcc-ism after all.

({ }) and "typeof" are of course non-portable, but one can write
a more portable variant without them, as I have in GNU PSPP:

/* Given expressions A and B, both of which have pointer type,
   expands to a void expression that causes a compiler warning if
   A and B are not pointers to qualified or unqualified versions
   of compatible types.

   Examples similar to those given for CHECK_POINTER_HAS_TYPE,
   above, can easily be devised. */
#define CHECK_POINTER_COMPATIBILITY(A, B) ((void) sizeof ((A) == (B)))

/* Given POINTER, a pointer to the given MEMBER within structure
   STRUCT, returns the address of the STRUCT. */
#define UP_CAST(POINTER, STRUCT, MEMBER)                                \
        (CHECK_POINTER_COMPATIBILITY (&((STRUCT *) 0)->MEMBER, POINTER), \
         (STRUCT *) ((char *) (POINTER) - offsetof (STRUCT, MEMBER)))

-- 
"Debugging is twice as hard as writing the code in the first place.
 Therefore, if you write the code as cleverly as possible, you are,
 by definition, not smart enough to debug it."
--Brian Kernighan

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.c

csiph-web