Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.std.c > #1355 > unrolled thread

Structure Packing.

Back to article view | Back to comp.std.c

Contents

  Structure Packing. John Kearney <dethrophes@googlemail.com> - 2012-06-06 17:27 -0700
    Re: Structure Packing. James Kuyper <jameskuyper@verizon.net> - 2012-06-06 21:19 -0400
      Re: Structure Packing. Walter Banks <walter@bytecraft.com> - 2012-06-11 22:39 -0400
        Re: Structure Packing. Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-06-12 09:21 -0400
          Re: Structure Packing. Walter Banks <walter@bytecraft.com> - 2012-06-12 11:51 -0400
            Re: Structure Packing. Hans-Bernhard Bröker <HBBroeker@t-online.de> - 2012-06-12 18:37 +0200
              Re: Structure Packing. Walter Banks <walter@bytecraft.com> - 2012-06-12 15:06 -0400
                Re: Structure Packing. Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-06-12 16:22 -0400
                  Re: Structure Packing. Walter Banks <walter@bytecraft.com> - 2012-06-12 17:44 -0400
                    Re: Structure Packing. Hans-Bernhard Bröker <HBBroeker@t-online.de> - 2012-06-13 15:53 +0200
                      Re: Structure Packing. Phil Carmody <thefatphil_demunged@yahoo.co.uk> - 2012-06-18 12:37 +0300
          Re: Structure Packing. Martin Shobe <martin.shobe@yahoo.com> - 2012-06-12 16:22 +0000
            Re: Structure Packing. Walter Banks <walter@bytecraft.com> - 2012-06-12 15:24 -0400
              Re: Structure Packing. Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-06-12 16:33 -0400
                Re: Structure Packing. Walter Banks <walter@bytecraft.com> - 2012-06-12 17:17 -0400
    Re: Structure Packing. Keith Thompson <kst-u@mib.org> - 2012-06-06 18:35 -0700
      Re: Structure Packing. John Kearney <dethrophes@googlemail.com> - 2012-06-06 21:58 -0700
        Re: Structure Packing. Keith Thompson <kst-u@mib.org> - 2012-06-06 23:54 -0700
          Re: Structure Packing. John Kearney <dethrophes@googlemail.com> - 2012-06-07 07:47 -0700
            Re: Structure Packing. James Kuyper <jameskuyper@verizon.net> - 2012-06-07 12:39 -0400
      Re: Structure Packing. jacob navia <jacob@spamsink.net> - 2012-06-07 11:14 +0200
        Re: Structure Packing. Keith Thompson <kst-u@mib.org> - 2012-06-07 09:46 -0700
          Re: Structure Packing. John Kearney <dethrophes@googlemail.com> - 2012-06-07 11:57 -0700

Page 1 of 2  [1] 2  Next page →

#1355 — Structure Packing.

Hi hopefully just a quick question.
struct s{
   uint16_t var1;
   uint16_t var2;
};

would it ever be acceptable for a compiler to align var2 on a 4/8 byte boundary, for speed, architectural or indeed any reason.

I just always feel uneasy when I see code that relies on the above structure being packed together without an explicit pack pragma.
is this just my paranoia?

Sorry if this is a silly question.

Thanks
John

[toc] | [next] | [standalone]

#1356

On 06/06/2012 08:27 PM, John Kearney wrote:
> Hi hopefully just a quick question.
> struct s{
>    uint16_t var1;
>    uint16_t var2;
> };
> 
> would it ever be acceptable for a compiler to align var2 on a 4/8 byte boundary, for speed, architectural or indeed any reason.

Yes, the standard allows padding between members and after the last
member (but not before the first member). It's rare for an
implementation to insert such padding between two members with the same
alignment requirements, but perfectly permissible.

> I just always feel uneasy when I see code that relies on the above structure being packed together without an explicit pack pragma.
> is this just my paranoia?

No, it's quite reasonable to be worried about this.
-- 
James Kuyper

[toc] | [prev] | [next] | [standalone]

#1365

James Kuyper wrote:

> Yes, the standard allows padding between members and after the last
> member (but not before the first member). It's rare for an
> implementation to insert such padding between two members with the same
> alignment requirements, but perfectly permissible.
>
> > I just always feel uneasy when I see code that relies on the above structure being packed together without an explicit pack pragma.
> > is this just my paranoia?
>
> No, it's quite reasonable to be worried about this.

A current compiler project I am working on potentially has a completely different twist on struct placement.

There is quite a list of hardware based alignment requirements based on boundaries. It is a RAM poor processor and I am being very
tempted to assign global variables in the holes left behind in global structs.  My guess the need for RAM space will outweigh other
considerations.

Walter..

[toc] | [prev] | [next] | [standalone]

#1366

On 6/11/2012 10:39 PM, Walter Banks wrote:
>
>
> James Kuyper wrote:
>
>> Yes, the standard allows padding between members and after the last
>> member (but not before the first member). It's rare for an
>> implementation to insert such padding between two members with the same
>> alignment requirements, but perfectly permissible.
>>
>>> I just always feel uneasy when I see code that relies on the above structure being packed together without an explicit pack pragma.
>>> is this just my paranoia?
>>
>> No, it's quite reasonable to be worried about this.
>
> A current compiler project I am working on potentially has a completely different twist on struct placement.
>
> There is quite a list of hardware based alignment requirements based on boundaries. It is a RAM poor processor and I am being very
> tempted to assign global variables in the holes left behind in global structs.  My guess the need for RAM space will outweigh other
> considerations.

     Don't do it: Operations on the named elements of a struct are
not guaranteed to preserve the contents of the holes.  For example

	struct s {
	    unsigned char flag;
	    // possible hole here
	    unsigned int value;
	} x;
	unsigned char *hole = &x.flag + 1;
	size_t hole_size = (unsigned char*)&x.value - hole;
	if (hole_size > 0) *hole = 0xFF;
	flag = 42;
	if (hole_size > 0) assert(*hole == 0xFF);  // may fail

	struct s y = x;
	assert(memcmp(&x, &y, sizeof x) == 0);  // may fail

If you need the variable, give it a name and make it an ordinary
struct element.

     If you're worried about different alignments on different
systems and it's difficult to find one arrangement that works
efficiently for all, consider generating system-specific struct
definitions from a data file during your build process.  You can
then give each system its own arrangement of the struct elements
to pack things as efficiently as possible.  (Of course, this won't
work for struct elements that "must be first" or "must appear in
such-and-such order" -- but if memory scarcity is extreme, that
sacrifice may be worthwhile.)

-- 
Eric Sosman
esosman@ieee-dot-org.invalid

[toc] | [prev] | [next] | [standalone]

#1367

Eric Sosman wrote:

> > A current compiler project I am working on potentially has a completely different twist on struct placement.
> >
> > There is quite a list of hardware based alignment requirements based on boundaries. It is a RAM poor processor and I am being very
> > tempted to assign global variables in the holes left behind in global structs.  My guess the need for RAM space will outweigh other
> > considerations.
>
>      Don't do it: Operations on the named elements of a struct are
> not guaranteed to preserve the contents of the holes.  For example
>
>         struct s {
>             unsigned char flag;
>             // possible hole here
>             unsigned int value;
>         } x;
>         unsigned char *hole = &x.flag + 1;
>         size_t hole_size = (unsigned char*)&x.value - hole;
>         if (hole_size > 0) *hole = 0xFF;
>         flag = 42;
>         if (hole_size > 0) assert(*hole == 0xFF);  // may fail
>
>         struct s y = x;
>         assert(memcmp(&x, &y, sizeof x) == 0);  // may fail
>
> If you need the variable, give it a name and make it an ordinary
> struct element.
>
>      If you're worried about different alignments on different
> systems and it's difficult to find one arrangement that works
> efficiently for all, consider generating system-specific struct
> definitions from a data file during your build process.  You can
> then give each system its own arrangement of the struct elements
> to pack things as efficiently as possible.  (Of course, this won't
> work for struct elements that "must be first" or "must appear in
> such-and-such order" -- but if memory scarcity is extreme, that
> sacrifice may be worthwhile.)

Your points are well taken. I have resolved every issue in this specific
processor except memcpy and memcmp both of which are rarely
used in embedded systems.

I am not worried about different alignments on different processors.

In this specific case first fit in a struct allocation is significantly tighter
allocation than next available that will fit. I am uncomfortable even
with packing the struct elements out of order.

I am about to re-write the struct allocator (within the next day or two)
and probably will at least use first fit in the struct or perhaps a
tetris style packer.

At that point I will see what hole metrics come out of a
bunch of regression tests with actual application code.


Thanks for the comments

w..

[toc] | [prev] | [next] | [standalone]

#1369

On 12.06.2012 17:51, Walter Banks wrote:

> I have resolved every issue in this specific processor except memcpy
> and memcmp both of which are rarely used in embedded systems.

I rather doubt the correctness of that assessment.  And even if it were 
correct, "rarely used" would not help you --- you would need "never 
used".  For tricks like the one you're considering to be viable, you 
would have to intercept not just every call of memcpy(), but worse, 
every single one of the roll-your-own implementations of it hidden 
somewhere in the code --- including those that aren't even written in C 
to begin with.  Danger, Will Robinson!

Sticking unrelated statics into the padding gaps might be viable for 
struct constants kept in ROM, but there's really no reliable way that 
can work in struct variables.

> I am uncomfortable even with packing the struct elements out of
> order.

You should be more than uncomfortable --- that's actually disallowed by 
the standard.  To quote C99 6.7.2.1#16:

> Within a structure object, the non-bit-field members and the units in which bit-fields
> reside have addresses that increase in the order in which they are declared.

[toc] | [prev] | [next] | [standalone]

#1370

Hans-Bernhard Bröker wrote:

> On 12.06.2012 17:51, Walter Banks wrote:
>
> > I have resolved every issue in this specific processor except memcpy
> > and memcmp both of which are rarely used in embedded systems.
>
> I rather doubt the correctness of that assessment.  And even if it were
> correct, "rarely used" would not help you --- you would need "never
> used".  For tricks like the one you're considering to be viable, you
> would have to intercept not just every call of memcpy(), but worse,
> every single one of the roll-your-own implementations of it hidden
> somewhere in the code --- including those that aren't even written in C
> to begin with.  Danger, Will Robinson!

That isn't going to happen the  memcpy and memcmp related issues
would be a show stopper.

> > I am uncomfortable even with packing the struct elements out of
> > order.
>
> You should be more than uncomfortable --- that's actually disallowed by
> the standard.  To quote C99 6.7.2.1#16:

> > Within a structure object, the non-bit-field members and the units in which bit-fields
> > reside have addresses that increase in the order in which they are declared.

I  represented Canada at ISO working on C99 and am aware of the
wording and know the reasons. It is the reason I am so uncomfortable.

We have never done it and this is a case where a struct empty space
sometime exceeds the amount of ram space used. Arrays of structs
are a disaster in some cases.

Something that I just thought of that might work is to add an create an
information message as part of the listing summary that identifies teh
struct and a better data order declaration. The hardware in this case
is imposing the constraints.

The current implementation allocates next fit for each structure element.

As the day goes on I am getting convinced that it is a bad idea.

Walter Banks

[toc] | [prev] | [next] | [standalone]

#1372

On 6/12/2012 3:06 PM, Walter Banks wrote:
>
>
> Hans-Bernhard Bröker wrote:
>
>> On 12.06.2012 17:51, Walter Banks wrote:
>>
>>> I have resolved every issue in this specific processor except memcpy
>>> and memcmp both of which are rarely used in embedded systems.
>>
>> I rather doubt the correctness of that assessment.  And even if it were
>> correct, "rarely used" would not help you --- you would need "never
>> used".  For tricks like the one you're considering to be viable, you
>> would have to intercept not just every call of memcpy(), but worse,
>> every single one of the roll-your-own implementations of it hidden
>> somewhere in the code --- including those that aren't even written in C
>> to begin with.  Danger, Will Robinson!
>
> That isn't going to happen the  memcpy and memcmp related issues
> would be a show stopper.
>
>>> I am uncomfortable even with packing the struct elements out of
>>> order.
>>
>> You should be more than uncomfortable --- that's actually disallowed by
>> the standard.  To quote C99 6.7.2.1#16:
>
>>> Within a structure object, the non-bit-field members and the units in which bit-fields
>>> reside have addresses that increase in the order in which they are declared.
>
> I  represented Canada at ISO working on C99 and am aware of the
> wording and know the reasons. It is the reason I am so uncomfortable.
>
> We have never done it and this is a case where a struct empty space
> sometime exceeds the amount of ram space used. Arrays of structs
> are a disaster in some cases.
>
> Something that I just thought of that might work is to add an create an
> information message as part of the listing summary that identifies teh
> struct and a better data order declaration. The hardware in this case
> is imposing the constraints.
>
> The current implementation allocates next fit for each structure element.
>
> As the day goes on I am getting convinced that it is a bad idea.

     Perhaps structs are not the right tool for the job?  Instead of

	struct {
	    unsigned char flag;
	    double value;
	} data[1000];

... you might do better to write

	unsigned char data_flag[1000];
	double data_value[1000];

("Parallel arrays" were bread-and-butter for languages that didn't
have struct-like things, but seem to have fallen into disuse.  The
struct is a great help, but it is not the only way to skin cats.)

-- 
Eric Sosman
esosman@ieee-dot-org.invalid

[toc] | [prev] | [next] | [standalone]

#1375

Eric Sosman wrote:

> > The current implementation allocates next fit for each structure element.
> >
> > As the day goes on I am getting convinced that it is a bad idea.
>
>      Perhaps structs are not the right tool for the job?  Instead of
>
>         struct {
>             unsigned char flag;
>             double value;
>         } data[1000];
>
> ... you might do better to write
>
>         unsigned char data_flag[1000];
>         double data_value[1000];
>
> ("Parallel arrays" were bread-and-butter for languages that didn't
> have struct-like things, but seem to have fallen into disuse.  The
> struct is a great help, but it is not the only way to skin cats.)

My job is to take what our customers write in their application
code and optimize within their expectations and the standards
documents.

This whole question started to show itself with a piece of customer
code that has a large struct array with about 25 % of the ram
space un-used. This specific customer is happy by moving two lines
and saving (as it turns out) all of his hole space.

This conversation a few days ago started me looking into
how far I could push the envelope from the compiler end.
to just make the problem go away.

From a customer point of view there isn't an easy way to
identify the solution to the problem without knowing a quite
a bit about both this specific hardware and the C compiler
constraints allocating struct space. There are several data
types involved. It is information a programmer should not
need to know at that level of detail for most routine applications

Having the compiler make an intelligent suggestion is a
workable solution. If there is any message in this thread it is
don't even consider messing with standards and conventions.

w..



w..

[toc] | [prev] | [next] | [standalone]

#1376

On 12.06.2012 23:44, Walter Banks wrote:
> Eric Sosman wrote:

>>       Perhaps structs are not the right tool for the job?  Instead of
>>
>>          struct {
>>              unsigned char flag;
>>              double value;
>>          } data[1000];
>>
>> ... you might do better to write
>>
>>          unsigned char data_flag[1000];
>>          double data_value[1000];

Or the slightly less obscure

	struct {
	    unsigned char flag[1000];
	    double value[1000];
	} data;

I guess one big reason that array-of-structs is favoured over this 
struct-of-arrays approach quite a lot these days is that all those 
caches favour locality over dense packing.

> This whole question started to show itself with a piece of customer
> code that has a large struct array with about 25 % of the ram
> space un-used.

> From a customer point of view there isn't an easy way to
> identify the solution to the problem without knowing a quite
> a bit about both this specific hardware and the C compiler
> constraints allocating struct space.

Well, writing tight code does tend to require such knowledge anyway.  A 
rule-of-thumb like "big things first in structs" covers most of the 
ground, and should be known to anyone seriously working with this kind 
of processor.

> There are several data types involved. It is information a programmer
> should not need to know at that level of detail for most routine
> applications

OTOH "routine applications" should not cut their RAM needs quite so 
close to the bone that this really matters.

> Having the compiler make an intelligent suggestion is a
> workable solution.

I agree that the optimal solution here would be an elective info/warning 
message from the compiler as it sees sub-optimal packing; particularly 
if optimization is turned on to favour size over speed.  GCC's -Wpadded 
comes to mind.

[toc] | [prev] | [next] | [standalone]

#1378

Hans-Bernhard Bröker <HBBroeker@t-online.de> writes:
> On 12.06.2012 23:44, Walter Banks wrote:
> > Eric Sosman wrote:
> 
> >>       Perhaps structs are not the right tool for the job?  Instead of
> >>
> >>          struct {
> >>              unsigned char flag;
> >>              double value;
> >>          } data[1000];
> >>
> >> ... you might do better to write
> >>
> >>          unsigned char data_flag[1000];
> >>          double data_value[1000];
> 
> Or the slightly less obscure
> 
> 	struct {
> 	    unsigned char flag[1000];
> 	    double value[1000];
> 	} data;
> 
> I guess one big reason that array-of-structs is favoured over this
> struct-of-arrays approach quite a lot these days is that all those
> caches favour locality over dense packing.

Hopping from index to index or running through linked lists to only
access a limited subset of the fields would have improved locality
as well as dense packing. A locality of reference argument which
assumes that each time you want to access any part of a struct you
want to access it all is a very flawed one. However, that's highly
domain specific, and neither the language nor the compiler can 
assist you post-facto, you simply have to select the data structure
you think (or measure) is best. OK, it helps by letting you 
abstract the data layout via zero-cost accessor functions (or macros
if you're stuck in the 80s), but some might find that too clumsy, 
YMMV.

Phil
-- 
> I'd argue that there is much evidence for the existence of a God.
Pics or it didn't happen.
-- Tom (/. uid 822)

[toc] | [prev] | [next] | [standalone]

#1368

Eric Sosman wrote:

> On 6/11/2012 10:39 PM, Walter Banks wrote:
>>
>>
>> James Kuyper wrote:
>>
>>> Yes, the standard allows padding between members and after the last
>>> member (but not before the first member). It's rare for an
>>> implementation to insert such padding between two members with the same
>>> alignment requirements, but perfectly permissible.
>>>
>>>> I just always feel uneasy when I see code that relies on the above structure being packed together without an explicit pack pragma.
>>>> is this just my paranoia?
>>>
>>> No, it's quite reasonable to be worried about this.
>>
>> A current compiler project I am working on potentially has a completely different twist on struct placement.
>>
>> There is quite a list of hardware based alignment requirements based on boundaries. It is a RAM poor processor and I am being very
>> tempted to assign global variables in the holes left behind in global structs.  My guess the need for RAM space will outweigh other
>> considerations.
>
>      Don't do it: Operations on the named elements of a struct are
> not guaranteed to preserve the contents of the holes.  For example
>
> 	struct s {
> 	    unsigned char flag;
> 	    // possible hole here
> 	    unsigned int value;
> 	} x;
> 	unsigned char *hole = &x.flag + 1;
> 	size_t hole_size = (unsigned char*)&x.value - hole;
> 	if (hole_size > 0) *hole = 0xFF;
> 	flag = 42;
> 	if (hole_size > 0) assert(*hole == 0xFF);  // may fail
>
> 	struct s y = x;
> 	assert(memcmp(&x, &y, sizeof x) == 0);  // may fail
>
> If you need the variable, give it a name and make it an ordinary
> struct element.
>
>      If you're worried about different alignments on different
> systems and it's difficult to find one arrangement that works
> efficiently for all, consider generating system-specific struct
> definitions from a data file during your build process.  You can
> then give each system its own arrangement of the struct elements
> to pack things as efficiently as possible.  (Of course, this won't
> work for struct elements that "must be first" or "must appear in
> such-and-such order" -- but if memory scarcity is extreme, that
> sacrifice may be worthwhile.)
>

I don't think you understand what's being proposed.
(Of course, it might be my understanding that's faulty). The project is
to build a compiler for a system that has some strict alignment
restrictions, and very little RAM. As such, there isn't an issue with
different alignments on different systems, and the fact that those
asserts may fail actually helps.

The issue I see is that the asserts in the following must not fail.

struct s {
    unsigned char flag;
    // hole here that is exactly the right size for
    // an unsigned char.
    unsigned int value;
} x;

// y has been placed by the compiler into the hole in x immediately
// after x.flag
unsigned char y;

int main()
{
    unsigned char * hole = &x.flag + 1;

    y = 42;
    *hole = 0xFF;
    assert(*hole == 0xFF);
    assert(y == 42);

    return 0;
}

Martin Shobe

[toc] | [prev] | [next] | [standalone]

#1371

Martin Shobe wrote:

> I don't think you understand what's being proposed.
> (Of course, it might be my understanding that's faulty). The project is
> to build a compiler for a system that has some strict alignment
> restrictions, and very little RAM. As such, there isn't an issue with
> different alignments on different systems, and the fact that those
> asserts may fail actually helps.
>
> The issue I see is that the asserts in the following must not fail.
>
> struct s {
>     unsigned char flag;
>     // hole here that is exactly the right size for
>     // an unsigned char.
>     unsigned int value;
> } x;
>
> // y has been placed by the compiler into the hole in x immediately
> // after x.flag
> unsigned char y;
>
> int main()
> {
>     unsigned char * hole = &x.flag + 1;
>
>     y = 42;
>     *hole = 0xFF;
>     assert(*hole == 0xFF);
>     assert(y == 42);
>
>     return 0;
> }

This is the exact problem.

I am less sure that pointer math in the asserts
should be expected to reference the holes.


The following struct illustrates the problem
Assume that ints need to be on even boundaries
(assume for a moment that ints are 16 bits char is 8 bits)
the following struct will take a total of 6 bytes two are unused

struct  {
   unsigned char flag;
   unsigned int value;
   unsigned char flag2;
} z;

The following struct contains the same information
but only requires 4 bytes of ram to store the
information with no unused bytes. This is a
change the compiler struct allocator 'could' do.

struct  {
   unsigned char flag;
  unsigned char flag2;
   unsigned int value;
 } o;

Compatibility is not an issue even that could be
covered with an allocation rules pragma.

I am totally convinced that using the holes for other
globals is a bad idea.

I am less convinced but very uncomfortable to
changing a structs element order to first fit within
a struct.

Intetresting  problem

Walter..

[toc] | [prev] | [next] | [standalone]

#1373

On 6/12/2012 3:24 PM, Walter Banks wrote:
>
>
> Martin Shobe wrote:
>
>> I don't think you understand what's being proposed.
>> (Of course, it might be my understanding that's faulty). The project is
>> to build a compiler for a system that has some strict alignment
>> restrictions, and very little RAM. As such, there isn't an issue with
>> different alignments on different systems, and the fact that those
>> asserts may fail actually helps.
>>
>> The issue I see is that the asserts in the following must not fail.
>>
>> struct s {
>>      unsigned char flag;
>>      // hole here that is exactly the right size for
>>      // an unsigned char.
>>      unsigned int value;
>> } x;
>>
>> // y has been placed by the compiler into the hole in x immediately
>> // after x.flag
>> unsigned char y;
>>
>> int main()
>> {
>>      unsigned char * hole = &x.flag + 1;
>>
>>      y = 42;
>>      *hole = 0xFF;
>>      assert(*hole == 0xFF);
>>      assert(y == 42);
>>
>>      return 0;
>> }
>
> This is the exact problem.
>
> I am less sure that pointer math in the asserts
> should be expected to reference the holes.
>
>
> The following struct illustrates the problem
> Assume that ints need to be on even boundaries
> (assume for a moment that ints are 16 bits char is 8 bits)
> the following struct will take a total of 6 bytes two are unused
>
> struct  {
>     unsigned char flag;
>     unsigned int value;
>     unsigned char flag2;
> } z;
>
> The following struct contains the same information
> but only requires 4 bytes of ram to store the
> information with no unused bytes. This is a
> change the compiler struct allocator 'could' do.
>
> struct  {
>     unsigned char flag;
>    unsigned char flag2;
>     unsigned int value;
>   } o;
>
> Compatibility is not an issue even that could be
> covered with an allocation rules pragma.
>
> I am totally convinced that using the holes for other
> globals is a bad idea.
>
> I am less convinced but very uncomfortable to
> changing a structs element order to first fit within
> a struct.

     That would make me uncomfortable, too.  Remember, it's always
allowed to convert a pointer to a struct to a pointer to its first
element and vice versa, and if your compiler rearranges the element
order this could break rather badly.  Think of all those linked
lists where the "next" pointer is assumed to be the first thing in
the struct (so "generic" functions can find it), and think of the
havoc that could ensue if the compiler silently shuffled a "double"
into first place in the name of storage efficiency ...

     One thing you might consider would be to have your compiler
emit a warning message for inefficiently-arranged structs.  (The
compiler is always permitted to whine, even about perfectly legal
code.)  This would draw people's attention to wasteful constructs
without breaking C conformance.

-- 
Eric Sosman
esosman@ieee-dot-org.invalid

[toc] | [prev] | [next] | [standalone]

#1374

Eric Sosman wrote:

>
>
>      That would make me uncomfortable, too.  Remember, it's always
> allowed to convert a pointer to a struct to a pointer to its first
> element and vice versa, and if your compiler rearranges the element
> order this could break rather badly.  Think of all those linked
> lists where the "next" pointer is assumed to be the first thing in
> the struct (so "generic" functions can find it), and think of the
> havoc that could ensue if the compiler silently shuffled a "double"
> into first place in the name of storage efficiency ...

The first element is never going to be a problem in this case.
One of the very few really good things is that I will never need
to pad the first element. It can for example be either a char
or int and in every case be referenced by it actual first
element address.

>      One thing you might consider would be to have your compiler
> emit a warning message for inefficiently-arranged structs.  (The
> compiler is always permitted to whine, even about perfectly legal
> code.)  This would draw people's attention to wasteful constructs
> without breaking C conformance.

I am coming to that conclusion. Even better it is reasonably easy for
the compiler to define a cleaner declaration and make the whining helpful.

Thanks

w..

[toc] | [prev] | [next] | [standalone]

#1357

John Kearney <dethrophes@googlemail.com> writes:
> Hi hopefully just a quick question.
> struct s{
>    uint16_t var1;
>    uint16_t var2;
> };
>
> would it ever be acceptable for a compiler to align var2 on a 4/8 byte boundary, for speed, architectural or indeed any reason.
>
> I just always feel uneasy when I see code that relies on the above structure being packed together without an explicit pack pragma.
> is this just my paranoia?

I'm not sure just what you mean by "a 4/8 boundary". I'm going to
guess that you mean either 4-byte or 8-byte alignment.

I'll also assume 8-byte bytes (CHAR_BIT==8). The standard requires
CHAR_BIT>=8. Any system you're likely to encounter, unless you work
with DSPs, is almost certain to have CHAR_BIT==8 exactly.

Given that the two members are of the same type, it's very likely that
there will be no padding between them. If you declared an array of
uint16_t, padding between elements is forbidden, so the compiler is
certainly able to generate code to access var1 and var1 if there's no
padding.

Compilers are *permitted* to add arbitrary padding between any
two struct members, or after the last one. The intended purpose
of such padding is to enable better code for accessing members
-- but the standard imposes no such restrictions.  For example,
a conforming compiler could make sizeof (struct s) == 8, with 2
bytes of padding after var1 and another 2 after var2. But it's
unlikely that any real-world compiler would do so.

Bottom line: a compiler *could* add padding, but in practice it's
vanishingly unlikely to do so.

If you're concerned, you could add something like:

    assert(sizeof (struct s) == 4);

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
    Will write code for food.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#1358

On Wednesday, 6 June 2012 21:35:59 UTC-4, Keith Thompson  wrote:
> John Kearney <dethrophes@googlemail.com> writes:
> > Hi hopefully just a quick question.
> > struct s{
> >    uint16_t var1;
> >    uint16_t var2;
> > };
> >
> > would it ever be acceptable for a compiler to align var2 on a 4/8 byte boundary, for speed, architectural or indeed any reason.
> >
> > I just always feel uneasy when I see code that relies on the above structure being packed together without an explicit pack pragma.
> > is this just my paranoia?
> 
> I'm not sure just what you mean by "a 4/8 boundary". I'm going to
> guess that you mean either 4-byte or 8-byte alignment.
> 
> I'll also assume 8-byte bytes (CHAR_BIT==8). The standard requires
> CHAR_BIT>=8. Any system you're likely to encounter, unless you work
> with DSPs, is almost certain to have CHAR_BIT==8 exactly.
> 
> Given that the two members are of the same type, it's very likely that
> there will be no padding between them. If you declared an array of
> uint16_t, padding between elements is forbidden, so the compiler is
> certainly able to generate code to access var1 and var1 if there's no
> padding.
> 
> Compilers are *permitted* to add arbitrary padding between any
> two struct members, or after the last one. The intended purpose
> of such padding is to enable better code for accessing members
> -- but the standard imposes no such restrictions.  For example,
> a conforming compiler could make sizeof (struct s) == 8, with 2
> bytes of padding after var1 and another 2 after var2. But it's
> unlikely that any real-world compiler would do so.
> 
> Bottom line: a compiler *could* add padding, but in practice it's
> vanishingly unlikely to do so.
> 
> If you're concerned, you could add something like:
> 
>     assert(sizeof (struct s) == 4);
> 
> -- 
> Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
>     Will write code for food.
> "We must do something.  This is something.  Therefore, we must do this."
>     -- Antony Jay and Jonathan Lynn, "Yes Minister"

:| Sorry your assumptions are correct and thank you for the clarification.
Actually I think the compiler would be right to reserve a native uint space for each, as it would take less code space to access it, and be quicker.

[toc] | [prev] | [next] | [standalone]

#1359

John Kearney <dethrophes@googlemail.com> writes:
> On Wednesday, 6 June 2012 21:35:59 UTC-4, Keith Thompson  wrote:
>> John Kearney <dethrophes@googlemail.com> writes:
>> > Hi hopefully just a quick question.
>> > struct s{
>> >    uint16_t var1;
>> >    uint16_t var2;
>> > };
>> >
>> > would it ever be acceptable for a compiler to align var2 on a 4/8
>> > byte boundary, for speed, architectural or indeed any reason.
[snip]
>> Bottom line: a compiler *could* add padding, but in practice it's
>> vanishingly unlikely to do so.
>> 
>> If you're concerned, you could add something like:
>> 
>>     assert(sizeof (struct s) == 4);
>
> :| Sorry your assumptions are correct and thank you for the
> clarification.  Actually I think the compiler would be right to
> reserve a native uint space for each, as it would take less code space
> to access it, and be quicker.

Assuming uint16_t is 2 bytes and unsigned int is 4 bytes, I don't think
that approach would work.  var1 and var2 *must* be 2 bytes each (again,
assuming CHAR_BIT==8), and if there's padding following them, then that
padding isn't part of var1 or var2.  I'm fairly sure it wouldn't work on
a big-endian system.  It *might* work on a little-endian system, but I
get the feeling that there are some corner cases that would fail.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
    Will write code for food.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#1361

On Thursday, 7 June 2012 02:54:55 UTC-4, Keith Thompson  wrote:
> John Kearney <dethrophes@googlemail.com> writes:
> > On Wednesday, 6 June 2012 21:35:59 UTC-4, Keith Thompson  wrote:
> >> John Kearney <dethrophes@googlemail.com> writes:
> >> > Hi hopefully just a quick question.
> >> > struct s{
> >> >    uint16_t var1;
> >> >    uint16_t var2;
> >> > };
> >> >
> >> > would it ever be acceptable for a compiler to align var2 on a 4/8
> >> > byte boundary, for speed, architectural or indeed any reason.
> [snip]
> >> Bottom line: a compiler *could* add padding, but in practice it's
> >> vanishingly unlikely to do so.
> >> 
> >> If you're concerned, you could add something like:
> >> 
> >>     assert(sizeof (struct s) == 4);
> >
> > :| Sorry your assumptions are correct and thank you for the
> > clarification.  Actually I think the compiler would be right to
> > reserve a native uint space for each, as it would take less code space
> > to access it, and be quicker.
> 
> Assuming uint16_t is 2 bytes and unsigned int is 4 bytes, I don't think
> that approach would work.  var1 and var2 *must* be 2 bytes each (again,
> assuming CHAR_BIT==8), and if there's padding following them, then that
> padding isn't part of var1 or var2.  I'm fairly sure it wouldn't work on
> a big-endian system.  It *might* work on a little-endian system, but I
> get the feeling that there are some corner cases that would fail.
> 
> -- 
> Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
>     Will write code for food.
> "We must do something.  This is something.  Therefore, we must do this."
>     -- Antony Jay and Jonathan Lynn, "Yes Minister"

I only really work on x86,ppc,arm,mips so for me char == 1 byte == 8bits. :) 

Well its quiet normal for compilers to have arch specific optimizations, stepping specific even in some cases. 
Admittedly I can see how it would be problematic to do this on BE system and be spec conform, it is a feature I would like to see. there is so much senseless overuse of uint16_t and char for things like counters etc.. where people really don't seem to understand the cost in code space and senseless expand operations. Its even worse when you consider in a lot of cases due to alignment at least an native int will be taken anyway. 
I mean unless its an array it prob wouldn't take more data space, it would save quiet a bit of code space, and it would execute quicker. :) win win win. Ok I've oversimplified a bit but hopefully you get my point. I looked into it recently arm and ppc both use an extra 4 bytes of code when reading an undersized int, to expand it to a full int, mips uses 2 bytes, and even x86 costs more due to type modifiers etc.
I mean if I explicitly want to pack data I can use an array or specify my pack requirements on a per structure basis. 


what I see a lot is.
uint16_t readio_16(uint16_t addr);

void func(void)
   uint16_t lclCpy;
   lclCpy=readio_16(0xXXXX)

Ok you see why its being done, but its pointless overhead.
and this is the sort of thing that a compiler could and should (if practical), kindly ignore, and just use an uint. 


Now I did look into using uint16fast_t and uint16least_t but in my compilers and on my targets they seem to be mapped the same as uint_t. which frustrated me I don't get why a uint8fast_t uint16fast_t isn't mapped to a native uint even if it is 32bit. this is actually what I expected from these.
I mean normally I just want to make sure its at least 16bits I could care less if its actually 32bits to 64 bits being used for what ever reason. 

Sorry for the rant.

[toc] | [prev] | [next] | [standalone]

#1362

On 06/07/2012 10:47 AM, John Kearney wrote:
...
> Well its quiet normal for compilers to have arch specific
> optimizations, stepping specific even in some cases. Admittedly I can
> see how it would be problematic to do this on BE system and be spec
> conform, it is a feature I would like to see. there is so much
> senseless overuse of uint16_t and char for things like counters etc..
> where people really don't seem to understand the cost in code space
> and senseless expand operations. Its even worse when you consider in
> a lot of cases due to alignment at least an native int will be taken
> anyway. I mean unless its an array it prob wouldn't take more data
> space, it would save quiet a bit of code space, and it would execute
> quicker. :) win win win. Ok I've oversimplified a bit but hopefully
> you get my point. I looked into it recently arm and ppc both use an
> extra 4 bytes of code when reading an undersized int, to expand it to
> a full int, mips uses 2 bytes, and even x86 costs more due to type
> modifiers etc.

Before the final draft of C99 standard was finished, I anticipated that
people would behave that way. I argued for changing the name from
uint16_t to uint_exact16_t, and the name uint_fast16_t to uint16_t (and
similarly for the other sizes), so as to encourage the routine use of
the fast types, and to discourage unnecessary use of exact-sized types.
I didn't win many converts, and anyway, by the time I made the
suggestion it was already too late to make changes that big in the proposal.

...
> Now I did look into using uint16fast_t and uint16least_t but in my
> compilers and on my targets they seem to be mapped the same as
> uint_t. which frustrated me I don't get why a uint8fast_t
> uint16fast_t isn't mapped to a native uint even if it is 32bit. this
> is actually what I expected from these.

The implication of choosing the same type for uint16_t and uint_fast16_t
is that the person who made that choice believes
that uint16_t also happens to be the fastest supported unsigned type
with at least 16 bits. Your complaint implies that you disagree. Do you
have hard evidence to back up your opinion? If so, you should present it
to whoever provided <stdint.h>.

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.std.c

csiph-web