Re: Struct with unaligned fields

From	James Kuyper <jameskuyper@verizon.net>
Newsgroups	comp.lang.c
Subject	Re: Struct with unaligned fields
Date	2013-08-22 16:45 -0400
Organization	Self
Message-ID	<5216785C.8070306@verizon.net> (permalink)
References	<kv5jvm$7mk$1@dont-email.me> <52165F15.3000207@verizon.net> <kv5qup$h9h$1@dont-email.me>

Show all headers | View raw

On 08/22/2013 04:05 PM, James Harris wrote:
> "James Kuyper" <jameskuyper@verizon.net> wrote in message 
> news:52165F15.3000207@verizon.net...
>> On 08/22/2013 02:06 PM, James Harris wrote:
>>> A file allocation table is an on-disk structure that was designed in the
>>> days before people took much care with field alignment. FAT file systems 
>>> are
>>> still prevalent today. I need to write some code to work with such
>>> structures so have to deal with fields that will not be aligned.
>>>
>>> To illustrate, the FAT header starts like this:
>>>
>>>   8-byte BS_OEMName
>>>   2-byte BPB_BytsPerSec
>>>   1-byte BPB_SecPerClus
>>>   2-byte BPB_RsvdSecCnt
>>>   ...
>>>
>>> As you can see, it is impossible to align the last of those fields,
>>> BPB_RsvdSecCnt. The same is true of some of the fields that follow. (For
>>> anyone who is interested the full list of fields can be seen in a 
>>> document
>>> called fatgen103. Copies are freely available on the internet.)
>>>
>>> I've seen the c faq at http://c-faq.com/struct/padding.html and know that
>>> there are no ideal solutions. This post is to ask for suggestions as to 
>>> how
>>> to address this well and portably.
>>
>> If portable is your goal, structures are NOT the right tool to achieve
>> it, and that article gives one of main reasons why.
> 
> You mean that there's no guarantee of how a struct's fields are laid out?

There are some guarantees, but not enough to be useful:

"11 An implementation may allocate any addressable storage unit large
enough to hold a bitfield. If enough space remains, a bit-field that
immediately follows another bit-field in a structure shall be packed
into adjacent bits of the same unit. If insufficient space remains,
whether a bit-field that does not fit is put into the next unit or
overlaps adjacent units is implementation-defined. The order of
allocation of bit-fields within a unit (high-order to low-order or
low-order to high-order) is implementation-defined. The alignment of the
addressable storage unit is unspecified."
"12 A bit-field declaration with no declarator, but only a colon and a
width, indicates an unnamed bit-field. As a special case, a bit-field
structure member with a width of 0 indicates that no further bit-field
is to be packed into the unit in which the previous bitfield,
if any, was placed."
....
"14 Each non-bit-field member of a structure or union object is aligned
in an implementation defined manner appropriate to its type.
"15 Within a structure object, the non-bit-field members and the units
in which bit-fields reside have addresses that increase in the order in
which they are declared. A pointer to a structure object, suitably
converted, points to its initial member (or if that member is a
bit-field, then to the unit in which it resides), and vice versa. There
may be unnamed padding within a structure object, but not at its
beginning."
...
"17 There may be unnamed padding at the end of a structure or union."
(6.7.2.1p11-12,14-15,17)

Pay careful attention to how many things are "implementation-defined" or
"unspecified". Notice all of the words that give implementors additional
freedom: "any", "appropriate", "increase", "suitably", "may".

> The thing is - or the things are: 1) Structs are frequently used in C code 
> for many structures of the types I have to deal with and I have no choice as 
> to padding. The layout of fields is defined by the hardware or prior 
> standards. And 2) C has nothing else that's particularly helpful for 
> describing storage layout. So this type of code tends to rely on the 
> compilers laying out structs as wanted (as long as some sensible rules are 
> followed).

All of that's true - but don't expect it to be portable. Where it works,
it's convenient - but be aware of the fact that it's not required to work.

> Perhaps I could use some form of assertion to check structure sizes or 
> similar. That could at least throw an error of some sort if things were not 
> as they should be.

If there were an authoritative statement about how "things should be",
such an assert would be unnecessary - thing either would be that way, or
it wouldn't be a conforming implementation. The right phrase would be
"as I expected". However, such an assertion needs to incorporate all the
details about what you were expecting that is not actually guaranteed to
be true. In order to make use of structs for this purpose, that's a LOT
of details, no matter what it was you were expecting.

> (I also need to work with specific endianness but that's another topic and I 
> don't want to go off-topic in this thread. Suffice to say I am aware that 
> the two things - alignment and endianness - need to be correct.)
> 
>> On the other hand, I find it hard to imagine that a tool which works
>> with FATs would need to be very portable. It could only be useful on
>> systems which use FATs; I wouldn't expect C implementations targeting
>> such systems to have a lot of variation in how they deal with alignment
>> issue.
> 
> That's sensible However, you could consider a vfat filesystem as part of a 
> disk image file. I was working with one today (or trying to) on an Arm CPU 
> and an x86 (as a user, not at the programming level). I'm not sure whether 
> the Arm hardware requires alignment or not but I think the example 
> illustrates that a FAT file system could still be used on machines which 
> have none of the hardware or operating systems one normally associates with 
> such file structures.

You're still talking about an x86 platform- there's only a limited
amount of variation in how alignment is handled by C compilers targeting
x86 machines (at least, it's limited by comparison with the rest of the
C world). Find out what the precise range of variation is (I don't know,
and a newsgroup not targeted at x86 machines is not a good place to find
out), and write your code to deal with that range of variation.

My own code has portability requirements that would render it pointless
to worry about such things. I must extract fields byte by byte,
reconstructing the value that multi-byte fields represent, and using
mask and shift operations to extract the equivalent of bit-fields; no
simpler approach will work on all the platforms my code is required to
be portable to.

Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

Struct with unaligned fields "James Harris" <james.harris.1@gmail.com> - 2013-08-22 19:06 +0100
  Re: Struct with unaligned fields glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2013-08-22 18:18 +0000
  Re: Struct with unaligned fields James Kuyper <jameskuyper@verizon.net> - 2013-08-22 14:57 -0400
    Re: Struct with unaligned fields "James Harris" <james.harris.1@gmail.com> - 2013-08-22 21:05 +0100
      Re: Struct with unaligned fields James Kuyper <jameskuyper@verizon.net> - 2013-08-22 16:45 -0400
      Re: Struct with unaligned fields Keith Thompson <kst-u@mib.org> - 2013-08-22 15:12 -0700
    Re: Struct with unaligned fields glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2013-08-22 22:22 +0000
    Re: Struct with unaligned fields Les Cargill <lcargill99@comcast.com> - 2013-08-22 22:07 -0500
  Re: Struct with unaligned fields Stephen Sprunk <stephen@sprunk.org> - 2013-08-22 14:13 -0500
    Re: Struct with unaligned fields Les Cargill <lcargill99@comcast.com> - 2013-08-22 22:08 -0500
      Re: Struct with unaligned fields "James Harris" <james.harris.1@gmail.com> - 2013-08-23 09:36 +0100
        Re: Struct with unaligned fields Les Cargill <lcargill99@comcast.com> - 2013-08-23 12:48 -0500
    Re: Struct with unaligned fields "James Harris" <james.harris.1@gmail.com> - 2013-08-23 09:34 +0100
      Re: Struct with unaligned fields Stephen Sprunk <stephen@sprunk.org> - 2013-08-23 04:47 -0500
        Re: Struct with unaligned fields "James Harris" <james.harris.1@gmail.com> - 2013-08-23 15:31 +0100
          Re: Struct with unaligned fields Stephen Sprunk <stephen@sprunk.org> - 2013-08-23 13:31 -0500
            Re: Struct with unaligned fields "James Harris" <james.harris.1@gmail.com> - 2013-08-23 20:46 +0100
            Re: Struct with unaligned fields glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2013-08-23 20:48 +0000
  Re: Struct with unaligned fields Jorgen Grahn <grahn+nntp@snipabacken.se> - 2013-08-22 20:01 +0000
    Re: Struct with unaligned fields "James Harris" <james.harris.1@gmail.com> - 2013-08-23 12:54 +0100
      Re: Struct with unaligned fields Eric Sosman <esosman@comcast-dot-net.invalid> - 2013-08-23 08:17 -0400
      Re: Struct with unaligned fields Jorgen Grahn <grahn+nntp@snipabacken.se> - 2013-08-23 20:00 +0000
      Re: Struct with unaligned fields Tim Rentsch <txr@alumni.caltech.edu> - 2013-08-24 16:50 -0700
  Re: Struct with unaligned fields "BartC" <bc@freeuk.com> - 2013-08-22 21:59 +0100
    Re: Struct with unaligned fields Keith Thompson <kst-u@mib.org> - 2013-08-22 15:02 -0700
    Re: Struct with unaligned fields Tim Rentsch <txr@alumni.caltech.edu> - 2013-08-24 16:42 -0700
  Re: Struct with unaligned fields Siri Cruise <chine.bleu@yahoo.com> - 2013-08-22 14:21 -0700
  Re: Struct with unaligned fields Eric Sosman <esosman@comcast-dot-net.invalid> - 2013-08-22 17:23 -0400
    Re: Struct with unaligned fields "James Harris" <james.harris.1@gmail.com> - 2013-08-23 13:03 +0100
      Re: Struct with unaligned fields Eric Sosman <esosman@comcast-dot-net.invalid> - 2013-08-23 08:43 -0400
        Re: Struct with unaligned fields "James Harris" <james.harris.1@gmail.com> - 2013-08-23 20:20 +0100
          Re: Struct with unaligned fields James Kuyper <jameskuyper@verizon.net> - 2013-08-23 16:15 -0400
          Re: Struct with unaligned fields Ian Collins <ian-news@hotmail.com> - 2013-08-24 11:47 +1200
            Re: Struct with unaligned fields "James Harris" <james.harris.1@gmail.com> - 2013-08-24 14:01 +0100
              Re: Struct with unaligned fields Robert Wessel <robertwessel2@yahoo.com> - 2013-08-24 14:26 -0500
                Re: Struct with unaligned fields Ian Collins <ian-news@hotmail.com> - 2013-08-25 08:55 +1200
                Re: Struct with unaligned fields Robert Wessel <robertwessel2@yahoo.com> - 2013-08-25 04:17 -0500
                Re: Struct with unaligned fields Ian Collins <ian-news@hotmail.com> - 2013-08-25 21:22 +1200
                Re: Struct with unaligned fields Stephen Sprunk <stephen@sprunk.org> - 2013-08-24 16:16 -0500
                Re: Struct with unaligned fields Robert Wessel <robertwessel2@yahoo.com> - 2013-08-25 04:43 -0500
                Re: Struct with unaligned fields David Thompson <dave.thompson2@verizon.net> - 2013-08-28 22:27 -0400
                Re: Struct with unaligned fields Stephen Sprunk <stephen@sprunk.org> - 2013-08-29 14:30 -0500
      Re: Struct with unaligned fields "BartC" <bc@freeuk.com> - 2013-08-23 23:55 +0100
  Re: Struct with unaligned fields Les Cargill <lcargill99@comcast.com> - 2013-08-22 22:05 -0500
    Re: Struct with unaligned fields Stephen Sprunk <stephen@sprunk.org> - 2013-08-23 00:29 -0500
      Re: Struct with unaligned fields Les Cargill <lcargill99@comcast.com> - 2013-08-23 12:53 -0500
        Re: Struct with unaligned fields Stephen Sprunk <stephen@sprunk.org> - 2013-08-23 15:15 -0500
          Re: Struct with unaligned fields Les Cargill <lcargill99@comcast.com> - 2013-08-23 22:14 -0500
            Re: Struct with unaligned fields Ian Collins <ian-news@hotmail.com> - 2013-08-24 16:24 +1200
              Re: Struct with unaligned fields Les Cargill <lcargill99@comcast.com> - 2013-08-23 23:47 -0500
            Re: Struct with unaligned fields Keith Thompson <kst-u@mib.org> - 2013-08-24 01:16 -0700
              Re: Struct with unaligned fields Les Cargill <lcargill99@comcast.com> - 2013-08-24 09:45 -0500
                Re: Struct with unaligned fields Keith Thompson <kst-u@mib.org> - 2013-08-24 13:52 -0700
                Re: Struct with unaligned fields Tim Prince <tprince@computer.org> - 2013-08-24 17:07 -0400
                Re: Struct with unaligned fields Les Cargill <lcargill99@comcast.com> - 2013-08-24 16:35 -0500
                Re: Struct with unaligned fields Ian Collins <ian-news@hotmail.com> - 2013-08-25 09:51 +1200
                Re: Struct with unaligned fields Keith Thompson <kst-u@mib.org> - 2013-08-24 16:04 -0700
                Re: Struct with unaligned fields Les Cargill <lcargill99@comcast.com> - 2013-08-24 21:03 -0500
          Re: Struct with unaligned fields Eric Sosman <esosman@comcast-dot-net.invalid> - 2013-08-24 08:22 -0400
  Re: Struct with unaligned fields Nobody <nobody@nowhere.com> - 2013-08-23 17:51 +0100
  Re: Struct with unaligned fields falk@rahul.net (Edward A. Falk) - 2013-08-24 01:43 +0000

csiph-web