Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #153367 > unrolled thread

Are bitfields useful?

Back to article view | Back to comp.lang.c

Contents

  Are bitfields useful? Frederick Gotham <cauldwell.thomas@gmail.com> - 2020-07-30 02:10 -0700
    Re: Are bitfields useful? Jorgen Grahn <grahn+nntp@snipabacken.se> - 2020-07-30 09:50 +0000
      Re: Are bitfields useful? jacobnavia <jacob@jacob.remcomp.fr> - 2020-07-30 12:18 +0200
        Re: Are bitfields useful? Frederick Gotham <cauldwell.thomas@gmail.com> - 2020-07-30 04:01 -0700
      Re: Are bitfields useful? David Brown <david.brown@hesbynett.no> - 2020-08-03 12:26 +0200
    Re: Are bitfields useful? Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2020-07-30 03:10 -0700
    Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-30 12:11 +0100
    Re: Are bitfields useful? Richard Damon <Richard@Damon-Family.org> - 2020-07-30 07:54 -0400
      Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-30 13:14 +0100
        Re: Are bitfields useful? Richard Damon <Richard@Damon-Family.org> - 2020-07-30 21:18 -0400
    Re: Are bitfields useful? jadill33@gmail.com - 2020-07-30 09:57 -0700
    Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-30 18:37 -0400
      Re: Are bitfields useful? Lew Pitcher <lew.pitcher@digitalfreehold.ca> - 2020-07-30 19:15 -0400
        Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-30 20:36 -0400
    Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-30 19:01 -0400
      Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-31 01:49 +0100
        Re: Are bitfields useful? Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2020-07-30 18:08 -0700
        Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-30 19:28 -0700
          Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-31 11:15 +0100
            Re: Are bitfields useful? Jorgen Grahn <grahn+nntp@snipabacken.se> - 2020-07-31 11:47 +0000
            Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-31 05:00 -0700
              Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-31 13:53 +0100
                Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-31 07:59 -0700
                  Re: Are bitfields useful? Lew Pitcher <lew.pitcher@digitalfreehold.ca> - 2020-07-31 11:32 -0400
                    Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-31 13:55 -0400
                  Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-31 17:30 +0100
                    Re: Are bitfields useful? scott@slp53.sl.home (Scott Lurndal) - 2020-07-31 17:59 +0000
                    Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-31 11:19 -0700
                      Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-31 20:54 +0100
                        Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-31 15:14 -0700
                          Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-08-01 00:31 +0100
                            Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-31 16:55 -0700
                              Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-08-01 01:16 +0100
                                Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-31 20:31 -0700
                    Re: Are bitfields useful? Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2020-07-31 12:55 -0700
                      Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-31 21:04 +0100
              Re: Are bitfields useful? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-08-23 16:16 -0700
          Re: Are bitfields useful? Richard Damon <Richard@Damon-Family.org> - 2020-07-31 08:23 -0400
            Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-31 14:20 +0100
            Re: Are bitfields useful? Philipp Klaus Krause <pkk@spth.de> - 2020-08-04 17:21 +0200
        Re: Are bitfields useful? Kaz Kylheku <793-849-0957@kylheku.com> - 2020-08-25 00:13 +0000
          Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-08-25 12:42 +0100
          Re: Are bitfields useful? scott@slp53.sl.home (Scott Lurndal) - 2020-08-25 15:27 +0000
            Re: Are bitfields useful? Kaz Kylheku <793-849-0957@kylheku.com> - 2020-08-25 16:05 +0000
    Re: Are bitfields useful? Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2020-07-30 17:26 -0700
    Re: Are bitfields useful? Andrey Tarasevich <andreytarasevich@hotmail.com> - 2020-07-31 22:32 -0700
      Re: Are bitfields useful? Richard Damon <Richard@Damon-Family.org> - 2020-08-01 10:20 -0400
      Re: Are bitfields useful? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-08-23 07:54 -0700
      Re: Are bitfields useful? Kaz Kylheku <793-849-0957@kylheku.com> - 2020-08-25 00:26 +0000
        Re: Are bitfields useful? Kaz Kylheku <793-849-0957@kylheku.com> - 2020-08-25 23:55 +0000

Page 3 of 3 — ← Prev page 1 2 [3]

#153999

On 2020-07-31, Bart <bc@freeuk.com> wrote:
> When I tried this struct:
>
>   typedef struct {
>        char a:1;
>        long long int b:33;
>   } s2;
>
> then I variously got sizes of 8 or 16 (including both from different 
> versions of gcc), with one compiler rejecting it.

Which version/port of GCC produced a value of 16?

I don't see how it can be, unless sizeof (long long) == 16.

> So, what are the rules that will let me predict those results? For 
> example because I need to create a duplicate struct from another 
> language. Because I can't figure it out.

As far as GCC goes, I have tried to reverse engineer its the bitfield
allocation rules, while working on the FFI for TXR Lisp.

This was by means of black-box testing.

Rather than just coding something that works, I also documented my
assumptions, producing a "Bitfield Allocation Rules" section of the
document:

https://www.nongnu.org/txr/txr-manpage.html#TOC-10.10

>   typedef struct {
>        short a:9;
>        char b:7;
>   } s3;

What port of GCC to Windows produced a size of 4 for the last one?

4 doesn't make any sense, unless short is four bytes.

The Cygwin compilers, 32 and 64 bit, produce 2.

Aha! I figured it out. This is almost certianly coming from the
-mms-bitfields option. That is enabled in some MinGW compilers.

The following is under Cygwin:

Default:

  0:rocktron:~/txr$ gcc align.c -o align
  0:rocktron:~/txr$ ./align
  4 1 2

With Microsoft bitfield rules:

  0:rocktron:~/txr$ gcc -mms-bitfields align.c -o align
  0:rocktron:~/txr$ ./align.exe
  4 1 4

The reason is that according to the Microsoft rules, because the
two integer types (short, char) are different, the second
bitfield is not packed together with the previous one.

So that is to say, by flipping from short to char, we are saying,
"okay, we are done with the short type; short-sized allocation cell is
is done, and any bits are allocated after that cell". So the bits
of member b are allocated into the third byte.

Then, the whole thing is padded for short alignment, becoming four
bytes.

Given that the Cygwin compilers don't do this (and "everything" is
fine), it seems pretty retarded. This option should not even exist,
as far as I'm concerned. It's a recent addition that we got along just
fine without.

Under the regular GCC rules, there is so much control, that you can
conform to a Microsoft layout.

For instance, to finish the current "short" cell and move past it,
it can be done like this:

  typedef struct {
       short a:9;
  #ifdef __GNUC__ /* conform to MS bitfield layout */
       short  :0;
  #endif
       char b:7;
  } s3;

Now we get a size of 4 without -mms-bitfields.

-- 
TXR Programming Lanuage: http://nongnu.org/txr
Music DIY Mailing List:  http://www.kylheku.com/diy
ADA MP-1 Mailing List:   http://www.kylheku.com/mp1

[toc] | [prev] | [next] | [standalone]

#154006

On 25/08/2020 01:13, Kaz Kylheku wrote:
> On 2020-07-31, Bart <bc@freeuk.com> wrote:
>> When I tried this struct:
>>
>>    typedef struct {
>>         char a:1;
>>         long long int b:33;
>>    } s2;
>>
>> then I variously got sizes of 8 or 16 (including both from different
>> versions of gcc), with one compiler rejecting it.
> 
> Which version/port of GCC produced a value of 16?

It was on Windows:

  C:\c>type c.c
  #include <stdio.h>
  typedef struct {
        char a:1;
        long long int b:33;
  } s2;

  int main(void) {
      printf("%d\n",(int)sizeof(s2));
  }

  C:\c>gcc c.c

  C:\c>a
  16

  C:\c>gcc --version
  gcc (x86_64-posix-seh-rev0, Built by MinGW-W64 project) 8.1.0

> I don't see how it can be, unless sizeof (long long) == 16.

long long is 8 bytes. Presumably it doesn't combine the two bitfields 
for whatever reason, so it has to align b at offset +8, leading to 8 
bytes for 'char a:1', and 8 bytes for the long long.


>> So, what are the rules that will let me predict those results? For
>> example because I need to create a duplicate struct from another
>> language. Because I can't figure it out.
> 
> As far as GCC goes, I have tried to reverse engineer its the bitfield
> allocation rules, while working on the FFI for TXR Lisp.
> 
> This was by means of black-box testing.
> 
> Rather than just coding something that works, I also documented my
> assumptions, producing a "Bitfield Allocation Rules" section of the
> document:
> 
> https://www.nongnu.org/txr/txr-manpage.html#TOC-10.10
> 
>>    typedef struct {
>>         short a:9;
>>         char b:7;
>>    } s3;
> 
> What port of GCC to Windows produced a size of 4 for the last one?
> 
> 4 doesn't make any sense, unless short is four bytes.

Same as above. Short is the regular 2 bytes. However it will combine 
them if both are short, into a single short field. For some reason, this 
gcc doesn't combine bitfields when their host types are different.

> The reason is that according to the Microsoft rules, because the
> two integer types (short, char) are different, the second
> bitfield is not packed together with the previous one.

I've given up trying to understand it. Whatever a compiler decides to do 
is apparently conforming and therefore not actually wrong.

But I noticed it being used in structs exported by a major library (GTK) 
where it might just work by luck (eg. bitfields have a consistent host 
type).

[toc] | [prev] | [next] | [standalone]

#154009

Kaz Kylheku <793-849-0957@kylheku.com> writes:
>On 2020-07-31, Bart <bc@freeuk.com> wrote:
>> When I tried this struct:
>>
>>   typedef struct {
>>        char a:1;
>>        long long int b:33;
>>   } s2;
>>
>> then I variously got sizes of 8 or 16 (including both from different 
>> versions of gcc), with one compiler rejecting it.
>
>Which version/port of GCC produced a value of 16?
>
>I don't see how it can be, unless sizeof (long long) == 16.

It should be 16 for any 64-bit system conforming to the standard
x86 ABI.   The long long needs to be 8-byte aligned.  This leaves
a 7-byte hole after the 1-bit field.

[toc] | [prev] | [next] | [standalone]

#154010

On 2020-08-25, Scott Lurndal <scott@slp53.sl.home> wrote:
> Kaz Kylheku <793-849-0957@kylheku.com> writes:
>>On 2020-07-31, Bart <bc@freeuk.com> wrote:
>>> When I tried this struct:
>>>
>>>   typedef struct {
>>>        char a:1;
>>>        long long int b:33;
>>>   } s2;
>>>
>>> then I variously got sizes of 8 or 16 (including both from different 
>>> versions of gcc), with one compiler rejecting it.
>>
>>Which version/port of GCC produced a value of 16?
>>
>>I don't see how it can be, unless sizeof (long long) == 16.
>
> It should be 16 for any 64-bit system conforming to the standard
> x86 ABI.   The long long needs to be 8-byte aligned.  This leaves
> a 7-byte hole after the 1-bit field.

The actual reason is that the MinGW64 toolchain has enabled Microsoft
bitfield rules (-mms-bitfields) by default.

Without -mms-bitfields, regular GCC bitfield allocation rules apply and
the size of the structure is 8.

The Microsoft rules call for adjacent bitfields of different type not to
be packed into the same allocation unit.

So above, the "char a:1" allocates one bit already, in a byte-wide
allocation cell.

Under regular GCC rules, the subsequent "long long int b:33" will
ignore the "char" type. It will consider the "long long" sized allocation
unit from which 1 bit has already been claimed. It will see that it
still has 63 bits left in that unit, and put 34 more into it.

Under Microsoft rules, the type of the previuos bitfield matters. It's
a different type, and so it will start a new allocation unit of
long long type. That will be aligned at offset 8.

Since the long long bitfield is named, it contributes to alignment:
the struct must be 8 byte aligned, and so in the one case it will be
8 bytes, and in the other 16.

Note that the Microsoft rules skirt the C standard which has wording
very similar to "[i]f enough space remains, a bit-field that immediately
follows another bit-field in a structure shall be packed into adjacent
bits of the same unit."

I am also noticing that, under the Microsoft rules (-mms-bitfields),
there is no difference whether the bitfield is named or not: it
contributes to alignment. This does not appear to be documented!

So that is to say, the following has size 5 under regular rules:

  typedef struct {
    char a:1;
    long long :34; /* no name -> size 5; name -> size 8 */
  } s1;

But it has size 16 under -mms-bitfields. I don't see any reference to
this in the GCC documentation for -mms-bitfields.

Under regular rules, an unnamed bitfield has no alignment. 35 bits
have been allocated into the structure, and that is covered by five
bytes. There is no alignment padding, so five bytes it is.

Under the Microsoft rules, if only those effects take place which are
documented, I would expect the unnamed bitfield to claim five bytes
starting at offset 8 for a size of 13.  I suppose that it is nonsensical
to deliberately start a new, *aligned* allocation unit for the bitfield,
but then turn around and pretend that it's not aligned and not generate
any padding.

-- 
TXR Programming Lanuage: http://nongnu.org/txr
Music DIY Mailing List:  http://www.kylheku.com/diy
ADA MP-1 Mailing List:   http://www.kylheku.com/mp1

[toc] | [prev] | [next] | [standalone]

#153379

Frederick Gotham <cauldwell.thomas@gmail.com> writes:
[...]
> I want to set the member "transparent" on both platforms, and so I
> started out with code like this:
>
>     void Set_Flag_For_Transparency(Manager &e)
>     {
>         float *const p_z = &e.z;
>         u8 *const p_offset = reinterpret_cast<u8*>(p_z + 1u);
>         u8 *const p_byte_containing_transparent = p_offset + 1u;
>         *p_byte_containing_transparent |= 0x80;  //Set the high bit
>     }
[...]

Why are you posting C++-specific code in comp.lang.c?

-- 
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Philips Healthcare
void Void(void) { Void(); } /* The recursive call of the void */

[toc] | [prev] | [next] | [standalone]

#153406

On 7/30/2020 2:10 AM, Frederick Gotham wrote:
> Are bitfields pretty much useless other than for limiting the max value of an integer type (e.g. 3 bits for 7, 4 bits for 15, 5 bits for 31)?

They are "useless" if you attempt to use them to tailor memory layout of 
your struct type to some predefined external specification.

But that is not what bit-fields are for.

Bit-fields exist in the language as means of saving memory consumed by 
mass-instantiated objects. For such objects the exact memory layout does 
not matter. All that matters is that bit-fields are _packed_ in some 
implementation-defined way. They could be packed completely differently 
from implementation to implementation, but that's something that can be 
said about struct layout in general.

It is true that language specification does not even guarantee that 
bit-fields are packed somehow, but that is the original _intent_ behind 
bit-fields and all self-respecting implementations follow that intent.

With bit-fields we want packing and the resultant memory savings. We 
don't care about how they are packed. And we don't care about the 
consistency of that packing (from implementation to implementation).

This is something you have to keep in mind when using bit-fields. If at 
any moment you ask more from them, it means that you are using them 
incorrectly.

Using bit-fields for fine-tuning bit-layout of a struct type has never 
been the intent. Don't attempt to use bit-fields for that purpose.

So, to resume, your assumption that bit-fields are somehow "useless" is 
based on your misconceptions about their intended application. No, they 
are not useless at all.

-- 
Bets regards,
Andrey Tarasevich

[toc] | [prev] | [next] | [standalone]

#153408

On 8/1/20 1:32 AM, Andrey Tarasevich wrote:
> On 7/30/2020 2:10 AM, Frederick Gotham wrote:
>> Are bitfields pretty much useless other than for limiting the max
>> value of an integer type (e.g. 3 bits for 7, 4 bits for 15, 5 bits for
>> 31)?
> 
> They are "useless" if you attempt to use them to tailor memory layout of
> your struct type to some predefined external specification.
> 
> But that is not what bit-fields are for.
> 
> Bit-fields exist in the language as means of saving memory consumed by
> mass-instantiated objects. For such objects the exact memory layout does
> not matter. All that matters is that bit-fields are _packed_ in some
> implementation-defined way. They could be packed completely differently
> from implementation to implementation, but that's something that can be
> said about struct layout in general.
> 
> It is true that language specification does not even guarantee that
> bit-fields are packed somehow, but that is the original _intent_ behind
> bit-fields and all self-respecting implementations follow that intent.
> 
> With bit-fields we want packing and the resultant memory savings. We
> don't care about how they are packed. And we don't care about the
> consistency of that packing (from implementation to implementation).
> 
> This is something you have to keep in mind when using bit-fields. If at
> any moment you ask more from them, it means that you are using them
> incorrectly.
> 
> Using bit-fields for fine-tuning bit-layout of a struct type has never
> been the intent. Don't attempt to use bit-fields for that purpose.
> 
> So, to resume, your assumption that bit-fields are somehow "useless" is
> based on your misconceptions about their intended application. No, they
> are not useless at all.
> 

At the time the language was being developed, many/most machines did NOT
have memory mapped I/O, but specific instructions to talk to I/O devices
in a different address space. As such, mapping a bit-field into memory
for an I/O device didn't make sense.

One problem with trying to conform a struct to an external definition is
that often to do so would you would need to allows for access patterns
that the target machine just didn't provide, wrong byte order, unaligned
multi-byte structures, etc. Typically what was done, was you took that
'external' packet and copied it into an internal structure that was
organized to store the data in a format that the processor could
actually handle efficiently.

Remember also, at this time memory transfers were fast, often a single
processor cycle (basically the whole program would live in Level 1
cache), so copying data from the external buffer to an internal format
was cheap.

[toc] | [prev] | [next] | [standalone]

#153964

Andrey Tarasevich <andreytarasevich@hotmail.com> writes:

<snip>

> It is true that language specification does not even guarantee that
> bit-fields are packed somehow, [...]

The ISO C standard doesn't guarantee anything.  It does, however,
require that conforming implementations pack adjacent bit-fields in
a certain way, subject to a small number of implementation-defined
choices.

[toc] | [prev] | [next] | [standalone]

#154000

On 2020-08-01, Andrey Tarasevich <andreytarasevich@hotmail.com> wrote:
> On 7/30/2020 2:10 AM, Frederick Gotham wrote:
>> Are bitfields pretty much useless other than for limiting the max value of an integer type (e.g. 3 bits for 7, 4 bits for 15, 5 bits for 31)?
>
> They are "useless" if you attempt to use them to tailor memory layout of 
> your struct type to some predefined external specification.

I've seen plenty of device drivers use bitfields this way, with at worst
some #ifdef for endiannness:

   struct wonderdev_status_reg {
   #ifdef BIG_ENDIAN
      unsigned resetting : 1;
      unsigned meditating : 1;
      unsigned : 5; /* reserved */
      unsigned interrupt : 1;
      ....
    #else
      ....
      unsigned interrupt : 1;
      unsigned : 5; /* reserved */
      unsigned meditating : 1;
      unsigned resetting : 1;
    #endif
   };

> But that is not what bit-fields are for.
>
> Bit-fields exist in the language as means of saving memory consumed by 
> mass-instantiated objects.

And so then the unnamed bitfields (including the zero width one) must
exist for wasting some of that memory again, in case the rampant saving
gets out of hand.

[toc] | [prev] | [next] | [standalone]

#154011

On 2020-08-25, Kaz Kylheku <793-849-0957@kylheku.com> wrote:
> On 2020-08-01, Andrey Tarasevich <andreytarasevich@hotmail.com> wrote:
>> On 7/30/2020 2:10 AM, Frederick Gotham wrote:
>>> Are bitfields pretty much useless other than for limiting the max value of an integer type (e.g. 3 bits for 7, 4 bits for 15, 5 bits for 31)?
>>
>> They are "useless" if you attempt to use them to tailor memory layout of 
>> your struct type to some predefined external specification.
>
> I've seen plenty of device drivers use bitfields this way, with at worst
> some #ifdef for endiannness:
>
>    struct wonderdev_status_reg {
>    #ifdef BIG_ENDIAN
>       unsigned resetting : 1;
>       unsigned meditating : 1;
>       unsigned : 5; /* reserved */
>       unsigned interrupt : 1;
>       ....
>     #else
>       ....
>       unsigned interrupt : 1;
>       unsigned : 5; /* reserved */
>       unsigned meditating : 1;
>       unsigned resetting : 1;
>     #endif
>    };

IP header from Linux kernel:

    struct iphdr {
    #if defined(__LITTLE_ENDIAN_BITFIELD)
            __u8    ihl:4,
                    version:4;
    #elif defined (__BIG_ENDIAN_BITFIELD)
            __u8    version:4,
                    ihl:4;
    #else
    #error  "Please fix <asm/byteorder.h>"
    #endif
            __u8    tos;
            __be16  tot_len;
            __be16  id;
            __be16  frag_off;
            __u8    ttl;
            __u8    protocol;
            __sum16 check;
            __be32  saddr;
            __be32  daddr;
            /*The options start here. */
    };

-- 
TXR Programming Lanuage: http://nongnu.org/txr
Music DIY Mailing List:  http://www.kylheku.com/diy
ADA MP-1 Mailing List:   http://www.kylheku.com/mp1

[toc] | [prev] | [standalone]

Page 3 of 3 — ← Prev page 1 2 [3]

Back to top | Article view | comp.lang.c

csiph-web