Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #153367 > unrolled thread

Are bitfields useful?

Back to article view | Back to comp.lang.c

Contents

  Are bitfields useful? Frederick Gotham <cauldwell.thomas@gmail.com> - 2020-07-30 02:10 -0700
    Re: Are bitfields useful? Jorgen Grahn <grahn+nntp@snipabacken.se> - 2020-07-30 09:50 +0000
      Re: Are bitfields useful? jacobnavia <jacob@jacob.remcomp.fr> - 2020-07-30 12:18 +0200
        Re: Are bitfields useful? Frederick Gotham <cauldwell.thomas@gmail.com> - 2020-07-30 04:01 -0700
      Re: Are bitfields useful? David Brown <david.brown@hesbynett.no> - 2020-08-03 12:26 +0200
    Re: Are bitfields useful? Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2020-07-30 03:10 -0700
    Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-30 12:11 +0100
    Re: Are bitfields useful? Richard Damon <Richard@Damon-Family.org> - 2020-07-30 07:54 -0400
      Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-30 13:14 +0100
        Re: Are bitfields useful? Richard Damon <Richard@Damon-Family.org> - 2020-07-30 21:18 -0400
    Re: Are bitfields useful? jadill33@gmail.com - 2020-07-30 09:57 -0700
    Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-30 18:37 -0400
      Re: Are bitfields useful? Lew Pitcher <lew.pitcher@digitalfreehold.ca> - 2020-07-30 19:15 -0400
        Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-30 20:36 -0400
    Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-30 19:01 -0400
      Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-31 01:49 +0100
        Re: Are bitfields useful? Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2020-07-30 18:08 -0700
        Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-30 19:28 -0700
          Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-31 11:15 +0100
            Re: Are bitfields useful? Jorgen Grahn <grahn+nntp@snipabacken.se> - 2020-07-31 11:47 +0000
            Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-31 05:00 -0700
              Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-31 13:53 +0100
                Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-31 07:59 -0700
                  Re: Are bitfields useful? Lew Pitcher <lew.pitcher@digitalfreehold.ca> - 2020-07-31 11:32 -0400
                    Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-31 13:55 -0400
                  Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-31 17:30 +0100
                    Re: Are bitfields useful? scott@slp53.sl.home (Scott Lurndal) - 2020-07-31 17:59 +0000
                    Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-31 11:19 -0700
                      Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-31 20:54 +0100
                        Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-31 15:14 -0700
                          Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-08-01 00:31 +0100
                            Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-31 16:55 -0700
                              Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-08-01 01:16 +0100
                                Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-31 20:31 -0700
                    Re: Are bitfields useful? Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2020-07-31 12:55 -0700
                      Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-31 21:04 +0100
              Re: Are bitfields useful? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-08-23 16:16 -0700
          Re: Are bitfields useful? Richard Damon <Richard@Damon-Family.org> - 2020-07-31 08:23 -0400
            Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-31 14:20 +0100
            Re: Are bitfields useful? Philipp Klaus Krause <pkk@spth.de> - 2020-08-04 17:21 +0200
        Re: Are bitfields useful? Kaz Kylheku <793-849-0957@kylheku.com> - 2020-08-25 00:13 +0000
          Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-08-25 12:42 +0100
          Re: Are bitfields useful? scott@slp53.sl.home (Scott Lurndal) - 2020-08-25 15:27 +0000
            Re: Are bitfields useful? Kaz Kylheku <793-849-0957@kylheku.com> - 2020-08-25 16:05 +0000
    Re: Are bitfields useful? Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2020-07-30 17:26 -0700
    Re: Are bitfields useful? Andrey Tarasevich <andreytarasevich@hotmail.com> - 2020-07-31 22:32 -0700
      Re: Are bitfields useful? Richard Damon <Richard@Damon-Family.org> - 2020-08-01 10:20 -0400
      Re: Are bitfields useful? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-08-23 07:54 -0700
      Re: Are bitfields useful? Kaz Kylheku <793-849-0957@kylheku.com> - 2020-08-25 00:26 +0000
        Re: Are bitfields useful? Kaz Kylheku <793-849-0957@kylheku.com> - 2020-08-25 23:55 +0000

Page 1 of 3  [1] 2 3  Next page →

#153367 — Are bitfields useful?

This week I'm working on two different products that link with the same library, however one product links with a newer version of the library. The older library has a struct like this:

    typedef struct {
        u16 scale;
        u8 window;
        u8 a;
        u8 b;
        u8 c;
        float x;
        float y;
        float z;
        u32 reserved[4];
    } Manager;


And the newer library has the same struct like this:

    typedef struct {
        u16 scale;
        u8 window;
        u8 a;
        u8 b;
        u8 c;
        float x;
        float y;
        float z;
        s8 offset;
        u8 transparent : 1;
        u8 reserved0 : 7;
        u8 reserved1[2];
        u32 reserved2[3];
    } Manager;

I want to set the member "transparent" on both platforms, and so I started out with code like this:

    void Set_Flag_For_Transparency(Manager &e)
    {
        float *const p_z = &e.z;
        u8 *const p_offset = reinterpret_cast<u8*>(p_z + 1u);
        u8 *const p_byte_containing_transparent = p_offset + 1u;
        *p_byte_containing_transparent |= 0x80;  //Set the high bit
    }

After doing a little reading up on bitfields, it seems that ISO/ANSI standards give compilers a lot of freedom as to how bitfields are implemented -- so much freedom in fact that they don't have much use in portable code. For example, in my function just above, the 'transparent' bit might be 0x80 or it might be 0x01. Bits might not straddle bytes, and really the only thing that's guaranteed is the number of bits of precision (e.g. if it's 3 bits then the max value is 7).

Are bitfields pretty much useless other than for limiting the max value of an integer type (e.g. 3 bits for 7, 4 bits for 15, 5 bits for 31)?

The people who wrote the library I'm working with either:
(A) Expect me to use the same compiler as them
(B) Expect me to use a compiler that implements bitfields the same way
(C) Don't know that bitfields aren't portable for this purpose

[toc] | [next] | [standalone]

#153368

On Thu, 2020-07-30, Frederick Gotham wrote:
>
> This week I'm working on two different products that link with the
> same library, however one product links with a newer version of the
> library. The older library has a struct like this:
>
>     typedef struct {
>         u16 scale;
>         u8 window;
>         u8 a;
>         u8 b;
>         u8 c;
>         float x;
>         float y;
>         float z;
>         u32 reserved[4];
>     } Manager;
>
>
> And the newer library has the same struct like this:
>
>     typedef struct {
>         u16 scale;
>         u8 window;
>         u8 a;
>         u8 b;
>         u8 c;
>         float x;
>         float y;
>         float z;
>         s8 offset;
>         u8 transparent : 1;
>         u8 reserved0 : 7;
>         u8 reserved1[2];
>         u32 reserved2[3];
>     } Manager;
>
> I want to set the member "transparent" on both platforms,

The only way to do that with the old library is to read its
documentation on the subject.  Sounds a bit unlikely that they would
have support for transparency if they didn't bother to include it in
the API.

> and so I
> started out with code like this:
>
>     void Set_Flag_For_Transparency(Manager &e)
>     {
>         float *const p_z = &e.z;
>         u8 *const p_offset = reinterpret_cast<u8*>(p_z + 1u);
>         u8 *const p_byte_containing_transparent = p_offset + 1u;
>         *p_byte_containing_transparent |= 0x80;  //Set the high bit
>     }

Now you're assuming both libraries really just transfer the same chunk
of bits, and provides Manager (the struct) just for convenience. That
assumption could be unfounded.

> After doing a little reading up on bitfields, it seems that ISO/ANSI
> standards give compilers a lot of freedom as to how bitfields are
> implemented -- so much freedom in fact that they don't have much use
> in portable code. For example, in my function just above, the
> 'transparent' bit might be 0x80 or it might be 0x01. Bits might not
> straddle bytes, and really the only thing that's guaranteed is the
> number of bits of precision (e.g. if it's 3 bits then the max value
> is 7).
>
> Are bitfields pretty much useless other than for limiting the max
> value of an integer type (e.g. 3 bits for 7, 4 bits for 15, 5 bits
> for 31)?

No. Just don't pretend to know they're laid out in memory in any
specific way, and you'll be fine.

I don't see it as very different from how you can't know how an int is
laid out in memory in terms of bytes.

> The people who wrote the library I'm working with either:
> (A) Expect me to use the same compiler as them

Yes. You have to have tools that share the same ABI.  But neither of
you have to know its exact rules (not until you use a debugger,
anyway).

It's nothing that most people have to worry about, because there's
typically only one ABI on a given system.

> (B) Expect me to use a compiler that implements bitfields the same way
> (C) Don't know that bitfields aren't portable for this purpose

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#153370

Le 30/07/2020 à 11:50, Jorgen Grahn a écrit :
> The only way to do that with the old library is to read its
> documentation on the subject.  Sounds a bit unlikely that they would
> have support for transparency if they didn't bother to include it in
> the API.

+1

The old version of the library doesn't support transparency!

OBVIOUSLY!

[toc] | [prev] | [next] | [standalone]

#153371

On Thursday, July 30, 2020 at 11:18:19 AM UTC+1, jacobnavia wrote:
> Le 30/07/2020 à 11:50, Jorgen Grahn a écrit :
> > The only way to do that with the old library is to read its
> > documentation on the subject.  Sounds a bit unlikely that they would
> > have support for transparency if they didn't bother to include it in
> > the API.
> 
> +1
> 
> The old version of the library doesn't support transparency!
> 
> OBVIOUSLY!


In my job, we are occasionally supplied with a new binary for a library, e.g. libmonkey.so, before we're given the updated header file, e.g. monkey.h.

Anyway I need my code to compile and work properly no matter which version of the library it's using (i.e. I need a common codebase for all architectures and versions of the library).

[toc] | [prev] | [next] | [standalone]

#153418

On 30/07/2020 11:50, Jorgen Grahn wrote:
> On Thu, 2020-07-30, Frederick Gotham wrote:
>>

>> Are bitfields pretty much useless other than for limiting the max
>> value of an integer type (e.g. 3 bits for 7, 4 bits for 15, 5 bits
>> for 31)?
> 
> No. Just don't pretend to know they're laid out in memory in any
> specific way, and you'll be fine.
> 

More accurately, don't assume that the layout is portable in any way. 
But sometimes you do know the way the layout is done for a particular 
target and compiler (and perhaps even compiler options), and that can be 
useful - as long as you are very careful about the lack of portability.

> I don't see it as very different from how you can't know how an int is
> laid out in memory in terms of bytes.
> 

[toc] | [prev] | [next] | [standalone]

#153369

On Thursday, 30 July 2020 10:10:13 UTC+1, Frederick Gotham  wrote:
> This week I'm working on two different products that link with the same library, however one product links with a newer version of the library. The older library has a struct like this:
> 
>     typedef struct {
>         u16 scale;
>         u8 window;
>         u8 a;
>         u8 b;
>         u8 c;
>         float x;
>         float y;
>         float z;
>         u32 reserved[4];
>     } Manager;
> 
> 
> And the newer library has the same struct like this:
> 
>     typedef struct {
>         u16 scale;
>         u8 window;
>         u8 a;
>         u8 b;
>         u8 c;
>         float x;
>         float y;
>         float z;
>         s8 offset;
>         u8 transparent : 1;
>         u8 reserved0 : 7;
>         u8 reserved1[2];
>         u32 reserved2[3];
>     } Manager;
> 
So what you want to do is declare a struct "HackManager" with the same layout 
as the second version of "Manager". Then set the bit in "HackManager" and
copy the entire structure to a "Manager" before passing it to the library.

That way you will always hit the right bit even if "Manager" is the old form
of the struct.

[toc] | [prev] | [next] | [standalone]

#153372

On 30/07/2020 10:10, Frederick Gotham wrote:
> 
> This week I'm working on two different products that link with the same library, however one product links with a newer version of the library. The older library has a struct like this:
> 
>      typedef struct {
>          u16 scale;
>          u8 window;
>          u8 a;
>          u8 b;
>          u8 c;
>          float x;
>          float y;
>          float z;
>          u32 reserved[4];
>      } Manager;
> 
> 
> And the newer library has the same struct like this:
> 
>      typedef struct {
>          u16 scale;
>          u8 window;
>          u8 a;
>          u8 b;
>          u8 c;
>          float x;
>          float y;
>          float z;
>          s8 offset;
>          u8 transparent : 1;
>          u8 reserved0 : 7;
>          u8 reserved1[2];
>          u32 reserved2[3];
>      } Manager;
> 
> I want to set the member "transparent" on both platforms, and so I started out with code like this:
> 
>      void Set_Flag_For_Transparency(Manager &e)
>      {
>          float *const p_z = &e.z;
>          u8 *const p_offset = reinterpret_cast<u8*>(p_z + 1u);
>          u8 *const p_byte_containing_transparent = p_offset + 1u;
>          *p_byte_containing_transparent |= 0x80;  //Set the high bit
>      }
> 
> After doing a little reading up on bitfields, it seems that ISO/ANSI standards give compilers a lot of freedom as to how bitfields are implemented -- so much freedom in fact that they don't have much use in portable code. For example, in my function just above, the 'transparent' bit might be 0x80 or it might be 0x01. Bits might not straddle bytes, and really the only thing that's guaranteed is the number of bits of precision (e.g. if it's 3 bits then the max value is 7).
> 
> Are bitfields pretty much useless other than for limiting the max value of an integer type (e.g. 3 bits for 7, 4 bits for 15, 5 bits for 31)?
> 
> The people who wrote the library I'm working with either:
> (A) Expect me to use the same compiler as them
> (B) Expect me to use a compiler that implements bitfields the same way
> (C) Don't know that bitfields aren't portable for this purpose
> 

I /think/ there is supposed to be compatibility between compilers for 
the same platform. I don't know if that is language enforced or just a 
general agreement, or defined by an ABI.

I know that a critical data type of the GTK library is a struct with 
bitfields. Those bitfields define the overall size of the struct which 
is the important bit.

I was trying to use that library via a different language, not even C, 
so it presented some problems. I temporarily got around by writing a C 
program to display the size of the struct, then arranged for my own 
struct to be the same size. But this only worked because my test didn't 
need to access the bitfield elements.

 > Are bitfields pretty much useless other than for limiting the max 
value of an integer type (e.g. 3 bits for 7, 4 bits for 15, 5 bits for 31)?

Mainly they are mainly used to be able to pack things more efficiently 
when it is known that values have a limited range. Your example would 
need 12 bits (requiring 16 bits in the struct, with 4 spare) compared 
with 24 bits if three char types were used.

[toc] | [prev] | [next] | [standalone]

#153373

On 7/30/20 5:10 AM, Frederick Gotham wrote:
> After doing a little reading up on bitfields, it seems that ISO/ANSI standards give compilers a lot of freedom as to how bitfields are implemented -- so much freedom in fact that they don't have much use in portable code. For example, in my function just above, the 'transparent' bit might be 0x80 or it might be 0x01. Bits might not straddle bytes, and really the only thing that's guaranteed is the number of bits of precision (e.g. if it's 3 bits then the max value is 7).
> 
> Are bitfields pretty much useless other than for limiting the max value of an integer type (e.g. 3 bits for 7, 4 bits for 15, 5 bits for 31)?
> 
> The people who wrote the library I'm working with either:
> (A) Expect me to use the same compiler as them
> (B) Expect me to use a compiler that implements bitfields the same way
> (C) Don't know that bitfields aren't portable for this purpose

When compiling code to link with code complied by someone else, it is
ALWAYS required that either you use the same compiler (invoked with the
same options) or use another compiler/option set that is known to be
compatible. PERIOD. That is one purposes of defining the ABI for a
system, it sets the ground rules for these sorts of decisions.

Note, the ABI is not the C Standard, but the ABI, in effect, makes a lot
of the definitions that the C Standard indicates are Implementation
Defined (or sometime just Unspecified) and locks those choices for all
implementations that wish to claim compliance to that ABI as well as the
Standard.

Once you understand that, then, it can be seen that bitfields are very
useful, and usable in portable code, as long as you use them in a
portable manner. Using a bitfield as part of a union (or your effective
union by assuming two structures are 'the same') has some non-portable
behavior, and in fact there is no promise that your two structs will be
even the same size.

The biggest portable use of bit-fields is that by putting fields into a
bit field, you likely reduce the memory cost to store those items (but
likely increase the memory cost to access them, so best for things that
are stored a lot, in arrays for example).

On many systems, due to the ABI, bit-fields become VERY useful to match
hardware structures, but because this use depends on the ABI, its
portability is limited systems that match that ABI, but then the use of
hardware structures is pretty much not portable to a wide number of
different machines.

They are less useful for matching software format structures (file
formats or communication packet formats), as these may be used on a
variety of machines, with different ABIs, so their use for this sort of
thing is limited to implementations with matching ABIs for at least this
sort of thing.

[toc] | [prev] | [next] | [standalone]

#153374

On 30/07/2020 12:54, Richard Damon wrote:
> On 7/30/20 5:10 AM, Frederick Gotham wrote:
>> After doing a little reading up on bitfields, it seems that ISO/ANSI standards give compilers a lot of freedom as to how bitfields are implemented -- so much freedom in fact that they don't have much use in portable code. For example, in my function just above, the 'transparent' bit might be 0x80 or it might be 0x01. Bits might not straddle bytes, and really the only thing that's guaranteed is the number of bits of precision (e.g. if it's 3 bits then the max value is 7).
>>
>> Are bitfields pretty much useless other than for limiting the max value of an integer type (e.g. 3 bits for 7, 4 bits for 15, 5 bits for 31)?
>>
>> The people who wrote the library I'm working with either:
>> (A) Expect me to use the same compiler as them
>> (B) Expect me to use a compiler that implements bitfields the same way
>> (C) Don't know that bitfields aren't portable for this purpose
> 
> When compiling code to link with code complied by someone else, it is
> ALWAYS required that either you use the same compiler (invoked with the
> same options) or use another compiler/option set that is known to be
> compatible. PERIOD. That is one purposes of defining the ABI for a
> system, it sets the ground rules for these sorts of decisions.
> 
> Note, the ABI is not the C Standard, but the ABI, in effect, makes a lot
> of the definitions that the C Standard indicates are Implementation
> Defined (or sometime just Unspecified) and locks those choices for all
> implementations that wish to claim compliance to that ABI as well as the
> Standard.

Do the ABIs (for say x64 for Windows and Linux) specify in detail how C 
bitfields should be allocated?

They do seem to me to be heavily influenced by how C works.

> Once you understand that, then, it can be seen that bitfields are very
> useful, and usable in portable code, as long as you use them in a
> portable manner.

I added bitfields in my non-C language in a way that is much more 
portable (eg. defined only within 8/16/32/64-bit ints).

I was unable to implement C bitfields a few years ago partly because 
they were so poorly defined.

[toc] | [prev] | [next] | [standalone]

#153383

On 7/30/20 8:14 AM, Bart wrote:
> On 30/07/2020 12:54, Richard Damon wrote:
>> On 7/30/20 5:10 AM, Frederick Gotham wrote:
>>> After doing a little reading up on bitfields, it seems that ISO/ANSI
>>> standards give compilers a lot of freedom as to how bitfields are
>>> implemented -- so much freedom in fact that they don't have much use
>>> in portable code. For example, in my function just above, the
>>> 'transparent' bit might be 0x80 or it might be 0x01. Bits might not
>>> straddle bytes, and really the only thing that's guaranteed is the
>>> number of bits of precision (e.g. if it's 3 bits then the max value
>>> is 7).
>>>
>>> Are bitfields pretty much useless other than for limiting the max
>>> value of an integer type (e.g. 3 bits for 7, 4 bits for 15, 5 bits
>>> for 31)?
>>>
>>> The people who wrote the library I'm working with either:
>>> (A) Expect me to use the same compiler as them
>>> (B) Expect me to use a compiler that implements bitfields the same way
>>> (C) Don't know that bitfields aren't portable for this purpose
>>
>> When compiling code to link with code complied by someone else, it is
>> ALWAYS required that either you use the same compiler (invoked with the
>> same options) or use another compiler/option set that is known to be
>> compatible. PERIOD. That is one purposes of defining the ABI for a
>> system, it sets the ground rules for these sorts of decisions.
>>
>> Note, the ABI is not the C Standard, but the ABI, in effect, makes a lot
>> of the definitions that the C Standard indicates are Implementation
>> Defined (or sometime just Unspecified) and locks those choices for all
>> implementations that wish to claim compliance to that ABI as well as the
>> Standard.
> 
> Do the ABIs (for say x64 for Windows and Linux) specify in detail how C
> bitfields should be allocated?
> 
> They do seem to me to be heavily influenced by how C works.
> 
>> Once you understand that, then, it can be seen that bitfields are very
>> useful, and usable in portable code, as long as you use them in a
>> portable manner.
> 
> I added bitfields in my non-C language in a way that is much more
> portable (eg. defined only within 8/16/32/64-bit ints).
> 
> I was unable to implement C bitfields a few years ago partly because
> they were so poorly defined.
> 

The ABI's typically define these sorts of things, so the definitions
used by system calls (and 3rd party libraries) can use those features.
They also tend to define things like how structs get padded, and lots of
similar details. Ultimately, almost anything that an implementation
could make a decision on relating to how data is laid out, tends to get
specified in the ABI, as well as the details of how a function call works.

One problem with the detail of a typical ABI is that people who learn on
a single system then to end up assuming ALL systems make the same
assumptions, and THAT is where the problem lie.

[toc] | [prev] | [next] | [standalone]

#153375

On Thursday, July 30, 2020 at 5:10:13 AM UTC-4, Frederick Gotham wrote:
> This week I'm working on two different products that link with the same library, however one product links with a newer version of the library. The older library has a struct like this:
> 
>     typedef struct {
>         u16 scale;
>         u8 window;
>         u8 a;
>         u8 b;
>         u8 c;
>         float x;
>         float y;
>         float z;
>         u32 reserved[4];
>     } Manager;
> 
> 
> And the newer library has the same struct like this:
> 
>     typedef struct {
>         u16 scale;
>         u8 window;
>         u8 a;
>         u8 b;
>         u8 c;
>         float x;
>         float y;
>         float z;
>         s8 offset;
>         u8 transparent : 1;
>         u8 reserved0 : 7;
>         u8 reserved1[2];
>         u32 reserved2[3];
>     } Manager;
> 
> I want to set the member "transparent" on both platforms, and so I started out with code like this:
> 
>     void Set_Flag_For_Transparency(Manager &e)
>     {
>         float *const p_z = &e.z;
>         u8 *const p_offset = reinterpret_cast<u8*>(p_z + 1u);
>         u8 *const p_byte_containing_transparent = p_offset + 1u;
>         *p_byte_containing_transparent |= 0x80;  //Set the high bit
>     }
> 
> After doing a little reading up on bitfields, it seems that ISO/ANSI standards give compilers a lot of freedom as to how bitfields are implemented -- so much freedom in fact that they don't have much use in portable code. For example, in my function just above, the 'transparent' bit might be 0x80 or it might be 0x01. Bits might not straddle bytes, and really the only thing that's guaranteed is the number of bits of precision (e.g. if it's 3 bits then the max value is 7).
> 
> Are bitfields pretty much useless other than for limiting the max value of an integer type (e.g. 3 bits for 7, 4 bits for 15, 5 bits for 31)?
> 
> The people who wrote the library I'm working with either:
> (A) Expect me to use the same compiler as them
> (B) Expect me to use a compiler that implements bitfields the same way
> (C) Don't know that bitfields aren't portable for this purpose

Bitfields are useful for compressing information down when you have a
constraint of limited resources.  This could me memory for embedded devices,
bandwidth constraints for serial or network connections, etc.  You're willing
to trade computing power of which you have an excess to essentially compress
and decompress these bits of information over a limited transmission medium.
I can imagine that the Mars rover uses bitfields or at least bit-level
definitions of information extensively.

Another useful purpose is mapping memory to physical hardware operation,
like you'd find in low level device drivers that connect equipment to a PC
or if you do any sort of embedded 8 or 16-bit processor development.

A lot of the time, the library is targeted for one architecture which has
the order pre-defined.  Less commonly, you may have a library that supports
multiple hardware and acts as a integration layer.  For example, you may
have a low level library that writes graphics for a selective group of
small pixel displays LCD panels.  In my case it was 128x64 LCD panel graphics
library.

Typically for these purposes, you have one or more #defines in a file that
defines the orientation based on your local architecture.  This can be done
manually based on the compiler options, architecture, etc.  This #define
could be in a config.h kind of file.  The configuration of the LCD graphics
library was done by including the right vendor provided include files for
the architecture and orientation of the display of your target, which can
be configured by a build system.

If this support is unavailable, you can usually set up a script that is
called by your build program before compiling your project that writes a
sample test program to a file, compiles it with your compiler and options,
executes the program, then redirects the output to an include file, which
is references from the other source files in your project.

This test would likely contains a union of a bitfield and an integer.  You
set the bitfield, then use the integer portion of the union, check it's
value, and write a config.h file of what you discovered.

This could be done with Makefiles, autoconf, CMake, etc, but it adds
considerable complexity you may not want to introduce.  If you already have
a build system in place, it may not be that bad to add.  YMMV.

Best regards,
John D.

[toc] | [prev] | [next] | [standalone]

#153376

On Thursday, July 30, 2020 at 5:10:13 AM UTC-4, Frederick Gotham wrote:
> This week I'm working on two different products that link with the same library, however one product links with a newer version of the library. The older library has a struct like this:
> 
>     typedef struct {
>         u16 scale;
>         u8 window;
>         u8 a;
>         u8 b;
>         u8 c;
>         float x;
>         float y;
>         float z;
>         u32 reserved[4];
>     } Manager;
> 
> 
> And the newer library has the same struct like this:
> 
>     typedef struct {
>         u16 scale;
>         u8 window;
>         u8 a;
>         u8 b;
>         u8 c;
>         float x;
>         float y;
>         float z;
>         s8 offset;
>         u8 transparent : 1;
>         u8 reserved0 : 7;
>         u8 reserved1[2];
>         u32 reserved2[3];
>     } Manager;
> 
> I want to set the member "transparent" on both platforms, and so I started out with code like this:\\

The fundamental problem you're facing is that the member 'transparent'
doesn't even exist on one of the two platforms. I'm not just referring
to the obvious fact that no such member is declared. I'm referring to
the likelihood that the platform where that member isn't included in the
declaration also probably doesn't store that information in that
location - it might not store it anywhere - it might not even provide
the corresponding feature. If it does store such information, there's a
very good chance that its stored in a different location.

If the library doesn't provide a function for setting that flag, you
really have no way of setting it that will have any reliable effect on
the behavior of that library, on the platform where that member doesn't
exist.

> 
>     void Set_Flag_For_Transparency(Manager &e)
>     {
>         float *const p_z = &e.z;
>         u8 *const p_offset = reinterpret_cast<u8*>(p_z + 1u);
>         u8 *const p_byte_containing_transparent = p_offset + 1u;
>         *p_byte_containing_transparent |= 0x80;  //Set the high bit
>     }
> 
> After doing a little reading up on bitfields, it seems that ISO/ANSI standards give compilers a lot of freedom as to how bitfields are implemented -- so much freedom in fact that they don't have much use in portable code. For example, in my function just above, the 'transparent' bit might be 0x80 or it might be 0x01. Bits might not straddle bytes, and really the only thing that's guaranteed is the number of bits of precision (e.g. if it's 3 bits then the max value is 7).
> 
> Are bitfields pretty much useless other than for limiting the max value of an integer type (e.g. 3 bits for 7, 4 bits for 15, 5 bits for 31)?


No, they can be a very useful way of storing data in less space than
would otherwise be required. They simply cannot be used to communicate
between programs compiled by compilers that make different choices with
regard to the issues. That is a serious limitation, but not necessarily
as severe a limitation as it sounds - most compilers that target a given
platform tend to make the same choices, for that platform.

A key point you don't seem to be aware of is that your problem doesn't
involve any such communication.

You're assuming that there is a "transparent" bit even in the version of
the struct that doesn't explicitly declare one, and that this bit
happens to be stored in the same location as it is in the version that
does declare it. I doubt that this is true; it's certainly not
guaranteed to be true. However, if it does happen to be true, there's a
better way to set that bit:


    typedef struct {
        u16 scale;
        u8 window;
        u8 a;
        u8 b;
        u8 c;
        float x;
        float y;
        float z;
        s8 offset;
        u8 transparent : 1;
        u8 reserved0 : 7;
        u8 reserved1[2];
        u32 reserved2[3];
    } myManager;

    void Set_Flag_For_Transparency(Manager &e)
    {
        myManager *m = reinterpret_cast<myManager*> &e;
        m->transparent = 1;
    }

Technically, this has undefined behavior, just like your original code.
However, pragmatically, if your assumption is correct, it's likely to
work - in particular, it's somewhat more likely to work than your code.
For instance, in the unlikely case that the compiler inserts any padding
between 'z' and 'offset', or between 'offset' and 'transparent', it's
likely to be the correct amount of padding, whether or not the compiler
inserts any padding between 'z' and 'reserved' in the other version of
the struct.

The reason is that myManager and Manager are being interpreted by the
same compiler, and the member named transparent is therefore very likely
to refer to the correct bit, if such a bit exists.

[toc] | [prev] | [next] | [standalone]

#153378

Hi, James

On July 30, 2020 18:37, James Kuyper wrote:
[snip]
> You're assuming that there is a "transparent" bit even in the version of
> the struct that doesn't explicitly declare one, and that this bit
> happens to be stored in the same location as it is in the version that
> does declare it. I doubt that this is true; it's certainly not
> guaranteed to be true. However, if it does happen to be true, there's a
> better way to set that bit:
> 
> 
>     typedef struct {
>         u16 scale;
>         u8 window;
>         u8 a;
>         u8 b;
>         u8 c;
>         float x;
>         float y;
>         float z;
>         s8 offset;
>         u8 transparent : 1;
>         u8 reserved0 : 7;
>         u8 reserved1[2];
>         u32 reserved2[3];
>     } myManager;


I don't write code that uses /all/ the features of C, and I don't recognize 
something in your code. Could you tell me about

>     void Set_Flag_For_Transparency(Manager &e)
>     {
>         myManager *m = reinterpret_cast<myManager*> &e;
>         m->transparent = 1;
>     }

The questions I have are
1) in the parameter list of the Set_Flag_For_Transparency function
   declaration, what does the "&e" signify? I would expect the ampersand to
   signify a "take the address of" operation, but that only applies to an
   expression, and a function parameter list is not an expression.
2) in the init_declarator_list of the myManager *m declaration, you use an
   idiom that I'm unfamiliar with
      reinterpret_cast<myManager*>
   Normally, less-than and greater-than are used in expressions, and result
   in a truth value indicating the magnitude relationship between two
   values. But here, you seem to use them as type of bracket that I'm
   unfamiliar with. What do they signify?

[snip]

-- 
Lew Pitcher
"In Skills, We Trust"

[toc] | [prev] | [next] | [standalone]

#153380

On 7/30/20 7:15 PM, Lew Pitcher wrote:
> Hi, James
> 
> On July 30, 2020 18:37, James Kuyper wrote:
> [snip]
>> You're assuming that there is a "transparent" bit even in the version of
>> the struct that doesn't explicitly declare one, and that this bit
>> happens to be stored in the same location as it is in the version that
>> does declare it. I doubt that this is true; it's certainly not
>> guaranteed to be true. However, if it does happen to be true, there's a
>> better way to set that bit:
>>
>>
>>     typedef struct {
>>         u16 scale;
>>         u8 window;
>>         u8 a;
>>         u8 b;
>>         u8 c;
>>         float x;
>>         float y;
>>         float z;
>>         s8 offset;
>>         u8 transparent : 1;
>>         u8 reserved0 : 7;
>>         u8 reserved1[2];
>>         u32 reserved2[3];
>>     } myManager;
> 
> 
> I don't write code that uses /all/ the features of C, and I don't recognize 
> something in your code. Could you tell me about
> 
>>     void Set_Flag_For_Transparency(Manager &e)
>>     {
>>         myManager *m = reinterpret_cast<myManager*> &e;
>>         m->transparent = 1;
>>     }
> 
> The questions I have are
> 1) in the parameter list of the Set_Flag_For_Transparency function
>    declaration, what does the "&e" signify? I would expect the ampersand to

It signifies that this question was posted separately to both
comp.lang.c and comp.lang.c++, and that I saw it in both locations, and
thought, while I was composing my response, that I was responding to the
one posted in comp.lang.c++. I distinctly remember selecting a message
from comp.lang.c++ to respond to, so I'm not quite sure how this happened.

I'd love to blame my age, but I made some equally stupid mistakes (and,
in fact, several that were much worse) when I was only half my current age.

The C version would look like the following:

    void Set_Flag_For_Transparency(Manager *pe)
    {
        myManager *m = (myManager*) pe;
        m->transparent = 1;
    }

The issue he's raised is unaffected by those differences: the layout of
bit-fields is so poorly constrained in both C and C++ that it's not
worth the trouble of discussing which one is more poorly constrained.

[toc] | [prev] | [next] | [standalone]

#153377

On Thursday, July 30, 2020 at 5:10:13 AM UTC-4, Frederick Gotham wrote:
...
>     typedef struct {
>         u16 scale;
>         u8 window;
>         u8 a;
>         u8 b;
>         u8 c;
>         float x;
>         float y;
>         float z;
>         s8 offset;
>         u8 transparent : 1;
>         u8 reserved0 : 7;
>         u8 reserved1[2];
>         u32 reserved2[3];
>     } Manager;
...
> After doing a little reading up on bitfields, it seems that ISO/ANSI standards give compilers a lot of freedom as to how bitfields are implemented -- so much freedom in fact that they don't have much use in portable code. For example, in my function just above, the 'transparent' bit might be 0x80 or it might be 0x01. Bits might not straddle bytes, and really the only thing that's guaranteed is the number of bits of precision (e.g. if it's 3 bits then the max value is 7).
That's not quite true.

"An implementation may allocate any addressable storage unit large
enough to hold a bit-field." (6.7.2.1p11).

"Within a structure object, the non-bit-field members and the units in
which bit-fields reside have addresses that increase in the order in
which they are declared." (6.7.2.1p15)

Also, "a bit-field structure member with a width of 0 indicates that no
further bit-field is to be packed into the unit in which the previous
bit-field, if any, was placed." (6.7.2.1p12).

You posted this in both comp.lang.c and comp.lang.c++. C++ has those
same restrictions, though worded a bit differently. However, there's on
additional requirement that only applies in C:

"If enough space remains, a bit-field that immediately follows another
bit-field in a structure shall be packed into adjacent bits of the same
unit." (6.7.2.1p11).

Since the storage unit must be addressable, and CHAR_BIT must be at
least 8, there's guaranteed to be enough space to store 'transparent'
and 'reserved0' in the same storage unit, and that must, therefore, be done.

That isn't much in the way of restrictions on the layout, but there's
more guaranteed about the layout than you suggested.

[toc] | [prev] | [next] | [standalone]

#153381

On 31/07/2020 00:01, James Kuyper wrote:
> On Thursday, July 30, 2020 at 5:10:13 AM UTC-4, Frederick Gotham wrote:
> ...
>>      typedef struct {
>>          u16 scale;
>>          u8 window;
>>          u8 a;
>>          u8 b;
>>          u8 c;
>>          float x;
>>          float y;
>>          float z;
>>          s8 offset;
>>          u8 transparent : 1;
>>          u8 reserved0 : 7;
>>          u8 reserved1[2];
>>          u32 reserved2[3];
>>      } Manager;
> ...
>> After doing a little reading up on bitfields, it seems that ISO/ANSI standards give compilers a lot of freedom as to how bitfields are implemented -- so much freedom in fact that they don't have much use in portable code. For example, in my function just above, the 'transparent' bit might be 0x80 or it might be 0x01. Bits might not straddle bytes, and really the only thing that's guaranteed is the number of bits of precision (e.g. if it's 3 bits then the max value is 7).
> That's not quite true.
> 
> "An implementation may allocate any addressable storage unit large
> enough to hold a bit-field." (6.7.2.1p11).
> 
> "Within a structure object, the non-bit-field members and the units in
> which bit-fields reside have addresses that increase in the order in
> which they are declared." (6.7.2.1p15)
> 
> Also, "a bit-field structure member with a width of 0 indicates that no
> further bit-field is to be packed into the unit in which the previous
> bit-field, if any, was placed." (6.7.2.1p12).
> 
> You posted this in both comp.lang.c and comp.lang.c++. C++ has those
> same restrictions, though worded a bit differently. However, there's on
> additional requirement that only applies in C:
> 
> "If enough space remains, a bit-field that immediately follows another
> bit-field in a structure shall be packed into adjacent bits of the same
> unit." (6.7.2.1p11).
> 
> Since the storage unit must be addressable, and CHAR_BIT must be at
> least 8, there's guaranteed to be enough space to store 'transparent'
> and 'reserved0' in the same storage unit, and that must, therefore, be done.
> 
> That isn't much in the way of restrictions on the layout, but there's
> more guaranteed about the layout than you suggested.
> 

I got these results from the C program below:

Tiny C          4 1 2
gcc (Windows)   4 1 4
lccwin          4 4 4
DMC             4 1 4
gcc (rextester) 4 1 2
msvc(rextester) 4 1 4
bcc             8 2 4 (does not do bitfields)

So 3 different results even from those that implement bitfields.

When I tried this struct:

  typedef struct {
       char a:1;
       long long int b:33;
  } s2;

then I variously got sizes of 8 or 16 (including both from different 
versions of gcc), with one compiler rejecting it.

So, what are the rules that will let me predict those results? For 
example because I need to create a duplicate struct from another 
language. Because I can't figure it out.


----------------------------
  typedef struct {
       int a:1;
       int b:7;
  } s1;

  typedef struct {
       char a:1;
       char b:7;
  } s2;

  typedef struct {
       short a:9;
       char b:7;
  } s3;

#include <stdio.h>

int main(void) {
     printf("%d %d %d\n",(int)sizeof(s1),(int)sizeof(s2),
                         (int)sizeof(s3));
}

[toc] | [prev] | [next] | [standalone]

#153382

Bart <bc@freeuk.com> writes:
[...]
> So, what are the rules that will let me predict those results? For
> example because I need to create a duplicate struct from another
> language. Because I can't figure it out.
>
>
> ----------------------------
>  typedef struct {
>       int a:1;
>       int b:7;
>  } s1;
>
>  typedef struct {
>       char a:1;
>       char b:7;
>  } s2;
>
>  typedef struct {
>       short a:9;
>       char b:7;
>  } s3;
>
> #include <stdio.h>
>
> int main(void) {
>     printf("%d %d %d\n",(int)sizeof(s1),(int)sizeof(s2),
>                         (int)sizeof(s3));
> }

Keep in mind that support for bit-fields of types other than
_Bool, signed int, or unsigned int is implementation-defined.
(A bit-field of type int is treated as signed int or unsigned int;
the choice is implementation-defined.)

Most aspects of bit-field layout are implementation-defined.
See N1570 6.7.2p11 for the details.  So the rules should be in the
documentation for whatever implementation you're using.

-- 
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Philips Healthcare
void Void(void) { Void(); } /* The recursive call of the void */

[toc] | [prev] | [next] | [standalone]

#153384

On Thursday, July 30, 2020 at 8:50:18 PM UTC-4, Bart wrote:
> On 31/07/2020 00:01, James Kuyper wrote:
> > On Thursday, July 30, 2020 at 5:10:13 AM UTC-4, Frederick Gotham wrote:
...
> >> After doing a little reading up on bitfields, it seems that ISO/ANSI standards give compilers a lot of freedom as to how bitfields are implemented -- so much freedom in fact that they don't have much use in portable code. For example, in my function just above, the 'transparent' bit might be 0x80 or it might be 0x01. Bits might not straddle bytes, and really the only thing that's guaranteed is the number of bits of precision (e.g. if it's 3 bits then the max value is 7).
> > That's not quite true.
...
> > That isn't much in the way of restrictions on the layout, but there's
> > more guaranteed about the layout than you suggested.
> > 
> 
> I got these results from the C program below:
> 
> Tiny C          4 1 2
> gcc (Windows)   4 1 4
> lccwin          4 4 4
> DMC             4 1 4
> gcc (rextester) 4 1 2
> msvc(rextester) 4 1 4
> bcc             8 2 4 (does not do bitfields)
> 
> So 3 different results even from those that implement bitfields.
> 
> When I tried this struct:
> 
>   typedef struct {
>        char a:1;
>        long long int b:33;
>   } s2;
> 
> then I variously got sizes of 8 or 16 (including both from different 
> versions of gcc), with one compiler rejecting it.
> 
> So, what are the rules that will let me predict those results?

The rule is relatively simple: "read the documentation for the specific
implementation you're using.".

If you want to access data in a specified layout in a more portable
fashion, you'll have to access the raw data as an array of unsigned
char, and use mask and shift operations to implement bit-fields. What
the C standard fails to mandate about the value of CHAR_BIT, endianess,
the representation of signed types, and padding bits means that even
that approach isn't completely portable.

I agree with your implied criticism of C. I consider the fact that so
many aspects of struct layouts are implementation-defined rather than
standard-defined to be one of the major defects of C (and C++ is no
better - actually, it's somewhat worse). But I don't see any point in
pretending that you don't know this already.

[toc] | [prev] | [next] | [standalone]

#153386

On 31/07/2020 03:28, James Kuyper wrote:
> On Thursday, July 30, 2020 at 8:50:18 PM UTC-4, Bart wrote:

>> I got these results from the C program below:
>>
>> Tiny C          4 1 2
>> gcc (Windows)   4 1 4
>> lccwin          4 4 4
>> DMC             4 1 4
>> gcc (rextester) 4 1 2
>> msvc(rextester) 4 1 4
>> bcc             8 2 4 (does not do bitfields)
>>
>> So 3 different results even from those that implement bitfields.
>>
>> When I tried this struct:
>>
>>    typedef struct {
>>         char a:1;
>>         long long int b:33;
>>    } s2;
>>
>> then I variously got sizes of 8 or 16 (including both from different
>> versions of gcc), with one compiler rejecting it.
>>
>> So, what are the rules that will let me predict those results?
> 
> The rule is relatively simple: "read the documentation for the specific
> implementation you're using.".

The problem is not /my/ implementation, but the one used by the library 
I'm trying to utilise. Which would /that/ be?


> If you want to access data in a specified layout in a more portable
> fashion, you'll have to access the raw data as an array of unsigned
> char, and use mask and shift operations to implement bit-fields. What
> the C standard fails to mandate about the value of CHAR_BIT, endianess,
> the representation of signed types, and padding bits means that even
> that approach isn't completely portable.
> 
> I agree with your implied criticism of C. I consider the fact that so
> many aspects of struct layouts are implementation-defined rather than
> standard-defined to be one of the major defects of C (and C++ is no
> better - actually, it's somewhat worse). But I don't see any point in
> pretending that you don't know this already.

Actually, struct layout seems to be done the same way by different 
compilers, at least by platform. I know the rules for that, so that in 
my language, I can use this special attribute:

     record dummy = $caligned
         byte a
         int64 b
     end

to end up with with a struct of 16 bytes, with b at offset 8, rather 
than 9 bytes and offset 1 since I normally work with the equivalent of 
pack(1).

It's the rules for bitfields I have a problem with. I think it is also a 
mistake for a library to depend on bitfield size and layout for exported 
struct types.

[toc] | [prev] | [next] | [standalone]

#153387

On Fri, 2020-07-31, Bart wrote:
> On 31/07/2020 03:28, James Kuyper wrote:
...
>> I agree with your implied criticism of C. I consider the fact that so
>> many aspects of struct layouts are implementation-defined rather than
>> standard-defined to be one of the major defects of C (and C++ is no
>> better - actually, it's somewhat worse). But I don't see any point in
>> pretending that you don't know this already.
>
> Actually, struct layout seems to be done the same way by different 
> compilers, at least by platform.

The key word is "ABI".  And yes, since a common ABI lets you share
tools and libraries, a certain OS version on a certain CPU
architecture tends to have only one ABI.

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

Page 1 of 3  [1] 2 3  Next page →

Back to top | Article view | comp.lang.c

csiph-web