Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #153367 > unrolled thread

Are bitfields useful?

Back to article view | Back to comp.lang.c

Contents

  Are bitfields useful? Frederick Gotham <cauldwell.thomas@gmail.com> - 2020-07-30 02:10 -0700
    Re: Are bitfields useful? Jorgen Grahn <grahn+nntp@snipabacken.se> - 2020-07-30 09:50 +0000
      Re: Are bitfields useful? jacobnavia <jacob@jacob.remcomp.fr> - 2020-07-30 12:18 +0200
        Re: Are bitfields useful? Frederick Gotham <cauldwell.thomas@gmail.com> - 2020-07-30 04:01 -0700
      Re: Are bitfields useful? David Brown <david.brown@hesbynett.no> - 2020-08-03 12:26 +0200
    Re: Are bitfields useful? Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2020-07-30 03:10 -0700
    Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-30 12:11 +0100
    Re: Are bitfields useful? Richard Damon <Richard@Damon-Family.org> - 2020-07-30 07:54 -0400
      Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-30 13:14 +0100
        Re: Are bitfields useful? Richard Damon <Richard@Damon-Family.org> - 2020-07-30 21:18 -0400
    Re: Are bitfields useful? jadill33@gmail.com - 2020-07-30 09:57 -0700
    Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-30 18:37 -0400
      Re: Are bitfields useful? Lew Pitcher <lew.pitcher@digitalfreehold.ca> - 2020-07-30 19:15 -0400
        Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-30 20:36 -0400
    Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-30 19:01 -0400
      Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-31 01:49 +0100
        Re: Are bitfields useful? Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2020-07-30 18:08 -0700
        Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-30 19:28 -0700
          Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-31 11:15 +0100
            Re: Are bitfields useful? Jorgen Grahn <grahn+nntp@snipabacken.se> - 2020-07-31 11:47 +0000
            Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-31 05:00 -0700
              Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-31 13:53 +0100
                Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-31 07:59 -0700
                  Re: Are bitfields useful? Lew Pitcher <lew.pitcher@digitalfreehold.ca> - 2020-07-31 11:32 -0400
                    Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-31 13:55 -0400
                  Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-31 17:30 +0100
                    Re: Are bitfields useful? scott@slp53.sl.home (Scott Lurndal) - 2020-07-31 17:59 +0000
                    Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-31 11:19 -0700
                      Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-31 20:54 +0100
                        Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-31 15:14 -0700
                          Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-08-01 00:31 +0100
                            Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-31 16:55 -0700
                              Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-08-01 01:16 +0100
                                Re: Are bitfields useful? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-07-31 20:31 -0700
                    Re: Are bitfields useful? Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2020-07-31 12:55 -0700
                      Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-31 21:04 +0100
              Re: Are bitfields useful? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-08-23 16:16 -0700
          Re: Are bitfields useful? Richard Damon <Richard@Damon-Family.org> - 2020-07-31 08:23 -0400
            Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-07-31 14:20 +0100
            Re: Are bitfields useful? Philipp Klaus Krause <pkk@spth.de> - 2020-08-04 17:21 +0200
        Re: Are bitfields useful? Kaz Kylheku <793-849-0957@kylheku.com> - 2020-08-25 00:13 +0000
          Re: Are bitfields useful? Bart <bc@freeuk.com> - 2020-08-25 12:42 +0100
          Re: Are bitfields useful? scott@slp53.sl.home (Scott Lurndal) - 2020-08-25 15:27 +0000
            Re: Are bitfields useful? Kaz Kylheku <793-849-0957@kylheku.com> - 2020-08-25 16:05 +0000
    Re: Are bitfields useful? Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2020-07-30 17:26 -0700
    Re: Are bitfields useful? Andrey Tarasevich <andreytarasevich@hotmail.com> - 2020-07-31 22:32 -0700
      Re: Are bitfields useful? Richard Damon <Richard@Damon-Family.org> - 2020-08-01 10:20 -0400
      Re: Are bitfields useful? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-08-23 07:54 -0700
      Re: Are bitfields useful? Kaz Kylheku <793-849-0957@kylheku.com> - 2020-08-25 00:26 +0000
        Re: Are bitfields useful? Kaz Kylheku <793-849-0957@kylheku.com> - 2020-08-25 23:55 +0000

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

#153388

On Friday, July 31, 2020 at 6:15:51 AM UTC-4, Bart wrote:
> On 31/07/2020 03:28, James Kuyper wrote:
> > On Thursday, July 30, 2020 at 8:50:18 PM UTC-4, Bart wrote:
> 
> >> I got these results from the C program below:
> >>
> >> Tiny C          4 1 2
> >> gcc (Windows)   4 1 4
> >> lccwin          4 4 4
> >> DMC             4 1 4
> >> gcc (rextester) 4 1 2
> >> msvc(rextester) 4 1 4
> >> bcc             8 2 4 (does not do bitfields)
> >>
> >> So 3 different results even from those that implement bitfields.
> >>
> >> When I tried this struct:
> >>
> >>    typedef struct {
> >>         char a:1;
> >>         long long int b:33;
> >>    } s2;
> >>
> >> then I variously got sizes of 8 or 16 (including both from different
> >> versions of gcc), with one compiler rejecting it.
> >>
> >> So, what are the rules that will let me predict those results?
> > 
> > The rule is relatively simple: "read the documentation for the specific
> > implementation you're using.".
> 
> The problem is not /my/ implementation, but the one used by the library 
> I'm trying to utilise. Which would /that/ be?

When I said "the implementation you're using", "you" referred to the
person compiling the code, and the implementation I was referring to was
the one used for that compilation. If you built the library, ask
yourself which implementation you used. If someone else built it, ask
them.

> > If you want to access data in a specified layout in a more portable
> > fashion, you'll have to access the raw data as an array of unsigned
> > char, and use mask and shift operations to implement bit-fields. What
> > the C standard fails to mandate about the value of CHAR_BIT, endianess,
> > the representation of signed types, and padding bits means that even
> > that approach isn't completely portable.
> > 
> > I agree with your implied criticism of C. I consider the fact that so
> > many aspects of struct layouts are implementation-defined rather than
> > standard-defined to be one of the major defects of C (and C++ is no
> > better - actually, it's somewhat worse). But I don't see any point in
> > pretending that you don't know this already.
> 
> Actually, struct layout seems to be done the same way by different 
> compilers, at least by platform. I know the rules for that, so that in 
> my language, I can use this special attribute:
> 
>      record dummy = $caligned
>          byte a
>          int64 b
>      end
> 
> to end up with with a struct of 16 bytes, with b at offset 8, rather 
> than 9 bytes and offset 1 since I normally work with the equivalent of 
> pack(1).
> 
> It's the rules for bitfields I have a problem with.

Bit-fields only occur in structs; bit-field layout is a subset of struct layout. If different compilers layout bit-fields differently, then they do
struct layout differently. Compilers targeting the same platform tend to
lay out bit-fields the same way, for the same reason that they tend to
do other aspects of struct layout the same: because it's very
inconvenient if they don't. However, as with other aspects of struct
layout, there's no guarantee that they will in fact do bit-field layout
the same.

> ... I think it is also a 
> mistake for a library to depend on bitfield size and layout for exported 
> struct types.

No more so than it is a mistake to rely upon other aspects of struct
layout. Which is to say that, yes, it can be a mistake to rely upon any
aspect of struct layout when communicating between code compiled using
different compilers.

[toc] | [prev] | [next] | [standalone]

#153390

On 31/07/2020 13:00, James Kuyper wrote:
> On Friday, July 31, 2020 at 6:15:51 AM UTC-4, Bart wrote:
>> On 31/07/2020 03:28, James Kuyper wrote:
>>> On Thursday, July 30, 2020 at 8:50:18 PM UTC-4, Bart wrote:
>>
>>>> I got these results from the C program below:
>>>>
>>>> Tiny C          4 1 2
>>>> gcc (Windows)   4 1 4
>>>> lccwin          4 4 4
>>>> DMC             4 1 4
>>>> gcc (rextester) 4 1 2
>>>> msvc(rextester) 4 1 4
>>>> bcc             8 2 4 (does not do bitfields)
>>>>
>>>> So 3 different results even from those that implement bitfields.
>>>>
>>>> When I tried this struct:
>>>>
>>>>     typedef struct {
>>>>          char a:1;
>>>>          long long int b:33;
>>>>     } s2;
>>>>
>>>> then I variously got sizes of 8 or 16 (including both from different
>>>> versions of gcc), with one compiler rejecting it.
>>>>
>>>> So, what are the rules that will let me predict those results?
>>>
>>> The rule is relatively simple: "read the documentation for the specific
>>> implementation you're using.".
>>
>> The problem is not /my/ implementation, but the one used by the library
>> I'm trying to utilise. Which would /that/ be?
> 
> When I said "the implementation you're using", "you" referred to the
> person compiling the code, and the implementation I was referring to was
> the one used for that compilation. If you built the library, ask
> yourself which implementation you used. If someone else built it, ask
> them.

I'm not sure you get it. Take the GTK library again. It is far too 
massive a library to just casually rebuild.

So you have to use existing binaries, and in fact some 34 DLLs are 
provided in the download 'bundle'.

These binaries will have been built with compiler X. You will build your 
own applications, that use the same headers, with compiler Y. X and Y 
may lay out the bitfields differently.

Not that GTK is supposed to be cross-platform and usable from multiple 
languages via the bindings that are available. It doesn't seem right 
that you should then only be able to use C compiler X, whatever that is.


>>> If you want to access data in a specified layout in a more portable
>>> fashion, you'll have to access the raw data as an array of unsigned
>>> char, and use mask and shift operations to implement bit-fields. What
>>> the C standard fails to mandate about the value of CHAR_BIT, endianess,
>>> the representation of signed types, and padding bits means that even
>>> that approach isn't completely portable.
>>>
>>> I agree with your implied criticism of C. I consider the fact that so
>>> many aspects of struct layouts are implementation-defined rather than
>>> standard-defined to be one of the major defects of C (and C++ is no
>>> better - actually, it's somewhat worse). But I don't see any point in
>>> pretending that you don't know this already.
>>
>> Actually, struct layout seems to be done the same way by different
>> compilers, at least by platform. I know the rules for that, so that in
>> my language, I can use this special attribute:
>>
>>       record dummy = $caligned
>>           byte a
>>           int64 b
>>       end
>>
>> to end up with with a struct of 16 bytes, with b at offset 8, rather
>> than 9 bytes and offset 1 since I normally work with the equivalent of
>> pack(1).
>>
>> It's the rules for bitfields I have a problem with.
> 
> Bit-fields only occur in structs; bit-field layout is a subset of struct layout. If different compilers layout bit-fields differently, then they do
> struct layout differently.

The layout of non-bitfields mainly follows rules for alignment. There is 
less such need for bitfields, except perhaps where a bitfield could 
straddle two separate memory accesses.

> Compilers targeting the same platform tend to
> lay out bit-fields the same way, for the same reason that they tend to
> do other aspects of struct layout the same: because it's very
> inconvenient if they don't. However, as with other aspects of struct
> layout, there's no guarantee that they will in fact do bit-field layout
> the same.
> 
>> ... I think it is also a
>> mistake for a library to depend on bitfield size and layout for exported
>> struct types.
> 
> No more so than it is a mistake to rely upon other aspects of struct
> layout. Which is to say that, yes, it can be a mistake to rely upon any
> aspect of struct layout when communicating between code compiled using
> different compilers.

The specific problem I had with GTK (it's too massive a project to try 
pinpoint the exact issue again), was in determining a struct size. 
(Maybe at some point I would need to access those bitfields; I didn't 
get far into it.)

I'm now looking at that library (now increased to 760 headers), it uses 
bitfields extensively, but many of those uses are gratuituous.

For example, a struct containing just one :1 field, right in the middle 
and followed by pointers. Or containing just one :16 field. Which are 
both pointless; no space will be saved by that 1-bit field, as padding 
will be needed. And the 16-bit field might as well be a short.

[toc] | [prev] | [next] | [standalone]

#153392

On Friday, July 31, 2020 at 8:54:21 AM UTC-4, Bart wrote:
> On 31/07/2020 13:00, James Kuyper wrote:
> > On Friday, July 31, 2020 at 6:15:51 AM UTC-4, Bart wrote:
> >> On 31/07/2020 03:28, James Kuyper wrote:
> >>> On Thursday, July 30, 2020 at 8:50:18 PM UTC-4, Bart wrote:
...
> >>>> So, what are the rules that will let me predict those results?
> >>>
> >>> The rule is relatively simple: "read the documentation for the specific
> >>> implementation you're using.".
> >>
> >> The problem is not /my/ implementation, but the one used by the library
> >> I'm trying to utilise. Which would /that/ be?
> > 
> > When I said "the implementation you're using", "you" referred to the
> > person compiling the code, and the implementation I was referring to was
> > the one used for that compilation. If you built the library, ask
> > yourself which implementation you used. If someone else built it, ask
> > them.
> 
> I'm not sure you get it. Take the GTK library again. It is far too 
> massive a library to just casually rebuild.
> 
> So you have to use existing binaries, and in fact some 34 DLLs are 
> provided in the download 'bundle'.
> 
> These binaries will have been built with compiler X.

So? As I said, that means that compiler X is the one whose documentation you will need to examine to predict the layout.

> ... You will build your 
> own applications, that use the same headers, with compiler Y. X and Y 
> may lay out the bitfields differently.

Correct - though if Y targets the same platform, there's a good chance that it won't. To be certain, you need to read Y's documentation.


> Not that GTK is supposed to be cross-platform and usable from multiple 
> languages via the bindings that are available. It doesn't seem right 
> that you should then only be able to use C compiler X, whatever that is.

If you're using platform with an ABI specification, all compilers that
generate code conforming to that ABI should be able to interoperate - in
particular, this means that all of those compilers will have to obey
the ABI's specifications with regard to bit-field layout.

If there is no unique ABI for a given platform, or are using an
implementation that produces code that doesn't conform to the ABI,
that's a problem - and I didn't say anything to suggest otherwise -
quite the contrary.

...
> I'm now looking at that library (now increased to 760 headers), it uses 
> bitfields extensively, but many of those uses are gratuituous.
> 
> For example, a struct containing just one :1 field, right in the middle 
> and followed by pointers. Or containing just one :16 field. Which are 
> both pointless; no space will be saved by that 1-bit field, as padding 
> will be needed. And the 16-bit field might as well be a short.

That's not necessarily gratuitious. It's possible (and I would suggest,
likely) that they're thinking ahead, and reserving the remaining space
in that storage unit to possible future bit-fields.

[toc] | [prev] | [next] | [standalone]

#153393

On July 31, 2020 10:59, James Kuyper wrote:

> On Friday, July 31, 2020 at 8:54:21 AM UTC-4, Bart wrote:
>> On 31/07/2020 13:00, James Kuyper wrote:
>> > On Friday, July 31, 2020 at 6:15:51 AM UTC-4, Bart wrote:
>> >> On 31/07/2020 03:28, James Kuyper wrote:
>> >>> On Thursday, July 30, 2020 at 8:50:18 PM UTC-4, Bart wrote:
> ...
>> >>>> So, what are the rules that will let me predict those results?
>> >>>
>> >>> The rule is relatively simple: "read the documentation for the
>> >>> specific implementation you're using.".
>> >>
>> >> The problem is not /my/ implementation, but the one used by the
>> >> library I'm trying to utilise. Which would /that/ be?
>> > 
>> > When I said "the implementation you're using", "you" referred to the
>> > person compiling the code, and the implementation I was referring to
>> > was the one used for that compilation. If you built the library, ask
>> > yourself which implementation you used. If someone else built it, ask
>> > them.
>> 
>> I'm not sure you get it. Take the GTK library again. It is far too
>> massive a library to just casually rebuild.
>> 
>> So you have to use existing binaries, and in fact some 34 DLLs are
>> provided in the download 'bundle'.
>> 
>> These binaries will have been built with compiler X.
> 
> So? As I said, that means that compiler X is the one whose documentation
> you will need to examine to predict the layout.

Likely so, but only incidentaly. The operating environment, that "compiler 
X" lives in, sets rules that "compiler X" must obey (in order to interface 
with that environment) /and/ ensure that resulting programs also obey 
(again, in order to interface with the environment). If the resulting 
program's target environment is not the same as the environment in which the 
compiler lives (a "cross-compilation"), "Compiler X" may chose different 
rules for it's output than it uses itself, but still must ensure that those 
rules obey the requirements of the target environment.

In other words, the compiler /must/ implement, in it's output, the 
interfaces that the target environment requires. And, the order and 
placement of bitfields is one of those requirements dictated by the target 
environment.

For instance, the Linux x86 kernel ABI dictates that:
 ‘Plain’’ bit-fields (that is, those neither signed nor unsigned) always
 have non-negative values.  Although they may have type char, short, int,
 or long (which can have negative values), these bit-fields have the same
 range as a bit-field of the same size with the corresponding unsigned type.
 Bit-fields obey the same size and alignment rules as other structure and
 union members, with the following additions:
 - Bit-fields are allocated from right to left (least to most significant).
 - A bit-field must entirely reside in a storage unit appropriate for its
   declared type.  Thus a bit-field never crosses its unit boundary.
 - Bit-fields may share a storage unit with other struct / union
   members, including members that are not bit-fields.  Of course, struct
   members occupy different parts of the storage unit.
 - Unnamed bit-fields’ types do not affect the alignment of a structure or
   union, although individual bit-fields’ member offsets obey the alignment
   constraints. 

(https://refspecs.linuxfoundation.org/elf/abi386-4.pdf p 32)

A compiler targeting the Linux X86 platform /must/ handle bitfields this 
way, no matter the preference of the compiler's designer. (The argument that 
the above spec only applies to the kernel, and thus can be ignored when 
compiling non-kernel code, is suspect, as the compiler would not be able to 
reliably distinguish non-kernel from kernel code.)

[snip]

-- 
Lew Pitcher
"In Skills, We Trust"

[toc] | [prev] | [next] | [standalone]

#153395

On 7/31/20 11:32 AM, Lew Pitcher wrote:
> On July 31, 2020 10:59, James Kuyper wrote:
...
>> So? As I said, that means that compiler X is the one whose documentation
>> you will need to examine to predict the layout.
> 
> Likely so, but only incidentaly. The operating environment, that "compiler 
> X" lives in, sets rules that "compiler X" must obey (in order to interface 
> with that environment) /and/ ensure that resulting programs also obey 
> (again, in order to interface with the environment). If the resulting 
> program's target environment is not the same as the environment in which the 
> compiler lives (a "cross-compilation"), "Compiler X" may chose different 
> rules for it's output than it uses itself, but still must ensure that those 
> rules obey the requirements of the target environment.
> 
> In other words, the compiler /must/ implement, in it's output, the 
> interfaces that the target environment requires. And, the order and 
> placement of bitfields is one of those requirements dictated by the target 
> environment.

I've used compilers that could generate executables that used 4
different incompatible APIs for the same target platform. It's not
sufficient to know the operating environment - it's essential to also
know which of the possible ABIs the compiler actually implements (when
given a particular set of compiler options), and only the compiler's
documentation can answer that question.

[toc] | [prev] | [next] | [standalone]

#153394

On 31/07/2020 15:59, James Kuyper wrote:
> On Friday, July 31, 2020 at 8:54:21 AM UTC-4, Bart wrote:

>> I'm not sure you get it. Take the GTK library again. It is far too
>> massive a library to just casually rebuild.
>>
>> So you have to use existing binaries, and in fact some 34 DLLs are
>> provided in the download 'bundle'.
>>
>> These binaries will have been built with compiler X.
> 
> So? As I said, that means that compiler X is the one whose documentation you will need to examine to predict the layout.

Perhaps you didn't read what I wrote. With a cross-platform 
cross-language library like GTK, are you really expected to seek out the 
compiler and version used to build the binaries, and use the exact same 
one to compile the headers?

>> ... You will build your
>> own applications, that use the same headers, with compiler Y. X and Y
>> may lay out the bitfields differently.
> 
> Correct - though if Y targets the same platform, there's a good chance that it won't. To be certain, you need to read Y's documentation.

Did you read my post where several C compilers for Windows gave 
different results for the sizes of various bitfield combinations?

Maybe bitfields as used in GTK will yield consistent results, but I 
can't be sure. The best I can do is to use gcc, as I can assume it's 
been tested with that, and write test programs using that will just 
print the size of each struct.

I can't use my semi-automatic tool to convert headers, because it 
doesn't support bitfields, and it doesn't support them partly because 
they are so ill-defined. Even if I try and work out what gcc does, gcc 
gave different results between Windows and Linux! And that's without 
introducing struct member types that could vary.

[toc] | [prev] | [next] | [standalone]

#153396

Bart <bc@freeuk.com> writes:
>On 31/07/2020 15:59, James Kuyper wrote:
>> On Friday, July 31, 2020 at 8:54:21 AM UTC-4, Bart wrote:
>
>>> I'm not sure you get it. Take the GTK library again. It is far too
>>> massive a library to just casually rebuild.
>>>
>>> So you have to use existing binaries, and in fact some 34 DLLs are
>>> provided in the download 'bundle'.
>>>
>>> These binaries will have been built with compiler X.
>> 
>> So? As I said, that means that compiler X is the one whose documentation you will need to examine to predict the layout.
>
>Perhaps you didn't read what I wrote. With a cross-platform 
>cross-language library like GTK, are you really expected to seek out the 
>compiler and version used to build the binaries, and use the exact same 
>one to compile the headers?
>

No, since all compilers  and tools that generate object files on a particular
implementation will be required to follow the same application binary
interface (ABI) in order to interoperate properly with the operating
environment (libraries, system calls, IPC, etc.).

For Unix/Linux systems,  there is a processor ABI that describes
what's necessary for interoperability for each of ARM, Intel/AMD (32 and 64),
PowerPC, SPARC, MIPS, 88100, i860/i960, VAX, etc).   This includes register
use conventions, parameter passing conventions, how bitfields are arranged
in memory, minimum alignment constraints on the stack, and on various
data types, thread-specific data, run-time linking,  und so weiter.

[toc] | [prev] | [next] | [standalone]

#153397

On Friday, July 31, 2020 at 12:22:15 PM UTC-4, Bart wrote:
> On 31/07/2020 15:59, James Kuyper wrote:
> > On Friday, July 31, 2020 at 8:54:21 AM UTC-4, Bart wrote:
> 
> >> I'm not sure you get it. Take the GTK library again. It is far too
> >> massive a library to just casually rebuild.
> >>
> >> So you have to use existing binaries, and in fact some 34 DLLs are
> >> provided in the download 'bundle'.
> >>
> >> These binaries will have been built with compiler X.
> > 
> > So? As I said, that means that compiler X is the one whose documentation you will need to examine to predict the layout.
> 
> Perhaps you didn't read what I wrote. With a cross-platform 
> cross-language library like GTK, are you really expected to seek out the 
> compiler and version used to build the binaries, and use the exact same 
> one to compile the headers?

What you're expected to do is up to the designers of GTK. I expect that
they expected you to choose a compiler that made compatible choices with
regard to the layout of any struct that is part of their interface, and
that they didn't expect it to be difficult for you to find one. If the
instructions that came with the library didn't even mention the issue, I
imagine they they thought it would be difficult, if not impossible, to
find a compiler that wasn't compatible. Have you found any?

> >> ... You will build your
> >> own applications, that use the same headers, with compiler Y. X and Y
> >> may lay out the bitfields differently.
> > 
> > Correct - though if Y targets the same platform, there's a good chance that it won't. To be certain, you need to read Y's documentation.
> 
> Did you read my post where several C compilers for Windows gave 
> different results for the sizes of various bitfield combinations?

Yes. Did you have problems using GTK with any of those compilers? If so,
you should complain to the GTK developers about that fact. They might
tell you that GTK isn't intended to be used with that compiler,
or they might change their code so it can be compatible with that
compiler.

I am not by any means defending this aspect of the C standard. I'm
merely explaining it.

[toc] | [prev] | [next] | [standalone]

#153398

On 31/07/2020 19:19, James Kuyper wrote:
> On Friday, July 31, 2020 at 12:22:15 PM UTC-4, Bart wrote:

>>
>> Did you read my post where several C compilers for Windows gave
>> different results for the sizes of various bitfield combinations?
> 
> Yes. Did you have problems using GTK with any of those compilers? If so,
> you should complain to the GTK developers about that fact. They might
> tell you that GTK isn't intended to be used with that compiler,
> or they might change their code so it can be compatible with that
> compiler.
> 
> I am not by any means defending this aspect of the C standard. I'm
> merely explaining it.
> 

Compiling even just '#include <gtk.h>' is quite challenging, using bare 
compilers. I can do it with gcc and bcc, but not tcc or others, for 
other reasons.

It would therefore take a long time to find structs that compile to a 
different size across compilers, as I'd have to recreate each struct 
within a separate test program that works with them all.

Take however this struct, compiled for a 64-bit target so that pointers 
are 64 bits:

struct new      // renamed from original as I modified this copy
{
   GParamSpec    parent_instance;
   gchar        *default_value;
   gchar        *cset_first;
   gchar        *cset_nth;
   gchar         substitutor;
   guint         null_fold_if_empty : 1;
   guint         ensure_non_null : 1;
};

parent_instance is 72 bytes, the next 3 are 24 bytes, total 96 bytes.

With 'substitutor', that's 97 bytes.

As it is, the total size is 97 bytes plus the bitfields (4 bytes on gcc, 
1 or tcc?), rounded up to 104 bytes. Take out the bitfields, and it's 
still rounded to 104 bytes. Here the size of the bitfields makes no 
difference.

But tweak it to this:

struct new
{
   GParamSpec    parent_instance;
   gchar        *default_value;
   gchar        *cset_first;
   gchar        *cset_nth;
   gchar         substitutor[7];           // EXTRA 6 BYTES
   guint         null_fold_if_empty : 1;
   guint         ensure_non_null : 1;
};

Size is now 103 bytes without the bitfields. Both tcc and gcc round this 
to 104 bytes. Add the bitfields back in: tcc adds only one byte (so 
still 104), gcc adds 4 (?), which is then rounded to 112.

Two different sizes for the same struct on two compilers for the same 
target.

Are there are actual structs in the 760 headers of GTK where this 
actually happens? I don't know. Does it matter if the sizes are 
different? I don't know. Does it matter if the offsets of some fields 
are different? I don't know (obviously it will do if you try and access 
them from the application).

But it did matter in my test a couple of years ago (I think the struct 
was cleared using my too-large size, but the struct was smaller; all I 
remember was that the wrong size caused it not to work).

It's a poor show.

[toc] | [prev] | [next] | [standalone]

#153401

On Friday, July 31, 2020 at 3:54:44 PM UTC-4, Bart wrote:
> On 31/07/2020 19:19, James Kuyper wrote:
> > On Friday, July 31, 2020 at 12:22:15 PM UTC-4, Bart wrote:
> 
> >>
> >> Did you read my post where several C compilers for Windows gave
> >> different results for the sizes of various bitfield combinations?
> > 
> > Yes. Did you have problems using GTK with any of those compilers? If so,
> > you should complain to the GTK developers about that fact. They might
> > tell you that GTK isn't intended to be used with that compiler,
> > or they might change their code so it can be compatible with that
> > compiler.
> > 
> > I am not by any means defending this aspect of the C standard. I'm
> > merely explaining it.
> > 
> 
> Compiling even just '#include <gtk.h>' is quite challenging, using bare 
> compilers. I can do it with gcc and bcc, but not tcc or others, for 
> other reasons.

The reason I asked whether you had any actual problems is that GTK is a
fairly popular, widely used package. Therefore, I'd naively expect that
if two different commonly used compilers targeting a platform supported
by GTK lay out some structs differently, it probably does something to
avoid the problems that might otherwise be expected. It might use
structs carefully designed to avoid triggering such differences. It
might use the preprocessor to present different declarations for the
same struct type when that type is seen by different compilers. It might
do something so clever I'm not even aware of what it might be.

Unless you've run into actual problems executing a program linked to the
GTK library using different compilers, you can't be sure that those
differences actually matter.

It also seems likely, from past experience, that you've done something
bizarre when installing GTK, something that might have disabled the
"magic" that would normally allow it to work with both of those
compilers.

[toc] | [prev] | [next] | [standalone]

#153402

On 31/07/2020 23:14, James Kuyper wrote:
> On Friday, July 31, 2020 at 3:54:44 PM UTC-4, Bart wrote:

>> Compiling even just '#include <gtk.h>' is quite challenging, using bare
>> compilers. I can do it with gcc and bcc, but not tcc or others, for
>> other reasons.
> 
> The reason I asked whether you had any actual problems is that GTK is a
> fairly popular, widely used package. Therefore, I'd naively expect that
> if two different commonly used compilers targeting a platform supported
> by GTK lay out some structs differently, it probably does something to
> avoid the problems that might otherwise be expected. It might use
> structs carefully designed to avoid triggering such differences. It
> might use the preprocessor to present different declarations for the
> same struct type when that type is seen by different compilers. It might
> do something so clever I'm not even aware of what it might be.
> 
> Unless you've run into actual problems executing a program linked to the
> GTK library using different compilers, you can't be sure that those
> differences actually matter.
> 
> It also seems likely, from past experience, that you've done something
> bizarre when installing GTK, something that might have disabled the
> "magic" that would normally allow it to work with both of those
> compilers.
> 

Yes, I had a problem using my compiler. But it doesn't implement 
bitfields, or at least does so in an implementation defined manner 
(namely, each bitfield occupies a whole int whatever the width).

This caused actual problems due to struct sizes being mismatched.

The experiments today showed that even compilers that implemented 
bitfields properly could still result in different struct sizes, 
depending on what's in the rest of the struct. Or could do so in future 
if the struct is changed.

Maybe it happens that none of the estimated 700 structs visible via 
gtk.h has just the right layout to trigger a discrepancy between 
compilers. Or if there is, it's one that doesn't cause problems, because 
it's only used as an opaque type passed via a pointer.

So there could well be a bug, or potential for one, lurking in the code.

Apart from not working with my bcc C compiler unless I used a tweak 
(hard-coding the expected struct size), I also had problems in 
determining the exact behaviour which I needed to do to create bindings 
for my own language, which doesn't have C-style unrestained bitfields.

Eg. whether that earlier example should be 104 bytes or 112 bytes.

If you think it's easy, please tell me what the size of this struct 
ought to be, and why you think that:

  struct s1 {
    int a:1;
    char b;
   };

I get mixed results on Windows, and mixed results on rextester.com.


BTW I still haven't found an example in GTK where bitfields are actually 
warranted, because they either save no space, or it's insignificant.

[toc] | [prev] | [next] | [standalone]

#153403

On Friday, July 31, 2020 at 7:32:22 PM UTC-4, Bart wrote:
...
> Maybe it happens that none of the estimated 700 structs visible via 
> gtk.h has just the right layout to trigger a discrepancy between 
> compilers.

If so, it probably didn't just "happen" - it's probably the result of
careful design.

[toc] | [prev] | [next] | [standalone]

#153404

On 01/08/2020 00:55, James Kuyper wrote:
> On Friday, July 31, 2020 at 7:32:22 PM UTC-4, Bart wrote:
> ...
>> Maybe it happens that none of the estimated 700 structs visible via
>> gtk.h has just the right layout to trigger a discrepancy between
>> compilers.
> 
> If so, it probably didn't just "happen" - it's probably the result of
> careful design.
> 

Not backed up by any comments to that effect in the source code. If 
somebody decides to add 6 bytes to that example I posted, then gcc and 
tcc WILL use different sizes for it, and one of them (most probably tcc) 
might not work (although tcc has problems anyway for other reasons).

BTW you haven't told me that that struct size should have been. On 
Windows, I get 4,8,4,8. On Rextester, 4,4,8. On Linux 4,4.

This for something that should fit into 2 bytes. What appears to happen 
is that on some compilers, the following char field fits into the spare 
bits of the int used in 'int a:1', even though it is not a bitfield, 
giving 4 bytes in all. And the full size of the 'int' is honoured even 
though at most 16 bits are needed for the struct.

While others allocate a full 32 bits for the bit-field, and start a new 
field for the char, but the struct size must be padded to keep field 'a' 
aligned in arrays.

It seems to me more guidance from the language would have helped. It 
seems like bitfield-handling was a free-for-all before, and the language 
couldn't retrospectively change things.

So at present, gcc on Windows gave 8 bytes for my example, and gcc on 
Linux gave 4 bytes.

[toc] | [prev] | [next] | [standalone]

#153405

On Friday, July 31, 2020 at 8:17:01 PM UTC-4, Bart wrote:
...
> BTW you haven't told me that that struct size should have been. On 
> Windows, I get 4,8,4,8. On Rextester, 4,4,8. On Linux 4,4.

Oh - sorry! I wasn't paying much attention that part of your message.

The bit-field must be allocated from an addressable storage unit, which
cannot overlap the other member of the struct. The alignment requirement
of the storage unit is explicitly unspecified. In C2011 and later, any
alignment requirement is required to be a non-negative integer power of
2 - however, I suspect that most of the compilers you tested conform, if
at all, to an older version of the standard, in which the alignment
requirement could be any positive integer.

The size of the storage unit is unspecified by omission of any explicit
specification. However, the general rules for alignment requirements
require that the storage unit have a size which is an integer multiple
of the alignment requirement of the storage unit.

_Alignof(struct s1) must be an integer multiple of the alignment
requirement of the storage unit, and in C2011 must be an non-negative
integer power of 2. sizeof(struct s1) must be an integer multiple of
_Alignof(struct s1) that is larger than the size of the storage unit.
It's likely to be the smallest such multiple, but that's not required.
sizeof(struct s1) can be as small as 2, and has no upper limit - it can
be any integer >= 2.

That's all that can be said about what the size "should be". If it
doesn't seem very useful, that could mean that you've correctly
understood what I've said.

> This for something that should fit into 2 bytes. What appears to happen 
> is that on some compilers, the following char field fits into the spare 
> bits of the int used in 'int a:1', even though it is not a bitfield, 

That can be explained by having a storage unit whose size is smaller
than sizeof(int).

> While others allocate a full 32 bits for the bit-field, and start a new 
> field for the char, but the struct size must be padded to keep field 'a' 
> aligned in arrays.

That corresponds to a storage unit with a size and alignment requirement
of 32 bits.

> It seems to me more guidance from the language would have helped.

The C standard seldom provides guidance - all it normally provides is
requirements. I agree that a few more requirements would have helped.

> It 
> seems like bitfield-handling was a free-for-all before, and the language 
> couldn't retrospectively change things.

I'm not sure that they couldn't have imposed stricter requirements - but
it's definitely the case that they didn't impose stricter requirements.

[toc] | [prev] | [next] | [standalone]

#153399

On Friday, 31 July 2020 17:22:15 UTC+1, Bart  wrote:
> 
> Did you read my post where several C compilers for Windows gave 
> different results for the sizes of various bitfield combinations?
> 
> Maybe bitfields as used in GTK will yield consistent results, but I 
> can't be sure. The best I can do is to use gcc, as I can assume it's 
> been tested with that, and write test programs using that will just 
> print the size of each struct.
> 
Structs need to have the same layout in memory for application code
to call libraries. It's not specifically a bitfield issue, though
bitfields are where the conventions are likely to break, if they
break at all.

If Windows compilers are giving different results for structures that
are identical in source code, then there's definitely a problem. I've
never noticed the issue myself, but then whilst I do quite a lot
of Windows programming, my interactions with the Windows API and third
party libraries are pretty light, and I always use a Microsoft compiler.
 

[toc] | [prev] | [next] | [standalone]

#153400

On 31/07/2020 20:55, Malcolm McLean wrote:
> On Friday, 31 July 2020 17:22:15 UTC+1, Bart  wrote:
>>
>> Did you read my post where several C compilers for Windows gave
>> different results for the sizes of various bitfield combinations?
>>
>> Maybe bitfields as used in GTK will yield consistent results, but I
>> can't be sure. The best I can do is to use gcc, as I can assume it's
>> been tested with that, and write test programs using that will just
>> print the size of each struct.
>>
> Structs need to have the same layout in memory for application code
> to call libraries. It's not specifically a bitfield issue, though
> bitfields are where the conventions are likely to break, if they
> break at all.
> 
> If Windows compilers are giving different results for structures that
> are identical in source code, then there's definitely a problem. I've
> never noticed the issue myself, but then whilst I do quite a lot
> of Windows programming, my interactions with the Windows API and third
> party libraries are pretty light, and I always use a Microsoft compiler.
>   
> 

The layout of structs is driven by the need to align fields according 
their size. It is assumed the structs starts at a suitable alignment.

There are a few rules which affect padding between members and at the 
end to ensure an array of such stucts will still have the right 
alignment for each one.

Compilers all seem to do the same thing, which is necessary for 
libraries such as WinAPI otherwise they couldn't be used from C. So all 
C compilers work.

There appears to be some variability with bitfields, but they are not 
used in WinAPI. They're rare in libraries which are expected to be portable.

[toc] | [prev] | [next] | [standalone]

#153972

James Kuyper <jameskuyper@alumni.caltech.edu> writes:

> Bit-fields only occur in structs;  bit-field layout is a subset of
> struct layout.  [...]

Both C and C++ also allow bit-field members in unions.

[toc] | [prev] | [next] | [standalone]

#153389

On 7/30/20 10:28 PM, James Kuyper wrote:

> I agree with your implied criticism of C. I consider the fact that so
> many aspects of struct layouts are implementation-defined rather than
> standard-defined to be one of the major defects of C (and C++ is no
> better - actually, it's somewhat worse). But I don't see any point in
> pretending that you don't know this already.
> 

Actually, in one sense the looseness of the definitions for structure
layouts is one of the strengths of C too. C gives implementations the
flexibility to define things to allow the generation of more efficient
code for a particular machine. With more rigid definitions, then C could
have been much harder to implement on some of the more unusual machines.

There is perhaps grounds to say that in today's largely mono-culture of
CPU architecture (even if not all the exact same CPU, there are a number
of attributes they have in common, like 8 bit byte addressable, two's
complement machines without padding bits, maybe even adding flat
monolithic address space), and Appendix or additional standard to cover
these common cases might make sense. I think the C Standard committee
has decided that this sort of has already been done with things like
platform ABIs and things like the POSIX standard, so it doesn't by them
that much to fold such things into the C Standard.

[toc] | [prev] | [next] | [standalone]

#153391

On 31/07/2020 13:23, Richard Damon wrote:
> On 7/30/20 10:28 PM, James Kuyper wrote:
> 
>> I agree with your implied criticism of C. I consider the fact that so
>> many aspects of struct layouts are implementation-defined rather than
>> standard-defined to be one of the major defects of C (and C++ is no
>> better - actually, it's somewhat worse). But I don't see any point in
>> pretending that you don't know this already.
>>
> 
> Actually, in one sense the looseness of the definitions for structure
> layouts is one of the strengths of C too. C gives implementations the
> flexibility to define things to allow the generation of more efficient
> code for a particular machine. With more rigid definitions, then C could
> have been much harder to implement on some of the more unusual machines.
> 
> There is perhaps grounds to say that in today's largely mono-culture of
> CPU architecture (even if not all the exact same CPU, there are a number
> of attributes they have in common, like 8 bit byte addressable, two's
> complement machines without padding bits, maybe even adding flat
> monolithic address space), and Appendix or additional standard to cover
> these common cases might make sense. I think the C Standard committee
> has decided that this sort of has already been done with things like
> platform ABIs and things like the POSIX standard, so it doesn't by them
> that much to fold such things into the C Standard.
> 

Here's how I implement bitfields; given this C example (which had a size 
of either 2 or 4 bytes):

     typedef struct {
         short a:9;
         char b:7;
     } s3;

I define it like this:

     record s3 =
        word16 attribs : (a:9, b:7)
     end

* The bitfields are only defined inside a conventional 'host' member

* That member will here ALWAYS be 16 bits, so the struct size will
   always be 16 bits

* The alighment of the first bitfield is tied to the alignment of
   the enclosing member

* The group of bitfields can be read, written, passed, assigned etc
   as a whole via the 'attribs' member

* Allocation of bitfields always starts from bit 0 (least sig) of
   the host member

* Overflow of the host (eg. if b was 8 bits) is reported. (C will just
   use more fields. If you'd intended a+b to be 16 bits, but were in
   fact 17 bits, you wouldn't know.)

* Bitfields here are always unsigned.

* Bitfield access is related to more general bitfield syntax (they are
   implemented with the same mechanims), so that p.a is equivalent to
   p.attribs.[0..8]. Such accesses are unsigned so named bitfields are
   unsigned.

* Bitfields can be up to 64 bits long (or 63 bits in practice as a
   64-bit one is just a regular int). (Actually, 128 bits within a
   128-bit host member, but this is less useful so may be restricted.)

Better than C, yes?

[toc] | [prev] | [next] | [standalone]

#153431

Am 31.07.20 um 14:23 schrieb Richard Damon:
> There is perhaps grounds to say that in today's largely mono-culture of
> CPU architecture (even if not all the exact same CPU, there are a number
> of attributes they have in common, like 8 bit byte addressable, two's
> complement machines without padding bits, maybe even adding flat
> monolithic address space), and Appendix or additional standard to cover
> these common cases might make sense.

A bit of that can be found in Annex J.5.

[toc] | [prev] | [next] | [standalone]

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

Back to top | Article view | comp.lang.c

csiph-web