Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #77000 > unrolled thread

Aliasing through a union pointer

Back to article view | Back to comp.lang.c

Contents

  Aliasing through a union pointer Ian Pilcher <arequipeno@gmail.com> - 2015-11-23 10:14 -0600
    Re: Aliasing through a union pointer Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-11-23 11:43 -0500
      Re: Aliasing through a union pointer Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-11-23 11:49 -0500
      Re: Aliasing through a union pointer Ian Pilcher <arequipeno@gmail.com> - 2015-11-23 13:22 -0600
        Re: Aliasing through a union pointer Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-11-23 14:38 -0500
          Re: Aliasing through a union pointer Tim Rentsch <txr@alumni.caltech.edu> - 2015-11-25 10:38 -0800
            Re: Aliasing through a union pointer Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-11-25 14:17 -0500
              Re: Aliasing through a union pointer Tim Rentsch <txr@alumni.caltech.edu> - 2015-11-27 13:16 -0800
        Re: Aliasing through a union pointer David Brown <david.brown@hesbynett.no> - 2015-11-23 21:46 +0100
          Re: Aliasing through a union pointer Jorgen Grahn <grahn+nntp@snipabacken.se> - 2015-11-23 22:20 +0000
            Re: Aliasing through a union pointer Stephen Sprunk <stephen@sprunk.org> - 2015-11-23 16:52 -0600
              Re: Aliasing through a union pointer Jorgen Grahn <grahn+nntp@snipabacken.se> - 2015-11-23 23:10 +0000
                Re: Aliasing through a union pointer Stephen Sprunk <stephen@sprunk.org> - 2015-11-23 19:05 -0600
            Re: Aliasing through a union pointer Nick Bowler <nbowler@draconx.ca> - 2015-11-23 22:51 +0000
            Re: Aliasing through a union pointer Keith Thompson <kst-u@mib.org> - 2015-11-23 15:49 -0800
              Re: Aliasing through a union pointer supercat@casperkitty.com - 2015-11-23 16:01 -0800
            Re: Aliasing through a union pointer David Brown <david.brown@hesbynett.no> - 2015-11-24 09:55 +0100
              Re: Aliasing through a union pointer Jorgen Grahn <grahn+nntp@snipabacken.se> - 2015-11-24 14:29 +0000
                Re: Aliasing through a union pointer David Brown <david.brown@hesbynett.no> - 2015-11-24 22:58 +0100
                  Re: Aliasing through a union pointer Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-11-24 17:30 -0500
                    Re: Aliasing through a union pointer supercat@casperkitty.com - 2015-11-24 15:27 -0800
                    Re: Aliasing through a union pointer Ian Collins <ian-news@hotmail.com> - 2015-11-25 12:57 +1300
                      Re: Aliasing through a union pointer supercat@casperkitty.com - 2015-11-24 16:31 -0800
                        Re: Aliasing through a union pointer Ian Collins <ian-news@hotmail.com> - 2015-11-25 18:28 +1300
                          Re: Aliasing through a union pointer supercat@casperkitty.com - 2015-11-25 06:54 -0800
                            Re: Aliasing through a union pointer Ian Collins <ian-news@hotmail.com> - 2015-11-26 08:59 +1300
                              Re: Aliasing through a union pointer supercat@casperkitty.com - 2015-11-25 13:01 -0800
                                Re: Aliasing through a union pointer Ian Collins <ian-news@hotmail.com> - 2015-11-26 12:58 +1300
                                  Re: Aliasing through a union pointer supercat@casperkitty.com - 2015-11-25 16:44 -0800
                                    Re: Aliasing through a union pointer Ian Collins <ian-news@hotmail.com> - 2015-11-26 14:12 +1300
                                      Re: Aliasing through a union pointer supercat@casperkitty.com - 2015-11-25 21:34 -0800
                                        Re: Aliasing through a union pointer Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-11-25 21:55 -0800
                                        Re: Aliasing through a union pointer Ian Collins <ian-news@hotmail.com> - 2015-11-26 20:51 +1300
                                          Re: Aliasing through a union pointer supercat@casperkitty.com - 2015-11-26 11:39 -0800
                    Re: Aliasing through a union pointer Jorgen Grahn <grahn+nntp@snipabacken.se> - 2015-11-26 00:06 +0000
              Re: Aliasing through a union pointer Keith Thompson <kst-u@mib.org> - 2015-11-24 11:51 -0800
                Re: Aliasing through a union pointer supercat@casperkitty.com - 2015-11-24 12:20 -0800
                  Re: Aliasing through a union pointer Keith Thompson <kst-u@mib.org> - 2015-11-24 21:52 -0800
                    Re: Aliasing through a union pointer supercat@casperkitty.com - 2015-11-25 08:30 -0800
                Re: Aliasing through a union pointer David Brown <david.brown@hesbynett.no> - 2015-11-24 23:04 +0100
    Re: Aliasing through a union pointer Tim Rentsch <txr@alumni.caltech.edu> - 2015-11-25 10:10 -0800
      Re: Aliasing through a union pointer Richard Damon <Richard@Damon-Family.org> - 2015-11-25 11:02 -0800
        Re: Aliasing through a union pointer Philip Lantz <prl@canterey.us> - 2015-11-26 13:06 -0800
          Re: Aliasing through a union pointer Richard Damon <Richard@Damon-Family.org> - 2015-11-26 13:09 -0800
        Re: Aliasing through a union pointer Tim Rentsch <txr@alumni.caltech.edu> - 2015-11-27 13:08 -0800

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

#77135

On Tuesday, November 24, 2015 at 4:31:06 PM UTC-6, Eric Sosman wrote:
>      Not only does the separation of forms avoid all the alignment
> issues discussed elsethread, but it also offers a convenient hook for
> dealing with other matters.  Does the external form require a specific
> endianness that may or may not match your machine's?

IMHO, a good systems programming language (which is what C tried to be, at
least for awhile) should allow a programmer to specify what is required from
various types, and what aspects a compiler should be free to optimize as
convenient.  Unless a compiler needs to run on a machine with less than a meg
of memory (which would be rare nowadays, even if the target might have less
than a kilobyte) there really shouldn't be any difficulty with a compiler
allowing code to define structures very precisely in terms of physical layout
and then generating code to access members appropriately.  Compilers should
be able to do a much better job of producing efficient code from such things
than user code would be able to do, especially if code which made use of
aliasing was required to include explicit aliasing barriers [but the language
actually included the barriers that were required].  Local machine endianness
needn't be a problem, because the definition of the structure could specify
what endianness was actually needed.

[toc] | [prev] | [next] | [standalone]

#77137

Eric Sosman wrote:
> On 11/24/2015 4:58 PM, David Brown wrote:
>> [... "packed" structs and similar underhanded artifices ...]
>> Also explicit packing "dummies" have the advantage that there is no
>> chance of undefined, implementation-defined, unspecified or generally
>> unexpected or undesired behaviour when they are accessed.  (When storing
>> something in a struct, padding bytes get unspecified values.)  This is
>> particularly important if you want to compare structs, or do something
>> like calculate a checksum over them.
>
>       It's been my experience that these questions of packing and so on
> nearly always arise in connection with an externally-imposed format of
> some kind: You want to use a struct to represent an IP header or a
> disk partition table entry or something of that sort.  If that's the
> situation, I've almost always found that trying to sculpt the struct
> into the desired shape is inferior to just letting the struct take its
> own preferred form and using a mediating buffer to translate to and
> from the imposed format.  Use separate "native" and "serialized" forms,
> and you're quite likely to be better off than by trying to force both
> forms into one object.
>
>       Not only does the separation of forms avoid all the alignment
> issues discussed elsethread, but it also offers a convenient hook for
> dealing with other matters.  Does the external form require a specific
> endianness that may or may not match your machine's?  Does it use a
> weird representation for negative numbers, or for floating-point
> quantities?  Are text strings supposed to be a count word followed
> by a payload in some strange five-bit encoding?  The import() and
> export() functions can handle all such idiosyncracies cleanly and
> quietly, so the rest of the program can deal with perfectly normal,
> native-format structs with native-format numbers, zero-terminated
> native-encoded strings, and so on.  The translation functions are
> also good places to compute and verify things like checksums, which
> are usually specified in terms of the external format.
>
>       Somewhere, somebody is shrieking "Efficiency!  All that copying?
> Why should I un-pickle an entire object just to test the one field
> that tells me whether I care about the others?  Eeeek!"  This shrieker
> is overly fearful and insufficiently imaginative; there are plenty of
> ways to attack such issues if they should prove important.  (Also,
> for some reason the shrieker seldom yowls about the speed penalties
> of using the misaligned elements in his packed structs.  Go figure.)
>
>       There are many situations where it is tempting to map an imposed
> format with a struct, but resisting temptation is nearly always a
> virtue, and a virtue with rewards.

I agree with you here.  I leaned that lesson many years ago when testing 
embedded code on SPARC systems.  The cost of a misaligned access was way 
more than the cost of unpacking the messages (and there were a lot of 
them!).

The "Why should I un-pickle an entire object just to test the one 
field?" problem can be solved by lazy extraction: only extract that one 
field and keep the raw buffer around to be processed later.

-- 
Ian Collins

[toc] | [prev] | [next] | [standalone]

#77139

On Tuesday, November 24, 2015 at 5:58:04 PM UTC-6, Ian Collins wrote:
> The "Why should I un-pickle an entire object just to test the one 
> field?" problem can be solved by lazy extraction: only extract that one 
> field and keep the raw buffer around to be processed later.

It could be solved even more nicely if a language provided built-in support
for such a semantics.  I would aver that:

1. While working with data in externally-imposed formats isn't necessary in
   all projects, nor is it an obscure requirement that's only needed on a
   few specialized systems.  Such semantics often play an important rule in
   things like audio-visual decoding, where performance can be very
   important.

2. It would be possible to formulate a list of required features that would
   handle the vast majority of applicable usage cases.

3. It would greatly ease porting of code between machines with different
   architectures.

4. In many cases, the most efficient way to code something will vary
   enormously based not just on the target architecture, but also on context.
   If a field's value will be needed twice, it may make sense to cache it
   in a native-format register, or to simply fetch it when needed, but a
   programmer can't be expected to know which approach will be faster.
   By contrast, if a typed-memory-barrier directive were required to separate
   accesses to a memory-imaged type from accesses to any other type which
   might alias it, a compiler would be able to apply optimizations much more
   effectively than a programmer could.

IMHO, rather than trying to push optimization at the expense of everything
else, it would be more helpful for C to offer more ways for programmers to
specify their requirements declaratively rather than imperatively, and let
the compiler figure out how to satisfy such declarative requirements rather
than trying to infer requirements from imperative-language code.

[toc] | [prev] | [next] | [standalone]

#77145

supercat@casperkitty.com wrote:
> On Tuesday, November 24, 2015 at 5:58:04 PM UTC-6, Ian Collins wrote:
>> The "Why should I un-pickle an entire object just to test the one
>> field?" problem can be solved by lazy extraction: only extract that one
>> field and keep the raw buffer around to be processed later.
>
> It could be solved even more nicely if a language provided built-in support
> for such a semantics.  I would aver that:
>
> 1. While working with data in externally-imposed formats isn't necessary in
>     all projects, nor is it an obscure requirement that's only needed on a
>     few specialized systems.  Such semantics often play an important rule in
>     things like audio-visual decoding, where performance can be very
>     important.
>
> 2. It would be possible to formulate a list of required features that would
>     handle the vast majority of applicable usage cases.
>
> 3. It would greatly ease porting of code between machines with different
>     architectures.
>
> 4. In many cases, the most efficient way to code something will vary
>     enormously based not just on the target architecture, but also on context.
>     If a field's value will be needed twice, it may make sense to cache it
>     in a native-format register, or to simply fetch it when needed, but a
>     programmer can't be expected to know which approach will be faster.
>     By contrast, if a typed-memory-barrier directive were required to separate
>     accesses to a memory-imaged type from accesses to any other type which
>     might alias it, a compiler would be able to apply optimizations much more
>     effectively than a programmer could.
>
> IMHO, rather than trying to push optimization at the expense of everything
> else, it would be more helpful for C to offer more ways for programmers to
> specify their requirements declaratively rather than imperatively, and let
> the compiler figure out how to satisfy such declarative requirements rather
> than trying to infer requirements from imperative-language code.

The language doesn't because it can't.

Some targets simply don't support misaligned access and others support 
it at great cost (a trap in the case of SPARC).  On those that do, 
compiler extensions (such as those offered by gcc) are the recognised 
and appropriate solution.  I would argue that it is important for the 
programmer to be aware that they are entering the realm of 
implementation defined behaviour in this case.

-- 
Ian Collins

[toc] | [prev] | [next] | [standalone]

#77165

On Tuesday, November 24, 2015 at 11:28:29 PM UTC-6, Ian Collins wrote:
> The language doesn't because it can't.
> 
> Some targets simply don't support misaligned access and others support 
> it at great cost (a trap in the case of SPARC).  On those that do, 
> compiler extensions (such as those offered by gcc) are the recognised 
> and appropriate solution.  I would argue that it is important for the 
> programmer to be aware that they are entering the realm of 
> implementation defined behaviour in this case.

If "foo" was defined as a pointer to a big-endian 32-bit integer stored using
8 bits in each of four "char" values, what processor would not be able to, at
worst, accommodate a compiler replacing "unsigned long l = *foo;" with

    unsigned char *p = foo;
    l = (p[1] << 8) | p[3] | ((unsigned long)p[2] << 16) |
        ((unsigned long)p[0] << 24);

A programmer would want to avoid using image types unnecessarily, but in
cases where it's necessary to process data according to a memory image,
a compiler would often be in a better position than a programmer to make
decisions about efficiency.

There would be some possible image descriptions that would be meaningful on
some processors but not others (e.g. storing a 36-bit unsigned integer as
four 9-bit characters), but the solution for that is to say that *if code
compiles* it must behave as indicated, while allowing for the fact that not
all compilers will be able to compile all code.  Having code which won't
compile on all implementations, but will run correctly on all platforms that
can compile it, would be miles better than having code which compiles on all
platforms, runs correctly on some, and arbitrarily malfunctions on the rest.

[toc] | [prev] | [next] | [standalone]

#77183

supercat@casperkitty.com wrote:
> On Tuesday, November 24, 2015 at 11:28:29 PM UTC-6, Ian Collins wrote:
>> The language doesn't because it can't.
>>
>> Some targets simply don't support misaligned access and others support
>> it at great cost (a trap in the case of SPARC).  On those that do,
>> compiler extensions (such as those offered by gcc) are the recognised
>> and appropriate solution.  I would argue that it is important for the
>> programmer to be aware that they are entering the realm of
>> implementation defined behaviour in this case.
>
> If "foo" was defined as a pointer to a big-endian 32-bit integer stored using
> 8 bits in each of four "char" values, what processor would not be able to, at
> worst, accommodate a compiler replacing "unsigned long l = *foo;" with
>
>      unsigned char *p = foo;
>      l = (p[1] << 8) | p[3] | ((unsigned long)p[2] << 16) |
>          ((unsigned long)p[0] << 24);

I'm sure any compiler could do that, but it would have to do so for any 
assignment through a pointer to a a type other than (unsigned) char 
unless it could prove the address was correctly aligned.  An alternative 
is to trap on an access violation and perform the access byte by byte 
(as on SPARC).  Neither should happen without the programmer making the 
choice (as they do on SPARC).

-- 
Ian Collins

[toc] | [prev] | [next] | [standalone]

#77188

On Wednesday, November 25, 2015 at 1:59:58 PM UTC-6, Ian Collins wrote:
> I'm sure any compiler could do that, but it would have to do so for any 
> assignment through a pointer to a a type other than (unsigned) char 
> unless it could prove the address was correctly aligned.  An alternative 
> is to trap on an access violation and perform the access byte by byte 
> (as on SPARC).  Neither should happen without the programmer making the 
> choice (as they do on SPARC).

Perhaps you misunderstood my intention.  The idea was not to say that all
pointers to integer types should tolerate unaligned loads, but rather that
there should be a means of defining types with precise layout and alignment
specifications, and that accesses *made through pointers to such types*
should do whatever is necessary to accommodate them.  Generally code should
try to minimize use of such types within tight loops, but if it's necessary
to operate upon a large foreign-layout data structure that consists of lots
of little pieces, it may sometimes be faster to operate on the structure in
its native format than to convert to a native representation, process
everything, and then convert back, especially if much of the data will be
untouched.

In cases where a small embedded processor and a PC will exchange data in the
native format of the embedded processor, the embedded processor may not be
able to afford the code or CPU time necessary to manipulate everything using
character types, but the PC certainly would in most cases easily be able to
afford both.  I would suggest that even code designed around the embedded
systems' storage formats would be much less efficient on the PC than would
code that was designed around the PC's storage formats, there would still be
great value in being able to have the PC "leisurely stroll" the same code
that the embedded system could run.

[toc] | [prev] | [next] | [standalone]

#77194

supercat@casperkitty.com wrote:
> On Wednesday, November 25, 2015 at 1:59:58 PM UTC-6, Ian Collins wrote:
>> I'm sure any compiler could do that, but it would have to do so for any
>> assignment through a pointer to a a type other than (unsigned) char
>> unless it could prove the address was correctly aligned.  An alternative
>> is to trap on an access violation and perform the access byte by byte
>> (as on SPARC).  Neither should happen without the programmer making the
>> choice (as they do on SPARC).
>
> Perhaps you misunderstood my intention.  The idea was not to say that all
> pointers to integer types should tolerate unaligned loads, but rather that
> there should be a means of defining types with precise layout and alignment
> specifications, and that accesses *made through pointers to such types*
> should do whatever is necessary to accommodate them.

I see where you are coming from, but I don't think your suggestion goes 
far enough.  Consider passing the address of second to f() in this snippet:

packed struct S
{
   uint8_t  first;
   uint32_t second;
};

void f( uint32_t* );

<snip>

> In cases where a small embedded processor and a PC will exchange data in the
> native format of the embedded processor, the embedded processor may not be
> able to afford the code or CPU time necessary to manipulate everything using
> character types, but the PC certainly would in most cases easily be able to
> afford both.  I would suggest that even code designed around the embedded
> systems' storage formats would be much less efficient on the PC than would
> code that was designed around the PC's storage formats, there would still be
> great value in being able to have the PC "leisurely stroll" the same code
> that the embedded system could run.

There is - compiler extensions!  Supporting misaligned access impacts 
the entire program, not just specific accesses.

I have done a lot of work similar to what you describe above as well as 
running parts of the embedded code on a workstation host.  My solution 
on SPARC (which can't do packed structs) was to compile the embedded 
code as C++ and use customised types for the stricture members to handle 
the packing an unpacking.  A more general solution would be to use 
opaque types and accessor functions.

At the time, enabling misaligned access on the SPARC build cased the 
code to run slower than it did on the embedded target....

-- 
Ian Collins

[toc] | [prev] | [next] | [standalone]

#77197

On Wednesday, November 25, 2015 at 5:58:21 PM UTC-6, Ian Collins wrote:
> I see where you are coming from, but I don't think your suggestion goes 
> far enough.  Consider passing the address of second to f() in this snippet:
> 
> __packed struct S
> {
>    uint8_t  first;
>    uint32_t second;
> };

> void f( uint32_t* );

With regard to the above code, I'd say that the type of &thing.second should
be "uint32_t __packed *", which is not compatible with "uint32_t*", but could
be safely passed to "f" if the function had been declared

  void f( uint32_t __packed* p);

or

  void f( uint32_t __packed* restrict p);

Of course, declaring the argument type as "uint32_t __packed *" rather than
"uint32_t*" would degrade performance, especially in the absence of a
restrict qualifier, but (especially with "restrict") it may still be able
to outperform any user-written code for the same purpose.  Consider, for
example, that if the type is little-endian, a compiler could optimize
"*p |= 768;" to simply "((uint8_t*)p)[1] |= 3;" whereas user code which
wasn't designed around the value being a multiple of 256 would more likely
have to read all four bytes, do the "or", and then write all four bytes
back, and a compiler would be far less likely to notice that it could skip
three of the reads and three of the writes.

> There is - compiler extensions!  Supporting misaligned access impacts 
> the entire program, not just specific accesses.

If the small system and big system alike both used new types whose storage
format and behavior were specified in precise terms, why would the existence
of such types need to affect any code using normal types on the PC?

> I have done a lot of work similar to what you describe above as well as 
> running parts of the embedded code on a workstation host.  My solution 
> on SPARC (which can't do packed structs) was to compile the embedded 
> code as C++ and use customised types for the stricture members to handle 
> the packing an unpacking.  A more general solution would be to use 
> opaque types and accessor functions.

I've done something similar so I could run the same code on both the embedded
target using C, and on a PC using C++; unfortunately, the two languages are
diverging from each other rather than converging.

> At the time, enabling misaligned access on the SPARC build cased the 
> code to run slower than it did on the embedded target....

Was it slower because the code was trying to do unaligned reads which then
generated traps, or because the code was using sequences of byte reads and
putting them together?

[toc] | [prev] | [next] | [standalone]

#77199

supercat@casperkitty.com wrote:
> On Wednesday, November 25, 2015 at 5:58:21 PM UTC-6, Ian Collins wrote:
>> I see where you are coming from, but I don't think your suggestion goes
>> far enough.  Consider passing the address of second to f() in this snippet:
>>
>> __packed struct S
>> {
>>     uint8_t  first;
>>     uint32_t second;
>> };
>
>> void f( uint32_t* );
>
> With regard to the above code, I'd say that the type of &thing.second should
> be "uint32_t __packed *", which is not compatible with "uint32_t*", but could
> be safely passed to "f" if the function had been declared
>
>    void f( uint32_t __packed* p);
>
> or
>
>    void f( uint32_t __packed* restrict p);

But that would preclude calling any (library) function that didn't have 
the qualification.  That's why I said supporting misaligned access 
impacts the entire program.

<snip>

>> There is - compiler extensions!  Supporting misaligned access impacts
>> the entire program, not just specific accesses.
>
> If the small system and big system alike both used new types whose storage
> format and behavior were specified in precise terms, why would the existence
> of such types need to affect any code using normal types on the PC?

Because misaligned pointers can leak.

>> I have done a lot of work similar to what you describe above as well as
>> running parts of the embedded code on a workstation host.  My solution
>> on SPARC (which can't do packed structs) was to compile the embedded
>> code as C++ and use customised types for the stricture members to handle
>> the packing an unpacking.  A more general solution would be to use
>> opaque types and accessor functions.
>
> I've done something similar so I could run the same code on both the embedded
> target using C, and on a PC using C++; unfortunately, the two languages are
> diverging from each other rather than converging.

That depends on how much of the "new" stuff in C99 you use :)

>> At the time, enabling misaligned access on the SPARC build cased the
>> code to run slower than it did on the embedded target....
>
> Was it slower because the code was trying to do unaligned reads which then
> generated traps, or because the code was using sequences of byte reads and
> putting them together?

The former.

-- 
Ian Collins

[toc] | [prev] | [next] | [standalone]

#77201

On Wednesday, November 25, 2015 at 7:13:05 PM UTC-6, Ian Collins wrote:
> But that would preclude calling any (library) function that didn't have 
> the qualification.  That's why I said supporting misaligned access 
> impacts the entire program.

And I'd say that library functions would be limited to using normal pointers;
I'm curious for which library problems you think that would be a problem.
if one wants to read a number from the a file into a bitfield, one can't use
fscanf to load data into the bit-field directly--one must read it into a
temporary variable and then copy that manually.  How many library routines
are there that accept a non-character pointer type and write a lot of data
to it?

> Because misaligned pointers can leak.

Only if the type system allows them to be given to code that expects normal
pointers.

> That depends on how much of the "new" stuff in C99 you use :)

The languages also have different rules about things like aliasing and
various other oddities.

>> Was it slower because the code was trying to do unaligned reads which then
>> generated traps, or because the code was using sequences of byte reads and
>> putting them together?
> 
> The former.

Well don't do that. >:*3  If one tells the compiler to *expect* unaligned
data (possible in at least some compilers, and IMHO should be defined in the
Standard even if compilers were allowed to refuse compilation of programs
specifying it) the performance impact would be greatly reduced to be no
worse than (and in many cases not as bad as) the cost of user code building
up larger types from smaller ones.

[toc] | [prev] | [next] | [standalone]

#77202

On Thursday, November 26, 2015 at 5:34:53 AM UTC, supe...@casperkitty.com wrote:
> On Wednesday, November 25, 2015 at 7:13:05 PM UTC-6, Ian Collins wrote:
> > But that would preclude calling any (library) function that didn't have 
> > the qualification.  That's why I said supporting misaligned access 
> > impacts the entire program.
> 
> And I'd say that library functions would be limited to using normal 
> pointers;
> I'm curious for which library problems you think that would be a problem.
> if one wants to read a number from the a file into a bitfield, one can't  > use fscanf to load data into the bit-field directly--one must read it 
> into a temporary variable and then copy that manually.  How many library
> routines are there that accept a non-character pointer type and write a
> lot of data to it?
> 
memcpy takes a void *, which is only rarely a string buffer (strcpy()
is used instead), and is even more rarely not 4 byte aligned.
But it has to support any combination of size and alignment. It's
easy enough to patch a compiler to insert fast copies, although
for a long time you often got a simple byte-copying loop.

However often the reason memcpy() is called is because the data has
been abstracted away and passed in as a void *. So it can be hard
to apply the patch. 

[toc] | [prev] | [next] | [standalone]

#77204

supercat@casperkitty.com wrote:
> On Wednesday, November 25, 2015 at 7:13:05 PM UTC-6, Ian Collins wrote:
>> But that would preclude calling any (library) function that didn't have
>> the qualification.  That's why I said supporting misaligned access
>> impacts the entire program.
>
> And I'd say that library functions would be limited to using normal pointers;
> I'm curious for which library problems you think that would be a problem.
> if one wants to read a number from the a file into a bitfield, one can't use
> fscanf to load data into the bit-field directly--one must read it into a
> temporary variable and then copy that manually.  How many library routines
> are there that accept a non-character pointer type and write a lot of data
> to it?

You don't have to.  A single read or write is all that is required.  If 
I understand your concept correctly and you have a new qualifier, you 
will need to copy data between qualified and unqualified pointers, which 
is little different from copying data to and from packed external 
message formats.

>> Because misaligned pointers can leak.
>
> Only if the type system allows them to be given to code that expects normal
> pointers.

See above.

>> That depends on how much of the "new" stuff in C99 you use :)
>
> The languages also have different rules about things like aliasing and
> various other oddities.

None of which make any practical difference in normal code.  I mix C and 
C++ often and I must admit I would have to go looking for any such 
differences.

>>> Was it slower because the code was trying to do unaligned reads which then
>>> generated traps, or because the code was using sequences of byte reads and
>>> putting them together?
>>
>> The former.
>
> Well don't do that. >:*3  If one tells the compiler to *expect* unaligned
> data (possible in at least some compilers, and IMHO should be defined in the
> Standard even if compilers were allowed to refuse compilation of programs
> specifying it) the performance impact would be greatly reduced to be no
> worse than (and in many cases not as bad as) the cost of user code building
> up larger types from smaller ones.

Compilers for machines such as SPARC would have to refuse due to 
hardware restrictions.  So you don't gain any portability benefits over 
compiler extensions.

-- 
Ian Collins

[toc] | [prev] | [next] | [standalone]

#77221

On Thursday, November 26, 2015 at 1:51:59 AM UTC-6, Ian Collins wrote:
> You don't have to.  A single read or write is all that is required.  If 
> I understand your concept correctly and you have a new qualifier, you 
> will need to copy data between qualified and unqualified pointers, which 
> is little different from copying data to and from packed external 
> message formats.

Requiring code to occasionally use a temporary variable when exchanging data
between a library routine which wants a pointer to a native integer and a
data type that holds something else would be less bad than having to do
translation all the time.  Besides, in cases where the called routine applies
a restrict qualifier (which most library routines actually should) a compiler
could safely generate and use the temporary variable automatically if
needed, or could pass the address of a compiler-generated temporary if the
compiler needs one for every purposes.

BTW, it's too bad there's no variation of "restrict" which would conveniently
indicate that two (or more) parameters may alias each other but will not
alias anything else in the universe that is used during function execution.
If memmove() had restrict-qualified parameters, it could actually behave just
as it does now while allowing callers to make more optimizations than are
presently possible, though implementations with which have weak pointer-
overlap detection could not be as efficient as they can be without the
qualifier (those which either have no ability to use relational operators
with pointers, or those where relational operators can check for arbitrary
pointer-region overlap, could be just as efficient as they are now, even
while abiding by the rules of "restrict").

[toc] | [prev] | [next] | [standalone]

#77195

On Tue, 2015-11-24, Eric Sosman wrote:
> On 11/24/2015 4:58 PM, David Brown wrote:
>> [... "packed" structs and similar underhanded artifices ...]
>> Also explicit packing "dummies" have the advantage that there is no
>> chance of undefined, implementation-defined, unspecified or generally
>> unexpected or undesired behaviour when they are accessed.  (When storing
>> something in a struct, padding bytes get unspecified values.)  This is
>> particularly important if you want to compare structs, or do something
>> like calculate a checksum over them.
>
>      It's been my experience that these questions of packing and so on
> nearly always arise in connection with an externally-imposed format of
> some kind: You want to use a struct to represent an IP header or a
> disk partition table entry or something of that sort.  If that's the
> situation, I've almost always found that trying to sculpt the struct
> into the desired shape is inferior to just letting the struct take its
> own preferred form and using a mediating buffer to translate to and
> from the imposed format.  Use separate "native" and "serialized" forms,
> and you're quite likely to be better off than by trying to force both
> forms into one object.

When I wrote "I prefer reading uint8_t by uint8_t from a buffer in
most cases where I could have used a struct 'overlay'" upthread, I was
referring to the first part of that.

I'm not so sure about the second part -- having

  struct Foo deserialise(source);
  void serialise(struct Foo, target);

In many cases, yes. But in other cases I find that the code
/on top of/ that gets nicer if I use some other approach.  For
example, the data may not map well to a struct.  And that's not
primarily an efficiency argument.

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#77122

David Brown <david.brown@hesbynett.no> writes:
> On 23/11/15 23:20, Jorgen Grahn wrote:
>> On Mon, 2015-11-23, David Brown wrote:
>> ...
>>> I'd advise against __attribute__((packed)) (or equivalents on other 
>>> compilers) - either it makes no difference because there is no padding, 
>>> or you will end up with structures with fields that don't match their 
>>> alignment.  On some targets, that means run-time errors - on others, it 
>>> means seriously degraded code speed.  So it is rarely a good idea.
>> 
>> I think I agree, but are there really cases where you say
>> __attribute__((packed)) and then suffer a crash because of it?  The
>> compiler shouldn't generate a struct, and code to access it which will
>> crash if it's ever run.
>
> The attribute tells gcc that it should generate the struct layout
> exactly as /you/ tell it - /not/ as the target system normally requires
> with respect to alignment.

I wouldn't quite put it that way.  Even with __attribute__((packed)),
you're not specifying the exact layout.  You're just declaring the names
and types of the members, and letting the compiler determine the layout.
__attribute__((packed)) just causes the compiler to follow a different
set of rules in doing so.

>                             On some systems, misaligned accesses leads
> to a bus fault or similar hardware exception - a crash, in most cases.
> On some other systems, you will not get a hardware exception but you
> will access incorrect data (for example, a 16-bit access will simply
> ignore the lowest bit of the address).  That means disaster of some sort.

If gcc knows that it's accessing a member of a packed struct, it will do
whatever is necessary, perhaps generating code to access it a byte at a
time.  Problems occur only when you take the address of a member.

> If you use "packed", then /you/ take full responsibility for making sure
> the accesses are correct.  Perhaps you want to create a pointer to that
> struct that is not fully aligned (that's going to involve some
> implementation-dependent pointer casts).  Perhaps you use memcpy() to
> access the fields, or some other method that you know is safe.

No, the compiler takes *some* responsibility for making sure accesses
are correct.  It's just not able to do so in all cases.

(That's not a general language rule, but it's how gcc behaves, and we're
talking about a gcc-specific feature.)

[...]

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#77123

On Tuesday, November 24, 2015 at 1:51:52 PM UTC-6, Keith Thompson wrote:
> If gcc knows that it's accessing a member of a packed struct, it will do
> whatever is necessary, perhaps generating code to access it a byte at a
> time.  Problems occur only when you take the address of a member.

Many compilers (I believe gcc is among them) have distinct types for "packed"
and "unpacked" pointers; problems arise when storing a value of a "packed"
pointer type into a variable of a "normal" pointer type.  I'm actually
not quite sure why such stores are allowed without a cast, since the mere
act of storing a misaligned pointer into a "normal" pointer type may knock a
program off the rails even if the pointer is only used in ways which would
be legal for a "packed" pointer.  For example, while memcpy does not normally
impose any alignment restrictions on its operands, a compiler may
legitimately generate code for

    void intcpy(int *dest, int *src)
    {
      memcpy(dest, src, sizeof int);
    }

which fails if src or dest were not int-aligned.  If one has a structure
which contains a packed member of type "int" which satisfies the alignment
requirement of neither "int" nor the structure itself, there is no way in
which code could store the address of that member into a normal pointer
without the program going off the rails.  If the itself does not require
int alignment, it would be possible for the struct to have an unusual
alignment that makes the member alignment correct, but if the struct itself
requires integer alignment that can't happen.

[toc] | [prev] | [next] | [standalone]

#77146

supercat@casperkitty.com writes:
> On Tuesday, November 24, 2015 at 1:51:52 PM UTC-6, Keith Thompson wrote:
>> If gcc knows that it's accessing a member of a packed struct, it will do
>> whatever is necessary, perhaps generating code to access it a byte at a
>> time.  Problems occur only when you take the address of a member.
>
> Many compilers (I believe gcc is among them) have distinct types for "packed"
> and "unpacked" pointers; problems arise when storing a value of a "packed"
> pointer type into a variable of a "normal" pointer type.
[...]

As far as I know, gcc doesn't support "packed" pointers.

If it did, they would be relevant to this question on Stack Overflow:
    http://stackoverflow.com/q/8568432/827263
and nobody has commented on such a thing in the 4 years since I
posted it.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#77168

On Tuesday, November 24, 2015 at 11:52:37 PM UTC-6, Keith Thompson wrote:
> > Many compilers (I believe gcc is among them) have distinct types for "packed"
> > and "unpacked" pointers; problems arise when storing a value of a "packed"
> > pointer type into a variable of a "normal" pointer type.
> [...]
> 
> As far as I know, gcc doesn't support "packed" pointers.

Oh well.  The Keil compiler I use does support them; the code necessary to
access them is basically the same as accessing unaligned structure members,
so I don't see any reason gcc shouldn't be able to support them as well.

On the Keil compiler, compiling for Cortex-m0 (no unaligned-read support)

  typedef struct {int i1; char c2; int __packed i2;} S1;

  int use_s1(S1 *p1, int __packed *p2)
  {
   return p1->i1 + p1->i2 + *p2;
  }

yields

;;;168    int use_s1(S1 *p1, int __packed *p2)
000000  b570              PUSH     {r4-r6,lr}
;;;169    {
000002  4604              MOV      r4,r0
000004  460d              MOV      r5,r1
;;;170    	return p1->i1 + p1->i2 + *p2;
000006  1d40              ADDS     r0,r0,#5
000008  f7fffffe          BL       __aeabi_uread4
00000c  6821              LDR      r1,[r4,#0]
00000e  1844              ADDS     r4,r0,r1
000010  4628              MOV      r0,r5
000012  f7fffffe          BL       __aeabi_uread4
000016  1820              ADDS     r0,r4,r0
;;;171    }
000018  bd70              POP      {r4-r6,pc}

From a type-system perspective, recognizing a qualifier on a variable type
would seem like it should be simple enough.  I wonder why gcc wouldn't
support that?

[toc] | [prev] | [next] | [standalone]

#77130

On 24/11/15 20:51, Keith Thompson wrote:
> David Brown <david.brown@hesbynett.no> writes:
>> On 23/11/15 23:20, Jorgen Grahn wrote:
>>> On Mon, 2015-11-23, David Brown wrote:
>>> ...
>>>> I'd advise against __attribute__((packed)) (or equivalents on other
>>>> compilers) - either it makes no difference because there is no padding,
>>>> or you will end up with structures with fields that don't match their
>>>> alignment.  On some targets, that means run-time errors - on others, it
>>>> means seriously degraded code speed.  So it is rarely a good idea.
>>>
>>> I think I agree, but are there really cases where you say
>>> __attribute__((packed)) and then suffer a crash because of it?  The
>>> compiler shouldn't generate a struct, and code to access it which will
>>> crash if it's ever run.
>>
>> The attribute tells gcc that it should generate the struct layout
>> exactly as /you/ tell it - /not/ as the target system normally requires
>> with respect to alignment.
>
> I wouldn't quite put it that way.  Even with __attribute__((packed)),
> you're not specifying the exact layout.  You're just declaring the names
> and types of the members, and letting the compiler determine the layout.
> __attribute__((packed)) just causes the compiler to follow a different
> set of rules in doing so.

As often happens, your wording is more precise than mine.

>
>>                              On some systems, misaligned accesses leads
>> to a bus fault or similar hardware exception - a crash, in most cases.
>> On some other systems, you will not get a hardware exception but you
>> will access incorrect data (for example, a 16-bit access will simply
>> ignore the lowest bit of the address).  That means disaster of some sort.
>
> If gcc knows that it's accessing a member of a packed struct, it will do
> whatever is necessary, perhaps generating code to access it a byte at a
> time.  Problems occur only when you take the address of a member.

That is true with gcc, as far as I know - but it may not be the case on 
other compilers with similar "packing" extensions.  (I would need to 
check with the old documentation, but I believe I have used a compiler 
that supported packed structures but could generate code that crashed 
when accessing them directly.)  Also note that on some architectures, 
such as the ARM, unaligned accesses are allowed (and slow, but more 
efficient than multiple byte accesses) on some versions but not on 
others.  If code is supposed to be object-code portable between 
different versions, it is easy to get this wrong.

So accessing packed members is /probably/ safe, but 
implementation-dependent and something that needs to be checked.

>
>> If you use "packed", then /you/ take full responsibility for making sure
>> the accesses are correct.  Perhaps you want to create a pointer to that
>> struct that is not fully aligned (that's going to involve some
>> implementation-dependent pointer casts).  Perhaps you use memcpy() to
>> access the fields, or some other method that you know is safe.
>
> No, the compiler takes *some* responsibility for making sure accesses
> are correct.  It's just not able to do so in all cases.
>
> (That's not a general language rule, but it's how gcc behaves, and we're
> talking about a gcc-specific feature.)
>

A good many compilers support structure packing in some way - several 
using gcc attribute syntax.  So while gcc is the focus here, it's worth 
considering general cases too.

[toc] | [prev] | [next] | [standalone]

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

Back to top | Article view | comp.lang.c

csiph-web