Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #30023 > unrolled thread

Data alignment questin, structures

Back to article view | Back to comp.lang.c

Contents

  Data alignment questin, structures mathog <mathog@caltech.edu> - 2013-01-12 10:15 -0800
    Re: Data alignment questin, structures Ben Bacarisse <ben.usenet@bsb.me.uk> - 2013-01-12 18:54 +0000
      Re: Data alignment questin, structures Jorgen Grahn <grahn+nntp@snipabacken.se> - 2013-01-15 20:14 +0000
        Re: Data alignment questin, structures Eric Sosman <esosman@comcast-dot-net.invalid> - 2013-01-15 16:29 -0500
    Re: Data alignment questin, structures Eric Sosman <esosman@comcast-dot-net.invalid> - 2013-01-12 15:41 -0500
      Re: Data alignment questin, structures mathog <dmathog@gmail.com> - 2013-01-18 10:24 -0800
        Re: Data alignment questin, structures James Kuyper <jameskuyper@verizon.net> - 2013-01-18 14:05 -0500
        Re: Data alignment questin, structures Eric Sosman <esosman@comcast-dot-net.invalid> - 2013-01-18 14:46 -0500
        Re: Data alignment questin, structures glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2013-01-18 21:11 +0000
        Re: Data alignment questin, structures Shao Miller <sha0.miller@gmail.com> - 2013-01-18 17:14 -0500
          Re: Data alignment questin, structures glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2013-01-18 23:09 +0000
        Re: Data alignment questin, structures Keith Thompson <kst-u@mib.org> - 2013-01-18 16:23 -0800
        Re: Data alignment questin, structures mathog <mathog@caltech.edu> - 2013-01-19 21:02 -0800
          Re: Data alignment questin, structures Shao Miller <sha0.miller@gmail.com> - 2013-01-20 01:25 -0500
          Re: Data alignment questin, structures glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2013-01-20 07:53 +0000
          Re: Data alignment questin, structures Ben Bacarisse <ben.usenet@bsb.me.uk> - 2013-01-20 14:02 +0000
          Re: Data alignment questin, structures Keith Thompson <kst-u@mib.org> - 2013-01-20 07:35 -0800
            Re: Data alignment questin, structures 88888 Dihedral <dihedral88888@googlemail.com> - 2013-01-20 11:49 -0800
          Re: Data alignment questin, structures James Kuyper <jameskuyper@verizon.net> - 2013-01-20 12:30 -0500
            Re: Data alignment questin, structures mathog <mathog@caltech.edu> - 2013-01-20 10:26 -0800
              Re: Data alignment questin, structures Shao Miller <sha0.miller@gmail.com> - 2013-01-20 15:11 -0500
              Re: Data alignment questin, structures "BartC" <bc@freeuk.com> - 2013-01-20 20:34 +0000
              Re: Data alignment questin, structures James Kuyper <jameskuyper@verizon.net> - 2013-01-20 15:53 -0500
                Re: Data alignment questin, structures 88888 Dihedral <dihedral88888@googlemail.com> - 2013-01-20 13:16 -0800
                Re: Data alignment questin, structures mathog <mathog@caltech.edu> - 2013-01-20 20:27 -0800
                  Re: Data alignment questin, structures Shao Miller <sha0.miller@gmail.com> - 2013-01-21 00:37 -0500
                    Re: Data alignment questin, structures Dr Nick <nospam-4@temporary-address.org.uk> - 2013-01-21 10:07 +0000
                      Re: Data alignment questin, structures mathog <mathog@caltech.edu> - 2013-01-21 10:21 -0800
                        Re: Data alignment questin, structures Tim Rentsch <txr@alumni.caltech.edu> - 2013-01-21 10:34 -0800
                        Re: Data alignment questin, structures Keith Thompson <kst-u@mib.org> - 2013-01-21 13:07 -0800
                          Re: Data alignment questin, structures "BartC" <bc@freeuk.com> - 2013-01-21 22:24 +0000
                          Re: Data alignment questin, structures mathog <dmathog@gmail.com> - 2013-01-22 10:18 -0800
                            Re: Data alignment questin, structures glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2013-01-22 20:57 +0000
                            Re: Data alignment questin, structures Keith Thompson <kst-u@mib.org> - 2013-01-22 14:51 -0800
                        Re: Data alignment questin, structures Ian Collins <ian-news@hotmail.com> - 2013-01-22 11:12 +1300
                      Re: Data alignment questin, structures glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2013-01-21 20:53 +0000
                  Re: Data alignment questin, structures Keith Thompson <kst-u@mib.org> - 2013-01-20 22:06 -0800
                    Re: Data alignment questin, structures Philip Lantz <prl@canterey.us> - 2013-01-22 00:27 -0800
                      Re: Data alignment questin, structures James Kuyper <jameskuyper@verizon.net> - 2013-01-22 06:44 -0500
                  Re: Data alignment questin, structures Bart van Ingen Schenau <bart@ingen.ddns.info.invalid> - 2013-01-21 10:19 +0000
              Re: Data alignment questin, structures glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2013-01-20 22:19 +0000
            Re: Data alignment questin, structures glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2013-01-20 22:14 +0000
            Re: Data alignment questin, structures "BartC" <bc@freeuk.com> - 2013-01-20 22:48 +0000
              Re: Data alignment questin, structures Les Cargill <lcargill99@comcast.com> - 2013-01-20 18:11 -0600
              Re: Data alignment questin, structures Ian Collins <ian-news@hotmail.com> - 2013-01-21 14:07 +1300
    Re: Data alignment questin, structures Shao Miller <sha0.miller@gmail.com> - 2013-01-12 17:45 -0500
      Re: Data alignment questin, structures mathog <mathog@caltech.edu> - 2013-01-13 10:17 -0800
        Re: Data alignment questin, structures Shao Miller <sha0.miller@gmail.com> - 2013-01-13 17:16 -0500

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

#30410

On 1/20/2013 13:26, mathog wrote:
> Exactly.  memstruct != struct.  A memstruct is a memory structure
> designed for accessing nonaligned data, the field organization is
> verbatim and cannot be distorted by the compiler to add padding or
> require alignments.  In a memstruct "ptr->two" means some equivalent of:
>
>     uint32_t tmp4;
>     memcpy(&tmp4,ptr + offsetof(Mystruct4c,two),4);
>
> especially including the cases where tmp4 is in a register, "memcpy" is
> some assembler operation(s) to copy from a random point in memory into
> that register, so that
>
>    &(ptr->two)
>
> would not be allowed.  Just like this:
>
>    register uint32_t tmp4;
>    printf("Pointer to:%p\n",&tmp4);
>
> is not valid. Presumably casting a memstruct to a struct and vice versa
> would also be forbidden.

You can't cast non-scalar types except for 'void', so it's always a 
constraint violation to try '(somestructtype) otherstruct'.

If you were talking about casting pointers to structs and your new 
memstructs, there is not a lot of discussion in the Standard that 
forbids casting pointers, so I'm not sure:

   1. Why you'd make that forbidden
   2. How it would be worded so as to be a special case (constraint 
violation or undefined behaviour?)

> That was the point of:
>
>  >  Mystruct4c  native;
>  >  Mystruct4b *foreign
>  >  /* field names/sizes must match for the following statement */
>  >  alternate_representations {Mystruct4b, Mystruct4c}
>
> which tells the compiler that it needs to know how to copy data in
> either direction between a specific memstruct and a specific struct.
> (And that would be illegal if the fields didn't match exactly.)
>

Ok, that would need some thought towards how that would be worded.  You 
couldn't use "compatible type," but you could reproduce the Standard's 
explanation of when two structure types in two different translation 
units are considered compatible types, but use it in your definition of 
whatever you'd call the relationship between a struct and a memstruct.

> I am not worried about endian issues or other floating point
> representations.  Both of these can be dealt with easily enough once the
> data has been mapped from nonaligned to aligned data structures.

Actually, I'm not sure what makes you say that.  If a member has a trap 
representation, the act of reading the stored value as the type of the 
member yields undefined behaviour.  You might be used to 
[non-C-Standard] 'htonl', but you can't really do that sort of thing if 
there's a trap representation, in general.

> As for
> bytes that are not 8 bits, does any modern hardware exist where that is
> not true?  Must we keep the requirement to run on archaic hardware
> forever going forward with newer versions of the language?
>

You can use limits.h's 'CHAR_BIT' to determine how many bits in a byte.

But again, why do you want to use this 'memstruct', which looks similar 
to a 'struct'?  Is it just because you like using the '.' and '->' 
operators?

-- 
- Shao Miller
--
"Thank you for the kind words; those are the kind of words I like to hear.

Cheerily,"  -- Richard Harter

[toc] | [prev] | [next] | [standalone]

#30413

"mathog" <mathog@caltech.edu> wrote in message 
news:kdhct5$bmb$1@dont-email.me...
> James Kuyper wrote:

> As for bytes that are not 8 bits, does any modern hardware exist where 
> that is not true?  Must we keep the requirement to run on archaic hardware 
> forever going forward with newer versions of the language?

There are still processors around that are not byte-addressable, so a C 
implementation might have a char width that is not 8 bits (but will likely 
be 16, 32 or 64). Sometimes an 8-bit char type might be emulated in such a 
case.

But these tend to be specialist processors and you might just ignore them if 
you know exactly what hardware you're going to run on.

-- 
Bartc 

[toc] | [prev] | [next] | [standalone]

#30414

On 01/20/2013 01:26 PM, mathog wrote:
...
> ...  As for 
> bytes that are not 8 bits, does any modern hardware exist where that is 
> not true?

I gather that there is VERY modern hardware in the embedded world where
CHAR_BIT == 16; in fact, it's a fairly common feature in that world.
It's not my field of expertise, so I can't give you specific examples,
but other people have brought them up on previous occasions when this
question was raised.
There have been people posting to this news group who expressed the
belief that all modern C programming is for embedded systems; I remember
having a very hard time convincing one of them that he was wrong.
-- 
James Kuyper

[toc] | [prev] | [next] | [standalone]

#30415

James Kuyper於 2013年1月21日星期一UTC+8上午4時53分10秒寫道：
> On 01/20/2013 01:26 PM, mathog wrote:
> 
> ...
> 
> > ...  As for 
> 
> > bytes that are not 8 bits, does any modern hardware exist where that is 
> 
> > not true?
> 
> 
> 
> I gather that there is VERY modern hardware in the embedded world where
> 
> CHAR_BIT == 16; in fact, it's a fairly common feature in that world.
> 
> It's not my field of expertise, so I can't give you specific examples,
> 
> but other people have brought them up on previous occasions when this
> 
> question was raised.
> 
> There have been people posting to this news group who expressed the
> 
> belief that all modern C programming is for embedded systems; I remember
> 
> having a very hard time convincing one of them that he was wrong.
> 
> -- 
> 
> James Kuyper

#define LASTBYTE(x) x&0xff
#define SECONDBYTE(x) (x&0xff00)>>8

LASTWORD and SECONDWORD are similar.

[toc] | [prev] | [next] | [standalone]

#30438

James Kuyper wrote:
> On 01/20/2013 01:26 PM, mathog wrote:
> ...
>> ...  As for
>> bytes that are not 8 bits, does any modern hardware exist where that is
>> not true?
>
> I gather that there is VERY modern hardware in the embedded world where
> CHAR_BIT == 16; in fact, it's a fairly common feature in that world.

On this hardware is memory access also 16 bits at a time? I wonder what 
it does if it reads a network packet with an odd number of 8 bit bytes, 
or worse yet, needs to send one?

A 16 bit char would make life fairly miserable because there would then 
be no way to avoid the endian issue.  The machine with that smallest 
data type might not consider its 16 bit char as either type of "endian", 
but incoming network data (for instance) must put its 8 bit bytes in one 
order or the other within it, so the description of a data structure in 
memory would have to be done conditionally for the 16 bit char system.

Admittedly pragma pack gets most of what I need from a memstruct in a 
struct.  Has that pragma made it into some version of the C standard, or 
is it still a compiler specific extension (albeit a common one)?

Thanks,

David Mathog

[toc] | [prev] | [next] | [standalone]

#30440

On 1/20/2013 23:27, mathog wrote:
>
> Admittedly pragma pack gets most of what I need from a memstruct in a
> struct.  Has that pragma made it into some version of the C standard, or
> is it still a compiler specific extension (albeit a common one)?
>

It's not in any Standard that I'm aware of, and it makes demands of 
implementers that could potentially make things difficult for their 
implementations.  Also, given that there are libraries to help with this 
sort of thing, I don't have any idea why it'd be a priority to standardize.

If network hardware really yields a 16-bit value, I'd expect that if 
8-bit values are desired for working with, there will be utility API 
available on the platform for working with these values, 8 bits at a 
time.  As an exercise, perhaps it'd be interesting to write a function 
which reads a file 16 bits at a time (maybe by reading two 8-bit bytes), 
but yields 8 bits at a time to the caller.

The high bits of a 16-bit byte can be empty, leaving the same range of 
values as an 8-bit byte.  The shift operators can be used to split a 
16-bit byte into 2 8-bit bytes.

-- 
- Shao Miller
--
"Thank you for the kind words; those are the kind of words I like to hear.

Cheerily,"  -- Richard Harter

[toc] | [prev] | [next] | [standalone]

#30444

Shao Miller <sha0.miller@gmail.com> writes:

> On 1/20/2013 23:27, mathog wrote:
>>
>> Admittedly pragma pack gets most of what I need from a memstruct in
> a
>> struct.  Has that pragma made it into some version of the C
> standard, or
>> is it still a compiler specific extension (albeit a common one)?
>>
>
> It's not in any Standard that I'm aware of, and it makes demands of
> implementers that could potentially make things difficult for their
> implementations.  Also, given that there are libraries to help with
> this sort of thing, I don't have any idea why it'd be a priority to
> standardize.

Structures are one of the areas where I think C has ended up in a rather
strange place.  They don't (as discussed here) work for mapping memory
to abstract data types and they don't work quite as well as they could
do for structuring program data.

I'd love to see something where I could tell the compiler "I've got
zillions of these things in my program, please order them in the most
memory efficient way" or "this is master configuration data that will be
used all over the place but there's only one copy, so optimise for rapid
access".  But in both cases I could lay out the structure when defined
in a way that made the code as self-documenting as possible.

Forcing my ordering, but allowing for indeterminate padding leaves us
somewhere between two stools.

[toc] | [prev] | [next] | [standalone]

#30468

Dr Nick wrote:

> Forcing my ordering, but allowing for indeterminate padding leaves us
> somewhere between two stools.

That's my view too.  The standards seem to think that one set of 
behavior for struct must fit all, and consequently it fits some common
applications not very well at all.  The idea of the "memstruct" I 
proposed was that rather than trying to make struct do everything, 
create a related construct specifically for handling data in memory, 
leaving the regular "struct" to be optimized in the other direction,
and the compiler knowing enough to to copy data back and forth between a 
memstruct and a struct, without the programmer having to know about the 
alignment, platform hardware, and the other nitty gritty details that 
are needed now.  Presumably such an extension could also handle issues 
with platforms having 16 or 32 bit chars.  (The programmer can with 
enough work, so the compiler should be able to do it too.)

On further consideration I think I would do memstruct slightly 
differently then I originally suggested.  Since it would hardly ever be 
used except in conjunction with its corresponding struct rather than 
duplicating all of the fields like this:

typedef memstruct {
   /*fields*/
} mymemstruct;

typedef struct {
   /* same fields as preceding */
} mystruct

just define the regular struct and then define both the memstruct and 
its relationship to the struct with a statement something like this:

typedef memstruct { mystruct; } mymemstruct;

(The syntax doesn't really matter, it just needs to tell the compiler to 
associate memstruct "mymemstruct" with struct "mystruct".)

Regards,

David Mathog

[toc] | [prev] | [next] | [standalone]

#30469

mathog <mathog@caltech.edu> writes:

> Dr Nick wrote:
>
>> Forcing my ordering, but allowing for indeterminate padding
>> leaves us somewhere between two stools.
>
> That's my view too.  The standards seem to think that one set
> of behavior for struct must fit all, and consequently it fits
> some common applications not very well at all.  The idea of the
> "memstruct" I proposed [...snip]

For the most part discussions in comp.lang.c discuss C as it is.
If you want to propose a new language feature, you may find that
comp.std.c is a better venue for that.

[toc] | [prev] | [next] | [standalone]

#30478

mathog <mathog@caltech.edu> writes:
> Dr Nick wrote:
>> Forcing my ordering, but allowing for indeterminate padding leaves us
>> somewhere between two stools.
>
> That's my view too.  The standards seem to think that one set of 
> behavior for struct must fit all, and consequently it fits some common
> applications not very well at all.  The idea of the "memstruct" I 
> proposed was that rather than trying to make struct do everything, 
> create a related construct specifically for handling data in memory, 
> leaving the regular "struct" to be optimized in the other direction,
> and the compiler knowing enough to to copy data back and forth between a 
> memstruct and a struct, without the programmer having to know about the 
> alignment, platform hardware, and the other nitty gritty details that 
> are needed now.
[snip]

How would your proposal handle this on hardware with strict alignment
requirements?

    #include <stdio.h>
    
    void func(int *ptr) {
        printf("*ptr = %d\n", *ptr);
    }
    
    int main(void) {
        memstruct foo {
            char c;
            int i;
        };
        memstruct foo obj = {'x', 42};
        func(&obj.i);
        return 0;
    }

The problem: the `i` member of `memstruct foo` may be misaligned, but
`func()` has no way of knowing that.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#30483

"Keith Thompson" <kst-u@mib.org> wrote in message
news:lntxqa5jpf.fsf@nuthaus.mib.org...
> mathog <mathog@caltech.edu> writes:

>> The idea of the "memstruct" I
>> proposed was that rather than trying to make struct do everything,
>> create a related construct specifically for handling data in memory,
>> leaving the regular "struct" to be optimized in the other direction,
>> and the compiler knowing enough to to copy data back and forth between a
>> memstruct and a struct, without the programmer having to know about the
>> alignment, platform hardware, and the other nitty gritty details that
>> are needed now.

> How would your proposal handle this on hardware with strict alignment
> requirements?

>    void func(int *ptr) {
>        printf("*ptr = %d\n", *ptr);
>    }
>
>    int main(void) {
>        memstruct foo {
>            char c;
>            int i;
>        };
>        memstruct foo obj = {'x', 42};
>        func(&obj.i);
>        return 0;
>    }
>
> The problem: the `i` member of `memstruct foo` may be misaligned, but
> `func()` has no way of knowing that.

It need not get that far. Pointers coulld have a compile-time attribute that 
indicates if they might be misaligned. Than passing such a pointer to func() 
would be an error (when it would cause the hardware to fail).

Dereferencing such pointers wouldn't be impossible either, but full
interaction with ordinary pointers would complicate the language. (I've seen
gcc do byte-at-a-byte accesses via pointers it thought might be misaligned.)

-- 
Bartc 

[toc] | [prev] | [next] | [standalone]

#30513

Keith Thompson wrote:
> mathog <mathog@caltech.edu> writes:
>
> How would your proposal handle this on hardware with strict alignment
> requirements?
>
>      #include <stdio.h>
>
>      void func(int *ptr) {
>          printf("*ptr = %d\n", *ptr);
>      }
>
>      int main(void) {
>          memstruct foo {
>              char c;
>              int i;
>          };
>          memstruct foo obj = {'x', 42};
>          func(&obj.i);
>          return 0;
>      }
>
> The problem: the `i` member of `memstruct foo` may be misaligned, but
> `func()` has no way of knowing that.

memstruct != struct

This is evolving as I go along here.  Originally I thought it best
to be able to dereference a memstruct member as "obj.i" in a read 
context to mean "copy (probably misaligned) bytes for i into a properly 
aligned location which can then be read normally".  Since this 
destination might be, and in this case probably would be, a register, 
the "&obj.i" would throw a compiler warning.  However, I'm now thinking 
it might be better simply to never allow a direct dereferencing of a 
memstruct or one of its members at all, except to the extent it defines 
a location in memory that is the location of another memstruct pointer 
(or, of course, when it maps data 1:1 to the corresponding normal 
struct).  In which case in the example above it would still throw a 
compiler warning, since it would be trying to pass a (pointer to 
memstruct of an int), not a (pointer to an int), and the two are not 
compatible.  The second method is a bit peculiar in that "obj.i" of a 
memstruct is something that has an address but does not have a value per 
se.  That is exactly what is going on now on some platforms, though,
where a misaligned uint32_t does have an address, but it is not one that 
may be dereferenced.  The advantage of putting this in the compiler is 
that the situation would be obvious on all platforms, whereas on an x86 
platform now this condition would not usually even
cause a hiccup.

That is
       #include <stdio.h>

      struct accumulator {
         uint32_t N;
         int32_t  sumN;
      }
      struct foo {
          uint8_t c;
          struct accumulator acc;
          uint32_t i;
      };

      memstruct foo         memfoo; /* defines memfoo memstruct type */
      memstruct accumulator memaccumulator;


       void func(int *ptr) {
           printf("*ptr = %d\n", *ptr);
       }

      void func2(memstruct memacumulator *acc){
         struct accumulator anacc;
         /* copy data to the corresponding struct.  This is
            one allowed form of dereferencing */
         anacc = *acc;
         printf("N:%d sumN:%d\n",anacc.N, anacc.sumN;
         /* this is OK, other allowed form of "dereferencing",
         actually just used to specify an address.  No type
         problem with memstruct * to void * */
         func3("Location of acc->N = ",&acc->N);
         /* this is NOT OK, memstruct * to int * */
         func(&acc->N);
      }
       void func3(char *string, void *ptr) {
           printf("%p\n", string, *ptr);
       }

       int main(void) {
           struct foo obj;
           memstruct memfoo memobj; /* this form _uses_ memfoo type    */
           /* fill memobj, equivalent to: memobj = {'x', {3,123}, 42};
           obj = memobj; /* copy data memstruct -> struct */
           func(&obj.i); /* no issue, this is a normal struct */
           /* pass address of acc within this memstruct as
              pointer to memstruct memaccumulator.  Types match,
              so no issue.    */
           func2(&memobj.acc);
           /* Next is not allowed, no dereferencing a memstruct
              field directly */
           printf("memobj i field:%d\n",memobj.i);
           return 0;
       }

I see now what some of you were getting at about byte order and floats. 
  Originally I expected the programmer to have routines to further clean 
up obj after it is loaded from memobj, since that is what the current 
code which triggered this thread does.  But it would make sense for the 
compiler to handle all of that that sort of thing
at the

    obj = memobj;

line.  That is, not just copy position and size in memory, but some 
types of value to value conversion as well.  That would need a couple of 
pragmas, one for the byte order of the data in memory 
(Big/End/Other/Native), and one for (IEEE float/not IEEE/Native), with 
the defaults being whatever is "Native" for each.  In some cases end 
user code would
still be needed, but only if "Native" was an odd machine.

Regards,

David Mathog

[toc] | [prev] | [next] | [standalone]

#30521

mathog <dmathog@gmail.com> wrote:

(snip)
>> The problem: the `i` member of `memstruct foo` may be misaligned, but
>> `func()` has no way of knowing that.
 
> memstruct != struct
 
> This is evolving as I go along here.  Originally I thought it best
> to be able to dereference a memstruct member as "obj.i" in a read 
> context to mean "copy (probably misaligned) bytes for i into a properly 
> aligned location which can then be read normally".  Since this 
> destination might be, and in this case probably would be, a register, 
> the "&obj.i" would throw a compiler warning.  However, I'm now thinking 
> it might be better simply to never allow a direct dereferencing of a 
> memstruct or one of its members at all, except to the extent it defines 
> a location in memory that is the location of another memstruct pointer 
> (or, of course, when it maps data 1:1 to the corresponding normal 
> struct).  

How about instead one could write a program that would read the
definition of a memstruct and write out C programs to convert
both directions between a memstruct (stored in an array 
of unsigned char) and a struct. It could at the same time do
endian conversion, floating point format conversion, word size
conversion (for machines with different or unusual word size).

Given that you have N machines with possibly different storage
methods (padding, word size, endianness, etc.) there are two
possibilities. One is to write the N**2 routines that will convert
between all pairs of machines. The other is to write 2N routines
that will convert between one intermediate form, from and to each
of the N formats.

If you do that, there is probably an easier to read and write
form than the memstruct. All you have to do is describe them,
generate the C routines, compile and link them, and you are 
done!

One implementation of the latter is Sun's XDR, designed along
with RPC, but can be used separately for other kinds of
data transfer problems.

-- glen

[toc] | [prev] | [next] | [standalone]

#30548

mathog <dmathog@gmail.com> writes:
> Keith Thompson wrote:
>> mathog <mathog@caltech.edu> writes:
>>
>> How would your proposal handle this on hardware with strict alignment
>> requirements?
>>
>>      #include <stdio.h>
>>
>>      void func(int *ptr) {
>>          printf("*ptr = %d\n", *ptr);
>>      }
>>
>>      int main(void) {
>>          memstruct foo {
>>              char c;
>>              int i;
>>          };
>>          memstruct foo obj = {'x', 42};
>>          func(&obj.i);
>>          return 0;
>>      }
>>
>> The problem: the `i` member of `memstruct foo` may be misaligned, but
>> `func()` has no way of knowing that.
>
> memstruct != struct

Perhaps the simplest way to describe it would be that members of a
memstruct act like bit fields.  You can access them directly, but you
can't take their addresses.

[snip]

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#30481

mathog wrote:
> Dr Nick wrote:
>
>> Forcing my ordering, but allowing for indeterminate padding leaves us
>> somewhere between two stools.
>
> That's my view too.  The standards seem to think that one set of
> behavior for struct must fit all, and consequently it fits some common
> applications not very well at all.

What else do you expect from a generic standard?  Compilers are free to 
add whatever extensions they like to support platform specific alignment 
requirements.  Embedded compilers invariably do.

-- 
Ian Collins

[toc] | [prev] | [next] | [standalone]

#30476

Dr Nick <nospam-4@temporary-address.org.uk> wrote:

(snip)

> Structures are one of the areas where I think C has ended up in a rather
> strange place.  They don't (as discussed here) work for mapping memory
> to abstract data types and they don't work quite as well as they could
> do for structuring program data.
 
> I'd love to see something where I could tell the compiler "I've got
> zillions of these things in my program, please order them in the most
> memory efficient way" or "this is master configuration data that will be
> used all over the place but there's only one copy, so optimise for rapid
> access".  

Fortran theoretically can do this by default for structures.
There is the SEQUENCE attribute when you don't want it to reorder
stucture members, though I don't know of any compilers that actually
do reorder them. Then there is the BIND(C) attribute when you want
it done exactly like C.

> But in both cases I could lay out the structure when defined
> in a way that made the code as self-documenting as possible.
 
> Forcing my ordering, but allowing for indeterminate padding leaves us
> somewhere between two stools.

Isn't that usual for any compromise?

-- glen

[toc] | [prev] | [next] | [standalone]

#30442

mathog <mathog@caltech.edu> writes:
> James Kuyper wrote:
>> On 01/20/2013 01:26 PM, mathog wrote:
>> ...
>>> ...  As for
>>> bytes that are not 8 bits, does any modern hardware exist where that is
>>> not true?
>>
>> I gather that there is VERY modern hardware in the embedded world where
>> CHAR_BIT == 16; in fact, it's a fairly common feature in that world.
>
> On this hardware is memory access also 16 bits at a time? I wonder what 
> it does if it reads a network packet with an odd number of 8 bit bytes, 
> or worse yet, needs to send one?

You're assuming it's able to read and write network packets using a
protocol that uses 8-bit bytes.

[...]

> Admittedly pragma pack gets most of what I need from a memstruct in a 
> struct.  Has that pragma made it into some version of the C standard, or 
> is it still a compiler specific extension (albeit a common one)?

No, neither pragma pack or anything that works like it is in any version
of the C standard.  There are some tricky corner cases that would have
to be resolved first.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#30499

Keith Thompson wrote:
> mathog writes:
> > James Kuyper wrote:
> >> mathog wrote:
> >>> ...  As for
> >>> bytes that are not 8 bits, does any modern hardware exist where that is
> >>> not true?
> >>
> >> I gather that there is VERY modern hardware in the embedded world where
> >> CHAR_BIT == 16; in fact, it's a fairly common feature in that world.
> >
> > On this hardware is memory access also 16 bits at a time? I wonder what 
> > it does if it reads a network packet with an odd number of 8 bit bytes, 
> > or worse yet, needs to send one?

This doesn't seem like a hard problem, given a definition of the 
interface to the network hardware and the implementation-defined 
capabilities of the C implementation. Without such definitions, we could 
only guess.

> You're assuming it's able to read and write network packets using a
> protocol that uses 8-bit bytes.

I would be *very* surprised if said 16-bit processors aren't used in 
platforms with Ethernet hardware that interoperate with platforms that 
send network packets with an odd number of octets. (In other words, I 
think it's a valid assumption.)

[toc] | [prev] | [next] | [standalone]

#30501

On 01/22/2013 03:27 AM, Philip Lantz wrote:
> Keith Thompson wrote:
...
>> You're assuming it's able to read and write network packets using a
>> protocol that uses 8-bit bytes.
> 
> I would be *very* surprised if said 16-bit processors aren't used in 
> platforms with Ethernet hardware that interoperate with platforms that 
> send network packets with an odd number of octets. (In other words, I 
> think it's a valid assumption.)

CHAR_BIT==16 would be extremely inconvenient in such environments. As I
understand it, that's why environments that violate such expectations
are the main places where you'll find implementations of C with
CHAR_BIT==16.
-- 
James Kuyper

[toc] | [prev] | [next] | [standalone]

#30446

On Sun, 20 Jan 2013 20:27:11 -0800, mathog wrote:

> James Kuyper wrote:
>> On 01/20/2013 01:26 PM, mathog wrote: ...
>>> ...  As for
>>> bytes that are not 8 bits, does any modern hardware exist where that
>>> is not true?
>>
>> I gather that there is VERY modern hardware in the embedded world where
>> CHAR_BIT == 16; in fact, it's a fairly common feature in that world.

See this answer on stackoverflow for some examples: 
http://stackoverflow.com/a/6971919/430719

> 
> On this hardware is memory access also 16 bits at a time?

Yes, in units of 16-bits or larger.

> I wonder what
> it does if it reads a network packet with an odd number of 8 bit bytes,
> or worse yet, needs to send one?

Those devices typically don't have to deal with that situation.
Many processors that have `CHAR_BIT == 16` (or event `CHAR_BIT == 32`) 
are DSPs (Digital Signal Processors) that don't communicate with octet-
oriented peripherals (such as networks).
And when they do, it turns out that all the relevant protocols were 
designed to be friendly for such processors (all multi-octet fields 
aligned to a multiple of their size, protocol headers a multiple of 4 
octets, etc.)

What I have also seen with such a processor is that each octet received 
from outside is put into a separate 16-bits char. It was up to the 
programmer to combine the values from multiple chars into a single value 
if the protocol actually contained a multi-octet value at that point.
No amount of memory mapping technique is going to help you there.

> 
> Admittedly pragma pack gets most of what I need from a memstruct in a
> struct.  Has that pragma made it into some version of the C standard, or
> is it still a compiler specific extension (albeit a common one)?

The #pragma directive is specified in the C standard, but in the most 
part it is defined to have implementation-defined effects.

> 
> Thanks,
> 
> David Mathog

Bart v Ingen Schenau

[toc] | [prev] | [next] | [standalone]

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

Back to top | Article view | comp.lang.c

csiph-web