Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
| From | glen herrmannsfeldt <gah@ugcs.caltech.edu> |
|---|---|
| Newsgroups | comp.lang.c |
| Subject | Re: Data alignment questin, structures |
| Date | 2013-01-20 07:53 +0000 |
| Organization | Aioe.org NNTP Server |
| Message-ID | <kdg7pg$b39$1@speranza.aioe.org> (permalink) |
| References | <kcs982$r3p$1@dont-email.me> <kcshq5$7e8$1@dont-email.me> <kdc426$f0i$1@dont-email.me> <kdftp6$i66$1@dont-email.me> |
mathog <mathog@caltech.edu> wrote:
>>>> Now, what happens when this struct is embedded in a binary file stored
>>>> in a character array "buffer[]" such that its position "i" is not
>>>> guaranteed to be aligned on the same boundary that
>>>> Mystract *instance2 = malloc(sizeof(Mystruct));
>>>> would have used, but will always be on a multiple of 2.
>>>> 1. Will this always work?
(snip)
> (Eric Sosman wrote:)
>>> Not "always," no. 6.3.2.3p7: "A pointer to an object type
>>> may be converted to a pointer to a different object type. If the
>>> resulting pointer is not correctly aligned for the referenced
>>> type, the behavior is undefined. [...]" There's no guarantee
>>> that the alignment of `buffer' suffices for `Mystruct'.
>> On further consideration, how does one know what "correctly aligned"
>> means for a given struct? Consider these two examples:
>> typedef struct {
>> uint16_t one;
>> uint16_t two;
>> uint16_t three;
>> uint16_t four;
>> } Mystruct4;
> Taking into account the various answers in this thread to the preceding
> question it would seem that there is still no safe (safe meaning here
> "code that will work on any platform") method to use a C struct
> to extract binary data directly from memory, since the compiler can
> throw in padding and alignment requirements whenever it feels like it.
While I believe that is true, it is still most likely that you can
use it on any appropriate boundary.
> Even if the data is reduced to byte representation, something like this
> (I know it isn't the same as Mystruct4 above):
> typedef struct {
> uint8_t one[2]; /* actually uint16_t */
> uint8_t two[4]; /* actually uint32_t */
> int8_t three[2]; /* etc. */
> int8_t four;
> } Mystruct4b;
> it seems not to be safe to pass a "Mystruct4b *" pointer to a function
> which references this data at an arbitrary location in memory. Instead
> the only safe method is to pass "char *" pointers and take the data
> apart with memcpy() at a very low level, moving it from memory to the
> structure, or vice versa.
Yes, that is always the most reliable way. Especially if you have
the possibility of different endianness. (Not to menion all the other
possible different representations.)
> What an odd situation. I would have thought with all of the data that
> does show up in C programming as "a series of bytes in a buffer arranged
> in some particular manner", that is, pretty much any data which is
> passed from machine to machine, C would have developed a method for
> simplifying this sort of code. Perhaps something along these lines
Well, pretty much it optimizes for passing data around within the
program.
For many years Fortran required, or at least it was believed to require,
that COMMON blocks be packed (no padding). On some machine, that was
just a little slow, on others it required run-time trap for the
misaligned access, copy the data, perform the operation, copy the data
back again, and then return. Much much slower.
Most RISC processors require data to be aligned, though some have
special instructions for access to misaligned data. (Faster than a byte
copy and performing the operation on the copy.)
> /* declare memory organization, no padding, no alignment requirement */
> typedef memstruct {
> uint16_t one;
> uint32_t two;
> int16_t three;
> uint8_t four;
> } Mystruct4c;
Doesn't help if the actual representation, such as endianness,
is different.
> Specifically so that one could pass a "Mystruct4c *" pointer to a
> function, like so:
> myfunction3a(Mystruct4c *ptr){
> ptr->two = 5;
> printf("value of three:%d\n",ptr->three);
> }
> and the compiler would do the "right thing", using memcpy or whatever,
> to hide all of the cruft that is platform specific, up to and including
> loading the magic N bytes of memory (as in the Alpha issue Glen
> mentioned), shifting, and masking and so forth, to handle data types
> smaller than is native for the CPU, without the programmer having to
> ever care about the details.
In the cases where the compiler knows in advance, it isn't
so bad. But then if you pass a pointer to misaligned data, then it is
referenced in a place that doesn't expect it to be misaligned.
Or, the compiler has to generate slow (maybe much slower) code
for all accesses.
> One might even dream that the compiler could be induced to do:
> Mystruct4c native,ncopy;
> Mystruct4b *foreign
> Mystruct4b acopy;
> /* field names/sizes must match for the following statement */
> alternate_representations {Mystruct4b, Mystruct4c}
Well, when C was new there were still plenty of 36 bit, 48 bit,
and 60 bit machines around, and probably others that I don't know
about. In addition, there is at least (still in the standard) allowance
for sign magnitude or ones complement representation, and finally
endianness. But yes, it could be done.
> char *buffer;
> /* fill buffer from a file or network */
> foreign = (Mystruct4b *)buffer[123];
> native = *foreign; /* field to field copy, NOT a memcpy*/
> /* change some data in native*/
> *foreign = native; /* field to field copy */
> ncopy = native; /* this is a memcpy */
> acopy = *foreign; /* as is this */
> *foreign = acopy; /* as is this */
> ncopy = acopy; /* field to field copy */
Well, there is XDR http://www.ietf.org/rfc/rfc4506.txt
which will do all the work for any reasonable, and also not
so reasonable representation.
> That is, when a memstruct pointer is dereferenced it does not mean quite
> the same thing as when a struct pointer is dereferenced. A few more
> rules for the compiler, a lot less work for the programmer.
Well, as in another thread, consider C a portable assembler.
It helps you, but you still have to do some of the work.
-- glen
Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Data alignment questin, structures mathog <mathog@caltech.edu> - 2013-01-12 10:15 -0800
Re: Data alignment questin, structures Ben Bacarisse <ben.usenet@bsb.me.uk> - 2013-01-12 18:54 +0000
Re: Data alignment questin, structures Jorgen Grahn <grahn+nntp@snipabacken.se> - 2013-01-15 20:14 +0000
Re: Data alignment questin, structures Eric Sosman <esosman@comcast-dot-net.invalid> - 2013-01-15 16:29 -0500
Re: Data alignment questin, structures Eric Sosman <esosman@comcast-dot-net.invalid> - 2013-01-12 15:41 -0500
Re: Data alignment questin, structures mathog <dmathog@gmail.com> - 2013-01-18 10:24 -0800
Re: Data alignment questin, structures James Kuyper <jameskuyper@verizon.net> - 2013-01-18 14:05 -0500
Re: Data alignment questin, structures Eric Sosman <esosman@comcast-dot-net.invalid> - 2013-01-18 14:46 -0500
Re: Data alignment questin, structures glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2013-01-18 21:11 +0000
Re: Data alignment questin, structures Shao Miller <sha0.miller@gmail.com> - 2013-01-18 17:14 -0500
Re: Data alignment questin, structures glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2013-01-18 23:09 +0000
Re: Data alignment questin, structures Keith Thompson <kst-u@mib.org> - 2013-01-18 16:23 -0800
Re: Data alignment questin, structures mathog <mathog@caltech.edu> - 2013-01-19 21:02 -0800
Re: Data alignment questin, structures Shao Miller <sha0.miller@gmail.com> - 2013-01-20 01:25 -0500
Re: Data alignment questin, structures glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2013-01-20 07:53 +0000
Re: Data alignment questin, structures Ben Bacarisse <ben.usenet@bsb.me.uk> - 2013-01-20 14:02 +0000
Re: Data alignment questin, structures Keith Thompson <kst-u@mib.org> - 2013-01-20 07:35 -0800
Re: Data alignment questin, structures 88888 Dihedral <dihedral88888@googlemail.com> - 2013-01-20 11:49 -0800
Re: Data alignment questin, structures James Kuyper <jameskuyper@verizon.net> - 2013-01-20 12:30 -0500
Re: Data alignment questin, structures mathog <mathog@caltech.edu> - 2013-01-20 10:26 -0800
Re: Data alignment questin, structures Shao Miller <sha0.miller@gmail.com> - 2013-01-20 15:11 -0500
Re: Data alignment questin, structures "BartC" <bc@freeuk.com> - 2013-01-20 20:34 +0000
Re: Data alignment questin, structures James Kuyper <jameskuyper@verizon.net> - 2013-01-20 15:53 -0500
Re: Data alignment questin, structures 88888 Dihedral <dihedral88888@googlemail.com> - 2013-01-20 13:16 -0800
Re: Data alignment questin, structures mathog <mathog@caltech.edu> - 2013-01-20 20:27 -0800
Re: Data alignment questin, structures Shao Miller <sha0.miller@gmail.com> - 2013-01-21 00:37 -0500
Re: Data alignment questin, structures Dr Nick <nospam-4@temporary-address.org.uk> - 2013-01-21 10:07 +0000
Re: Data alignment questin, structures mathog <mathog@caltech.edu> - 2013-01-21 10:21 -0800
Re: Data alignment questin, structures Tim Rentsch <txr@alumni.caltech.edu> - 2013-01-21 10:34 -0800
Re: Data alignment questin, structures Keith Thompson <kst-u@mib.org> - 2013-01-21 13:07 -0800
Re: Data alignment questin, structures "BartC" <bc@freeuk.com> - 2013-01-21 22:24 +0000
Re: Data alignment questin, structures mathog <dmathog@gmail.com> - 2013-01-22 10:18 -0800
Re: Data alignment questin, structures glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2013-01-22 20:57 +0000
Re: Data alignment questin, structures Keith Thompson <kst-u@mib.org> - 2013-01-22 14:51 -0800
Re: Data alignment questin, structures Ian Collins <ian-news@hotmail.com> - 2013-01-22 11:12 +1300
Re: Data alignment questin, structures glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2013-01-21 20:53 +0000
Re: Data alignment questin, structures Keith Thompson <kst-u@mib.org> - 2013-01-20 22:06 -0800
Re: Data alignment questin, structures Philip Lantz <prl@canterey.us> - 2013-01-22 00:27 -0800
Re: Data alignment questin, structures James Kuyper <jameskuyper@verizon.net> - 2013-01-22 06:44 -0500
Re: Data alignment questin, structures Bart van Ingen Schenau <bart@ingen.ddns.info.invalid> - 2013-01-21 10:19 +0000
Re: Data alignment questin, structures glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2013-01-20 22:19 +0000
Re: Data alignment questin, structures glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2013-01-20 22:14 +0000
Re: Data alignment questin, structures "BartC" <bc@freeuk.com> - 2013-01-20 22:48 +0000
Re: Data alignment questin, structures Les Cargill <lcargill99@comcast.com> - 2013-01-20 18:11 -0600
Re: Data alignment questin, structures Ian Collins <ian-news@hotmail.com> - 2013-01-21 14:07 +1300
Re: Data alignment questin, structures Shao Miller <sha0.miller@gmail.com> - 2013-01-12 17:45 -0500
Re: Data alignment questin, structures mathog <mathog@caltech.edu> - 2013-01-13 10:17 -0800
Re: Data alignment questin, structures Shao Miller <sha0.miller@gmail.com> - 2013-01-13 17:16 -0500
csiph-web