Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #9735 > unrolled thread

Structure with unsigned chars and internal alignment

Back to article view | Back to comp.lang.c

Contents

  Structure with unsigned chars and internal alignment pozz <pozzugno@gmail.com> - 2011-08-31 00:05 +0200
    Re: Structure with unsigned chars and internal alignment "Scott Fluhrer" <sfluhrer@ix.netcom.com> - 2011-08-30 18:46 -0400
      Re: Structure with unsigned chars and internal alignment pozz <pozzugno@gmail.com> - 2011-08-31 08:53 +0200
        Re: Structure with unsigned chars and internal alignment James Kuyper <jameskuyper@verizon.net> - 2011-08-31 07:42 -0400
      Re: Structure with unsigned chars and internal alignment lawrence.jones@siemens.com - 2011-08-31 12:56 -0400
    Re: Structure with unsigned chars and internal alignment Eric Sosman <esosman@ieee-dot-org.invalid> - 2011-08-30 21:39 -0400
      Re: Structure with unsigned chars and internal alignment pozz <pozzugno@gmail.com> - 2011-08-31 09:07 +0200
        Re: Structure with unsigned chars and internal alignment Eric Sosman <esosman@ieee-dot-org.invalid> - 2011-08-31 21:26 -0400
          Re: Structure with unsigned chars and internal alignment pozz <pozzugno@gmail.com> - 2011-09-01 07:59 +0200
            Re: Structure with unsigned chars and internal alignment "BartC" <bc@freeuk.com> - 2011-09-01 10:34 +0100
              Re: Structure with unsigned chars and internal alignment pozz <pozzugno@gmail.com> - 2011-09-01 13:07 +0200
              Re: Structure with unsigned chars and internal alignment Keith Thompson <kst-u@mib.org> - 2011-09-01 08:34 -0700
            Re: Structure with unsigned chars and internal alignment Keith Thompson <kst-u@mib.org> - 2011-09-01 08:46 -0700
      Re: Structure with unsigned chars and internal alignment NicStevens <nic.stevens@gmail.com> - 2011-09-12 12:09 -0700
        Re: Structure with unsigned chars and internal alignment "J. J. Farrell" <jjf@bcs.org.uk> - 2011-09-12 20:30 +0100
        Re: Structure with unsigned chars and internal alignment James Kuyper <jameskuyper@verizon.net> - 2011-09-12 15:32 -0400
          Re: Structure with unsigned chars and internal alignment NicStevens <nic.stevens@gmail.com> - 2011-09-14 13:28 -0700
    Re: Structure with unsigned chars and internal alignment "BartC" <bc@freeuk.com> - 2011-08-31 11:50 +0100
    Re: Structure with unsigned chars and internal alignment David Resnick <lndresnick@gmail.com> - 2011-08-31 06:31 -0700

#9735 — Structure with unsigned chars and internal alignment

I have a struct composed by two arrays of unsigned char.

struct myStruct {
   unsigned char field1[2];
   unsigned char field2[30];
};

Is myStruct *always* 32 bytes long?  Is field2 *always* starting after 
two bytes the pointer to myStruct (i.e., no padding is allowed between 
field1 and field2)?

In this case, in my application I'd like to read the size of myStruct 
(between 3 and 32) from a file.  field1 will be always 2-bytes long, 
field2 will be the size of myStruct minus 2 bytes of field1 (in the case 
no padding is present between field1 and field2).

I could allocate field2 array dynamically, but I haven't malloc/free on 
my embedded platform.  So I decided to statically allocate the biggest 
size (32 bytes, 2 bytes for field1 and 30 bytes for field2).

If the real size of myStruct, read from the configuration file, is 
struct_size, how can I deduce the size of field2?  Actually I use the 
following formula:

   field2_size = struct_size - 2

but I don't like it.  It would be wrong if I'll decide to change the 
size of field1 member, and it would be wrong if padding is present 
between the two fields.  Maybe the following is better?

   field2_size = struct_size - offsetof(struct myStruct, field2)

Do you have other suggestions?

[toc] | [next] | [standalone]

#9738

"pozz" <pozzugno@gmail.com> wrote in message 
news:j3jmrn$cae$1@nnrp.ngi.it...
>I have a struct composed by two arrays of unsigned char.
>
> struct myStruct {
>   unsigned char field1[2];
>   unsigned char field2[30];
> };
>
> Is myStruct *always* 32 bytes long?  Is field2 *always* starting after two 
> bytes the pointer to myStruct (i.e., no padding is allowed between field1 
> and field2)?

Not necessarily; the Standard allows the compiler to add padding between 
field1 and field2, and also after field2.

On the other hand, I've never seen a compiler that would insert padding 
between field1 and field2 in this case.  It's unlikely that a compiler would 
insert padding after field2 (because 32 is such a round number); however, if 
you later change field2 to be (say) 31 bytes (because you've got a new 
record type that's a bit bigger), I wouldn't be shocked if there do exist 
compilers that would pad it out to (say) a multiple of 4 bytes.

>
> In this case, in my application I'd like to read the size of myStruct 
> (between 3 and 32) from a file.  field1 will be always 2-bytes long, 
> field2 will be the size of myStruct minus 2 bytes of field1 (in the case 
> no padding is present between field1 and field2).
>
> I could allocate field2 array dynamically, but I haven't malloc/free on my 
> embedded platform.  So I decided to statically allocate the biggest size 
> (32 bytes, 2 bytes for field1 and 30 bytes for field2).
>
> If the real size of myStruct, read from the configuration file, is 
> struct_size, how can I deduce the size of field2?  Actually I use the 
> following formula:
>
>   field2_size = struct_size - 2
>
> but I don't like it.  It would be wrong if I'll decide to change the size 
> of field1 member,

Modification of the data format would imply a change in the program.  On the 
other hand, if you expect that such a change is even slightly possible, I'd 
suggest that you replace the 2 with a meaningful constant name (say, 
SIZE_FIELD1)

> and it would be wrong if padding is present between the two fields.  Maybe 
> the following is better?
>
>   field2_size = struct_size - offsetof(struct myStruct, field2)
>
> Do you have other suggestions?

If you're looking for something that makes less assumption on the compiler, 
I suppose you could do:

typedef unsigned char myArray[ 32 ];
#define field2_offset 2 

[toc] | [prev] | [next] | [standalone]

#9746

Il 31/08/2011 00:46, Scott Fluhrer ha scritto:
>> I have a struct composed by two arrays of unsigned char.
>>
>> struct myStruct {
>>    unsigned char field1[2];
>>    unsigned char field2[30];
>> };
>>
>> Is myStruct *always* 32 bytes long?  Is field2 *always* starting after two
>> bytes the pointer to myStruct (i.e., no padding is allowed between field1
>> and field2)?
>
> Not necessarily; the Standard allows the compiler to add padding between
> field1 and field2, and also after field2.

I thought unsigned char was always aligned, so it wasn't any need to add 
padding between fields of unsigned char or array of unsigned chars.


> On the other hand, I've never seen a compiler that would insert padding
> between field1 and field2 in this case.  It's unlikely that a compiler would
> insert padding after field2 (because 32 is such a round number); however, if
> you later change field2 to be (say) 31 bytes (because you've got a new
> record type that's a bit bigger), I wouldn't be shocked if there do exist
> compilers that would pad it out to (say) a multiple of 4 bytes.

I think the padding after field2 is used to align array of myStruct.


>> and it would be wrong if padding is present between the two fields.  Maybe
>> the following is better?
>>
>>    field2_size = struct_size - offsetof(struct myStruct, field2)
>>
>> Do you have other suggestions?
>
> If you're looking for something that makes less assumption on the compiler,
> I suppose you could do:
>
> typedef unsigned char myArray[ 32 ];
> #define field2_offset 2

Maybe this is the best solution.

[toc] | [prev] | [next] | [standalone]

#9754

On 08/31/2011 02:53 AM, pozz wrote:
> Il 31/08/2011 00:46, Scott Fluhrer ha scritto:
>>> I have a struct composed by two arrays of unsigned char.
>>>
>>> struct myStruct {
>>>    unsigned char field1[2];
>>>    unsigned char field2[30];
>>> };
>>>
>>> Is myStruct *always* 32 bytes long?  Is field2 *always* starting after two
>>> bytes the pointer to myStruct (i.e., no padding is allowed between field1
>>> and field2)?
>>
>> Not necessarily; the Standard allows the compiler to add padding between
>> field1 and field2, and also after field2.
> 
> I thought unsigned char was always aligned, so it wasn't any need to add 
> padding between fields of unsigned char or array of unsigned chars.

Yes, unsigned char is always aligned. Yes, there's no need for such
padding. As a result, such padding is unlikely to occur. But it is
allowed by the standard, even though it is not needed.
-- 
James Kuyper

[toc] | [prev] | [next] | [standalone]

#9774

Scott Fluhrer <sfluhrer@ix.netcom.com> wrote:
> 
> "pozz" <pozzugno@gmail.com> wrote in message 
> news:j3jmrn$cae$1@nnrp.ngi.it...
> >I have a struct composed by two arrays of unsigned char.
> >
> > struct myStruct {
> >   unsigned char field1[2];
> >   unsigned char field2[30];
> > };
> >
> > Is myStruct *always* 32 bytes long?  Is field2 *always* starting after two 
> > bytes the pointer to myStruct (i.e., no padding is allowed between field1 
> > and field2)?
> 
> Not necessarily; the Standard allows the compiler to add padding between 
> field1 and field2, and also after field2.
> 
> On the other hand, I've never seen a compiler that would insert padding 
> between field1 and field2 in this case.

I have.  I don't remember exactly which one, but I distinctly remember
using a compiler that aligned arrays on at least a 4-byte boundary.  I
presume it would also add trailing padding in this case to make the
struct 36 bytes long.
-- 
Larry Jones

Buddy, if you think I'm even going to BE here, you're crazy! -- Calvin

[toc] | [prev] | [next] | [standalone]

#9743

On 8/30/2011 6:05 PM, pozz wrote:
> I have a struct composed by two arrays of unsigned char.
>
> struct myStruct {
> unsigned char field1[2];
> unsigned char field2[30];
> };
>
> Is myStruct *always* 32 bytes long? Is field2 *always* starting after
> two bytes the pointer to myStruct (i.e., no padding is allowed between
> field1 and field2)?

     No and no.  Many compilers will lay out the struct as you hope,
but none are under any obligation to do so.  If you want thirty-two
consecutive bytes, consider `unsigned char field_both[32]'.

> In this case, in my application I'd like to read the size of myStruct
> (between 3 and 32) from a file. field1 will be always 2-bytes long,
> field2 will be the size of myStruct minus 2 bytes of field1 (in the case
> no padding is present between field1 and field2).

     You probably do *not* want to read "the size of myStruct" from
the file; you want to read "the size of some blob of data."  The two
environments (on-disk form and in-memory form) are not necessarily
identical, even if they're strongly related by intention.

     I said "probably," because perhaps your file actually does hold
"the size of myStruct."  This could be the case if an actual `struct
myStruct' was written to the file originally, complete with whatever
padding it might have included.  If you never, never need to move the
data to another system (not even for post-mortem analysis), you can
probably get away with this.

> I could allocate field2 array dynamically, but I haven't malloc/free on
> my embedded platform. So I decided to statically allocate the biggest
> size (32 bytes, 2 bytes for field1 and 30 bytes for field2).

     It's not clear that the presence or absence of malloc() has
anything to do with the presence or absence of padding.

> If the real size of myStruct, read from the configuration file, is
> struct_size, how can I deduce the size of field2? Actually I use the
> following formula:
>
> field2_size = struct_size - 2

     I guess `struct_size' is something you compute from the two-byte
array?  Well, it matters not: The validity of the formula depends not
on C but on the program that wrote the file in the first place.  What
formula did *that* program use?

     (A literal reading of your question leads to the answer "The size
of field2 is thirty, always."  This may sound nit-picky, but I have a
hunch that if you think about it hard enough you'll arrive at the
question you *should* be asking instead.)

> but I don't like it. It would be wrong if I'll decide to change the size
> of field1 member, and it would be wrong if padding is present between
> the two fields. Maybe the following is better?
>
> field2_size = struct_size - offsetof(struct myStruct, field2)

     Same problem: It could be right or wrong (or "not even wrong"),
because what you need to care about is who wrote the file and how.

-- 
Eric Sosman
esosman@ieee-dot-org.invalid

[toc] | [prev] | [next] | [standalone]

#9747

Il 31/08/2011 03:39, Eric Sosman ha scritto:
>> In this case, in my application I'd like to read the size of myStruct
>> (between 3 and 32) from a file. field1 will be always 2-bytes long,
>> field2 will be the size of myStruct minus 2 bytes of field1 (in the case
>> no padding is present between field1 and field2).
>
> You probably do *not* want to read "the size of myStruct" from
> the file; you want to read "the size of some blob of data." The two
> environments (on-disk form and in-memory form) are not necessarily
> identical, even if they're strongly related by intention.

This is my case.


> I said "probably," because perhaps your file actually does hold
> "the size of myStruct." This could be the case if an actual `struct
> myStruct' was written to the file originally, complete with whatever
> padding it might have included. If you never, never need to move the
> data to another system (not even for post-mortem analysis), you can
> probably get away with this.

I understand your point and I expaling what I'm trying to do.

I have to read a file, created by another application on another 
platform, that is composed by blocks of data (what you named "blob of 
data").  The size of these blocks (between 3 and 32) is written in the 
same file at the beginning.
A single block is composed by 2 bytes and (block_size - 2) bytes.

Because I don't know the size of blocks I'll read and I can't malloc the 
right size at run-time, I was trying to define the maximum size of 
block, splitting it in the two fields:

   struct myStruct {
     unsigned char field1[2];
     unsigned char field2[30];
   };

I thought I could have read the block and copy it directly to myStruct. 
Anyway if padding could be present in myStruct, I can't use this approach.

Maybe the best approach is:

   #define FIELD1_OFFSET  0
   #define FIELD1_SIZE    2
   #define FIELD2_OFFSET  FIELD1_SIZE
   #define BLOCK_MAXSIZE  32
   void read_block(struct myStruct *s, size_t block_size) {
     unsigned char block[BLOCK_MAXSIZE];
     <read BLOCK_MAXSIZE bytes and copy it in block array>
     memcpy(s->field1, &block[FIELD1_OFFSET], FIELD1_SIZE);
     memcpy(s->field2, &block[FIELD2_OFFSET], block_size - FIELD1_SIZE);
   }

Here block_size is passed as an argument, because I don't know it in 
advance.

[toc] | [prev] | [next] | [standalone]

#9790

On 8/31/2011 3:07 AM, pozz wrote:
> Il 31/08/2011 03:39, Eric Sosman ha scritto:
>>> In this case, in my application I'd like to read the size of myStruct
>>> (between 3 and 32) from a file. field1 will be always 2-bytes long,
>>> field2 will be the size of myStruct minus 2 bytes of field1 (in the case
>>> no padding is present between field1 and field2).
>>
>> You probably do *not* want to read "the size of myStruct" from
>> the file; you want to read "the size of some blob of data." The two
>> environments (on-disk form and in-memory form) are not necessarily
>> identical, even if they're strongly related by intention.
>
> This is my case.
>
>
>> I said "probably," because perhaps your file actually does hold
>> "the size of myStruct." This could be the case if an actual `struct
>> myStruct' was written to the file originally, complete with whatever
>> padding it might have included. If you never, never need to move the
>> data to another system (not even for post-mortem analysis), you can
>> probably get away with this.
>
> I understand your point and I expaling what I'm trying to do.

     I'm not so sure you understand my points.  For myself, I'm
*sure* I don't understand "expaling."

> I have to read a file, created by another application on another
> platform, that is composed by blocks of data (what you named "blob of
> data"). The size of these blocks (between 3 and 32) is written in the
> same file at the beginning.
> A single block is composed by 2 bytes and (block_size - 2) bytes.

     You haven't said so, but I guess that the initial two bytes
somehow encode `block_size'.  Whether the remaining bytes are all
"payload" or may themselves include padding may be known to you, but
remains a mystery to the rest of us.

> Because I don't know the size of blocks I'll read and I can't malloc the
> right size at run-time, I was trying to define the maximum size of
> block, splitting it in the two fields:

     Again the hangup over the absence of malloc().  If this has
anything at all to do with the problem, it has to do with some aspect
of the problem that you have not yet revealed.  Based on what you've
said and shown, the existence or non-existence of malloc() has zilch
to do with the matter.

> struct myStruct {
> unsigned char field1[2];
> unsigned char field2[30];
> };

     ... but here comes the "splitting it in the two fields" part,
which you're going about (as several people have told you) in an
unreliable way.

> I thought I could have read the block and copy it directly to myStruct.
> Anyway if padding could be present in myStruct, I can't use this approach.
>
> Maybe the best approach is:
>
> #define FIELD1_OFFSET 0
> #define FIELD1_SIZE 2
> #define FIELD2_OFFSET FIELD1_SIZE
> #define BLOCK_MAXSIZE 32
> void read_block(struct myStruct *s, size_t block_size) {
> unsigned char block[BLOCK_MAXSIZE];
> <read BLOCK_MAXSIZE bytes and copy it in block array>
> memcpy(s->field1, &block[FIELD1_OFFSET], FIELD1_SIZE);
> memcpy(s->field2, &block[FIELD2_OFFSET], block_size - FIELD1_SIZE);
> }
>
> Here block_size is passed as an argument, because I don't know it in
> advance.

     That's odd.  Where do you learn `block_size', *before* reading
the first two bytes of your blob?  And if `block_size' turns out to
be less than thirty-two, how does your "read BLOCK_MAXSIZE bytes"
avoid running off the end and into whatever follows the blob?

     Observation: It is premature to seek the "best" way to do
something when you have not yet come up with "any" way.  That's
premature optimization personified.

-- 
Eric Sosman
esosman@ieee-dot-org.invalid

[toc] | [prev] | [next] | [standalone]

#9795

Il 01/09/2011 03:26, Eric Sosman ha scritto:
>>> I said "probably," because perhaps your file actually does hold
>>> "the size of myStruct." This could be the case if an actual `struct
>>> myStruct' was written to the file originally, complete with whatever
>>> padding it might have included. If you never, never need to move the
>>> data to another system (not even for post-mortem analysis), you can
>>> probably get away with this.
>>
>> I understand your point and I expaling what I'm trying to do.
>
> I'm not so sure you understand my points. For myself, I'm
> *sure* I don't understand "expaling."

I wanted to write "explaining"... :-)

I think I understood your point of view.  Data in the file can be of two 
types:
- blob of data of exactly N bytes sized
- myStruct previously written to the file by the same software
   on the same platform (so with the same layout of padding and data)
I'm in the first case.  I know the file contains a sequence of blobs of 
the same size N.  N is constant for a file, but may vary from file to 
file.  So the software should be ready to read blobs of 10 or 15 or 30 
or 32 bytes.
How the software can know the size of blobs in the file?  There are two 
bytes at the beginning of the file (just one time and *not* for each 
blob) coded as a 16-bits unsigned integer in Big Endian.


>> I have to read a file, created by another application on another
>> platform, that is composed by blocks of data (what you named "blob of
>> data"). The size of these blocks (between 3 and 32) is written in the
>> same file at the beginning.
>> A single block is composed by 2 bytes and (block_size - 2) bytes.
>
> You haven't said so, but I guess that the initial two bytes
> somehow encode `block_size'. Whether the remaining bytes are all
> "payload" or may themselves include padding may be known to you, but
> remains a mystery to the rest of us.

Just to better explain the content of the file:

- 2 bytes that code the size N of all the subsequent blocks
- N bytes for block 1
- N bytes for block 2
- ...up to the end of the file

A single N-bytes block is composed by two field:
- 2 bytes for field1
- N-2 bytes for field2

field1 and field2 are application data that aren't important now for our 
discussion.


>> Because I don't know the size of blocks I'll read and I can't malloc the
>> right size at run-time, I was trying to define the maximum size of
>> block, splitting it in the two fields:
>
> Again the hangup over the absence of malloc(). If this has
> anything at all to do with the problem, it has to do with some aspect
> of the problem that you have not yet revealed. Based on what you've
> said and shown, the existence or non-existence of malloc() has zilch
> to do with the matter.

On a "full" operating system I could write:

   open the file
   read the first two bytes and calculate N
   dynamically allocate an array of N bytes
   read block 1 from file and copy it into array
   other instructions based on array content
   read block 2 from file and copy it into array
   other instructions based on array content
   ...up to the end of file

Because I can't use malloc on my system, I was thinking to use an array 
calibrated to the maximum value for N, that is 32.

   static declaration of an array of 32 bytes
   open the file
   read the first two bytes and calculate N
   read block 1 from file and copy it into array
   other instructions based on array content
   read block 2 from file and copy it into array
   other instructions based on array content
   ...up to the end of file


>> struct myStruct {
>> unsigned char field1[2];
>> unsigned char field2[30];
>> };
>
> ... but here comes the "splitting it in the two fields" part,
> which you're going about (as several people have told you) in an
> unreliable way.

Logically the block is splitted in two fields, so it is useful to have a 
struct like this.  But I understand I can't read bytes from file and 
directly copy them into the structure.  I have to use a function that 
reads exactly N bytes and copy the two fields into the structure.


>> I thought I could have read the block and copy it directly to myStruct.
>> Anyway if padding could be present in myStruct, I can't use this
>> approach.
>>
>> Maybe the best approach is:
>>
>> #define FIELD1_OFFSET 0
>> #define FIELD1_SIZE 2
>> #define FIELD2_OFFSET FIELD1_SIZE
>> #define BLOCK_MAXSIZE 32
>> void read_block(struct myStruct *s, size_t block_size) {
>> unsigned char block[BLOCK_MAXSIZE];
>> <read BLOCK_MAXSIZE bytes and copy it in block array>
>> memcpy(s->field1, &block[FIELD1_OFFSET], FIELD1_SIZE);
>> memcpy(s->field2, &block[FIELD2_OFFSET], block_size - FIELD1_SIZE);
>> }
>>
>> Here block_size is passed as an argument, because I don't know it in
>> advance.
>
> That's odd. Where do you learn `block_size', *before* reading
> the first two bytes of your blob?

Yes, now I think it's clearer. See above.


> And if `block_size' turns out to
> be less than thirty-two, how does your "read BLOCK_MAXSIZE bytes"
> avoid running off the end and into whatever follows the blob?

Indeed I was wrong.  The code below should be correct.

   #define FIELD1_OFFSET 0
   #define FIELD1_SIZE 2
   #define FIELD2_OFFSET FIELD1_SIZE
   #define BLOCK_MAXSIZE 32
   void read_block(struct myStruct *s, size_t block_size) {
     unsigned char block[BLOCK_MAXSIZE];
     <read block_size bytes and copy it in block array>
     memcpy(s->field1, &block[FIELD1_OFFSET], FIELD1_SIZE);
     memcpy(s->field2, &block[FIELD2_OFFSET], block_size - FIELD1_SIZE);
   }

[toc] | [prev] | [next] | [standalone]

#9802

"pozz" <pozzugno@gmail.com> wrote in message
news:j3n70j$qrf$1@nnrp.ngi.it...

> Just to better explain the content of the file:
>
> - 2 bytes that code the size N of all the subsequent blocks
> - N bytes for block 1
> - N bytes for block 2
> - ...up to the end of the file
>
> A single N-bytes block is composed by two field:
> - 2 bytes for field1
> - N-2 bytes for field2
>
> field1 and field2 are application data that aren't important now for our
> discussion.

So why is it necessary to read each block of N bytes in one go?

Why not read 2 bytes into field1, then N-2 bytes into field2? Then problems
of padding and alignment will disappear.

(And field1 does sound very much sound like a 16-bit numeric value; if it 
is, it might as well be declared as one, making it easier to work with.)

-- 
Bartc 

[toc] | [prev] | [next] | [standalone]

#9805

Il 01/09/2011 11:34, BartC ha scritto:
> "pozz" <pozzugno@gmail.com> wrote in message
> news:j3n70j$qrf$1@nnrp.ngi.it...
>
>> Just to better explain the content of the file:
>>
>> - 2 bytes that code the size N of all the subsequent blocks
>> - N bytes for block 1
>> - N bytes for block 2
>> - ...up to the end of the file
>>
>> A single N-bytes block is composed by two field:
>> - 2 bytes for field1
>> - N-2 bytes for field2
>>
>> field1 and field2 are application data that aren't important now for our
>> discussion.
>
> So why is it necessary to read each block of N bytes in one go?
>
> Why not read 2 bytes into field1, then N-2 bytes into field2? Then problems
> of padding and alignment will disappear.

Yes, it's good too.  Anyway the oversized field2 array (NMAX and not 
only N) remains.


> (And field1 does sound very much sound like a 16-bit numeric value; if
> it is, it might as well be declared as one, making it easier to work with.)

Yes, it is a numeric 16-bit value.  I think I'll declare it as unsigned 
int in myStruct and convert it to and from 2-bytes array when accessing 
the file.

[toc] | [prev] | [next] | [standalone]

#9821

"BartC" <bc@freeuk.com> writes:
> "pozz" <pozzugno@gmail.com> wrote in message
> news:j3n70j$qrf$1@nnrp.ngi.it...
>
>> Just to better explain the content of the file:
>>
>> - 2 bytes that code the size N of all the subsequent blocks
>> - N bytes for block 1
>> - N bytes for block 2
>> - ...up to the end of the file
>>
>> A single N-bytes block is composed by two field:
>> - 2 bytes for field1
>> - N-2 bytes for field2
>>
>> field1 and field2 are application data that aren't important now for our
>> discussion.
>
> So why is it necessary to read each block of N bytes in one go?
>
> Why not read 2 bytes into field1, then N-2 bytes into field2? Then problems
> of padding and alignment will disappear.

Those problems also disappear if you just read each block into an N-byte
array.

> (And field1 does sound very much sound like a 16-bit numeric value; if it 
> is, it might as well be declared as one, making it easier to work with.)

It's stored in big-endian format, so you can't just read it directly
into a 16-bit integer object (unless you use htons() and/or ntohs()).

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Nokia
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#9822

pozz <pozzugno@gmail.com> writes:
[...]
> I think I understood your point of view.  Data in the file can be of two 
> types:
> - blob of data of exactly N bytes sized
> - myStruct previously written to the file by the same software
>    on the same platform (so with the same layout of padding and data)
> I'm in the first case.  I know the file contains a sequence of blobs of 
> the same size N.  N is constant for a file, but may vary from file to 
> file.  So the software should be ready to read blobs of 10 or 15 or 30 
> or 32 bytes.
> How the software can know the size of blobs in the file?  There are two 
> bytes at the beginning of the file (just one time and *not* for each 
> blob) coded as a 16-bits unsigned integer in Big Endian.

[...]

> Just to better explain the content of the file:
>
> - 2 bytes that code the size N of all the subsequent blocks
> - N bytes for block 1
> - N bytes for block 2
> - ...up to the end of the file
>
> A single N-bytes block is composed by two field:
> - 2 bytes for field1
> - N-2 bytes for field2
>
> field1 and field2 are application data that aren't important now for our 
> discussion.

Given that the maximum size is only 32 bytes, I probably wouldn't
use malloc() even if it were available; the overhead is likely to
exceed any savings from allocating, say, 24 bytes rather than 32.

Here's how I'd approach it:

Read 2 bytes from the file.
Compute N ((byte0 << 8) + byte1)
loop
    Read N bytes into a 32-byte buffer (unsigned char buf[32];)
    Bytes 0..1 contain field1
    Bytes 2..N-1 contain field2

(I've omitted any error checking.)

If you need the data in a friendlier format than a 32-byte buffer, you
can copy it out of buf into whatever is more convenient.

If you're *extremely* short on available memory, you can read
2 bytes directly into field1 and N-2 bytes directly into field2
(this saves you the 32-byte buffer).  Assuming field1 represents
a 16-bit integer, don't forget about byte ordering.

Reading directly into a struct is a bad idea if you care about
portability (including to future versions of the same compiler).
If reading directly into a struct turns out to be the best approach
anyway, I'd add some asserts to ensure that the sizes and offsets
of field1 and field2 are what they need to be, so the program will
fail to run rather than operate incorrectly if the layout changes.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Nokia
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#10162

On Aug 30, 6:39 pm, Eric Sosman <esos...@ieee-dot-org.invalid> wrote:
> On 8/30/2011 6:05 PM, pozz wrote:
>
> > I have a struct composed by two arrays of unsigned char.
>
> > struct myStruct {
> > unsigned char field1[2];
> > unsigned char field2[30];
> > };
>
> > Is myStruct *always* 32 bytes long? Is field2 *always* starting after
> > two bytes the pointer to myStruct (i.e., no padding is allowed between
> > field1 and field2)?
>
>      No and no.  Many compilers will lay out the struct as you hope,
> but none are under any obligation to do so.  If you want thirty-two
> consecutive bytes, consider `unsigned char field_both[32]'.
>
> > In this case, in my application I'd like to read the size of myStruct
> > (between 3 and 32) from a file. field1 will be always 2-bytes long,
> > field2 will be the size of myStruct minus 2 bytes of field1 (in the case
> > no padding is present between field1 and field2).
>
>      You probably do *not* want to read "the size of myStruct" from
> the file; you want to read "the size of some blob of data."  The two
> environments (on-disk form and in-memory form) are not necessarily
> identical, even if they're strongly related by intention.
>
>      I said "probably," because perhaps your file actually does hold
> "the size of myStruct."  This could be the case if an actual `struct
> myStruct' was written to the file originally, complete with whatever
> padding it might have included.  If you never, never need to move the
> data to another system (not even for post-mortem analysis), you can
> probably get away with this.
>
> > I could allocate field2 array dynamically, but I haven't malloc/free on
> > my embedded platform. So I decided to statically allocate the biggest
> > size (32 bytes, 2 bytes for field1 and 30 bytes for field2).
>
>      It's not clear that the presence or absence of malloc() has
> anything to do with the presence or absence of padding.
>
> > If the real size of myStruct, read from the configuration file, is
> > struct_size, how can I deduce the size of field2? Actually I use the
> > following formula:
>
> > field2_size = struct_size - 2
>
>      I guess `struct_size' is something you compute from the two-byte
> array?  Well, it matters not: The validity of the formula depends not
> on C but on the program that wrote the file in the first place.  What
> formula did *that* program use?
>
>      (A literal reading of your question leads to the answer "The size
> of field2 is thirty, always."  This may sound nit-picky, but I have a
> hunch that if you think about it hard enough you'll arrive at the
> question you *should* be asking instead.)
>
> > but I don't like it. It would be wrong if I'll decide to change the size
> > of field1 member, and it would be wrong if padding is present between
> > the two fields. Maybe the following is better?
>
> > field2_size = struct_size - offsetof(struct myStruct, field2)
>
>      Same problem: It could be right or wrong (or "not even wrong"),
> because what you need to care about is who wrote the file and how.
>
> --
> Eric Sosman
> esos...@ieee-dot-org.invalid

may I ask what compiler offsetof works on? I tried gcc and msc and
both complain about offsetof(struct foo, bar)

[toc] | [prev] | [next] | [standalone]

#10163

NicStevens wrote:
> 
> may I ask what compiler offsetof works on? I tried gcc and msc and
> both complain about offsetof(struct foo, bar)

Every compiler which implements Standard C. It's a macro in <stddef.h>.

[toc] | [prev] | [next] | [standalone]

#10164

On 09/12/2011 03:09 PM, NicStevens wrote:
> On Aug 30, 6:39 pm, Eric Sosman <esos...@ieee-dot-org.invalid> wrote:
>> On 8/30/2011 6:05 PM, pozz wrote:
>>
>>> I have a struct composed by two arrays of unsigned char.
>>
>>> struct myStruct {
>>> unsigned char field1[2];
>>> unsigned char field2[30];
>>> };
...
>>> field2_size = struct_size - offsetof(struct myStruct, field2)
>>
>>      Same problem: It could be right or wrong (or "not even wrong"),
>> because what you need to care about is who wrote the file and how.
...
> may I ask what compiler offsetof works on? I tried gcc and msc and
> both complain about offsetof(struct foo, bar)

The following code should compile without complaint on any hosted
implementation of C which fully conforms to any version of the C
standard. I can't test it on msc, but it does compile without any
complaints  with gcc, gcc -std=c89, and gcc -std=c99. I'd be surprised
at anything that dares to call itself a C compiler which would complain
about it:

	#include <stddef.h>
	struct foo { int bar; };
	int main(void)
	{
	    return offsetof(struct foo, bar);
	}

If you're running into problems with offsetof(struct foo, bar), it's
probably not due to the offsetof() expression itself, but it's
connection with the rest of your program. For instance, did you remember
to #include <stddef.h>?
If that's not the problem, can you provide an complete, short, program
that demonstrates the problem, along with the command line that you used
to compile it, and the message that was generated complaining about it?

[toc] | [prev] | [next] | [standalone]

#10248

On Sep 12, 12:32 pm, James Kuyper <jameskuy...@verizon.net> wrote:
> On 09/12/2011 03:09 PM, NicStevens wrote:
>
>
>
>
>
>
>
>
>
> > On Aug 30, 6:39 pm, Eric Sosman <esos...@ieee-dot-org.invalid> wrote:
> >> On 8/30/2011 6:05 PM, pozz wrote:
>
> >>> I have a struct composed by two arrays of unsigned char.
>
> >>> struct myStruct {
> >>> unsigned char field1[2];
> >>> unsigned char field2[30];
> >>> };
> ...
> >>> field2_size = struct_size - offsetof(struct myStruct, field2)
>
> >>      Same problem: It could be right or wrong (or "not even wrong"),
> >> because what you need to care about is who wrote the file and how.
> ...
> > may I ask what compiler offsetof works on? I tried gcc and msc and
> > both complain about offsetof(struct foo, bar)
>
> The following code should compile without complaint on any hosted
> implementation of C which fully conforms to any version of the C
> standard. I can't test it on msc, but it does compile without any
> complaints  with gcc, gcc -std=c89, and gcc -std=c99. I'd be surprised
> at anything that dares to call itself a C compiler which would complain
> about it:
>
>         #include <stddef.h>
>         struct foo { int bar; };
>         int main(void)
>         {
>             return offsetof(struct foo, bar);
>         }
>
> If you're running into problems with offsetof(struct foo, bar), it's
> probably not due to the offsetof() expression itself, but it's
> connection with the rest of your program. For instance, did you remember
> to #include <stddef.h>?
> If that's not the problem, can you provide an complete, short, program
> that demonstrates the problem, along with the command line that you used
> to compile it, and the message that was generated complaining about it?

Works on MSC

[toc] | [prev] | [next] | [standalone]

#9753

"pozz" <pozzugno@gmail.com> wrote in message
news:j3jmrn$cae$1@nnrp.ngi.it...

> but I don't like it.  It would be wrong if I'll decide to change the size
> of field1 member, and it would be wrong if padding is present between the
> two fields.  Maybe the following is better?
>
>   field2_size = struct_size - offsetof(struct myStruct, field2)
>
> Do you have other suggestions?

There are any number of solutions. The following is based on contiguous
byte-arrays rather than structs. It assumes field1 is meant to represent a
number, and that it's width is 2, and that field2 is a zero-terminated
string (although these were mainly for my testing, not shown here):

#include <stdio.h>
#include <stdlib.h>

#define MAXRECSIZE 32
typedef short int16;

#define GETFIELD1(x) (*(int16*)x)        /* Extract 1st 2 bytes as int */
#define GETFIELD2(x) ((char*)&x[2])      /* Char pointer to data */
#define SETFIELD1(x,v) (*(int16*)x)=v    /* Write 1st 2 bytes */

int main(void) {

int field2_size;
int struct_size;
char record[MAXRECSIZE+1]={0};           /* Structs are read into this */

struct_size = readstructsize();          /* Obtain overall struct size */
field2_size = struct_size-2;             /* Assume field1 is 2 bytes */

readrecord(&record,struct_size);         /* Read one record */
record[field2_size+2]=0;                 /* Terminate if data is a string */

printf("Field1 is <%d>\n",GETFIELD1(record));
printf("Data is <%s>\n",GETFIELD2(record));
}

-- 
bartc 

[toc] | [prev] | [next] | [standalone]

#9759

On Aug 30, 6:05 pm, pozz <pozzu...@gmail.com> wrote:
> I have a struct composed by two arrays of unsigned char.
>
> struct myStruct {
>    unsigned char field1[2];
>    unsigned char field2[30];
>
> };
>
> Is myStruct *always* 32 bytes long?  Is field2 *always* starting after
> two bytes the pointer to myStruct (i.e., no padding is allowed between
> field1 and field2)?
>

As others have said, probably but not guaranteed.

If you are willing to leave portability behind, many (most?) compilers
have a specific way to indicate that you want a struct to be "packed",
with out padding.  That might suit your needs here too.  An example of
that is gcc's  __attribute__ ((packed)).

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.c

csiph-web