Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.c > #4178 > unrolled thread
| Started by | Pushpa <pushpa@nospam.invalid> |
|---|---|
| First post | 2011-05-17 19:39 +0000 |
| Last post | 2011-05-19 09:12 -0600 |
| Articles | 20 on this page of 92 — 26 participants |
Back to article view | Back to comp.lang.c
Malloc Query Pushpa <pushpa@nospam.invalid> - 2011-05-17 19:39 +0000
Re: Malloc Query blp@cs.stanford.edu (Ben Pfaff) - 2011-05-17 12:55 -0700
Re: Malloc Query "BartC" <bc@freeuk.com> - 2011-05-18 12:05 +0100
Re: Malloc Query Pushpa <pushpa@nospam.invalid> - 2011-05-18 17:11 +0000
Re: Malloc Query "BartC" <bc@freeuk.com> - 2011-05-18 22:42 +0100
Re: Malloc Query Eric Sosman <esosman@ieee-dot-org.invalid> - 2011-05-18 21:10 -0400
Re: Malloc Query Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-05-18 19:34 +0100
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-18 17:10 -0400
Re: Malloc Query Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-05-19 00:26 +0100
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-18 21:25 -0400
Re: Malloc Query James Kuyper <jameskuyper@verizon.net> - 2011-05-18 21:38 -0400
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-19 14:11 -0400
Re: Malloc Query James Kuyper <jameskuyper@verizon.net> - 2011-05-19 22:55 -0400
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-20 12:14 -0400
Re: Malloc Query Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-05-19 03:22 +0100
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-19 14:20 -0400
Re: Malloc Query Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-05-19 23:51 +0100
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-19 20:01 -0400
Re: Malloc Query luser- -droog <mijoryx@yahoo.com> - 2011-05-22 02:49 -0700
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-22 10:46 -0400
Re: Malloc Query Shao Miller <sha0.miller@gmail.com> - 2011-05-22 12:37 -0500
Re: Malloc Query Shao Miller <sha0.miller@gmail.com> - 2011-05-22 13:02 -0500
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-22 14:12 -0400
Re: Malloc Query Shao Miller <sha0.miller@gmail.com> - 2011-05-23 03:06 -0500
Re: Malloc Query luser- -droog <mijoryx@yahoo.com> - 2011-05-23 00:42 -0700
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-23 11:10 -0400
Re: Malloc Query Keith Thompson <kst-u@mib.org> - 2011-05-23 09:06 -0700
Re: Malloc Query Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-05-22 23:18 +0100
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-22 19:18 -0400
Re: Malloc Query Ian Collins <ian-news@hotmail.com> - 2011-05-23 11:22 +1200
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-23 01:57 -0400
Re: Malloc Query Ian Collins <ian-news@hotmail.com> - 2011-05-23 19:24 +1200
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-23 14:20 -0400
Re: Malloc Query Ian Collins <ian-news@hotmail.com> - 2011-05-24 07:41 +1200
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-23 16:24 -0400
Re: Malloc Query Ian Collins <ian-news@hotmail.com> - 2011-05-24 09:20 +1200
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-23 18:05 -0400
Re: Malloc Query Ian Collins <ian-news@hotmail.com> - 2011-05-24 10:37 +1200
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-23 19:34 -0400
Re: Malloc Query "robertwessel2@yahoo.com" <robertwessel2@yahoo.com> - 2011-05-23 17:48 -0700
Re: Malloc Query Ian Collins <ian-news@hotmail.com> - 2011-05-24 12:56 +1200
Re: Malloc Query James Kuyper <jameskuyper@verizon.net> - 2011-05-24 07:19 -0400
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-24 11:32 -0400
Re: Malloc Query Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-05-23 01:04 +0100
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-23 02:36 -0400
Re: Malloc Query luser- -droog <mijoryx@yahoo.com> - 2011-05-23 00:39 -0700
Re: Malloc Query Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-05-23 14:44 +0100
Re: Malloc Query Seebs <usenet-nospam@seebs.net> - 2011-05-23 15:16 +0000
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-23 11:41 -0400
Re: Malloc Query David Thompson <dave.thompson2@verizon.net> - 2011-06-01 16:33 -0400
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-19 18:34 -0400
Re: Malloc Query Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-05-20 02:36 +0100
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-20 12:26 -0400
Re: Malloc Query Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-05-20 23:32 +0100
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-20 21:17 -0400
Re: Malloc Query Ian Collins <ian-news@hotmail.com> - 2011-05-21 14:30 +1200
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-21 11:13 -0400
Re: Malloc Query Ian Collins <ian-news@hotmail.com> - 2011-05-22 08:50 +1200
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-21 20:03 -0400
Re: Malloc Query Ian Collins <ian-news@hotmail.com> - 2011-05-22 13:23 +1200
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-22 10:01 -0400
Re: Malloc Query "io_x" <a@b.c.invalid> - 2011-05-26 08:12 +0200
Re: Malloc Query Ian Collins <ian-news@hotmail.com> - 2011-05-26 19:16 +1200
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-26 15:03 -0400
Re: Malloc Query Ian Collins <ian-news@hotmail.com> - 2011-05-27 09:16 +1200
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-26 17:55 -0400
Re: Malloc Query "io_x" <a@b.c.invalid> - 2011-05-27 05:47 +0200
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-26 15:02 -0400
Re: Malloc Query "BartC" <bc@freeuk.com> - 2011-05-18 22:31 +0100
Re: Malloc Query Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-05-18 23:37 +0100
Re: Malloc Query "BartC" <bc@freeuk.com> - 2011-05-19 01:50 +0100
Re: Malloc Query Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-05-19 02:50 +0100
Re: Malloc Query "BartC" <bc@freeuk.com> - 2011-05-19 12:14 +0100
Re: Malloc Query Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-05-19 15:10 +0100
Re: Malloc Query "BartC" <bc@freeuk.com> - 2011-05-19 15:48 +0100
Re: Malloc Query cri@tiac.net (Richard Harter) - 2011-05-18 19:31 +0000
Re: Malloc Query Ike Naar <ike@iceland.freeshell.org> - 2011-05-18 21:43 +0000
Re: Malloc Query "BartC" <bc@freeuk.com> - 2011-05-18 22:58 +0100
Re: Malloc Query Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2011-05-17 14:01 -0600
Re: Malloc Query Columbus sailed the ocean China Blue <chine.bleu@yahoo.com> - 2011-05-17 13:21 -0700
Re: Malloc Query BGB <cr88192@hotmail.com> - 2011-05-17 14:08 -0700
Re: Malloc Query Michael Press <rubrum@pacbell.net> - 2011-05-17 19:49 -0700
Re: Malloc Query Lauri Alanko <la@iki.fi> - 2011-05-18 00:50 +0000
Re: Malloc Query Eric Sosman <esosman@ieee-dot-org.invalid> - 2011-05-17 21:21 -0400
Re: Malloc Query "io_x" <a@b.c.invalid> - 2011-05-18 19:31 +0200
Re: Malloc Query Jorgen Grahn <grahn+nntp@snipabacken.se> - 2011-05-18 21:36 +0000
Re: Malloc Query Eric Sosman <esosman@ieee-dot-org.invalid> - 2011-05-18 21:26 -0400
Re: Malloc Query Tim Rentsch <txr@alumni.caltech.edu> - 2011-05-19 02:07 -0700
Re: Malloc Query Ian Collins <ian-news@hotmail.com> - 2011-05-19 21:13 +1200
Re: Malloc Query Nobody <nobody@nowhere.com> - 2011-05-18 07:08 +0100
Re: Malloc Query David Resnick <lndresnick@gmail.com> - 2011-05-19 07:51 -0700
Re: Malloc Query Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2011-05-19 09:12 -0600
Page 2 of 5 — ← Prev page 1 [2] 3 4 5 Next page →
| From | Shao Miller <sha0.miller@gmail.com> |
|---|---|
| Date | 2011-05-22 12:37 -0500 |
| Message-ID | <irbe4q$mi9$1@dont-email.me> |
| In reply to | #4511 |
On 5/22/2011 9:46 AM, Uncle Steve wrote:
> On Sun, May 22, 2011 at 02:49:51AM -0700, luser- -droog wrote:
>>
>> I've been using integer offsets instead of pointers a lot lately and
>> I think you might be better off using the array formula explicitly:
>>
>> #define arena_obj_addr(x, n) ((void *) (x->arena + n * x->obj_size))
>>
>> It's harder to get wrong.
>
> Still, you're at the mercy of transcription errors. Philosophically,
> the arena is an array even though the program using it does not care.
I agree, but would also like to point out that it _is_ an array object.
Your 'arena' member points to its first element, if I understand your
code correctly.
> But it would be better still to define the x->arena structure element
> with the proper type of object so accesses would be type constrained
> by the compiler and you would therefore get a warning if you tried to
> assign the address to the wrong pointer. (The following is modified
> more from the real version, and less from the quick hack I posted.)
Heheh. Yes but that departs further from 'malloc()', which returns a
'void *'. It seems that you are suggesting that "better," in this case,
would be an allocator for every type.
>
> So,
>
> #define ARENA_TYPE struct foo
>
>
> struct arena_s {
> size_t arena_size;
> int free_list;
> size_t free;
> ARENA_TYPE * arena;
> };
>
> And your access macro becomes:
>
> #define arena_obj_addr(x, n)&x->arena[n]
>
> since the pointer will be of the expected type.
>
> Even simpler. But if your ARENA_TYPE is an odd number of bytes, this
> can create alignment issues on some platforms.
If 'ARENA_TYPE' is a structure, oughtn't the implementation to pad
_every_ struct such that it can be the element type of an array? I
really think so, so I don't understand your concern, here. My
impression is that the 'sizeof' any type must be an integer multiple of
the alignment requirement, to allow for arrays... Even unions ought to
contain such trailing padding, when needed.
> On x86 there is merely
> a small penalty presumably because the processor has to access two
> 32-bit words to resolve a structure element that crosses alignment
> boundaries. I've read that some architectures trap with a bus error
> when you do this, so the easy solution is unworkable in practice
> unless you carefully pad out the structure to satisfy alignment
> requirements.
Who does "you" refer to in this last sentence? Isn't it the
implementation's job to worry about padding?
> All I've done here is move the alignment calculations
> into the arena allocator logic, looking something like this:
>
> op->obj_pad_size = (op->obj_size % pad_factor) ?
> (op->obj_size + pad_factor - (op->obj_size % pad_factor)) :
> op->obj_size;
>
> And so we go back to something much more like what is in the
> previously posted benchmarking code:
>
> struct arena_s {
> size_t obj_size; // Size in bytes of the object
> size_t obj_pad_size; // Oobject size inclusive of padding
> size_t arena_size; // Number of obj_pad_size slots in pool
> int free_list; // Pointer to head of free list
> int free; // Number of slots remaining
> unsigned char *
> arena; //
> };
>
If you are using 'malloc()' to allocate the arena, and 'malloc()'
returns a pointer to memory suitably aligned for _any_ object, then that
would seem to include an _array_ object, so again, I don't understand
the padding concerns.
If someone tries to use your allocator with a size of "3", they had
better not be lying and requiring an alignment of "4", of which "3" is
not an integer multiple.
> typedef struct arena_s arena;
>
>
> I do in fact have a function to calculate arena_obj_addr(), and it
> necessarily contains
>
> &p->arena[n * p->obj_pad_size]
Or, equivalently:
p->arena + n * p->obj_pad_size
>
> which is of course necessary since we can't calculate obj_pad_size
> before we know what the target object is and the run-time platform
> constraints. No real real reason it couldn't be "&p->arena + n *
> p->obj_pad_size", but I didn't do it that way.
(Without the '&', else your element size is 'sizeof (ARENA_TYPE *)')
> I guess the pointer
> conversions don't cause my brain to lock-up the way they used to.
Using an 'int' and macro/function? Seems reasonable to me, though
perhaps a 'ptrdiff_t' could be pleasant, too. :)
[toc] | [prev] | [next] | [standalone]
| From | Shao Miller <sha0.miller@gmail.com> |
|---|---|
| Date | 2011-05-22 13:02 -0500 |
| Message-ID | <irbfj7$ku$1@dont-email.me> |
| In reply to | #4512 |
On 5/22/2011 12:37 PM, Shao Miller wrote: > On 5/22/2011 9:46 AM, Uncle Steve wrote: >> constraints. No real real reason it couldn't be "&p->arena + n * >> p->obj_pad_size", but I didn't do it that way. > > (Without the '&', else your element size is 'sizeof (ARENA_TYPE *)') Sorry, I meant 'sizeof (unsigned char *)' there.
[toc] | [prev] | [next] | [standalone]
| From | Uncle Steve <stevet810@gmail.com> |
|---|---|
| Date | 2011-05-22 14:12 -0400 |
| Message-ID | <7d06059576d74e6-223c1@gmail.com> |
| In reply to | #4512 |
On Sun, May 22, 2011 at 12:37:43PM -0500, Shao Miller wrote:
> On 5/22/2011 9:46 AM, Uncle Steve wrote:
> >On Sun, May 22, 2011 at 02:49:51AM -0700, luser- -droog wrote:
> >>
> >>I've been using integer offsets instead of pointers a lot lately and
> >>I think you might be better off using the array formula explicitly:
> >>
> >> #define arena_obj_addr(x, n) ((void *) (x->arena + n * x->obj_size))
> >>
> >>It's harder to get wrong.
> >
> >Still, you're at the mercy of transcription errors. Philosophically,
> >the arena is an array even though the program using it does not care.
>
> I agree, but would also like to point out that it _is_ an array object.
> Your 'arena' member points to its first element, if I understand your
> code correctly.
True in spirit, but the 'array' aspect is an implementation detail.
> >But it would be better still to define the x->arena structure element
> >with the proper type of object so accesses would be type constrained
> >by the compiler and you would therefore get a warning if you tried to
> >assign the address to the wrong pointer. (The following is modified
> >more from the real version, and less from the quick hack I posted.)
>
> Heheh. Yes but that departs further from 'malloc()', which returns a
> 'void *'. It seems that you are suggesting that "better," in this case,
> would be an allocator for every type.
Can do with some macro magic, although there is room for debate as to
the value of forcing type constraints when your program logic already
carries with it the assumption that it will be working with objects
of a certain size as specified when you build the arena.
> >
> >So,
> >
> >#define ARENA_TYPE struct foo
> >
> >
> >struct arena_s {
> > size_t arena_size;
> > int free_list;
> > size_t free;
> > ARENA_TYPE * arena;
> >};
> >
> >And your access macro becomes:
> >
> >#define arena_obj_addr(x, n)&x->arena[n]
> >
> >since the pointer will be of the expected type.
> >
> >Even simpler. But if your ARENA_TYPE is an odd number of bytes, this
> >can create alignment issues on some platforms.
>
> If 'ARENA_TYPE' is a structure, oughtn't the implementation to pad
> _every_ struct such that it can be the element type of an array? I
> really think so, so I don't understand your concern, here. My
> impression is that the 'sizeof' any type must be an integer multiple of
> the alignment requirement, to allow for arrays... Even unions ought to
> contain such trailing padding, when needed.
Well, if C implementations pad out structures this way automatically
that would be a big win, but I have no way of knowing ahead of time
whether this is true for any given C compiler. And for platform
like x86 where unaligned access works with a small cycle penalty you
can choose to pack your nine-byte structures so they don't require an
extra three bytes of alignment. That saves twenty percent or
thereabouts for each array slot, which adds up bigtime if you have
thousands or millions of data elements.
> >On x86 there is merely
> >a small penalty presumably because the processor has to access two
> >32-bit words to resolve a structure element that crosses alignment
> >boundaries. I've read that some architectures trap with a bus error
> >when you do this, so the easy solution is unworkable in practice
> >unless you carefully pad out the structure to satisfy alignment
> >requirements.
>
> Who does "you" refer to in this last sentence? Isn't it the
> implementation's job to worry about padding?
'You' is whomever is hypothetically padding out structures to satisfy
alignment concerns.
> >All I've done here is move the alignment calculations
> >into the arena allocator logic, looking something like this:
> >
> > op->obj_pad_size = (op->obj_size % pad_factor) ?
> > (op->obj_size + pad_factor - (op->obj_size % pad_factor)) :
> > op->obj_size;
> >
> >And so we go back to something much more like what is in the
> >previously posted benchmarking code:
> >
> >struct arena_s {
> > size_t obj_size; // Size in bytes of the object
> > size_t obj_pad_size; // Oobject size inclusive of padding
> > size_t arena_size; // Number of obj_pad_size slots in pool
> > int free_list; // Pointer to head of free list
> > int free; // Number of slots remaining
> > unsigned char *
> > arena; //
> >};
> >
>
> If you are using 'malloc()' to allocate the arena, and 'malloc()'
> returns a pointer to memory suitably aligned for _any_ object, then that
> would seem to include an _array_ object, so again, I don't understand
> the padding concerns.
Well, if one is using a platform that requires four-byte alignment to
access ints (for instance), but you know the algorithm only needs 24
bits of data, you /can/ pack it in to three bytes and save 25% of the
nominal storage requirements of the array.
> If someone tries to use your allocator with a size of "3", they had
> better not be lying and requiring an alignment of "4", of which "3" is
> not an integer multiple.
Except on x86, which does not demand strict alignment. I'm leaving
alignment constraints up to the application programmer to define. If
someone wants to shoot themselves in the foot on a platform that
requires strict alignment, who am I to argue against it? Bloody toes
might be part of the application spec., and would therefore represent
an administrative policy beyond the scope of the allocator proper.
> >typedef struct arena_s arena;
> >w
> >
> >I do in fact have a function to calculate arena_obj_addr(), and it
> >necessarily contains
> >
> > &p->arena[n * p->obj_pad_size]
>
> Or, equivalently:
>
> p->arena + n * p->obj_pad_size
>
> >
> >which is of course necessary since we can't calculate obj_pad_size
> >before we know what the target object is and the run-time platform
> >constraints. No real real reason it couldn't be "&p->arena + n *
> >p->obj_pad_size", but I didn't do it that way.
>
> (Without the '&', else your element size is 'sizeof (ARENA_TYPE *)')
Yes, sorry. The ampersand is superfluous in that usage.
> >I guess the pointer
> >conversions don't cause my brain to lock-up the way they used to.
>
> Using an 'int' and macro/function? Seems reasonable to me, though
> perhaps a 'ptrdiff_t' could be pleasant, too. :)
I doubt it. If you're careful not to do pointer arithmetic where you
might get a negative offset, life is good. While I would not
disparage the utility of negative pointer offsets in certain
situations, it isn't completely necessary. "ptr - unsigned int"
should do the right thing. So far, I have not come across a situation
where I would need negative pointer offsets, making the use of
ptrdiff_t a moot issue, IMO. YMMV.
Regards,
Uncle Steve
--
Should a professional politician be charged with molestation if he
kisses babies while attending political rallies?
[toc] | [prev] | [next] | [standalone]
| From | Shao Miller <sha0.miller@gmail.com> |
|---|---|
| Date | 2011-05-23 03:06 -0500 |
| Message-ID | <ird11p$gra$1@dont-email.me> |
| In reply to | #4516 |
On 5/22/2011 1:12 PM, Uncle Steve wrote:
> On Sun, May 22, 2011 at 12:37:43PM -0500, Shao Miller wrote:
>> On 5/22/2011 9:46 AM, Uncle Steve wrote:
>>> On Sun, May 22, 2011 at 02:49:51AM -0700, luser- -droog wrote:
>>>>
>>>> I've been using integer offsets instead of pointers a lot lately and
>>>> I think you might be better off using the array formula explicitly:
>>>>
>>>> #define arena_obj_addr(x, n) ((void *) (x->arena + n * x->obj_size))
>>>>
>>>> It's harder to get wrong.
>>>
>>> Still, you're at the mercy of transcription errors. Philosophically,
>>> the arena is an array even though the program using it does not care.
>>
>> I agree, but would also like to point out that it _is_ an array object.
>> Your 'arena' member points to its first element, if I understand your
>> code correctly.
>
> True in spirit, but the 'array' aspect is an implementation detail.
Heheheh. I was suggesting that it's more than "philosophically" and
more than "in spirit." I think you are implying that a user of your
allocator needn't be aware of the fact that the arena is an array, or
that there is an "arena" at all.
>> If 'ARENA_TYPE' is a structure, oughtn't the implementation to pad
>> _every_ struct such that it can be the element type of an array? I
>> really think so, so I don't understand your concern, here. My
>> impression is that the 'sizeof' any type must be an integer multiple of
>> the alignment requirement, to allow for arrays... Even unions ought to
>> contain such trailing padding, when needed.
>
> Well, if C implementations pad out structures this way automatically
> that would be a big win, but I have no way of knowing ahead of time
> whether this is true for any given C compiler.
I fail to understand why any C implementation would not pad structures
in this way. Suppose you have an implementation that doesn't. Suppose
an 'int' has a size and alignment of 4 bytes. Suppose you have:
struct foo {
int i; /* 0 through 3 */
char ca[3]; /* 4 through 6 */
/* No padding */
};
struct foo bar[2];
It would seem that the C implementation would have the 'bar' array's
second element's 'i' member starting at offset 7, counting from 0 and
relative to the first byte of the array. That would be misaligned.
Adding one byte of padding would do the trick.
Now I can understand if an implementation has a documented extension
that allows you to override such sensible padding and pack a structure
or specify your own padding, but use of such an extension would, in my
opinion, put the onus on the user to know what they're doing...
If they really want the structure above to be 7 bytes and you allocate
the second slot in your arena for such a structure and they try to
access the 'i' member at offset 7 from the beginning of your arena and
their CPU is angry about it... Too bad for the user, in my opinion.
They need to re-think using the "packing" extension for their scenario.
I don't believe that your allocator should need to be concerned about
padding, but I could be mistaken.
> And for platform
> like x86 where unaligned access works with a small cycle penalty you
> can choose to pack your nine-byte structures so they don't require an
> extra three bytes of alignment. That saves twenty percent or
> thereabouts for each array slot, which adds up bigtime if you have
> thousands or millions of data elements.
Ok. So if it works with a small cycle penalty, then the CPU won't be
angry about it and 'i' will be read at offset 7 (continuing my example
above) and your allocator still works without having been worried about
padding. When they chose to pack, they opted for the small cycle
penalty. Should your allocator be burdened to guess at and provide
additional padding in order to remove the small cycle penalty?
>
>>> On x86 there is merely
>>> a small penalty presumably because the processor has to access two
>>> 32-bit words to resolve a structure element that crosses alignment
>>> boundaries. I've read that some architectures trap with a bus error
>>> when you do this, so the easy solution is unworkable in practice
>>> unless you carefully pad out the structure to satisfy alignment
>>> requirements.
>>
>> Who does "you" refer to in this last sentence? Isn't it the
>> implementation's job to worry about padding?
>
> 'You' is whomever is hypothetically padding out structures to satisfy
> alignment concerns.
>
Ok.
>>> All I've done here is move the alignment calculations
>>> into the arena allocator logic, looking something like this:
>>>
>>> op->obj_pad_size = (op->obj_size % pad_factor) ?
>>> (op->obj_size + pad_factor - (op->obj_size % pad_factor)) :
>>> op->obj_size;
>>>
>>> And so we go back to something much more like what is in the
>>> previously posted benchmarking code:
>>>
>>> struct arena_s {
>>> size_t obj_size; // Size in bytes of the object
>>> size_t obj_pad_size; // Oobject size inclusive of padding
>>> size_t arena_size; // Number of obj_pad_size slots in pool
>>> int free_list; // Pointer to head of free list
>>> int free; // Number of slots remaining
>>> unsigned char *
>>> arena; //
>>> };
>>>
>>
>> If you are using 'malloc()' to allocate the arena, and 'malloc()'
>> returns a pointer to memory suitably aligned for _any_ object, then that
>> would seem to include an _array_ object, so again, I don't understand
>> the padding concerns.
>
> Well, if one is using a platform that requires four-byte alignment to
> access ints (for instance), but you know the algorithm only needs 24
> bits of data, you /can/ pack it in to three bytes and save 25% of the
> nominal storage requirements of the array.
>
If your allocator uses padding in the arena, where are the savings?
>> If someone tries to use your allocator with a size of "3", they had
>> better not be lying and requiring an alignment of "4", of which "3" is
>> not an integer multiple.
>
> Except on x86, which does not demand strict alignment. I'm leaving
> alignment constraints up to the application programmer to define. If
> someone wants to shoot themselves in the foot on a platform that
> requires strict alignment, who am I to argue against it? Bloody toes
> might be part of the application spec., and would therefore represent
> an administrative policy beyond the scope of the allocator proper.
>
Agreed. So why attempt to pad on their behalf within the allocator?
>> Using an 'int' and macro/function? Seems reasonable to me, though
>> perhaps a 'ptrdiff_t' could be pleasant, too. :)
>
> I doubt it. If you're careful not to do pointer arithmetic where you
> might get a negative offset, life is good. While I would not
> disparage the utility of negative pointer offsets in certain
> situations, it isn't completely necessary. "ptr - unsigned int"
> should do the right thing. So far, I have not come across a situation
> where I would need negative pointer offsets, making the use of
> ptrdiff_t a moot issue, IMO. YMMV.
I didn't mean to suggest any consideration for negative 'ptrdiff_t'
values. It's a signed integer type just as 'int' is a signed integer
type. But because a pointer to the top of your arena minus a pointer to
the bottom of it yields a 'ptrdiff_t', perhaps it could be enjoyable to
use 'ptrdiff_t' for the addition, just for the sake of symmetry. :)
[toc] | [prev] | [next] | [standalone]
| From | luser- -droog <mijoryx@yahoo.com> |
|---|---|
| Date | 2011-05-23 00:42 -0700 |
| Message-ID | <fd99cd6e-4a6d-44f3-b495-a1a19f88662c@p13g2000yqh.googlegroups.com> |
| In reply to | #4551 |
On May 23, 3:06 am, Shao Miller <sha0.mil...@gmail.com> wrote: > On 5/22/2011 1:12 PM, Uncle Steve wrote: > > > > > On Sun, May 22, 2011 at 12:37:43PM -0500, Shao Miller wrote: > > >> Using an 'int' and macro/function? Seems reasonable to me, though > >> perhaps a 'ptrdiff_t' could be pleasant, too. :) > > > I doubt it. If you're careful not to do pointer arithmetic where you > > might get a negative offset, life is good. While I would not > > disparage the utility of negative pointer offsets in certain > > situations, it isn't completely necessary. "ptr - unsigned int" > > should do the right thing. So far, I have not come across a situation > > where I would need negative pointer offsets, making the use of > > ptrdiff_t a moot issue, IMO. YMMV. > > I didn't mean to suggest any consideration for negative 'ptrdiff_t' > values. It's a signed integer type just as 'int' is a signed integer > type. But because a pointer to the top of your arena minus a pointer to > the bottom of it yields a 'ptrdiff_t', perhaps it could be enjoyable to > use 'ptrdiff_t' for the addition, just for the sake of symmetry. :) If ever there were a place to use one, it'd be here. :)
[toc] | [prev] | [next] | [standalone]
| From | Uncle Steve <stevet810@gmail.com> |
|---|---|
| Date | 2011-05-23 11:10 -0400 |
| Message-ID | <ceaa2bf4fcc86ac-50e1d@gmail.com> |
| In reply to | #4551 |
On Mon, May 23, 2011 at 03:06:29AM -0500, Shao Miller wrote:
> On 5/22/2011 1:12 PM, Uncle Steve wrote:
> >On Sun, May 22, 2011 at 12:37:43PM -0500, Shao Miller wrote:
> >>On 5/22/2011 9:46 AM, Uncle Steve wrote:
> >>>On Sun, May 22, 2011 at 02:49:51AM -0700, luser- -droog wrote:
> >>>>
> >>>>I've been using integer offsets instead of pointers a lot lately and
> >>>>I think you might be better off using the array formula explicitly:
> >>>>
> >>>> #define arena_obj_addr(x, n) ((void *) (x->arena + n * x->obj_size))
> >>>>
> >>>>It's harder to get wrong.
> >>>
> >>>Still, you're at the mercy of transcription errors. Philosophically,
> >>>the arena is an array even though the program using it does not care.
> >>
> >>I agree, but would also like to point out that it _is_ an array object.
> >> Your 'arena' member points to its first element, if I understand your
> >>code correctly.
> >
> >True in spirit, but the 'array' aspect is an implementation detail.
>
> Heheheh. I was suggesting that it's more than "philosophically" and
> more than "in spirit." I think you are implying that a user of your
> allocator needn't be aware of the fact that the arena is an array, or
> that there is an "arena" at all.
More like a willful suspension of disbelief. There are a few gotchas
with this arrangement, but the organization of the data behind the API
is mostly immaterial to any algorithm using it. realloc() can move
the array, so you have to account for that when using a pointer
calculated from arena_obj_pointer() macro/function, but that's about
it, modulo corner cases such as the situation that comes about with
co-processors or with DMA from external devices.
> >>If 'ARENA_TYPE' is a structure, oughtn't the implementation to pad
> >>_every_ struct such that it can be the element type of an array? I
> >>really think so, so I don't understand your concern, here. My
> >>impression is that the 'sizeof' any type must be an integer multiple of
> >>the alignment requirement, to allow for arrays... Even unions ought to
> >>contain such trailing padding, when needed.
> >
> >Well, if C implementations pad out structures this way automatically
> >that would be a big win, but I have no way of knowing ahead of time
> >whether this is true for any given C compiler.
>
> I fail to understand why any C implementation would not pad structures
> in this way. Suppose you have an implementation that doesn't. Suppose
> an 'int' has a size and alignment of 4 bytes. Suppose you have:
>
> struct foo {
> int i; /* 0 through 3 */
> char ca[3]; /* 4 through 6 */
> /* No padding */
> };
>
> struct foo bar[2];
>
> It would seem that the C implementation would have the 'bar' array's
> second element's 'i' member starting at offset 7, counting from 0 and
> relative to the first byte of the array. That would be misaligned.
> Adding one byte of padding would do the trick.
The question is whether the padding is added at the structure
definition or if it is accounted for when the bar[2] array is used.
No idea what the standard says about that. As gcc supplies a 'packed'
attribute to use, I can assume that it is safe in this regard.
> Now I can understand if an implementation has a documented extension
> that allows you to override such sensible padding and pack a structure
> or specify your own padding, but use of such an extension would, in my
> opinion, put the onus on the user to know what they're doing...
__attribute__(( packed )) applied to struct foo {...} ought to
misalign the structure elements properly.
> If they really want the structure above to be 7 bytes and you allocate
> the second slot in your arena for such a structure and they try to
> access the 'i' member at offset 7 from the beginning of your arena and
> their CPU is angry about it... Too bad for the user, in my opinion.
> They need to re-think using the "packing" extension for their scenario.
Yes, which translates to reading and understanding the API
documentation. I realize that is a nearly unacceptable burden to put
on the user, but I see no reasonable substitute for informed literacy.
> I don't believe that your allocator should need to be concerned about
> padding, but I could be mistaken.
I'm taking the easy way out and leaving that decision to the
programmer. A sensible default aligns to machine-word boundaries, but
allows it to be overridden.
> >And for platform
> >like x86 where unaligned access works with a small cycle penalty you
> >can choose to pack your nine-byte structures so they don't require an
> >extra three bytes of alignment. That saves twenty percent or
> >thereabouts for each array slot, which adds up bigtime if you have
> >thousands or millions of data elements.
>
> Ok. So if it works with a small cycle penalty, then the CPU won't be
> angry about it and 'i' will be read at offset 7 (continuing my example
> above) and your allocator still works without having been worried about
> padding. When they chose to pack, they opted for the small cycle
> penalty. Should your allocator be burdened to guess at and provide
> additional padding in order to remove the small cycle penalty?
There are two choices: yes and no. The allocator will do either one,
so the question becomes: which would you choose? MIPS users may have
a different preference from those who are using x86, but then they
have to know what they're doing. The Linux kernel apparently traps
unaligned accesses and emulates them:
http://bit.ly/mwGxRV
PowerPC has issues with unaligned floating-point access as well
according to IBM Developer Works documentation:
http://ibm.co/6OdYj (I guess we know who runs bit.ly?)
That article goes into some detail on the issue. One thing that stood
out is that atomic operations fail on unaligned access for PowerPC (If
I am reading it correctly), which could cause huge problems if that
were brought about by careless use of the allocator.
I think it's safe to say that there is no substitute for reading and
understanding the documentation for your system.
> >>>On x86 there is merely
> >>>a small penalty presumably because the processor has to access two
> >>>32-bit words to resolve a structure element that crosses alignment
> >>>boundaries. I've read that some architectures trap with a bus error
> >>>when you do this, so the easy solution is unworkable in practice
> >>>unless you carefully pad out the structure to satisfy alignment
> >>>requirements.
> >>
> >>Who does "you" refer to in this last sentence? Isn't it the
> >>implementation's job to worry about padding?
> >
> >'You' is whomever is hypothetically padding out structures to satisfy
> >alignment concerns.
>
> Ok.
>
> >>>All I've done here is move the alignment calculations
> >>>into the arena allocator logic, looking something like this:
> >>>
> >>> op->obj_pad_size = (op->obj_size % pad_factor) ?
> >>> (op->obj_size + pad_factor - (op->obj_size % pad_factor)) :
> >>> op->obj_size;
> >>>
> >>>And so we go back to something much more like what is in the
> >>>previously posted benchmarking code:
> >>>
> >>>struct arena_s {
> >>> size_t obj_size; // Size in bytes of the object
> >>> size_t obj_pad_size; // Oobject size inclusive of padding
> >>> size_t arena_size; // Number of obj_pad_size slots in pool
> >>> int free_list; // Pointer to head of free list
> >>> int free; // Number of slots remaining
> >>> unsigned char *
> >>> arena; //
> >>>};
> >>>
> >>
> >>If you are using 'malloc()' to allocate the arena, and 'malloc()'
> >>returns a pointer to memory suitably aligned for _any_ object, then that
> >>would seem to include an _array_ object, so again, I don't understand
> >>the padding concerns.
> >
> >Well, if one is using a platform that requires four-byte alignment to
> >access ints (for instance), but you know the algorithm only needs 24
> >bits of data, you /can/ pack it in to three bytes and save 25% of the
> >nominal storage requirements of the array.
>
> If your allocator uses padding in the arena, where are the savings?
It doesn't /have/ to use padding. That's up to the programmer.
Therefore the savings in the example above are about 25% RAM, and
possibly reduced d-cache pollution.
> >>If someone tries to use your allocator with a size of "3", they had
> >>better not be lying and requiring an alignment of "4", of which "3" is
> >>not an integer multiple.
> >
> >Except on x86, which does not demand strict alignment. I'm leaving
> >alignment constraints up to the application programmer to define. If
> >someone wants to shoot themselves in the foot on a platform that
> >requires strict alignment, who am I to argue against it? Bloody toes
> >might be part of the application spec., and would therefore represent
> >an administrative policy beyond the scope of the allocator proper.
>
> Agreed. So why attempt to pad on their behalf within the allocator?
It comes down to having a sensible default. As alignment is an issue
for some platforms, and since it is impossible to know beforehand the
platform the allocator will run on, the safe way aligns the array
slots to minimize the artifacts of misalignment. Aligning to VM page
boundaries is about the only way I can think of to avoid all possible
unwanted interactions, but then we are off into ludicrous territory.
> >>Using an 'int' and macro/function? Seems reasonable to me, though
> >>perhaps a 'ptrdiff_t' could be pleasant, too. :)
> >
> >I doubt it. If you're careful not to do pointer arithmetic where you
> >might get a negative offset, life is good. While I would not
> >disparage the utility of negative pointer offsets in certain
> >situations, it isn't completely necessary. "ptr - unsigned int"
> >should do the right thing. So far, I have not come across a situation
> >where I would need negative pointer offsets, making the use of
> >ptrdiff_t a moot issue, IMO. YMMV.
>
> I didn't mean to suggest any consideration for negative 'ptrdiff_t'
> values. It's a signed integer type just as 'int' is a signed integer
> type. But because a pointer to the top of your arena minus a pointer to
> the bottom of it yields a 'ptrdiff_t', perhaps it could be enjoyable to
> use 'ptrdiff_t' for the addition, just for the sake of symmetry. :)
I'm not sure how that will save me from the risks inherent in the use
of ints, but I'll look into it.
Regards,
Uncle Steve
--
Should a professional politician be charged with molestation if he
kisses babies while attending political rallies?
[toc] | [prev] | [next] | [standalone]
| From | Keith Thompson <kst-u@mib.org> |
|---|---|
| Date | 2011-05-23 09:06 -0700 |
| Message-ID | <lnoc2txuxt.fsf@nuthaus.mib.org> |
| In reply to | #4562 |
Uncle Steve <stevet810@gmail.com> writes:
[...]
> The question is whether the padding is added at the structure
> definition or if it is accounted for when the bar[2] array is used.
> No idea what the standard says about that. As gcc supplies a 'packed'
> attribute to use, I can assume that it is safe in this regard.
[...]
I would have thought so as well, but it turns out that it *isn't* safe.
Here's a test program I just wrote:
#include <stdio.h>
#include <stddef.h>
int main(void)
{
struct foo {
char c;
int x;
} __attribute__((packed));
struct foo arr[2] = { { 'a', 10 }, {'b', 20 } };
int *p0 = &arr[0].x;
int *p1 = &arr[1].x;
printf("sizeof(struct foo) = %d\n",
(int)sizeof(struct foo));
printf("offsetof(struct foo, c) = %d\n",
(int)offsetof(struct foo, c));
printf("offsetof(struct foo, x) = %d\n",
(int)offsetof(struct foo, x));
printf("arr[0].x = %d\n", arr[0].x);
printf("arr[1].x = %d\n", arr[1].x);
printf("p0 = %p\n", (void*)p0);
printf("p1 = %p\n", (void*)p1);
printf("*p0 = %d\n", *p0);
printf("*p1 = %d\n", *p1);
return 0;
}
On my Linux x86 system, with gcc 4.5.2, the output is:
sizeof(struct foo) = 5
offsetof(struct foo, c) = 0
offsetof(struct foo, x) = 1
arr[0].x = 10
arr[1].x = 20
p0 = 0xbfaa71ff
p1 = 0xbfaa7204
*p0 = 10
*p1 = 20
As you can see, p0 points to arr[0].x, which is at an odd address.
Since the x86 tolerates misaligned accesses, the program doesn't
misbehave.
Now here's the output of the same program on a Solaris SPARC system,
compiled with gcc 4.2.1:
sizeof(struct foo) = 5
offsetof(struct foo, c) = 0
offsetof(struct foo, x) = 1
arr[0].x = 10
arr[1].x = 20
p0 = ffbff327
p1 = ffbff32c
Bus error
If you refer to arr[0].x directly, the compiler knows that it's
a member of a packed structure, and can generate whatever extra
code is needed to access it. If you take its address, though,
you risk creating a misaligned pointer, and when you dereference
it the compiler has no way of knowing that it's misaligned. Kaboom.
(Oddly, I don't see a warning about this in the gcc documentation.)
--
Keith Thompson (The_Other_Keith) kst-u@mib.org <http://www.ghoti.net/~kst>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
[toc] | [prev] | [next] | [standalone]
| From | Ben Bacarisse <ben.usenet@bsb.me.uk> |
|---|---|
| Date | 2011-05-22 23:18 +0100 |
| Message-ID | <0.47b2d301a43a01009630.20110522231836BST.87fwo6nzur.fsf@bsb.me.uk> |
| In reply to | #4511 |
Uncle Steve <stevet810@gmail.com> writes:
> On Sun, May 22, 2011 at 02:49:51AM -0700, luser- -droog wrote:
>>
>> I've been using integer offsets instead of pointers a lot lately and
>> I think you might be better off using the array formula explicitly:
>>
>> #define arena_obj_addr(x, n) ((void *) (x->arena + n * x->obj_size))
Everyone is taking chances here. The x and n should be in parentheses
so that the macro expands as expected. This is one reason why functions
are usually preferred.
<snip>
> #define ARENA_TYPE struct foo
>
> struct arena_s {
> size_t arena_size;
> int free_list;
> size_t free;
> ARENA_TYPE * arena;
> };
>
> And your access macro becomes:
>
> #define arena_obj_addr(x, n) &x->arena[n]
Here the [n] is OK, but why risk it with x? The (x) costs nothing and,
I think, saves time reading the macro since the reader does no have to
start worrying about what might be passed.
> since the pointer will be of the expected type.
<snip>
--
Ben.
[toc] | [prev] | [next] | [standalone]
| From | Uncle Steve <stevet810@gmail.com> |
|---|---|
| Date | 2011-05-22 19:18 -0400 |
| Message-ID | <21ceccb2e03a4dc-edfd3@gmail.com> |
| In reply to | #4532 |
On Sun, May 22, 2011 at 1u1:18:36PM +0100, Ben Bacarisse wrote:
> Uncle Steve <stevet810@gmail.com> writes:
>
> > On Sun, May 22, 2011 at 02:49:51AM -0700, luser- -droog wrote:
> >>
> >> I've been using integer offsets instead of pointers a lot lately and
> >> I think you might be better off using the array formula explicitly:
> >>
> >> #define arena_obj_addr(x, n) ((void *) (x->arena + n * x->obj_size))
>
> Everyone is taking chances here. The x and n should be in parentheses
> so that the macro expands as expected. This is one reason why functions
> are usually preferred.
For a long time I've thought macros were preferable to functions for
the reason that a function call has to save registers, stuff it's arguments on the
stack, and so on. Macros should just fold into the program text, but
today inline functions are just as fast as macros. It has become a
matter of preference. gcc even has the ability to use register
calling conventions for function calls in some circumstances. The
distinction between a macro and a function call is becoming moot.
> <snip>
> > #define ARENA_TYPE struct foo
> > struct arena_s {
> > size_t arena_size;
> > int free_list;
> > size_t free;
> > ARENA_TYPE * arena;
> > };
> >
> > And your access macro becomes:
> >
> > #define arena_obj_addr(x, n) &x->arena[n]
>
> Here the [n] is OK, but why risk it with x? The (x) costs nothing and,
> I think, saves time reading the macro since the reader does no have to
> start worrying about what might be passed.
No worries here. I've edited for brevity in posting, but I always
protect macro arguments from side-effects these days. It doesn't look
as clean and concise, which is why I omitted those statements from
example code. No biggie.
In this newsgroup is it really preferable to see
#define arena_object_addr(x, n) ({ \
struct arena_s X = x; \
int N = n; \
*X->arena[N]; }) \
for every dumb example that is posted? Possibly not if people can be
reminded that all code posted in comp.lang.c is generally intended as
an example.
Regards,
Uncle Steve
--
Should a professional politician be charged with molestation if he
kisses babies while attending political rallies?
[toc] | [prev] | [next] | [standalone]
| From | Ian Collins <ian-news@hotmail.com> |
|---|---|
| Date | 2011-05-23 11:22 +1200 |
| Message-ID | <93tk6hFt2pU3@mid.individual.net> |
| In reply to | #4534 |
On 05/23/11 11:18 AM, Uncle Steve wrote: > On Sun, May 22, 2011 at 1u1:18:36PM +0100, Ben Bacarisse wrote: >> Uncle Steve<stevet810@gmail.com> writes: >> >>> On Sun, May 22, 2011 at 02:49:51AM -0700, luser- -droog wrote: >>>> >>>> I've been using integer offsets instead of pointers a lot lately and >>>> I think you might be better off using the array formula explicitly: >>>> >>>> #define arena_obj_addr(x, n) ((void *) (x->arena + n * x->obj_size)) >> >> Everyone is taking chances here. The x and n should be in parentheses >> so that the macro expands as expected. This is one reason why functions >> are usually preferred. > > For a long time I've thought macros were preferable to functions for > the reason that a function call has to save registers, stuff it's arguments on the > stack, and so on. Macros should just fold into the program text, but > today inline functions are just as fast as macros. It has become a > matter of preference. gcc even has the ability to use register > calling conventions for function calls in some circumstances. The > distinction between a macro and a function call is becoming moot. No it isn't. All the disadvantages of function like macros still exist, but the single (possible) disadvantage of function calls has gone. -- Ian Collins
[toc] | [prev] | [next] | [standalone]
| From | Uncle Steve <stevet810@gmail.com> |
|---|---|
| Date | 2011-05-23 01:57 -0400 |
| Message-ID | <c5fb887a0c3800b-63061@gmail.com> |
| In reply to | #4535 |
On Mon, May 23, 2011 at 11:22:57AM +1200, Ian Collins wrote: > On 05/23/11 11:18 AM, Uncle Steve wrote: > >On Sun, May 22, 2011 at 1u1:18:36PM +0100, Ben Bacarisse wrote: > >>Uncle Steve<stevet810@gmail.com> writes: > >> > >>>On Sun, May 22, 2011 at 02:49:51AM -0700, luser- -droog wrote: > >>>> > >>>>I've been using integer offsets instead of pointers a lot lately and > >>>>I think you might be better off using the array formula explicitly: > >>>> > >>>> #define arena_obj_addr(x, n) ((void *) (x->arena + n * x->obj_size)) > >> > >>Everyone is taking chances here. The x and n should be in parentheses > >>so that the macro expands as expected. This is one reason why functions > >>are usually preferred. > > > >For a long time I've thought macros were preferable to functions for > >the reason that a function call has to save registers, stuff it's > >arguments on the > >stack, and so on. Macros should just fold into the program text, but > >today inline functions are just as fast as macros. It has become a > >matter of preference. gcc even has the ability to use register > >calling conventions for function calls in some circumstances. The > >distinction between a macro and a function call is becoming moot. > > No it isn't. All the disadvantages of function like macros still exist, > but the single (possible) disadvantage of function calls has gone. Not so fast, Mr. Collins. Macros are useful in numerous situations where function calls would be inefficient. For my purposes, it is sometimes nice to write a macro which uses conditionals to select a particular code path through the macro. If the conditional can be resolved at compile-time, the dead code branches in the macro are pruned, saving space. A similarly structured function call won't be optimized in this fashion unless (maybe) it is inlined. Not enough? Is it smart to rely on the compiler to optimize the crap out of everything, or is it better to write efficient code at the outset? There are a number of C compilers out there that are not gcc, and which may or may not generate equivalent code for otherwise similar macros/functions. I'm O.K. with peppering my code with well-designed macros and ignoring functions for simple common procedures, but then I'm biased that way from habit. Regards, Uncle Steve -- Should a professional politician be charged with molestation if he kisses babies while attending political rallies?
[toc] | [prev] | [next] | [standalone]
| From | Ian Collins <ian-news@hotmail.com> |
|---|---|
| Date | 2011-05-23 19:24 +1200 |
| Message-ID | <93ugcsFt2pU6@mid.individual.net> |
| In reply to | #4543 |
On 05/23/11 05:57 PM, Uncle Steve wrote: > On Mon, May 23, 2011 at 11:22:57AM +1200, Ian Collins wrote: >> On 05/23/11 11:18 AM, Uncle Steve wrote: >>> On Sun, May 22, 2011 at 1u1:18:36PM +0100, Ben Bacarisse wrote: >>>> Uncle Steve<stevet810@gmail.com> writes: >>>> >>>>> On Sun, May 22, 2011 at 02:49:51AM -0700, luser- -droog wrote: >>>>>> >>>>>> I've been using integer offsets instead of pointers a lot lately and >>>>>> I think you might be better off using the array formula explicitly: >>>>>> >>>>>> #define arena_obj_addr(x, n) ((void *) (x->arena + n * x->obj_size)) >>>> >>>> Everyone is taking chances here. The x and n should be in parentheses >>>> so that the macro expands as expected. This is one reason why functions >>>> are usually preferred. >>> >>> For a long time I've thought macros were preferable to functions for >>> the reason that a function call has to save registers, stuff it's >>> arguments on the >>> stack, and so on. Macros should just fold into the program text, but >>> today inline functions are just as fast as macros. It has become a >>> matter of preference. gcc even has the ability to use register >>> calling conventions for function calls in some circumstances. The >>> distinction between a macro and a function call is becoming moot. >> >> No it isn't. All the disadvantages of function like macros still exist, >> but the single (possible) disadvantage of function calls has gone. > > Not so fast, Mr. Collins. Macros are useful in numerous situations > where function calls would be inefficient. The only real use for macros is passing __FILE__ and __LINE__ to debug logs! > For my purposes, it is > sometimes nice to write a macro which uses conditionals to select a > particular code path through the macro. If the conditional can be > resolved at compile-time, the dead code branches in the macro are > pruned, saving space. A similarly structured function call won't be > optimized in this fashion unless (maybe) it is inlined. Why not? Whether the function is called or in line, the conditional compiles are still there. > Not enough? Is it smart to rely on the compiler to optimize the crap > out of everything, or is it better to write efficient code at the > outset? Inlining is a very simple optimisation. You also overlook the opposite optimisation - not inlining. There are situations where a smaller executable performs better (or even fit the target device!). The compiler should be able to judge where it is appropriate to inline. Compilers targeted at embedded devices often have options to tune these heuristics. Decent compilers for hosted environments offer profile feedback to further tune optimisations. In short, the more choice you give the compiler, the better job it can do. > There are a number of C compilers out there that are not gcc, > and which may or may not generate equivalent code for otherwise > similar macros/functions. I'm O.K. with peppering my code with > well-designed macros and ignoring functions for simple common > procedures, but then I'm biased that way from habit. I guess working with C++ for many years has biased me the other way. Aggressive inlining has long been a feature of C++ compilers. It used to be a unique feature, but those days are long past. -- Ian Collins
[toc] | [prev] | [next] | [standalone]
| From | Uncle Steve <stevet810@gmail.com> |
|---|---|
| Date | 2011-05-23 14:20 -0400 |
| Message-ID | <2cc0f0e084f264b-656fa@gmail.com> |
| In reply to | #4554 |
On Mon, May 23, 2011 at 07:24:11PM +1200, Ian Collins wrote: > On 05/23/11 05:57 PM, Uncle Steve wrote: > >On Mon, May 23, 2011 at 11:22:57AM +1200, Ian Collins wrote: > >>On 05/23/11 11:18 AM, Uncle Steve wrote: > >>>On Sun, May 22, 2011 at 1u1:18:36PM +0100, Ben Bacarisse wrote: > >>>>Uncle Steve<stevet810@gmail.com> writes: > >>>> > >>>>>On Sun, May 22, 2011 at 02:49:51AM -0700, luser- -droog wrote: > >>>>>> > >>>>>>I've been using integer offsets instead of pointers a lot lately and > >>>>>>I think you might be better off using the array formula explicitly: > >>>>>> > >>>>>> #define arena_obj_addr(x, n) ((void *) (x->arena + n * x->obj_size)) > >>>> > >>>>Everyone is taking chances here. The x and n should be in parentheses > >>>>so that the macro expands as expected. This is one reason why functions > >>>>are usually preferred. > >>> > >>>For a long time I've thought macros were preferable to functions for > >>>the reason that a function call has to save registers, stuff it's > >>>arguments on the > >>>stack, and so on. Macros should just fold into the program text, but > >>>today inline functions are just as fast as macros. It has become a > >>>matter of preference. gcc even has the ability to use register > >>>calling conventions for function calls in some circumstances. The > >>>distinction between a macro and a function call is becoming moot. > >> > >>No it isn't. All the disadvantages of function like macros still exist, > >>but the single (possible) disadvantage of function calls has gone. > > > >Not so fast, Mr. Collins. Macros are useful in numerous situations > >where function calls would be inefficient. > > The only real use for macros is passing __FILE__ and __LINE__ to debug logs! No, not really. > >For my purposes, it is > >sometimes nice to write a macro which uses conditionals to select a > >particular code path through the macro. If the conditional can be > >resolved at compile-time, the dead code branches in the macro are > >pruned, saving space. A similarly structured function call won't be > >optimized in this fashion unless (maybe) it is inlined. > > Why not? Whether the function is called or in line, the conditional > compiles are still there. A function is only inlined if it is defined in the same source file. If you define a function in an external source file, it cannot be inlined anywhere else unless the function is also defined in an included header file. Doesn't everyone know this? > >Not enough? Is it smart to rely on the compiler to optimize the crap > >out of everything, or is it better to write efficient code at the > >outset? > > Inlining is a very simple optimisation. You also overlook the opposite No it isn't. > optimisation - not inlining. There are situations where a smaller > executable performs better (or even fit the target device!). The In <a8a6d14a9ed9b70-0c8ce@gmail.com> I wrote: Whether or not the code is inlined is going to depend more on whether you are interested in performance or code size. Fitting code in some machines' L1 cache is harder with a lot of gratuitous inlining, and you also pollute the l2 cache. Sometimes this matters, and sometimes not. All highly dependent on the specifics of the code and platform. > compiler should be able to judge where it is appropriate to inline. > Compilers targeted at embedded devices often have options to tune these > heuristics. Decent compilers for hosted environments offer profile > feedback to further tune optimisations. > > In short, the more choice you give the compiler, the better job it can do. Of course. > >There are a number of C compilers out there that are not gcc, > >and which may or may not generate equivalent code for otherwise > >similar macros/functions. I'm O.K. with peppering my code with > >well-designed macros and ignoring functions for simple common > >procedures, but then I'm biased that way from habit. > > I guess working with C++ for many years has biased me the other way. > Aggressive inlining has long been a feature of C++ compilers. It used > to be a unique feature, but those days are long past. I guess so. In the last twenty years there's been a lot of work on compilers, that's for sure. Regards, Uncle Steve -- Should a professional politician be charged with molestation if he kisses babies while attending political rallies?
[toc] | [prev] | [next] | [standalone]
| From | Ian Collins <ian-news@hotmail.com> |
|---|---|
| Date | 2011-05-24 07:41 +1200 |
| Message-ID | <93vrieFt2pU7@mid.individual.net> |
| In reply to | #4571 |
On 05/24/11 06:20 AM, Uncle Steve wrote: > On Mon, May 23, 2011 at 07:24:11PM +1200, Ian Collins wrote: >> On 05/23/11 05:57 PM, Uncle Steve wrote: > >>> For my purposes, it is >>> sometimes nice to write a macro which uses conditionals to select a >>> particular code path through the macro. If the conditional can be >>> resolved at compile-time, the dead code branches in the macro are >>> pruned, saving space. A similarly structured function call won't be >>> optimized in this fashion unless (maybe) it is inlined. >> >> Why not? Whether the function is called or in line, the conditional >> compiles are still there. > > A function is only inlined if it is defined in the same source file. Macros also have to be defined in the same compilation unit. > If you define a function in an external source file, it cannot be > inlined anywhere else unless the function is also defined in an > included header file. Doesn't everyone know this? I'm sure they do, what's your point? -- Ian Collins
[toc] | [prev] | [next] | [standalone]
| From | Uncle Steve <stevet810@gmail.com> |
|---|---|
| Date | 2011-05-23 16:24 -0400 |
| Message-ID | <f59ae793f719149-1c10a@gmail.com> |
| In reply to | #4581 |
On Tue, May 24, 2011 at 07:41:02AM +1200, Ian Collins wrote: > On 05/24/11 06:20 AM, Uncle Steve wrote: > >On Mon, May 23, 2011 at 07:24:11PM +1200, Ian Collins wrote: > >>On 05/23/11 05:57 PM, Uncle Steve wrote: > > > >>>For mu y purposes, it is > >>>sometimes nice to write a macro which uses conditionals to select a > >>>particular code path through the macro. If the conditional can be > >>>resolved at compile-time, the dead code branches in the macro are > >>>pruned, saving space. A similarly structured function call won't be > >>>optimized in this fashion unless (maybe) it is inlined. > >> > >>Why not? Whether the function is called or in line, the conditional > >>compiles are still there. > > > >A function is only inlined if it is defined in the same source file. > > Macros also have to be defined in the same compilation unit. > > >If you define a function in an external source file, it cannot be > >inlined anywhere else unless the function is also defined in an > >included header file. Doesn't everyone know this? > > I'm sure they do, what's your point? That function inlining, far from being the unfettered panacea some make it out to be, has inherent limitations owing to the way C compilers are designed to take their input. If your software is written as one monolithic source file, this may not be much of an issue, but I doubt that is the case for any non-trivial program. Given that reality, function-like macros can be included to inline code where it will do the most good. If the programmer is lacking in clue[tm] there's no telling what might happen, but smart people will learn not to abuse the facility. Regards, Uncle Steve -- Should a professional politician be charged with molestation if he kisses babies while attending political rallies?
[toc] | [prev] | [next] | [standalone]
| From | Ian Collins <ian-news@hotmail.com> |
|---|---|
| Date | 2011-05-24 09:20 +1200 |
| Message-ID | <9401d5Ft2pU8@mid.individual.net> |
| In reply to | #4587 |
On 05/24/11 08:24 AM, Uncle Steve wrote: > On Tue, May 24, 2011 at 07:41:02AM +1200, Ian Collins wrote: >> On 05/24/11 06:20 AM, Uncle Steve wrote: >>> On Mon, May 23, 2011 at 07:24:11PM +1200, Ian Collins wrote: >>>> On 05/23/11 05:57 PM, Uncle Steve wrote: >>> >>>>> For mu y purposes, it is >>>>> sometimes nice to write a macro which uses conditionals to select a >>>>> particular code path through the macro. If the conditional can be >>>>> resolved at compile-time, the dead code branches in the macro are >>>>> pruned, saving space. A similarly structured function call won't be >>>>> optimized in this fashion unless (maybe) it is inlined. >>>> >>>> Why not? Whether the function is called or in line, the conditional >>>> compiles are still there. >>> >>> A function is only inlined if it is defined in the same source file. >> >> Macros also have to be defined in the same compilation unit. >> >>> If you define a function in an external source file, it cannot be >>> inlined anywhere else unless the function is also defined in an >>> included header file. Doesn't everyone know this? >> >> I'm sure they do, what's your point? > > That function inlining, far from being the unfettered panacea some > make it out to be, has inherent limitations owing to the way C > compilers are designed to take their input. Which it shares with macros, so again, what is your point? The single compilation unit limitation will eventually be a thing of the past as cross module optimisation become more common. For example see http://gcc.gnu.org/wiki/LightweightIpo > If your software is > written as one monolithic source file, this may not be much of an > issue, but I doubt that is the case for any non-trivial program. > > Given that reality, function-like macros can be included to inline > code where it will do the most good. So can functions, so again, what is your point? -- Ian Collins
[toc] | [prev] | [next] | [standalone]
| From | Uncle Steve <stevet810@gmail.com> |
|---|---|
| Date | 2011-05-23 18:05 -0400 |
| Message-ID | <52abe5c90929121-7f162@gmail.com> |
| In reply to | #4596 |
On Tue, May 24, 2011 at 09:20:37AM +1200, Ian Collins wrote: > On 05/24/11 08:24 AM, Uncle Steve wrote: > >On Tue, May 24, 2011 at 07:41:02AM +1200, Ian Collins wrote: > >>On 05/24/11 06:20 AM, Uncle Steve wrote: > >>>On Mon, May 23, 2011 at 07:24:11PM +1200, Ian Collins wrote: > >>>>On 05/23/11 05:57 PM, Uncle Steve wrote: > >>> > >>>>>For mu y purposes, it is > >>>>>sometimes nice to write a macro which uses conditionals to select a > >>>>>particular code path through the macro. If the conditional can be > >>>>>resolved at compile-time, the dead code branches in the macro are > >>>>>pruned, saving space. A similarly structured function call won't be > >>>>>optimized in this fashion unless (maybe) it is inlined. > >>>> > >>>>Why not? Whether the function is called or in line, the conditional > >>>>compiles are still there. > >>> > >>>A function is only inlined if it is defined in the same source file. > >> > >>Macros also have to be defined in the same compilation unit. > >> > >>>If you define a function in an external source file, it cannot be > >>>inlined anywhere else unless the function is also defined in an > >>>included header file. Doesn't everyone know this? > >> > >>I'm sure they do, what's your point? > > > >That function inlining, far from being the unfettered panacea some > >make it out to be, has inherent limitations owing to the way C > >compilers are designed to take their input. > > Which it shares with macros, so again, what is your point? > > The single compilation unit limitation will eventually be a thing of the > past as cross module optimisation become more common. For example > see http://gcc.gnu.org/wiki/LightweightIpo Filed under "what will they think of next". > >If your software is > >written as one monolithic source file, this may not be much of an > >issue, but I doubt that is the case for any non-trivial program. > > > >Given that reality, function-like macros can be included to inline > >code where it will do the most good. > > So can functions, so again, what is your point? To annoy with impertinent questions? Regards, Uncle Steve -- Should a professional politician be charged with molestation if he kisses babies while attending political rallies?
[toc] | [prev] | [next] | [standalone]
| From | Ian Collins <ian-news@hotmail.com> |
|---|---|
| Date | 2011-05-24 10:37 +1200 |
| Message-ID | <9405t6Ft2pU13@mid.individual.net> |
| In reply to | #4610 |
On 05/24/11 10:05 AM, Uncle Steve wrote: > On Tue, May 24, 2011 at 09:20:37AM +1200, Ian Collins wrote: >> On 05/24/11 08:24 AM, Uncle Steve wrote: >>> >>> Given that reality, function-like macros can be included to inline >>> code where it will do the most good. >> >> So can functions, so again, what is your point? > > To annoy with impertinent questions? Oh you'll have to try a lot harder than that to annoy me. Amuse would be a better description thus far. -- Ian Collins
[toc] | [prev] | [next] | [standalone]
| From | Uncle Steve <stevet810@gmail.com> |
|---|---|
| Date | 2011-05-23 19:34 -0400 |
| Message-ID | <388617a67cfb02b-42734@gmail.com> |
| In reply to | #4616 |
On Tue, May 24, 2011 at 10:37:26AM +1200, Ian Collins wrote: > On 05/24/11 10:05 AM, Uncle Steve wrote: > >On Tue, May 24, 2011 at 09:20:37AM +1200, Ian Collins wrote: > >>On 05/24/11 08:24 AM, Uncle Steve wrote: > >>> > >>>Given that reality, function-like macros can be included to inline > >>>code where it will do the most good. > >> > >>So can functions, so again, what is your point? > > > >To annoy with impertinent questions? > > Oh you'll have to try a lot harder than that to annoy me. Amuse would > be a better description thus far. I'll see what I can do about that, heh heh heh. Regards, Uncle Steve -- Should a professional politician be charged with molestation if he kisses babies while attending political rallies?
[toc] | [prev] | [next] | [standalone]
| From | "robertwessel2@yahoo.com" <robertwessel2@yahoo.com> |
|---|---|
| Date | 2011-05-23 17:48 -0700 |
| Message-ID | <0e5227d8-e4b8-41ca-bb9c-4024a50e425c@z19g2000yqz.googlegroups.com> |
| In reply to | #4581 |
On May 23, 2:41 pm, Ian Collins <ian-n...@hotmail.com> wrote: > On 05/24/11 06:20 AM, Uncle Steve wrote: > > > On Mon, May 23, 2011 at 07:24:11PM +1200, Ian Collins wrote: > >> On 05/23/11 05:57 PM, Uncle Steve wrote: > > >>> For my purposes, it is > >>> sometimes nice to write a macro which uses conditionals to select a > >>> particular code path through the macro. If the conditional can be > >>> resolved at compile-time, the dead code branches in the macro are > >>> pruned, saving space. A similarly structured function call won't be > >>> optimized in this fashion unless (maybe) it is inlined. > > >> Why not? Whether the function is called or in line, the conditional > >> compiles are still there. > > > A function is only inlined if it is defined in the same source file. > > Macros also have to be defined in the same compilation unit. > > > If you define a function in an external source file, it cannot be > > inlined anywhere else unless the function is also defined in an > > included header file. Doesn't everyone know this? > > I'm sure they do, what's your point? Compilers that support link time code generation manage to inline functions defined in different translation units all the time.
[toc] | [prev] | [next] | [standalone]
Page 2 of 5 — ← Prev page 1 [2] 3 4 5 Next page →
Back to top | Article view | comp.lang.c
csiph-web