Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.c > #4178 > unrolled thread
| Started by | Pushpa <pushpa@nospam.invalid> |
|---|---|
| First post | 2011-05-17 19:39 +0000 |
| Last post | 2011-05-19 09:12 -0600 |
| Articles | 12 on this page of 92 — 26 participants |
Back to article view | Back to comp.lang.c
Malloc Query Pushpa <pushpa@nospam.invalid> - 2011-05-17 19:39 +0000
Re: Malloc Query blp@cs.stanford.edu (Ben Pfaff) - 2011-05-17 12:55 -0700
Re: Malloc Query "BartC" <bc@freeuk.com> - 2011-05-18 12:05 +0100
Re: Malloc Query Pushpa <pushpa@nospam.invalid> - 2011-05-18 17:11 +0000
Re: Malloc Query "BartC" <bc@freeuk.com> - 2011-05-18 22:42 +0100
Re: Malloc Query Eric Sosman <esosman@ieee-dot-org.invalid> - 2011-05-18 21:10 -0400
Re: Malloc Query Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-05-18 19:34 +0100
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-18 17:10 -0400
Re: Malloc Query Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-05-19 00:26 +0100
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-18 21:25 -0400
Re: Malloc Query James Kuyper <jameskuyper@verizon.net> - 2011-05-18 21:38 -0400
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-19 14:11 -0400
Re: Malloc Query James Kuyper <jameskuyper@verizon.net> - 2011-05-19 22:55 -0400
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-20 12:14 -0400
Re: Malloc Query Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-05-19 03:22 +0100
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-19 14:20 -0400
Re: Malloc Query Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-05-19 23:51 +0100
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-19 20:01 -0400
Re: Malloc Query luser- -droog <mijoryx@yahoo.com> - 2011-05-22 02:49 -0700
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-22 10:46 -0400
Re: Malloc Query Shao Miller <sha0.miller@gmail.com> - 2011-05-22 12:37 -0500
Re: Malloc Query Shao Miller <sha0.miller@gmail.com> - 2011-05-22 13:02 -0500
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-22 14:12 -0400
Re: Malloc Query Shao Miller <sha0.miller@gmail.com> - 2011-05-23 03:06 -0500
Re: Malloc Query luser- -droog <mijoryx@yahoo.com> - 2011-05-23 00:42 -0700
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-23 11:10 -0400
Re: Malloc Query Keith Thompson <kst-u@mib.org> - 2011-05-23 09:06 -0700
Re: Malloc Query Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-05-22 23:18 +0100
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-22 19:18 -0400
Re: Malloc Query Ian Collins <ian-news@hotmail.com> - 2011-05-23 11:22 +1200
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-23 01:57 -0400
Re: Malloc Query Ian Collins <ian-news@hotmail.com> - 2011-05-23 19:24 +1200
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-23 14:20 -0400
Re: Malloc Query Ian Collins <ian-news@hotmail.com> - 2011-05-24 07:41 +1200
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-23 16:24 -0400
Re: Malloc Query Ian Collins <ian-news@hotmail.com> - 2011-05-24 09:20 +1200
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-23 18:05 -0400
Re: Malloc Query Ian Collins <ian-news@hotmail.com> - 2011-05-24 10:37 +1200
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-23 19:34 -0400
Re: Malloc Query "robertwessel2@yahoo.com" <robertwessel2@yahoo.com> - 2011-05-23 17:48 -0700
Re: Malloc Query Ian Collins <ian-news@hotmail.com> - 2011-05-24 12:56 +1200
Re: Malloc Query James Kuyper <jameskuyper@verizon.net> - 2011-05-24 07:19 -0400
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-24 11:32 -0400
Re: Malloc Query Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-05-23 01:04 +0100
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-23 02:36 -0400
Re: Malloc Query luser- -droog <mijoryx@yahoo.com> - 2011-05-23 00:39 -0700
Re: Malloc Query Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-05-23 14:44 +0100
Re: Malloc Query Seebs <usenet-nospam@seebs.net> - 2011-05-23 15:16 +0000
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-23 11:41 -0400
Re: Malloc Query David Thompson <dave.thompson2@verizon.net> - 2011-06-01 16:33 -0400
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-19 18:34 -0400
Re: Malloc Query Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-05-20 02:36 +0100
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-20 12:26 -0400
Re: Malloc Query Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-05-20 23:32 +0100
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-20 21:17 -0400
Re: Malloc Query Ian Collins <ian-news@hotmail.com> - 2011-05-21 14:30 +1200
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-21 11:13 -0400
Re: Malloc Query Ian Collins <ian-news@hotmail.com> - 2011-05-22 08:50 +1200
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-21 20:03 -0400
Re: Malloc Query Ian Collins <ian-news@hotmail.com> - 2011-05-22 13:23 +1200
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-22 10:01 -0400
Re: Malloc Query "io_x" <a@b.c.invalid> - 2011-05-26 08:12 +0200
Re: Malloc Query Ian Collins <ian-news@hotmail.com> - 2011-05-26 19:16 +1200
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-26 15:03 -0400
Re: Malloc Query Ian Collins <ian-news@hotmail.com> - 2011-05-27 09:16 +1200
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-26 17:55 -0400
Re: Malloc Query "io_x" <a@b.c.invalid> - 2011-05-27 05:47 +0200
Re: Malloc Query Uncle Steve <stevet810@gmail.com> - 2011-05-26 15:02 -0400
Re: Malloc Query "BartC" <bc@freeuk.com> - 2011-05-18 22:31 +0100
Re: Malloc Query Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-05-18 23:37 +0100
Re: Malloc Query "BartC" <bc@freeuk.com> - 2011-05-19 01:50 +0100
Re: Malloc Query Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-05-19 02:50 +0100
Re: Malloc Query "BartC" <bc@freeuk.com> - 2011-05-19 12:14 +0100
Re: Malloc Query Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-05-19 15:10 +0100
Re: Malloc Query "BartC" <bc@freeuk.com> - 2011-05-19 15:48 +0100
Re: Malloc Query cri@tiac.net (Richard Harter) - 2011-05-18 19:31 +0000
Re: Malloc Query Ike Naar <ike@iceland.freeshell.org> - 2011-05-18 21:43 +0000
Re: Malloc Query "BartC" <bc@freeuk.com> - 2011-05-18 22:58 +0100
Re: Malloc Query Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2011-05-17 14:01 -0600
Re: Malloc Query Columbus sailed the ocean China Blue <chine.bleu@yahoo.com> - 2011-05-17 13:21 -0700
Re: Malloc Query BGB <cr88192@hotmail.com> - 2011-05-17 14:08 -0700
Re: Malloc Query Michael Press <rubrum@pacbell.net> - 2011-05-17 19:49 -0700
Re: Malloc Query Lauri Alanko <la@iki.fi> - 2011-05-18 00:50 +0000
Re: Malloc Query Eric Sosman <esosman@ieee-dot-org.invalid> - 2011-05-17 21:21 -0400
Re: Malloc Query "io_x" <a@b.c.invalid> - 2011-05-18 19:31 +0200
Re: Malloc Query Jorgen Grahn <grahn+nntp@snipabacken.se> - 2011-05-18 21:36 +0000
Re: Malloc Query Eric Sosman <esosman@ieee-dot-org.invalid> - 2011-05-18 21:26 -0400
Re: Malloc Query Tim Rentsch <txr@alumni.caltech.edu> - 2011-05-19 02:07 -0700
Re: Malloc Query Ian Collins <ian-news@hotmail.com> - 2011-05-19 21:13 +1200
Re: Malloc Query Nobody <nobody@nowhere.com> - 2011-05-18 07:08 +0100
Re: Malloc Query David Resnick <lndresnick@gmail.com> - 2011-05-19 07:51 -0700
Re: Malloc Query Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2011-05-19 09:12 -0600
Page 5 of 5 — ← Prev page 1 2 3 4 [5]
| From | BGB <cr88192@hotmail.com> |
|---|---|
| Date | 2011-05-17 14:08 -0700 |
| Message-ID | <iquo8p$if4$1@news.albasani.net> |
| In reply to | #4178 |
On 5/17/2011 12:39 PM, Pushpa wrote: > Hi all, > > Many users claim that malloc() and free() provided by the language > library results in considerable > amount of overhead in terms of calling the same. Can we really > eliminate overhead by using > used defined functions/user defined memory manager. Is there any > significant advantage in providing user defined malloc() and free(). I > want to exclude realtime systems or embedded systems which are memory > constrained here. > I am talking about general purpose applications. With a user defined > memory manager can it handle > memory allocation and de-allocation in a much better way than what OS > would have done. Any inputs/comments > on the same are highly appreciated. > it actually really depends... if one doesn't have a good grasp on what they are doing, then a custom allocator is usually a good deal worse than the native malloc()/free() WRT overhead and/or performance. however, special purpose allocators can do well for certain tasks, such as being faster, or being more compact for tiny objects, but usually there are trade-offs. the main reason, in my case, I do a custom allocator is mostly for: 1, I use GC (often both a friend and an enemy at times); 2, I use dynamic typing, and dynamic type-checking and malloc are not friends; 3, my heap tends to often be dominated by small objects (and so certain special-cases, such as "cons cells" have special handling, and certain other special types are not even in addressable memory, as the pointer itself encodes the value); ... for example: cons cells are 8 bytes on x86 (they only ever hold 2 pointers), so it makes sense not to waste extra bytes on things like having a header or padding the object to a multiple of 16 bytes. also, they can easily end up (in certain types of app, such as those making lots of use of list-processing) being the single most common data-type on the heap. a secondary common data-type is strings, which if one is using interned/immutable strings, then having compact storage (such as byte-aligned string tables) may make a good amount of sense (as strings are also one of those things which can easily eat *lots* of memory, hence my personal belief that UTF-16 is rarely a sensible option vs UTF-8 or similar). granted, in other cases, it also makes sense to have strings in a form where they may either be mutable or are subject to being freed (it does not make sense to intern character buffers or large one-off strings, for example...). but, the custom MM/GC is mostly for GC and dynamic type-checking, rather than to try to save bytes or be faster (although it may do this in certain cases). but, if I were just doing "generic" stuff, then malloc would probably be fine, and simply trying to use a custom malloc as some perceived way to make allocating/freeing structs or arrays faster (or save a few bytes) is, IMO, a clear case of premature optimization... or such...
[toc] | [prev] | [next] | [standalone]
| From | Michael Press <rubrum@pacbell.net> |
|---|---|
| Date | 2011-05-17 19:49 -0700 |
| Message-ID | <rubrum-9F55C7.19492317052011@news.albasani.net> |
| In reply to | #4190 |
In article <iquo8p$if4$1@news.albasani.net>, BGB <cr88192@hotmail.com> wrote: > On 5/17/2011 12:39 PM, Pushpa wrote: > > Hi all, > > > > Many users claim that malloc() and free() provided by the language > > library results in considerable > > amount of overhead in terms of calling the same. Can we really > > eliminate overhead by using > > used defined functions/user defined memory manager. Is there any > > significant advantage in providing user defined malloc() and free(). I > > want to exclude realtime systems or embedded systems which are memory > > constrained here. > > I am talking about general purpose applications. With a user defined > > memory manager can it handle > > memory allocation and de-allocation in a much better way than what OS > > would have done. Any inputs/comments > > on the same are highly appreciated. > > > > it actually really depends... > > if one doesn't have a good grasp on what they are doing, then a custom > allocator is usually a good deal worse than the native malloc()/free() > WRT overhead and/or performance. > > however, special purpose allocators can do well for certain tasks, such > as being faster, or being more compact for tiny objects, but usually > there are trade-offs. > > > the main reason, in my case, I do a custom allocator is mostly for: > 1, I use GC (often both a friend and an enemy at times); > 2, I use dynamic typing, and dynamic type-checking and malloc are not > friends; > 3, my heap tends to often be dominated by small objects (and so certain > special-cases, such as "cons cells" have special handling, and certain > other special types are not even in addressable memory, as the pointer > itself encodes the value); > ... > > for example: cons cells are 8 bytes on x86 (they only ever hold 2 > pointers), so it makes sense not to waste extra bytes on things like > having a header or padding the object to a multiple of 16 bytes. > > also, they can easily end up (in certain types of app, such as those > making lots of use of list-processing) being the single most common > data-type on the heap. > > a secondary common data-type is strings, which if one is using > interned/immutable strings, then having compact storage (such as > byte-aligned string tables) may make a good amount of sense (as strings > are also one of those things which can easily eat *lots* of memory, > hence my personal belief that UTF-16 is rarely a sensible option vs > UTF-8 or similar). granted, in other cases, it also makes sense to have > strings in a form where they may either be mutable or are subject to > being freed (it does not make sense to intern character buffers or large > one-off strings, for example...). > > but, the custom MM/GC is mostly for GC and dynamic type-checking, rather > than to try to save bytes or be faster (although it may do this in > certain cases). > > > but, if I were just doing "generic" stuff, then malloc would probably be > fine, and simply trying to use a custom malloc as some perceived way to > make allocating/freeing structs or arrays faster (or save a few bytes) > is, IMO, a clear case of premature optimization... > > > or such... It is difficult to impossible to fully understand your prose unless the reader already knows this stuff; and you are responding to somebody who does not know. The impediments to understanding are the ellipses. Every ellipsis makes the reader stop and try to extrapolate what you leave unsaid. -- Michael Press
[toc] | [prev] | [next] | [standalone]
| From | Lauri Alanko <la@iki.fi> |
|---|---|
| Date | 2011-05-18 00:50 +0000 |
| Message-ID | <iqv54b$m48$1@oravannahka.helsinki.fi> |
| In reply to | #4178 |
I don't think there is very often need to shun the standard allocator for performance reasons. Production malloc implementations are quite mature and have reasonably good performance for the most common scenarios. It's worth noting that whenever you allocate and free some memory, you are also likely to _do_ something with that memory, possibly quite a lot. The performance overhead of malloc and free may often be negligible compared to the actual computation that is done using the memory. Yet still, I can see some reasons for preferring custom allocators. One is space efficiency. Because free() is not passed the size of the object being freed, the allocator must somehow store this information, and this may result in space overhead (depending on implementation details). In particular, if you allocate a large number of very small objects, the bookkeeping overhead may be significant. A custom allocator can be written so that the size of an object need not be stored in memory, if it can be known statically. The other reason is to improve the quality of the code. Explicit calls to free() for every allocated object can be tedious to write and it is easy to get them wrong, resulting in memory leaks or duplicate frees or whatnot. With a memory pool -based allocator, you don't need to free every object individually, but instead you just free an entire pool, along with all the objects allocated from that pool. This style fits many programs, and it results in code that is cleaner (less free()-calls sprinkled everywhere) and more correct (you can't accidentally forget to free a single object). Of course, sometimes this may not be enough and you may instead need reference counting or even actual tracing garbage collection. But again, using more specialized memory allocation systems should be motivated by the structure of your programs, not by performance reasons. Lauri
[toc] | [prev] | [next] | [standalone]
| From | Eric Sosman <esosman@ieee-dot-org.invalid> |
|---|---|
| Date | 2011-05-17 21:21 -0400 |
| Message-ID | <iqv6va$ulc$1@dont-email.me> |
| In reply to | #4178 |
On 5/17/2011 3:39 PM, Pushpa wrote:
> Hi all,
>
> Many users claim that malloc() and free() provided by the language
> library results in considerable
> amount of overhead in terms of calling the same.
Ah, yes, that well-known and reputable authority "many users."
Almost as reliable as "I've heard" and "some say," nearly as
infallible as "it seems." Well, I've heard that some of these
many users say it seems the Moon is made of green cheese, but even
so I'm not planning to head skyward with my Roquefort dressing
recipe.
If you can produce a claim that has substance, we'll perhaps
have something worth talking about. But if all you can come up
with is "many users" and "considerable," well, ...
> Can we really
> eliminate overhead by using
> used defined functions/user defined memory manager.
No. Since any memory manager, no matter by whom it is written
or with what design principles, will have some overhead, it follows
that the only way to eliminate overhead is to use no memory manager.
> Is there any
> significant advantage in providing user defined malloc() and free().
No. In fact, the attempt to do so yields undefined behavior.
Providing custom-written work-alikes for malloc() et al. may be
beneficial, *if* (1) you have information not available to the
implementors of the "stock" version(s), and (2) you are able to make
effective use of that extra information. Or, of course, if (3) you
are lots smarter than the stock implementors. (This is in fact a
possibility. I recall a vendor whose malloc() was essentially a
copy of the toy implementation in K&R. K&R's version was just fine
for the 64-256K memory spaces of the computers it ran on, but had
one or two teeny-tiny scalability problems with memories four
thousand times bigger...)
> I am talking about general purpose applications. With a user defined
> memory manager can it handle
> memory allocation and de-allocation in a much better way than what OS
> would have done. Any inputs/comments
> on the same are highly appreciated.
Content-free questions get content-free answers. Be more specific.
--
Eric Sosman
esosman@ieee-dot-org.invalid
[toc] | [prev] | [next] | [standalone]
| From | "io_x" <a@b.c.invalid> |
|---|---|
| Date | 2011-05-18 19:31 +0200 |
| Message-ID | <4dd4022b$0$38652$4fafbaef@reader1.news.tin.it> |
| In reply to | #4205 |
"Eric Sosman" <esosman@ieee-dot-org.invalid> ha scritto nel messaggio news:iqv6va$ulc$1@dont-email.me... > On 5/17/2011 3:39 PM, Pushpa wrote: >> Hi all, >> >> Many users claim that malloc() and free() provided by the language >> library results in considerable >> amount of overhead in terms of calling the same. > > Ah, yes, that well-known and reputable authority "many users." > Almost as reliable as "I've heard" and "some say," nearly as > infallible as "it seems." Well, I've heard that some of these > many users say it seems the Moon is made of green cheese, but even > so I'm not planning to head skyward with my Roquefort dressing > recipe. > > If you can produce a claim that has substance, we'll perhaps > have something worth talking about. But if all you can come up > with is "many users" and "considerable," well, ... > >> Can we really >> eliminate overhead by using >> used defined functions/user defined memory manager. > > No. Since any memory manager, no matter by whom it is written > or with what design principles, will have some overhead, it follows > that the only way to eliminate overhead is to use no memory manager. > >> Is there any >> significant advantage in providing user defined malloc() and free(). > > No. In fact, the attempt to do so yields undefined behavior. > > Providing custom-written work-alikes for malloc() et al. may be > beneficial, *if* (1) you have information not available to the > implementors of the "stock" version(s), and (2) you are able to make > effective use of that extra information. Or, of course, if (3) you > are lots smarter than the stock implementors. if someone is smart, or if someone is not smart it is good to write it own malloc - free routines > (This is in fact a > possibility. I recall a vendor whose malloc() was essentially a > copy of the toy implementation in K&R. I think i could not write from 0 the K&R one; but it is very good code for learn at last i think that; in the end of the day: i never see code better than that. > K&R's version was just fine > for the 64-256K memory spaces of the computers it ran on, but had > one or two teeny-tiny scalability problems with memories four > thousand times bigger...) 4000 bigger? can you show some example so i can stress my K&R modification of malloc()-free() [modified in the sense i see it as a doble list so malloc goes in one way the free in the other way in the little i remember ] in this day i had the hope to find one error in the realloc() implementation but the error was out that routine :( >> I am talking about general purpose applications. With a user defined >> memory manager can it handle >> memory allocation and de-allocation in a much better way than what OS >> would have done. Any inputs/comments >> on the same are highly appreciated. > > Content-free questions get content-free answers. Be more specific. i not claim my version of the "toy" K&R one is better than original or better than the os one it is just a toy i like > -- > Eric Sosman > esosman@ieee-dot-org.invalid
[toc] | [prev] | [next] | [standalone]
| From | Jorgen Grahn <grahn+nntp@snipabacken.se> |
|---|---|
| Date | 2011-05-18 21:36 +0000 |
| Message-ID | <slrnit8eu9.gtj.grahn+nntp@frailea.sa.invalid> |
| In reply to | #4205 |
On Wed, 2011-05-18, Eric Sosman wrote: > On 5/17/2011 3:39 PM, Pushpa wrote: ... >> I am talking about general purpose applications. With a user defined >> memory manager can it handle >> memory allocation and de-allocation in a much better way than what OS >> would have done. Any inputs/comments >> on the same are highly appreciated. > > Content-free questions get content-free answers. Be more specific. What's so content-free about that? At least the part you quote here is clearly stated, and the answer is of course "no, not unless the standard malloc implementation is really bad, which is unlikely". /Jorgen -- // Jorgen Grahn <grahn@ Oo o. . . \X/ snipabacken.se> O o .
[toc] | [prev] | [next] | [standalone]
| From | Eric Sosman <esosman@ieee-dot-org.invalid> |
|---|---|
| Date | 2011-05-18 21:26 -0400 |
| Message-ID | <ir1rkr$j4l$1@dont-email.me> |
| In reply to | #4248 |
On 5/18/2011 5:36 PM, Jorgen Grahn wrote:
> On Wed, 2011-05-18, Eric Sosman wrote:
>> On 5/17/2011 3:39 PM, Pushpa wrote:
> ...
>>> I am talking about general purpose applications. With a user defined
>>> memory manager can it handle
>>> memory allocation and de-allocation in a much better way than what OS
>>> would have done. Any inputs/comments
>>> on the same are highly appreciated.
>>
>> Content-free questions get content-free answers. Be more specific.
>
> What's so content-free about that? At least the part you quote here
> is clearly stated, and the answer is of course "no, not unless the
> standard malloc implementation is really bad, which is unlikely".
From the original post, I quote
>>> Many users claim that malloc() and free() provided by the language
>>> library results in considerable
>>> amount of overhead in terms of calling the same.
That's content-free, because both "considerable" and "overhead"
are left completely open to individual interpretation. One
respondent might say "I've measured the overhead, and it's only
twenty-six milliquivers per kilofrag," while another might say
"YES! I've measured the HORRIBLE overhead, and it's a full TWENTY-
SIX MILLIQUIVERS PER KILOFRAG, OMG!!!!" And both could be right.
A question for which both Yes and No are correct answers is,
in my book, content-free.
--
Eric Sosman
esosman@ieee-dot-org.invalid
[toc] | [prev] | [next] | [standalone]
| From | Tim Rentsch <txr@alumni.caltech.edu> |
|---|---|
| Date | 2011-05-19 02:07 -0700 |
| Message-ID | <kfnvcx7ukh3.fsf@x-alumni2.alumni.caltech.edu> |
| In reply to | #4205 |
Eric Sosman <esosman@ieee-dot-org.invalid> writes: > On 5/17/2011 3:39 PM, Pushpa wrote: >> Hi all, >> >> Many users claim that malloc() and free() provided by the language >> library results in considerable >> amount of overhead in terms of calling the same. > > Ah, yes, that well-known and reputable authority "many users." > Almost as reliable as "I've heard" and "some say," nearly as > infallible as "it seems." Well, I've heard that some of these > many users say it seems the Moon is made of green cheese, but even > so I'm not planning to head skyward with my Roquefort dressing > recipe. [snip] IMO this response is inappropriate. The statement starting "Many users claim..." wasn't offered for its truth, but just to establish a context for asking the question. And there's nothing wrong with that.
[toc] | [prev] | [next] | [standalone]
| From | Ian Collins <ian-news@hotmail.com> |
|---|---|
| Date | 2011-05-19 21:13 +1200 |
| Message-ID | <93k597F1bbU4@mid.individual.net> |
| In reply to | #4307 |
On 05/19/11 09:07 PM, Tim Rentsch wrote: > Eric Sosman<esosman@ieee-dot-org.invalid> writes: > >> On 5/17/2011 3:39 PM, Pushpa wrote: >>> Hi all, >>> >>> Many users claim that malloc() and free() provided by the language >>> library results in considerable >>> amount of overhead in terms of calling the same. >> >> Ah, yes, that well-known and reputable authority "many users." >> Almost as reliable as "I've heard" and "some say," nearly as >> infallible as "it seems." Well, I've heard that some of these >> many users say it seems the Moon is made of green cheese, but even >> so I'm not planning to head skyward with my Roquefort dressing >> recipe. [snip] > > IMO this response is inappropriate. Yes, it is. Roquefort is a blue cheese. -- Ian Collins
[toc] | [prev] | [next] | [standalone]
| From | Nobody <nobody@nowhere.com> |
|---|---|
| Date | 2011-05-18 07:08 +0100 |
| Message-ID | <pan.2011.05.18.06.07.21.765000@nowhere.com> |
| In reply to | #4178 |
On Tue, 17 May 2011 19:39:15 +0000, Pushpa wrote: > Many users claim that malloc() and free() provided by the language > library results in considerable amount of overhead in terms of calling > the same. Can we really eliminate overhead by using used defined > functions/user defined memory manager. Is there any significant > advantage in providing user defined malloc() and free(). I want to > exclude realtime systems or embedded systems which are memory > constrained here. > I am talking about general purpose applications. With a user defined > memory manager can it handle memory allocation and de-allocation in a > much better way than what OS would have done. Any inputs/comments > on the same are highly appreciated. First, malloc() isn't part of the OS; it's part of the standard C library. The application typically obtains memory from the OS as entire pages, via e.g. sbrk() or mmap(). The efficiency problems with malloc() don't lie with a specific implementation but with the interface. The application doesn't tell malloc() anything beyond the amount of memory required. A custom allocator can offer the application either more or less control, either of which may translate to improved performance. E.g. 1. An allocator for objects of a specific size may be much simpler (and thus faster) than a general-purpose allocator. The downside is that once the allocator has claimed memory for its heap, any memory returned to that heap may only be usable for subsequent allocations from that heap; fragmentation may make it impossible to return memory to the global heap or to the OS. 2. An allocator which allows the caller to provide additional information about the nature of the data may be able to allocate memory for "related" blocks in the same page, reducing the probability of cache misses at the cost of making allocation and deallocation slower. 3. If the application can predict the lifetime of the data in advance, the allocator can avoid allocating long-lived blocks around a short-lived block, reducing heap fragmentation. In the extreme case, the heap becomes a stack, with no fragmentation at all. 4. If the application can tolerate the data being moved after allocation, heap fragmentation can be eliminated.
[toc] | [prev] | [next] | [standalone]
| From | David Resnick <lndresnick@gmail.com> |
|---|---|
| Date | 2011-05-19 07:51 -0700 |
| Message-ID | <aa22e9b9-e17e-4300-bf2d-ba651588cf17@dn9g2000vbb.googlegroups.com> |
| In reply to | #4178 |
On May 17, 3:39 pm, Pushpa <pus...@nospam.invalid> wrote: > Hi all, > > Many users claim that malloc() and free() provided by the language > library results in considerable > amount of overhead in terms of calling the same. Can we really > eliminate overhead by using > used defined functions/user defined memory manager. Is there any > significant advantage in providing user defined malloc() and free(). I > want to exclude realtime systems or embedded systems which are memory > constrained here. > I am talking about general purpose applications. With a user defined > memory manager can it handle > memory allocation and de-allocation in a much better way than what OS > would have done. Any inputs/comments > on the same are highly appreciated. > Lots of suggestions in the thread that suggest that for general use you are unlikely to procure a better allocator than the standard one, which I'd agree with. Getting malloc and free exactly right (avoiding fragmentation, making it work and be reasonable in a multi-threaded environment if you care about that aspect, etc) is best left to the library absent any really compelling reason to the contrary. I've done some work on a system where profilers showed malloc/free were a very serious issue (receiving and processing 50000 packets per second, any dynamic allocations tended to show up rather conspicuously). However, rather than using a custom allocator, turned out object (memory) reuse was a better choice, a common way to attack this problem. As in, if a profiler says that malloc is actually causing you woe, doing less of it may well be a better solution than doing it faster. -David
[toc] | [prev] | [next] | [standalone]
| From | Joe Pfeiffer <pfeiffer@cs.nmsu.edu> |
|---|---|
| Date | 2011-05-19 09:12 -0600 |
| Message-ID | <1btycqrafr.fsf@snowball.wb.pfeifferfamily.net> |
| In reply to | #4326 |
David Resnick <lndresnick@gmail.com> writes: > On May 17, 3:39 pm, Pushpa <pus...@nospam.invalid> wrote: >> Hi all, >> >> Many users claim that malloc() and free() provided by the language >> library results in considerable >> amount of overhead in terms of calling the same. Can we really >> eliminate overhead by using >> used defined functions/user defined memory manager. Is there any >> significant advantage in providing user defined malloc() and free(). I >> want to exclude realtime systems or embedded systems which are memory >> constrained here. >> I am talking about general purpose applications. With a user defined >> memory manager can it handle >> memory allocation and de-allocation in a much better way than what OS >> would have done. Any inputs/comments >> on the same are highly appreciated. >> > > Lots of suggestions in the thread that suggest that for general use > you are unlikely to procure a better allocator than the standard one, > which I'd agree with. Getting malloc and free exactly right (avoiding > fragmentation, making it work and be reasonable in a multi-threaded > environment if you care about that aspect, etc) is best left to the > library absent any really compelling reason to the contrary. > > I've done some work on a system where profilers showed malloc/free > were a very serious issue (receiving and processing 50000 packets per > second, any dynamic allocations tended to show up rather > conspicuously). However, rather than using a custom allocator, > turned out object (memory) reuse was a better choice, a common way to > attack this problem. As in, if a profiler says that malloc is > actually causing you woe, doing less of it may well be a better > solution than doing it faster. My earlier anecdote is a case in point -- and one might say that in general it's very likely that the best way to do a "custom allocator" is really to do object reuse on top of malloc(). And when I say "very likely" I mean "there might be some pathological case where malloc() is available but this isn't the case , but I can't think of one off-hand". -- "It used to be that the USA was pretty good at producing stuff teenaged boys could lose a finger or two playing with." -- James Nicoll
[toc] | [prev] | [standalone]
Page 5 of 5 — ← Prev page 1 2 3 4 [5]
Back to top | Article view | comp.lang.c
csiph-web