Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.c > #163420 > unrolled thread
| Started by | Meredith Montgomery <mmontgomery@levado.to> |
|---|---|
| First post | 2021-11-12 15:48 -0300 |
| Last post | 2021-11-19 16:07 -0300 |
| Articles | 20 on this page of 75 — 20 participants |
Back to article view | Back to comp.lang.c
on why declare a struct with a single array in it Meredith Montgomery <mmontgomery@levado.to> - 2021-11-12 15:48 -0300
Re: on why declare a struct with a single array in it Guillaume <message@bottle.org> - 2021-11-12 19:56 +0100
Re: on why declare a struct with a single array in it Ben Bacarisse <ben.usenet@bsb.me.uk> - 2021-11-12 19:31 +0000
Re: on why declare a struct with a single array in it David Brown <david.brown@hesbynett.no> - 2021-11-12 21:07 +0100
Re: on why declare a struct with a single array in it pozz <pozzugno@gmail.com> - 2021-11-18 15:53 +0100
Re: on why declare a struct with a single array in it David Brown <david.brown@hesbynett.no> - 2021-11-18 16:13 +0100
Re: on why declare a struct with a single array in it Manfred <noname@add.invalid> - 2021-11-18 16:26 +0100
Re: on why declare a struct with a single array in it pozz <pozzugno@gmail.com> - 2021-11-18 17:22 +0100
Re: on why declare a struct with a single array in it Manfred <noname@add.invalid> - 2021-11-18 18:04 +0100
Re: on why declare a struct with a single array in it Meredith Montgomery <mmontgomery@levado.to> - 2021-11-19 16:00 -0300
Re: on why declare a struct with a single array in it David Brown <david.brown@hesbynett.no> - 2021-11-18 22:58 +0100
Re: on why declare a struct with a single array in it Guillaume <message@bottle.org> - 2021-11-19 17:53 +0100
Re: on why declare a struct with a single array in it Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2021-11-19 11:27 -0800
Re: on why declare a struct with a single array in it scott@slp53.sl.home (Scott Lurndal) - 2021-11-19 20:46 +0000
Re: on why declare a struct with a single array in it Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2021-11-19 12:52 -0800
Re: on why declare a struct with a single array in it David Brown <david.brown@hesbynett.no> - 2021-11-20 13:36 +0100
Re: on why declare a struct with a single array in it Bart <bc@freeuk.com> - 2021-11-20 14:49 +0000
Re: on why declare a struct with a single array in it Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2021-11-20 15:52 -0800
Re: on why declare a struct with a single array in it Bart <bc@freeuk.com> - 2021-11-21 00:19 +0000
Re: on why declare a struct with a single array in it Richard Damon <Richard@Damon-Family.org> - 2021-11-20 20:44 -0500
Re: on why declare a struct with a single array in it David Brown <david.brown@hesbynett.no> - 2021-11-21 14:51 +0100
Re: on why declare a struct with a single array in it Manfred <noname@add.invalid> - 2021-11-21 19:32 +0100
Re: on why declare a struct with a single array in it David Brown <david.brown@hesbynett.no> - 2021-11-21 21:51 +0100
Re: on why declare a struct with a single array in it Manfred <noname@add.invalid> - 2021-11-22 03:17 +0100
Re: on why declare a struct with a single array in it David Brown <david.brown@hesbynett.no> - 2021-11-22 11:31 +0100
Re: on why declare a struct with a single array in it Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2021-12-10 04:21 -0800
Re: on why declare a struct with a single array in it David Brown <david.brown@hesbynett.no> - 2021-12-10 14:43 +0100
Re: on why declare a struct with a single array in it Bart <bc@freeuk.com> - 2021-11-22 11:56 +0000
Re: on why declare a struct with a single array in it Tim Rentsch <tr.17687@z991.linuxsc.com> - 2021-12-10 01:47 -0800
Re: on why declare a struct with a single array in it Bart <bc@freeuk.com> - 2021-12-10 10:55 +0000
Re: on why declare a struct with a single array in it David Brown <david.brown@hesbynett.no> - 2021-12-10 14:51 +0100
Re: on why declare a struct with a single array in it Tim Rentsch <tr.17687@z991.linuxsc.com> - 2021-12-11 06:56 -0800
Re: on why declare a struct with a single array in it Bart <bc@freeuk.com> - 2021-12-11 16:04 +0000
Re: on why declare a struct with a single array in it Bart <bc@freeuk.com> - 2021-12-11 17:23 +0000
Re: on why declare a struct with a single array in it Tim Rentsch <tr.17687@z991.linuxsc.com> - 2022-01-21 05:30 -0800
Re: on why declare a struct with a single array in it Bart <bc@freeuk.com> - 2022-01-21 13:49 +0000
Re: on why declare a struct with a single array in it "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2021-12-11 12:17 -0800
Re: on why declare a struct with a single array in it gazelle@shell.xmission.com (Kenny McCormack) - 2021-12-12 13:00 +0000
Re: on why declare a struct with a single array in it Tim Rentsch <tr.17687@z991.linuxsc.com> - 2021-12-12 08:25 -0800
Re: on why declare a struct with a single array in it gazelle@shell.xmission.com (Kenny McCormack) - 2021-12-12 17:38 +0000
Re: on why declare a struct with a single array in it Dick <thiebauddick2@aol.com> - 2021-12-12 16:19 -0500
Re: on why declare a struct with a single array in it gazelle@shell.xmission.com (Kenny McCormack) - 2021-12-13 08:06 +0000
Re: on why declare a struct with a single array in it Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2021-12-13 01:30 -0800
Re: on why declare a struct with a single array in it David Brown <david.brown@hesbynett.no> - 2021-12-13 09:34 +0100
Re: on why declare a struct with a single array in it Robert Latest <boblatest@yahoo.com> - 2021-12-13 18:53 +0000
Re: on why declare a struct with a single array in it gazelle@shell.xmission.com (Kenny McCormack) - 2021-12-13 20:38 +0000
Re: on why declare a struct with a single array in it Tim Rentsch <tr.17687@z991.linuxsc.com> - 2021-12-15 00:38 -0800
Re: on why declare a struct with a single array in it gazelle@shell.xmission.com (Kenny McCormack) - 2021-12-15 11:21 +0000
Re: on why declare a struct with a single array in it Tim Rentsch <tr.17687@z991.linuxsc.com> - 2021-12-15 10:26 -0800
Re: on why declare a struct with a single array in it Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2021-12-15 10:45 -0800
Re: on why declare a struct with a single array in it Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2021-12-10 10:52 -0800
Re: on why declare a struct with a single array in it Guillaume <message@bottle.org> - 2021-11-22 04:06 +0100
Re: on why declare a struct with a single array in it Bart <bc@freeuk.com> - 2021-11-22 11:32 +0000
Re: on why declare a struct with a single array in it Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2021-11-22 13:00 -0800
Re: on why declare a struct with a single array in it "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2021-11-21 13:08 -0800
Re: on why declare a struct with a single array in it Richard Damon <Richard@Damon-Family.org> - 2021-11-20 20:44 -0500
Re: on why declare a struct with a single array in it Bart <bc@freeuk.com> - 2021-11-21 12:22 +0000
Re: on why declare a struct with a single array in it Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2021-11-20 20:13 -0700
Re: on why declare a struct with a single array in it Bart <bc@freeuk.com> - 2021-11-21 12:35 +0000
Re: on why declare a struct with a single array in it Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2021-11-21 10:53 -0700
Re: on why declare a struct with a single array in it Bart <bc@freeuk.com> - 2021-11-21 18:49 +0000
Re: on why declare a struct with a single array in it "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2021-11-21 13:04 -0800
Re: on why declare a struct with a single array in it Bart <bc@freeuk.com> - 2021-11-21 14:42 +0000
Re: on why declare a struct with a single array in it Guillaume <message@bottle.org> - 2021-11-21 01:34 +0100
Re: on why declare a struct with a single array in it Bart <bc@freeuk.com> - 2021-11-21 00:48 +0000
Re: on why declare a struct with a single array in it "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2021-11-20 16:58 -0800
Re: on why declare a struct with a single array in it David Brown <david.brown@hesbynett.no> - 2021-11-21 14:40 +0100
Re: on why declare a struct with a single array in it "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2021-11-21 12:59 -0800
Re: on why declare a struct with a single array in it scott@slp53.sl.home (Scott Lurndal) - 2021-11-21 15:58 +0000
Re: on why declare a struct with a single array in it gazelle@shell.xmission.com (Kenny McCormack) - 2021-11-20 10:55 +0000
Re: on why declare a struct with a single array in it Jorgen Grahn <grahn+nntp@snipabacken.se> - 2021-11-13 08:47 +0000
Re: on why declare a struct with a single array in it Guillaume <message@bottle.org> - 2021-11-13 18:23 +0100
Re: on why declare a struct with a single array in it Kaz Kylheku <480-992-1380@kylheku.com> - 2021-11-13 17:55 +0000
Re: on why declare a struct with a single array in it Anton Shepelev <anton.txt@g{oogle}mail.com> - 2021-11-19 13:14 +0300
Re: on why declare a struct with a single array in it Meredith Montgomery <mmontgomery@levado.to> - 2021-11-19 16:07 -0300
Page 2 of 4 — ← Prev page 1 [2] 3 4 Next page →
| From | David Brown <david.brown@hesbynett.no> |
|---|---|
| Date | 2021-11-21 14:51 +0100 |
| Message-ID | <snditr$7l1$1@dont-email.me> |
| In reply to | #163538 |
On 21/11/2021 02:44, Richard Damon wrote: > On 11/20/21 7:19 PM, Bart wrote: >> It doesn't make sense. I don't want to make a habit of it, but I agree with Bart here :-) > > Maybe not to you, but I would sure hate to have to keep track of all > that data, and would likely just wrap this dumb malloc with something > smarter. And that is /fine/. For those few cases where it might be inconvenient to know the size at deallocation time, make a wrapper so that the size is tracked. If you need that functionality, write the code and pay the price - others that don't need the heap to record their allocation sizes shouldn't have to pay for your needs. It is no different from malloc and calloc - if you want the convenience of zeroing the memory, you can use calloc, but if you don't want it you can use malloc and not pay for the extra effort. Of course it is too late to make such untracked-size heap functions the standard for C - I see it as an unlucky choice made long ago. After all, if you had "malloc_untracked" and "free_untracked" then it would be a simple matter to implement traditional C "malloc" and "free" on top of it. > > If you need a lot of a fixed sized blocks, there are lots of algorithms > that work well for that with much less or zero overhead (if the blocks > are big enough for a pointer) Yes, of course there are many ways of efficiently handling fixed size blocks. But it would be more convenient if the common heap system didn't have this particular inefficiency.
[toc] | [prev] | [next] | [standalone]
| From | Manfred <noname@add.invalid> |
|---|---|
| Date | 2021-11-21 19:32 +0100 |
| Message-ID | <sne3b7$1mre$1@gioia.aioe.org> |
| In reply to | #163547 |
On 11/21/2021 2:51 PM, David Brown wrote: > On 21/11/2021 02:44, Richard Damon wrote: >> On 11/20/21 7:19 PM, Bart wrote: > >>> It doesn't make sense. > > I don't want to make a habit of it, but I agree with Bart here :-) > >> >> Maybe not to you, but I would sure hate to have to keep track of all >> that data, and would likely just wrap this dumb malloc with something >> smarter. > > And that is /fine/. For those few cases where it might be inconvenient > to know the size at deallocation time, make a wrapper so that the size > is tracked. If you need that functionality, write the code and pay the > price - others that don't need the heap to record their allocation sizes > shouldn't have to pay for your needs. It is no different from malloc > and calloc - if you want the convenience of zeroing the memory, you can > use calloc, but if you don't want it you can use malloc and not pay for > the extra effort. > > Of course it is too late to make such untracked-size heap functions the > standard for C - I see it as an unlucky choice made long ago. After > all, if you had "malloc_untracked" and "free_untracked" then it would be > a simple matter to implement traditional C "malloc" and "free" on top of it. The problem I see with this is that it assumes one and only type of metadata information to be associated to the pointer, specifically the block size. I believe there are countless implementations of malloc out there, and I guess this might not be the case, or it may be the case today but not tomorrow. The point is that if you set the interface to be pointer+size you also put constraints on the implementation, on top of the extra burden for the programmer. > >> >> If you need a lot of a fixed sized blocks, there are lots of algorithms >> that work well for that with much less or zero overhead (if the blocks >> are big enough for a pointer) > > Yes, of course there are many ways of efficiently handling fixed size > blocks. But it would be more convenient if the common heap system > didn't have this particular inefficiency. >
[toc] | [prev] | [next] | [standalone]
| From | David Brown <david.brown@hesbynett.no> |
|---|---|
| Date | 2021-11-21 21:51 +0100 |
| Message-ID | <snebg4$jtk$1@dont-email.me> |
| In reply to | #163552 |
On 21/11/2021 19:32, Manfred wrote: > On 11/21/2021 2:51 PM, David Brown wrote: >> On 21/11/2021 02:44, Richard Damon wrote: >>> On 11/20/21 7:19 PM, Bart wrote: >> >>>> It doesn't make sense. >> >> I don't want to make a habit of it, but I agree with Bart here :-) >> >>> >>> Maybe not to you, but I would sure hate to have to keep track of all >>> that data, and would likely just wrap this dumb malloc with something >>> smarter. >> >> And that is /fine/. For those few cases where it might be inconvenient >> to know the size at deallocation time, make a wrapper so that the size >> is tracked. If you need that functionality, write the code and pay the >> price - others that don't need the heap to record their allocation sizes >> shouldn't have to pay for your needs. It is no different from malloc >> and calloc - if you want the convenience of zeroing the memory, you can >> use calloc, but if you don't want it you can use malloc and not pay for >> the extra effort. >> >> Of course it is too late to make such untracked-size heap functions the >> standard for C - I see it as an unlucky choice made long ago. After >> all, if you had "malloc_untracked" and "free_untracked" then it would be >> a simple matter to implement traditional C "malloc" and "free" on top >> of it. > > The problem I see with this is that it assumes one and only type of > metadata information to be associated to the pointer, specifically the > block size. No. It merely relieves the heap implementation from having to store the size of the allocation. Any other metadata (such as for tracking free space lists, or handling efficient multi-threaded access) will still be needed. > I believe there are countless implementations of malloc out there, and I > guess this might not be the case, or it may be the case today but not > tomorrow. > The point is that if you set the interface to be pointer+size you also > put constraints on the implementation, on top of the extra burden for > the programmer. > How could this suggestion impose any constraints on the implementation? The interface for getting the memory is the same (albeit with a different name, in order to keep compatibility with existing code), and the interface for freeing memory gives the implementation more information. It would be perfectly possible (though a little unhelpful) for an implementation of the "_untracked" functions to be : #define malloc_untracked(size) malloc(size) #define free_untracked(ptr, size) free(ptr) An implementation can do better, given this new information, but there are no new limitations or restrictions.
[toc] | [prev] | [next] | [standalone]
| From | Manfred <noname@add.invalid> |
|---|---|
| Date | 2021-11-22 03:17 +0100 |
| Message-ID | <sneuje$1qm$1@gioia.aioe.org> |
| In reply to | #163554 |
On 11/21/2021 9:51 PM, David Brown wrote: > On 21/11/2021 19:32, Manfred wrote: >> On 11/21/2021 2:51 PM, David Brown wrote: >>> On 21/11/2021 02:44, Richard Damon wrote: >>>> On 11/20/21 7:19 PM, Bart wrote: >>> >>>>> It doesn't make sense. >>> >>> I don't want to make a habit of it, but I agree with Bart here :-) >>> >>>> >>>> Maybe not to you, but I would sure hate to have to keep track of all >>>> that data, and would likely just wrap this dumb malloc with something >>>> smarter. >>> >>> And that is /fine/. For those few cases where it might be inconvenient >>> to know the size at deallocation time, make a wrapper so that the size >>> is tracked. If you need that functionality, write the code and pay the >>> price - others that don't need the heap to record their allocation sizes >>> shouldn't have to pay for your needs. It is no different from malloc >>> and calloc - if you want the convenience of zeroing the memory, you can >>> use calloc, but if you don't want it you can use malloc and not pay for >>> the extra effort. >>> >>> Of course it is too late to make such untracked-size heap functions the >>> standard for C - I see it as an unlucky choice made long ago. After >>> all, if you had "malloc_untracked" and "free_untracked" then it would be >>> a simple matter to implement traditional C "malloc" and "free" on top >>> of it. >> >> The problem I see with this is that it assumes one and only type of >> metadata information to be associated to the pointer, specifically the >> block size. > > No. It merely relieves the heap implementation from having to store the > size of the allocation. Any other metadata (such as for tracking free > space lists, or handling efficient multi-threaded access) will still be > needed. But it means that the implementation has to manage memory blocks based on size instead of something else - if it had to store anything else with the pointer, delegating the size to the programmer would be pointless One alternative that pops to mind is a pointer to the next block, however this might work, or nothing at all and a lookup table, or anything else I couldn't think of right now, since I am not designing the allocator, which the committee does not design either. I think it makes sense that the language defines the interface strictly to fulfill the requirements, and keep as much as possible of the design internal to the implementation. Moreover, the requested size (used by the user) may be different from the actual size (used by the allocator), which poses some consistency issues - but that's a side note. > >> I believe there are countless implementations of malloc out there, and I >> guess this might not be the case, or it may be the case today but not >> tomorrow. >> The point is that if you set the interface to be pointer+size you also >> put constraints on the implementation, on top of the extra burden for >> the programmer. >> > How could this suggestion impose any constraints on the implementation? > The interface for getting the memory is the same (albeit with a > different name, in order to keep compatibility with existing code), and > the interface for freeing memory gives the implementation more > information. It would be perfectly possible (though a little unhelpful) > for an implementation of the "_untracked" functions to be : > > #define malloc_untracked(size) malloc(size) > #define free_untracked(ptr, size) free(ptr) > > An implementation can do better, given this new information, but there > are no new limitations or restrictions. > Well, requiring the programmer to keep track of the block size is a significant programming overhead - if it is unused, it's a significant waste I'd rather not pay for.
[toc] | [prev] | [next] | [standalone]
| From | David Brown <david.brown@hesbynett.no> |
|---|---|
| Date | 2021-11-22 11:31 +0100 |
| Message-ID | <snfrib$uje$1@dont-email.me> |
| In reply to | #163558 |
On 22/11/2021 03:17, Manfred wrote: > On 11/21/2021 9:51 PM, David Brown wrote: >> On 21/11/2021 19:32, Manfred wrote: >>> On 11/21/2021 2:51 PM, David Brown wrote: >>>> On 21/11/2021 02:44, Richard Damon wrote: >>>>> On 11/20/21 7:19 PM, Bart wrote: >>>> >>>>>> It doesn't make sense. >>>> >>>> I don't want to make a habit of it, but I agree with Bart here :-) >>>> >>>>> >>>>> Maybe not to you, but I would sure hate to have to keep track of all >>>>> that data, and would likely just wrap this dumb malloc with something >>>>> smarter. >>>> >>>> And that is /fine/. For those few cases where it might be inconvenient >>>> to know the size at deallocation time, make a wrapper so that the size >>>> is tracked. If you need that functionality, write the code and pay the >>>> price - others that don't need the heap to record their allocation >>>> sizes >>>> shouldn't have to pay for your needs. It is no different from malloc >>>> and calloc - if you want the convenience of zeroing the memory, you can >>>> use calloc, but if you don't want it you can use malloc and not pay for >>>> the extra effort. >>>> >>>> Of course it is too late to make such untracked-size heap functions the >>>> standard for C - I see it as an unlucky choice made long ago. After >>>> all, if you had "malloc_untracked" and "free_untracked" then it >>>> would be >>>> a simple matter to implement traditional C "malloc" and "free" on top >>>> of it. >>> >>> The problem I see with this is that it assumes one and only type of >>> metadata information to be associated to the pointer, specifically the >>> block size. >> >> No. It merely relieves the heap implementation from having to store the >> size of the allocation. Any other metadata (such as for tracking free >> space lists, or handling efficient multi-threaded access) will still be >> needed. > > But it means that the implementation has to manage memory blocks based > on size instead of something else - if it had to store anything else > with the pointer, delegating the size to the programmer would be pointless No, it does /not/ mean the implementation has to use the size. It means it /can/ use the size - not that it has to! And as I have said several times, and no one has (yet) given a convincing counter-argument, the user code almost always knows the size of the allocation already when it calls free. After all, it knows the size when it allocates the memory, and it knows the size when it /uses/ the memory, so it's a tiny or non-existent cost to know the size when it frees the memory. > > Moreover, the requested size (used by the user) may be different from > the actual size (used by the allocator), which poses some consistency > issues - but that's a side note. The "free_untracked" implementation would match the "malloc_untracked" algorithm - any changes made to the requested size (such as rounding up to an allocation unit size) would be known to the "free" routine. > >> >>> I believe there are countless implementations of malloc out there, and I >>> guess this might not be the case, or it may be the case today but not >>> tomorrow. >>> The point is that if you set the interface to be pointer+size you also >>> put constraints on the implementation, on top of the extra burden for >>> the programmer. >>> >> How could this suggestion impose any constraints on the implementation? >> The interface for getting the memory is the same (albeit with a >> different name, in order to keep compatibility with existing code), and >> the interface for freeing memory gives the implementation more >> information. It would be perfectly possible (though a little unhelpful) >> for an implementation of the "_untracked" functions to be : >> >> #define malloc_untracked(size) malloc(size) >> #define free_untracked(ptr, size) free(ptr) >> >> An implementation can do better, given this new information, but there >> are no new limitations or restrictions. >> > > Well, requiring the programmer to keep track of the block size is a > significant programming overhead - if it is unused, it's a significant > waste I'd rather not pay for. Does this mean you understand why this idea does not impose any constraints on how the heap is implemented? And can you give examples of where you have a pointer to a block of memory but do not know the size of the memory?
[toc] | [prev] | [next] | [standalone]
| From | Malcolm McLean <malcolm.arthur.mclean@gmail.com> |
|---|---|
| Date | 2021-12-10 04:21 -0800 |
| Message-ID | <11f9344a-7d81-49c3-be8e-2e7c1b46468dn@googlegroups.com> |
| In reply to | #163564 |
On Monday, 22 November 2021 at 10:31:52 UTC, David Brown wrote: > > And can you give examples of where you have a pointer to a block of > memory but do not know the size of the memory? > The only example I can think of is where you are passed a compressed data stream, and the "end of data" marker is retrieved as part of the decompression. However normally you would pass about a "memory size" parameter anyway, just to guard against truncated data.
[toc] | [prev] | [next] | [standalone]
| From | David Brown <david.brown@hesbynett.no> |
|---|---|
| Date | 2021-12-10 14:43 +0100 |
| Message-ID | <sovlhn$fbq$2@dont-email.me> |
| In reply to | #163765 |
On 10/12/2021 13:21, Malcolm McLean wrote: > On Monday, 22 November 2021 at 10:31:52 UTC, David Brown wrote: >> >> And can you give examples of where you have a pointer to a block of >> memory but do not know the size of the memory? >> > The only example I can think of is where you are passed a compressed > data stream, and the "end of data" marker is retrieved as part of the > decompression. Fair enough - that's a reasonable example. > However normally you would pass about a "memory size" parameter > anyway, just to guard against truncated data. > Indeed. And whichever part of the code is responsible for allocating the memory, knows the size of the buffer. I think it is not unreasonable for this size to be communicated to whatever part of the code is responsible for deallocating the memory (which may or may not be the bit that does the decompressing).
[toc] | [prev] | [next] | [standalone]
| From | Bart <bc@freeuk.com> |
|---|---|
| Date | 2021-11-22 11:56 +0000 |
| Message-ID | <sng0gm$kiq$1@dont-email.me> |
| In reply to | #163558 |
On 22/11/2021 02:17, Manfred wrote:
> On 11/21/2021 9:51 PM, David Brown wrote:
>> An implementation can do better, given this new information, but there
>> are no new limitations or restrictions.
>>
>
> Well, requiring the programmer to keep track of the block size is a
> significant programming overhead
Why is it a significant overhead for user-code, but not for the malloc
implementation?!
In any case, I've already mentioned some situations where there is
pretty much zero extra overhead for the programmer:
* When allocating a fixed sized struct or other fixed size type
* When allocating a block where the programmers needs to record the size
anyway
Or the programmer can determine the size:
* When allocating a zero-terminated string (they can use strlen if not
recording the size)
> - if it is unused, it's a significant waste I'd rather not pay for.
Exactly - it is an overhead of malloc that I'd rather not pay for either!
Just so programmers can do this:
p[i] = malloc(rand()); // or some varying size
....
free(p[i]);
I see that as a luxury in a lower-level language.
And also, goes against the 'spirit' of C where you are supposed to have
precise control over such things.
Potentially doubling (or sometimes quadrupling) heap memory usage for no
good reason sort of goes against that.
[toc] | [prev] | [next] | [standalone]
| From | Tim Rentsch <tr.17687@z991.linuxsc.com> |
|---|---|
| Date | 2021-12-10 01:47 -0800 |
| Message-ID | <86pmq47pvb.fsf@linuxsc.com> |
| In reply to | #163558 |
Manfred <noname@add.invalid> writes: > On 11/21/2021 9:51 PM, David Brown wrote: > >> On 21/11/2021 19:32, Manfred wrote: >> >>> On 11/21/2021 2:51 PM, David Brown wrote: >>> >>>> On 21/11/2021 02:44, Richard Damon wrote: >>>> >>>>> On 11/20/21 7:19 PM, Bart wrote: >>>>> >>>>>> It doesn't make sense. >>>> >>>> I don't want to make a habit of it, but I agree with Bart here :-) >>>> >>>>> Maybe not to you, but I would sure hate to have to keep track of all >>>>> that data, and would likely just wrap this dumb malloc with something >>>>> smarter. >>>> >>>> And that is /fine/. For those few cases where it might be inconvenient >>>> to know the size at deallocation time, make a wrapper so that the size >>>> is tracked. If you need that functionality, write the code and pay the >>>> price - others that don't need the heap to record their allocation sizes >>>> shouldn't have to pay for your needs. It is no different from malloc >>>> and calloc - if you want the convenience of zeroing the memory, you can >>>> use calloc, but if you don't want it you can use malloc and not pay for >>>> the extra effort. >>>> >>>> Of course it is too late to make such untracked-size heap functions the >>>> standard for C - I see it as an unlucky choice made long ago. After >>>> all, if you had "malloc_untracked" and "free_untracked" then it would be >>>> a simple matter to implement traditional C "malloc" and "free" on top >>>> of it. >>> >>> The problem I see with this is that it assumes one and only type of >>> metadata information to be associated to the pointer, specifically the >>> block size. >> >> No. It merely relieves the heap implementation from having to store the >> size of the allocation. Any other metadata (such as for tracking free >> space lists, or handling efficient multi-threaded access) will still be >> needed. > > But it means that the implementation has to manage memory blocks based > on size instead of something else - if it had to store anything else > with the pointer, delegating the size to the programmer would be > pointless > One alternative that pops to mind is a pointer to the next block, > however this might work, or nothing at all and a lookup table, or > anything else I couldn't think of right now, since I am not designing > the allocator, which the committee does not design either. > I think it makes sense that the language defines the interface > strictly to fulfill the requirements, and keep as much as possible of > the design internal to the implementation. > > Moreover, the requested size (used by the user) may be different from > the actual size (used by the allocator), which poses some consistency > issues - but that's a side note. > >>> I believe there are countless implementations of malloc out there, and I >>> guess this might not be the case, or it may be the case today but not >>> tomorrow. >>> The point is that if you set the interface to be pointer+size you also >>> put constraints on the implementation, on top of the extra burden for >>> the programmer. >> >> How could this suggestion impose any constraints on the implementation? >> The interface for getting the memory is the same (albeit with a >> different name, in order to keep compatibility with existing code), and >> the interface for freeing memory gives the implementation more >> information. It would be perfectly possible (though a little unhelpful) >> for an implementation of the "_untracked" functions to be : >> >> #define malloc_untracked(size) malloc(size) >> #define free_untracked(ptr, size) free(ptr) >> >> An implementation can do better, given this new information, but there >> are no new limitations or restrictions. > > Well, requiring the programmer to keep track of the block size is a > significant programming overhead - if it is unused, it's a significant > waste I'd rather not pay for. Anyone who seriously proposes having to supply a size argument to free memory for a general purpose allocator like malloc() either hasn't bothered to think or doesn't know how to think. Kind of par for the course for the long-running Bart and David show.
[toc] | [prev] | [next] | [standalone]
| From | Bart <bc@freeuk.com> |
|---|---|
| Date | 2021-12-10 10:55 +0000 |
| Message-ID | <sovbmf$b13$1@dont-email.me> |
| In reply to | #163761 |
On 10/12/2021 09:47, Tim Rentsch wrote: > Manfred <noname@add.invalid> writes: > >> On 11/21/2021 9:51 PM, David Brown wrote: >>> An implementation can do better, given this new information, but there >>> are no new limitations or restrictions. >> >> Well, requiring the programmer to keep track of the block size is a >> significant programming overhead - if it is unused, it's a significant >> waste I'd rather not pay for. > > Anyone who seriously proposes having to supply a size argument to > free memory for a general purpose allocator like malloc() either > hasn't bothered to think or doesn't know how to think. Kind of > par for the course for the long-running Bart and David show. Well, that is your opinion. You haven't really given any of the advantages of having this extra overhead in the allocator that a program may not want or need for any or all of its allocations. For example, some programs may never need to free memory until they terminate.
[toc] | [prev] | [next] | [standalone]
| From | David Brown <david.brown@hesbynett.no> |
|---|---|
| Date | 2021-12-10 14:51 +0100 |
| Message-ID | <sovm1b$k3i$1@dont-email.me> |
| In reply to | #163763 |
On 10/12/2021 11:55, Bart wrote: > On 10/12/2021 09:47, Tim Rentsch wrote: >> Manfred <noname@add.invalid> writes: >> >>> On 11/21/2021 9:51 PM, David Brown wrote: > >>>> An implementation can do better, given this new information, but there >>>> are no new limitations or restrictions. >>> >>> Well, requiring the programmer to keep track of the block size is a >>> significant programming overhead - if it is unused, it's a significant >>> waste I'd rather not pay for. >> >> Anyone who seriously proposes having to supply a size argument to >> free memory for a general purpose allocator like malloc() either >> hasn't bothered to think or doesn't know how to think. Kind of >> par for the course for the long-running Bart and David show. > > Well, that is your opinion. You haven't really given any of the > advantages of having this extra overhead in the allocator that a program > may not want or need for any or all of its allocations. Remember, this is Tim posting. It's just another drive-by shot-the-messenger (and perhaps innocent bystanders) post, coming along weeks after most people have forgotten about the thread. If you are lucky, you'll get a reply containing nothing but sarcasm, cryptic messages, and patronising insults sometime next year. He used to post interesting things, and had insights into details of C and the standards that most others missed. Maybe he still could, but he sees to prefer to play the grumpy old man and wave his Zimmer frame at people who disturb his slumber. (And yes, I know he'll ignore this post. He has burned that bridge, and many others, long ago.) > > For example, some programs may never need to free memory until they > terminate. > >
[toc] | [prev] | [next] | [standalone]
| From | Tim Rentsch <tr.17687@z991.linuxsc.com> |
|---|---|
| Date | 2021-12-11 06:56 -0800 |
| Message-ID | <86h7bf6vhd.fsf@linuxsc.com> |
| In reply to | #163763 |
Bart <bc@freeuk.com> writes: > On 10/12/2021 09:47, Tim Rentsch wrote: > >> Manfred <noname@add.invalid> writes: >> >>> On 11/21/2021 9:51 PM, David Brown wrote: >>> >>>> An implementation can do better, given this new information, but there >>>> are no new limitations or restrictions. >>> >>> Well, requiring the programmer to keep track of the block size is a >>> significant programming overhead - if it is unused, it's a significant >>> waste I'd rather not pay for. >> >> Anyone who seriously proposes having to supply a size argument to >> free memory for a general purpose allocator like malloc() either >> hasn't bothered to think or doesn't know how to think. Kind of >> par for the course for the long-running Bart and David show. > > Well, that is your opinion. More accurately, it is my belief. > You haven't really given any of the > advantages of having this extra overhead in the allocator that a > program may not want or need for any or all of its allocations. In point of fact, I haven't said anything at all about what is needed to implement either kind of allocator interface. Why don't you write and post code that implements the sort of interface you are advocating, so there can be meaningful discussion about it?
[toc] | [prev] | [next] | [standalone]
| From | Bart <bc@freeuk.com> |
|---|---|
| Date | 2021-12-11 16:04 +0000 |
| Message-ID | <sp2i63$qh3$1@dont-email.me> |
| In reply to | #163786 |
On 11/12/2021 14:56, Tim Rentsch wrote: > Bart <bc@freeuk.com> writes: > >> On 10/12/2021 09:47, Tim Rentsch wrote: >> >>> Manfred <noname@add.invalid> writes: >>> >>>> On 11/21/2021 9:51 PM, David Brown wrote: >>>> >>>>> An implementation can do better, given this new information, but there >>>>> are no new limitations or restrictions. >>>> >>>> Well, requiring the programmer to keep track of the block size is a >>>> significant programming overhead - if it is unused, it's a significant >>>> waste I'd rather not pay for. >>> >>> Anyone who seriously proposes having to supply a size argument to >>> free memory for a general purpose allocator like malloc() either >>> hasn't bothered to think or doesn't know how to think. Kind of >>> par for the course for the long-running Bart and David show. >> >> Well, that is your opinion. > > More accurately, it is my belief. > >> You haven't really given any of the >> advantages of having this extra overhead in the allocator that a >> program may not want or need for any or all of its allocations. > > In point of fact, I haven't said anything at all about what is > needed to implement either kind of allocator interface. > > Why don't you write and post code that implements the sort > of interface you are advocating, so there can be meaningful > discussion about it? It's a big job, and there are many ways of doing it, so I'm not going to attempt it here. Basically, instead of free(p), you might typically write newfree(p, size(*p)) instead, when p points to an object where the compiler knows the size (eg. to a struct). Otherwise you need to keep track of it. The only such implementation I have is part of my language's runtime, and was optimised for interpreters. It uses special allocators for objects up to 2KB, using pools obtained with malloc, and calls malloc directly (but with sizes pre-rounded to my own prefered limits) for larger blocks. And it is not in C. If I use a generated-C version of it and apply it to the Binary Trees benchmark, then it is 4 times as fast as using plain malloc/free using Windows/msvcrt, and about 2.5 times as fast under WSL. Regular version using malloc+free: https://github.com/sal55/langs/blob/master/binary.c Version using my allocator for tree nodes (note lots of ugly generated C code; this is not up for discussion, only the results): https://github.com/sal55/langs/blob/master/binary2.c (Benchmark code starts line 194; it differs on two lines from the other version: using pc_alloc for malloc, and pc_free for free.)
[toc] | [prev] | [next] | [standalone]
| From | Bart <bc@freeuk.com> |
|---|---|
| Date | 2021-12-11 17:23 +0000 |
| Message-ID | <sp2mqu$ltt$1@dont-email.me> |
| In reply to | #163787 |
On 11/12/2021 16:04, Bart wrote: > On 11/12/2021 14:56, Tim Rentsch wrote: >> Bart <bc@freeuk.com> writes: >> >>> On 10/12/2021 09:47, Tim Rentsch wrote: >>> >>>> Manfred <noname@add.invalid> writes: >>>> >>>>> On 11/21/2021 9:51 PM, David Brown wrote: >>>>> >>>>>> An implementation can do better, given this new information, but >>>>>> there >>>>>> are no new limitations or restrictions. >>>>> >>>>> Well, requiring the programmer to keep track of the block size is a >>>>> significant programming overhead - if it is unused, it's a significant >>>>> waste I'd rather not pay for. >>>> >>>> Anyone who seriously proposes having to supply a size argument to >>>> free memory for a general purpose allocator like malloc() either >>>> hasn't bothered to think or doesn't know how to think. Kind of >>>> par for the course for the long-running Bart and David show. >>> >>> Well, that is your opinion. >> >> More accurately, it is my belief. >> >>> You haven't really given any of the >>> advantages of having this extra overhead in the allocator that a >>> program may not want or need for any or all of its allocations. >> >> In point of fact, I haven't said anything at all about what is >> needed to implement either kind of allocator interface. >> >> Why don't you write and post code that implements the sort >> of interface you are advocating, so there can be meaningful >> discussion about it? > > It's a big job, and there are many ways of doing it, so I'm not going to > attempt it here. > > Basically, instead of free(p), you might typically write newfree(p, > size(*p)) instead, when p points to an object where the compiler knows > the size (eg. to a struct). Otherwise you need to keep track of it. > > The only such implementation I have is part of my language's runtime, > and was optimised for interpreters. It uses special allocators for > objects up to 2KB, using pools obtained with malloc, and calls malloc > directly (but with sizes pre-rounded to my own prefered limits) for > larger blocks. > > And it is not in C. If I use a generated-C version of it and apply it to > the Binary Trees benchmark, then it is 4 times as fast as using plain > malloc/free using Windows/msvcrt, and about 2.5 times as fast under WSL. > > > Regular version using malloc+free: > > https://github.com/sal55/langs/blob/master/binary.c > > Version using my allocator for tree nodes (note lots of ugly generated C > code; this is not up for discussion, only the results): > > https://github.com/sal55/langs/blob/master/binary2.c [Note needs 64-bit target] > (Benchmark code starts line 194; it differs on two lines from the other > version: using pc_alloc for malloc, and pc_free for free.) Note that for this benchmark, the size of the 'treenode' struct, 20 bytes [on x64] is rounded up to 32 bytes for both allocators (in the case of malloc, the extra space is likely used for the size as well as for alignment). So both use the same amount of memory. If I increase the struct to 32 bytes in the user program, then malloc will I think use 48 bytes per allocation; pc_alloc still uses 32 bytes. On Windows that doesn't make much difference (I don't know the maximum memory in use, maybe it's not that high). On WSL however, it makes the malloc-based version 20% slower, so taking 3 times as long as the pc_alloc version.
[toc] | [prev] | [next] | [standalone]
| From | Tim Rentsch <tr.17687@z991.linuxsc.com> |
|---|---|
| Date | 2022-01-21 05:30 -0800 |
| Message-ID | <86zgnpp665.fsf@linuxsc.com> |
| In reply to | #163787 |
Bart <bc@freeuk.com> writes: > On 11/12/2021 14:56, Tim Rentsch wrote: > >> Bart <bc@freeuk.com> writes: >> >>> On 10/12/2021 09:47, Tim Rentsch wrote: >>> >>>> Manfred <noname@add.invalid> writes: >>>> >>>>> On 11/21/2021 9:51 PM, David Brown wrote: >>>>> >>>>>> An implementation can do better, given this new information, but there >>>>>> are no new limitations or restrictions. >>>>> >>>>> Well, requiring the programmer to keep track of the block size is a >>>>> significant programming overhead - if it is unused, it's a significant >>>>> waste I'd rather not pay for. >>>> >>>> Anyone who seriously proposes having to supply a size argument to >>>> free memory for a general purpose allocator like malloc() either >>>> hasn't bothered to think or doesn't know how to think. Kind of >>>> par for the course for the long-running Bart and David show. >>> >>> Well, that is your opinion. >> >> More accurately, it is my belief. >> >>> You haven't really given any of the >>> advantages of having this extra overhead in the allocator that a >>> program may not want or need for any or all of its allocations. >> >> In point of fact, I haven't said anything at all about what is >> needed to implement either kind of allocator interface. >> >> Why don't you write and post code that implements the sort >> of interface you are advocating, so there can be meaningful >> discussion about it? > > It's a big job, and there are many ways of doing it, so I'm not > going to attempt it here. [summary results from a non-C > environment] Yes, as usual you aren't interested in a meaningful discussion. Just the usual bait-and-switch to offtopic.
[toc] | [prev] | [next] | [standalone]
| From | Bart <bc@freeuk.com> |
|---|---|
| Date | 2022-01-21 13:49 +0000 |
| Message-ID | <ssedlm$2tb$1@dont-email.me> |
| In reply to | #164503 |
On 21/01/2022 13:30, Tim Rentsch wrote: > Bart <bc@freeuk.com> writes: > >> On 11/12/2021 14:56, Tim Rentsch wrote: >> >>> Bart <bc@freeuk.com> writes: >>> >>>> On 10/12/2021 09:47, Tim Rentsch wrote: >>>> >>>>> Manfred <noname@add.invalid> writes: >>>>> >>>>>> On 11/21/2021 9:51 PM, David Brown wrote: >>>>>> >>>>>>> An implementation can do better, given this new information, but there >>>>>>> are no new limitations or restrictions. >>>>>> >>>>>> Well, requiring the programmer to keep track of the block size is a >>>>>> significant programming overhead - if it is unused, it's a significant >>>>>> waste I'd rather not pay for. >>>>> >>>>> Anyone who seriously proposes having to supply a size argument to >>>>> free memory for a general purpose allocator like malloc() either >>>>> hasn't bothered to think or doesn't know how to think. Kind of >>>>> par for the course for the long-running Bart and David show. >>>> >>>> Well, that is your opinion. >>> >>> More accurately, it is my belief. >>> >>>> You haven't really given any of the >>>> advantages of having this extra overhead in the allocator that a >>>> program may not want or need for any or all of its allocations. >>> >>> In point of fact, I haven't said anything at all about what is >>> needed to implement either kind of allocator interface. >>> >>> Why don't you write and post code that implements the sort >>> of interface you are advocating, so there can be meaningful >>> discussion about it? >> >> It's a big job, and there are many ways of doing it, so I'm not >> going to attempt it here. [summary results from a non-C >> environment] > > Yes, as usual you aren't interested in a meaningful discussion. > Just the usual bait-and-switch to offtopic. WTF? You asked me to write and post code, and I wrote and posted code, and IN C. (Which, since it was 6 weeks ago, has now been removed in a periodic clean-up.) What exactly did you want, a completed, production-ready memory allocation scheme? The scheme I demonstrated was one I was already using myself, and I show that it gave significant improvements over regular malloc/free. I notice you didn't post any code.. And what topic did I switch to? The sub-topic was memory allocators, and my code demonstrated an allocator using caller-retained size into to signicantly improve an allocation benchmark. You're a fucking idiot.
[toc] | [prev] | [next] | [standalone]
| From | "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> |
|---|---|
| Date | 2021-12-11 12:17 -0800 |
| Message-ID | <sp311h$jtr$1@dont-email.me> |
| In reply to | #163786 |
On 12/11/2021 6:56 AM, Tim Rentsch wrote: > Bart <bc@freeuk.com> writes: > >> On 10/12/2021 09:47, Tim Rentsch wrote: >> >>> Manfred <noname@add.invalid> writes: >>> >>>> On 11/21/2021 9:51 PM, David Brown wrote: >>>> >>>>> An implementation can do better, given this new information, but there >>>>> are no new limitations or restrictions. >>>> >>>> Well, requiring the programmer to keep track of the block size is a >>>> significant programming overhead - if it is unused, it's a significant >>>> waste I'd rather not pay for. >>> >>> Anyone who seriously proposes having to supply a size argument to >>> free memory for a general purpose allocator like malloc() either >>> hasn't bothered to think or doesn't know how to think. Kind of >>> par for the course for the long-running Bart and David show. >> >> Well, that is your opinion. > > More accurately, it is my belief. > >> You haven't really given any of the >> advantages of having this extra overhead in the allocator that a >> program may not want or need for any or all of its allocations. > > In point of fact, I haven't said anything at all about what is > needed to implement either kind of allocator interface. I am quite fond of the reap interface: https://people.cs.umass.edu/~emery/pubs/berger-oopsla2002.pdf > > Why don't you write and post code that implements the sort > of interface you are advocating, so there can be meaningful > discussion about it? >
[toc] | [prev] | [next] | [standalone]
| From | gazelle@shell.xmission.com (Kenny McCormack) |
|---|---|
| Date | 2021-12-12 13:00 +0000 |
| Message-ID | <sp4ros$1p0q9$1@news.xmission.com> |
| In reply to | #163786 |
In article <86h7bf6vhd.fsf@linuxsc.com>, Tim Rentsch <tr.17687@z991.linuxsc.com> wrote: ... >> Well, that is your opinion. >More accurately, it is my belief. There is no difference between an opinion and a belief. The two words are virtually synonymous. -- The randomly chosen signature file that would have appeared here is more than 4 lines long. As such, it violates one or more Usenet RFCs. In order to remain in compliance with said RFCs, the actual sig can be found at the following URL: http://user.xmission.com/~gazelle/Sigs/Seriously
[toc] | [prev] | [next] | [standalone]
| From | Tim Rentsch <tr.17687@z991.linuxsc.com> |
|---|---|
| Date | 2021-12-12 08:25 -0800 |
| Message-ID | <861r2h7pu4.fsf@linuxsc.com> |
| In reply to | #163799 |
gazelle@shell.xmission.com (Kenny McCormack) writes: > In article <86h7bf6vhd.fsf@linuxsc.com>, > Tim Rentsch <tr.17687@z991.linuxsc.com> wrote: > ... > >>> Well, that is your opinion. >> >> More accurately, it is my belief. > > There is no difference between an opinion and a belief. There is in the way that I use these two words.
[toc] | [prev] | [next] | [standalone]
| From | gazelle@shell.xmission.com (Kenny McCormack) |
|---|---|
| Date | 2021-12-12 17:38 +0000 |
| Message-ID | <sp5c2k$1p856$1@news.xmission.com> |
| In reply to | #163801 |
In article <861r2h7pu4.fsf@linuxsc.com>,
Tim Rentsch <tr.17687@z991.linuxsc.com> wrote:
>gazelle@shell.xmission.com (Kenny McCormack) writes:
>
>> In article <86h7bf6vhd.fsf@linuxsc.com>,
>> Tim Rentsch <tr.17687@z991.linuxsc.com> wrote:
>> ...
>>
>>>> Well, that is your opinion.
>>>
>>> More accurately, it is my belief.
>>
>> There is no difference between an opinion and a belief.
>
>There is in the way that I use these two words.
Give me an example of how you could have an opinion about something and not
have it be your belief.
And/or vice versa.
--
You are again heaping damnation upon your own head by your statements.
- Rick C Hodgin -
[toc] | [prev] | [next] | [standalone]
Page 2 of 4 — ← Prev page 1 [2] 3 4 Next page →
Back to top | Article view | comp.lang.c
csiph-web