Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #23460 > unrolled thread

Strange C developments

Back to article view | Back to comp.lang.c

Contents

  Strange C developments Jase Schick <nospam@nospam.com> - 2012-07-20 21:14 +0000
    Re: Strange C developments Les Cargill <lcargill99@comcast.com> - 2012-07-20 18:03 -0500
    Re: Strange C developments "christian.bau" <christian.bau@cbau.wanadoo.co.uk> - 2012-07-20 16:03 -0700
      Re: Strange C developments Tim Rentsch <txr@alumni.caltech.edu> - 2012-07-21 15:11 -0700
      Re: Strange C developments tom st denis <tom@iahu.ca> - 2012-08-02 06:08 -0700
        Re: Strange C developments Jens Gustedt <jens.gustedt@loria.fr> - 2012-08-02 21:23 +0200
          Re: Strange C developments tom st denis <tom@iahu.ca> - 2012-08-03 04:13 -0700
            Re: Strange C developments Noob <root@127.0.0.1> - 2012-08-03 13:19 +0200
            Re: Strange C developments Anders Wegge Keller <wegge@wegge.dk> - 2012-08-03 13:31 +0200
            Re: Strange C developments Jens Gustedt <jens.gustedt@loria.fr> - 2012-08-03 23:47 +0200
        Re: Strange C developments gwowen <gwowen@gmail.com> - 2012-08-03 02:20 -0700
    Re: Strange C developments Rui Maciel <rui.maciel@gmail.com> - 2012-07-21 01:25 +0100
    Re: Strange C developments BGB <cr88192@hotmail.com> - 2012-07-20 19:38 -0500
    Re: Strange C developments Nobody <nobody@nowhere.com> - 2012-07-21 10:32 +0100
      Re: Strange C developments BGB <cr88192@hotmail.com> - 2012-07-21 09:31 -0500
        Re: Strange C developments Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-07-21 17:38 +0100
          Re: Strange C developments BGB <cr88192@hotmail.com> - 2012-07-21 12:58 -0500
          Re: Strange C developments Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-07-21 11:42 -0700
            Re: Strange C developments Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-07-21 20:13 +0100
              Re: Strange C developments Nobody <nobody@nowhere.com> - 2012-07-21 20:48 +0100
                Re: Strange C developments Ben Bacarisse <ben.usenet@bsb.me.uk> - 2012-07-21 23:32 +0100
    Re: Strange C developments jacob navia <jacob@spamsink.net> - 2012-07-21 11:43 +0200
      Re: Strange C developments Quentin Pope <qp19433@hotmail.NOSPAM.com> - 2012-07-21 10:23 +0000
        Re: Strange C developments jacob navia <jacob@spamsink.net> - 2012-07-21 13:01 +0200
          Re: Strange C developments gordonb.y4pdz@burditt.org (Gordon Burditt) - 2012-07-21 08:57 -0500
            Re: Strange C developments BGB <cr88192@hotmail.com> - 2012-07-21 10:40 -0500
            Re: Strange C developments jacob navia <jacob@spamsink.net> - 2012-07-21 19:03 +0200
              Re: Strange C developments gordonb.5rhpu@burditt.org (Gordon Burditt) - 2012-07-22 02:08 -0500
                Re: Strange C developments jacob navia <jacob@spamsink.net> - 2012-07-22 12:32 +0200
                  Re: Strange C developments BGB <cr88192@hotmail.com> - 2012-07-22 08:26 -0500
              Re: Strange C developments Nick Keighley <nick_keighley_nospam@hotmail.com> - 2012-07-24 01:23 -0700
                Re: Strange C developments Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-07-24 05:36 -0700
                  Re: Strange C developments Nick Keighley <nick_keighley_nospam@hotmail.com> - 2012-07-30 02:04 -0700
                Re: Strange C developments BGB <cr88192@hotmail.com> - 2012-07-24 09:55 -0500
                  Re: Strange C developments 88888 Dihedral <dihedral88888@googlemail.com> - 2012-08-02 06:44 -0700
                    Re: Strange C developments BGB <cr88192@hotmail.com> - 2012-08-03 08:48 -0500
            Re: Strange C developments jacob navia <jacob@spamsink.net> - 2012-07-21 19:06 +0200
              Re: Strange C developments gordonb.ibds9@burditt.org (Gordon Burditt) - 2012-07-22 02:28 -0500
                Re: Strange C developments jacob navia <jacob@spamsink.net> - 2012-07-22 12:30 +0200
                Re: Strange C developments Philip Lantz <prl@canterey.us> - 2012-07-24 00:58 -0700
          Re: Strange C developments Nick Keighley <nick_keighley_nospam@hotmail.com> - 2012-07-24 01:13 -0700
        Re: Strange C developments Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-07-21 05:44 -0700
          Re: Strange C developments Tim Rentsch <txr@alumni.caltech.edu> - 2012-07-21 15:31 -0700
            Re: Strange C developments Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-07-21 16:59 -0700
              Re: Strange C developments Tim Rentsch <txr@alumni.caltech.edu> - 2012-07-21 23:46 -0700
                Re: Strange C developments "io_x" <a@b.c.invalid> - 2012-07-22 10:37 +0200
                  Re: Strange C developments Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-07-22 03:28 -0700
                  Re: Strange C developments BGB <cr88192@hotmail.com> - 2012-07-22 09:05 -0500
                  Re: Strange C developments Nick Keighley <nick_keighley_nospam@hotmail.com> - 2012-07-24 02:10 -0700
                    Re: Strange C developments "io_x" <a@b.c.invalid> - 2012-07-27 20:47 +0200
                      Re: Strange C developments Nick Keighley <nick_keighley_nospam@hotmail.com> - 2012-07-30 02:09 -0700
                        Re: Strange C developments "BartC" <bc@freeuk.com> - 2012-07-30 13:05 +0100
                          Re: Strange C developments BGB <cr88192@hotmail.com> - 2012-08-03 10:45 -0500
                        Re: Strange C developments "io_x" <a@b.c.invalid> - 2012-07-30 16:52 +0200
                          Re: Strange C developments "io_x" <a@b.c.invalid> - 2012-07-30 18:26 +0200
                        Re: Strange C developments Malcolm McLean <malcolm.mclean5@btinternet.com> - 2012-08-03 12:12 -0700
                          Re: Strange C developments "BartC" <bc@freeuk.com> - 2012-08-04 12:18 +0100
                            Re: Strange C developments Phil Carmody <thefatphil_demunged@yahoo.co.uk> - 2012-08-10 09:50 +0300
                  Re: Strange C developments Tim Rentsch <txr@alumni.caltech.edu> - 2012-09-07 11:19 -0700
                    Re: Strange C developments "io_x" <a@b.c.invalid> - 2012-09-09 12:06 +0200
                  Re: Strange C developments Kaz Kylheku <kaz@kylheku.com> - 2012-09-09 05:25 +0000
            Re: Strange C developments Melzzzzz <mel@zzzzz.com> - 2012-07-22 02:48 +0200
            Re: Strange C developments William Ahern <william@wilbur.25thandClement.com> - 2012-07-21 21:45 -0700
              Re: Strange C developments Tim Rentsch <txr@alumni.caltech.edu> - 2012-07-22 02:40 -0700
                Re: Strange C developments BGB <cr88192@hotmail.com> - 2012-07-22 10:38 -0500
            Re: Strange C developments William Ahern <william@wilbur.25thandClement.com> - 2012-07-21 22:38 -0700
              Re: Strange C developments jacob navia <jacob@spamsink.net> - 2012-07-22 08:43 +0200
              Re: Strange C developments "BartC" <bc@freeuk.com> - 2012-07-22 11:24 +0100
                Re: Strange C developments BGB <cr88192@hotmail.com> - 2012-08-03 09:09 -0500
                  Re: Strange C developments BGB <cr88192@hotmail.com> - 2012-08-03 16:54 -0500
            Re: Strange C developments Tim Rentsch <txr@alumni.caltech.edu> - 2012-07-21 23:16 -0700
              Re: Strange C developments Phil Carmody <thefatphil_demunged@yahoo.co.uk> - 2012-07-22 17:50 +0300
                Re: Strange C developments BGB <cr88192@hotmail.com> - 2012-07-22 11:27 -0500
                  Re: Strange C developments Ike Naar <ike@iceland.freeshell.org> - 2012-07-22 20:55 +0000
                    Re: Strange C developments Philip Lantz <prl@canterey.us> - 2012-07-24 00:56 -0700
                      Re: Strange C developments Nick Keighley <nick_keighley_nospam@hotmail.com> - 2012-07-24 01:57 -0700
                Re: Strange C developments Tim Rentsch <txr@alumni.caltech.edu> - 2012-09-07 11:20 -0700
        Re: Strange C developments Tim Rentsch <txr@alumni.caltech.edu> - 2012-07-21 15:13 -0700
    Re: Strange C developments John Bode <jfbode1029@gmail.com> - 2012-07-23 14:59 -0700

Page 2 of 4 — ← Prev page 1 [2] 3 4  Next page →

#23530

Nobody <nobody@nowhere.com> writes:

> On Sat, 21 Jul 2012 20:13:28 +0100, Ben Bacarisse wrote:
>
>>> In C a pointer is valid if saved to a file in binary form then read back
>>> in again. It's invalid if read back in to an second instance of the
>>> program. So the standard needs a tweak to make the first situation also
>>> invalid.
>> 
>> Why?  What would be the benefit?
>
> What would be the benefit in writing a pointer to a file, or to making
> that invalid?

To making it invalid.  That seems to be the suggesting that Malcolm is
making.

> Writing a pointer to a file may be useful for virtual memory systems such
> as that used by the Win16 API.
>
> Forbidding writing pointers to files would eliminate one possible
> mechanism whereby "transparent" GC would fail.

Right, but it's just one way, and not (in my experience at least) the
most common.

I can see where I went wrong though.  I should have said "is the benefit
really worth it?" because I can see a minuscule benefit, just not one
that could drive a change to the language.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#23479

Le 20/07/12 23:14, Jase Schick a écrit :
> Hi Can anyone explain why C has added support for pthreads, while NOT
> adding support for garbage collection? Convenient memory management would
> be a much greater enhancement than replicating a perfectly good existing
> library it seems to me.
>
> Jase
>
The lcc-win compiler offers a garbage collector (Boehm's) in its 
standard distribution. It is a very useful feature, used for instance in 
the debugger of lcc-win, in the IDE and several other applications. Of 
course it is used by many of the people that have downloaded lcc-win 
(more than 1 million)

[toc] | [prev] | [next] | [standalone]

#23480

On Sat, 21 Jul 2012 11:43:36 +0200, jacob navia wrote:
> Le 20/07/12 23:14, Jase Schick a écrit :
>> Hi Can anyone explain why C has added support for pthreads, while NOT
>> adding support for garbage collection? Convenient memory management
>> would be a much greater enhancement than replicating a perfectly good
>> existing library it seems to me.
>>
>> Jase
>>
> The lcc-win compiler offers a garbage collector (Boehm's) in its
> standard distribution. It is a very useful feature, used for instance in
> the debugger of lcc-win, in the IDE and several other applications. Of
> course it is used by many of the people that have downloaded lcc-win
> (more than 1 million)

Do you never get tired of spamming this group with advertising for your 
compiler?

Adding garbage collection would break a large amount of existing code.

Often the bottom couple of bits of pointers to memory with known 
alignment properties will be used to store information (the pointer than 
being and'd with ~0x3ul or similar prior to dereferencing).

Many code protection methods rely on storing pointers xor'd with an 
obfuscating mask. GCs are not sophisticated enough to track such pointers.

And what is the gain? With careful programming, there is no need 
whatsoever for this stupid overhead. Leave it for the kiddies programming 
JAVA.

//QP

[toc] | [prev] | [next] | [standalone]

#23481

Le 21/07/12 12:23, Quentin Pope a écrit :
> On Sat, 21 Jul 2012 11:43:36 +0200, jacob navia wrote:
>> Le 20/07/12 23:14, Jase Schick a écrit :
>>> Hi Can anyone explain why C has added support for pthreads, while NOT
>>> adding support for garbage collection? Convenient memory management
>>> would be a much greater enhancement than replicating a perfectly good
>>> existing library it seems to me.
>>>
>>> Jase
>>>
>> The lcc-win compiler offers a garbage collector (Boehm's) in its
>> standard distribution. It is a very useful feature, used for instance in
>> the debugger of lcc-win, in the IDE and several other applications. Of
>> course it is used by many of the people that have downloaded lcc-win
>> (more than 1 million)
>
> Do you never get tired of spamming this group with advertising for your
> compiler?
>
> Adding garbage collection would break a large amount of existing code.
>

To port existing code to a GC environment you do not need to change a 
single line. Just define malloc as gc_malloc and define free as a noop.


> Often the bottom couple of bits of pointers to memory with known
> alignment properties will be used to store information (the pointer than
> being and'd with ~0x3ul or similar prior to dereferencing).
>

??? That is not the case with the GC used by lcc-win.

> Many code protection methods rely on storing pointers xor'd with an
> obfuscating mask. GCs are not sophisticated enough to track such pointers.
>

Yes, that kind of code shouldn't be used with a GC.

> And what is the gain?

The gain is that instead of loosing endless hours tracking that dangling
pointer in the debugger you can concentrate on your application instead.


> With careful programming, there is no need
> whatsoever for this stupid overhead.

You fail to mention "With careful programming and not making any 
mistake. NEVER. A single moment of inattention and you are screwed.


Leave it for the kiddies programming
> JAVA.
>

JAVA, Lisp, C++, C, all the languages that can be used ith a collector.

[toc] | [prev] | [next] | [standalone]

#23483

>> Adding garbage collection would break a large amount of existing code.
>>
> 
> To port existing code to a GC environment you do not need to change a 
> single line. Just define malloc as gc_malloc and define free as a noop.

Jacob, what rules would have to be added for application writers
to use the GC in lcc-win besides the C standard and "don't invoke
undefined behavior"?  Hint:  the answer is not "none".  And that's
not intended as a put-down of lcc's GC or GC in general.

Or, to put it another way which I'm sure you will still think of
as a personal attack, list the things an application programmer
could do to break GC that no sane programmer would do (someone will
come up with a sane-sounding reason for doing it anyway) but the C
standard still allows.  If the C standard calls it undefined behavior
on the part of the application, GC is off the hook.

I'll define "break GC" as any one or more of the following:
- Collecting non-garbage
- Aborting the program with things like segfaults.
- A 1,000,000% slowdown
- A GC so conservative it never collects anything.


I'm pretty sure that writing pointers to a (potentially terabyte)
temporary file, erasing the pointers from memory, and later reading
the pointers back from the file (all done by the same run of the
same program) will either confuse or horribly slow down GC.  And I
think that's perfectly OK with the C standard.  

>> Often the bottom couple of bits of pointers to memory with known
>> alignment properties will be used to store information (the pointer than
>> being and'd with ~0x3ul or similar prior to dereferencing).
>>
> 
> ??? That is not the case with the GC used by lcc-win.

I take that to mean:  often an application program will abuse the bottom
couple of bits of pointers to memory with known alignment properties to
store information, and this will break the GC.  I think doing that to
pointers invokes undefined behavior, at least if you do stuff like
	void *ptr;

	ptr |= 3;
or 	ptr &= ~3;

so the GC has a perfectly good excuse for breaking.

>> Many code protection methods rely on storing pointers xor'd with an
>> obfuscating mask. GCs are not sophisticated enough to track such pointers.

Code protection methods are designed to break the application, but
the C standard doesn't forbid that.

I'm not sure the C standard allows that specific method of disguising
pointers, but I think you are allowed to format a pointer with
sprintf() and %p, erase the original pointer, then later feed the
buffer sprintf() wrote into sscanf() and %p, and get back the same
pointer.  While it's in text form, you could do all sorts of things
to it (encrypt, store it in a file, etc.) , as long as it eventually
gets back to the original text.

> Yes, that kind of code shouldn't be used with a GC.

People want to know how to recognize "that kind of code" and not
try to use GC with it, rather than trying it and figuring it out
the hard way.  And I don't really see a "don't disguise pointers"
rule as being too onerous if you can clearly define what *ISN'T*
disguised.  Probably 99.9% of programs would need no change.

>> And what is the gain?
> 
> The gain is that instead of loosing endless hours tracking that dangling
> pointer in the debugger you can concentrate on your application instead.

Oh, yes, another rule for a working GC is that the GC has to
occasionally actually collect some garbage if there's any around
to collect.  It doesn't have to catch everything immediately, but
code like:

#define malloc(x) gc_malloc(x)
int main(void)
{
	while(malloc(6)) { /* loop */ }
}
shouldn't terminate the loop because malloc() eventually returns NULL
due to insufficient collection of garbage.

[toc] | [prev] | [next] | [standalone]

#23486

On 7/21/2012 8:57 AM, Gordon Burditt wrote:
>>> Adding garbage collection would break a large amount of existing code.
>>>
>>
>> To port existing code to a GC environment you do not need to change a
>> single line. Just define malloc as gc_malloc and define free as a noop.
>
> Jacob, what rules would have to be added for application writers
> to use the GC in lcc-win besides the C standard and "don't invoke
> undefined behavior"?  Hint:  the answer is not "none".  And that's
> not intended as a put-down of lcc's GC or GC in general.
>
> Or, to put it another way which I'm sure you will still think of
> as a personal attack, list the things an application programmer
> could do to break GC that no sane programmer would do (someone will
> come up with a sane-sounding reason for doing it anyway) but the C
> standard still allows.  If the C standard calls it undefined behavior
> on the part of the application, GC is off the hook.
>
> I'll define "break GC" as any one or more of the following:
> - Collecting non-garbage
> - Aborting the program with things like segfaults.
> - A 1,000,000% slowdown
> - A GC so conservative it never collects anything.
>

in my case, the policy is basically just to give code the choice.
if it wants to use malloc, it can use malloc;
if it wants to use the GC, it can use the GC.

the GC wont generally trace through memory allocated via malloc though.
in my case, there is a "gcmalloc()" call, which is behaviorally similar 
to malloc (it wont automatically release the memory), except that the GC 
will trace through it.

putting pointers to malloc-managed memory in GC-managed objects works, 
just the GC will ignore them.

it is possible to register behavior hooks with the GC to allow 
custom-managed memory regions.


usually, the GC wont collect anything until after a certain amount of 
memory is used (can be set by the program), so if the app keeps its 
memory use below this limit, the GC wont run (this currently leads to a 
case of setting the limit at something like 1GB, which makes the GC 
running, except in cases where the app "springs a leak" fairly unlikely).


> I'm pretty sure that writing pointers to a (potentially terabyte)
> temporary file, erasing the pointers from memory, and later reading
> the pointers back from the file (all done by the same run of the
> same program) will either confuse or horribly slow down GC.  And I
> think that's perfectly OK with the C standard.

actually, more likely, it will become timing dependent:
if the GC runs during the time the pointers no longer exist, the objects 
may be freed;
if it does not, nothing happens (the pointers will just come back in 
pointing at the same objects).

in the case where the pointers are read back in for objects which have 
been freed, then essentially they are just dangling pointers (and the GC 
will either ignore them, or treat them as if they point to whatever new 
object was allocated at that address).


this case only really matters if the GC is being used to replace malloc 
or similar.

otherwise, a person could declare that the case of temporarily hiding 
and then restoring pointers is itself undefined-behavior.


>>> Often the bottom couple of bits of pointers to memory with known
>>> alignment properties will be used to store information (the pointer than
>>> being and'd with ~0x3ul or similar prior to dereferencing).
>>>
>>
>> ??? That is not the case with the GC used by lcc-win.
>
> I take that to mean:  often an application program will abuse the bottom
> couple of bits of pointers to memory with known alignment properties to
> store information, and this will break the GC.  I think doing that to
> pointers invokes undefined behavior, at least if you do stuff like
> 	void *ptr;
>
> 	ptr |= 3;
> or 	ptr &= ~3;
>
> so the GC has a perfectly good excuse for breaking.
>

my GC will handle this case just fine (my GC is more conservative, and 
so will by-default treat a pointer to anywhere in the object as if it 
were a GC reference). likewise, most GC operations will accept these 
pointers as well, and there is also a "gcGetBase()" operation mostly 
specially for this case: it allows taking a pointer into an object and 
getting the starting address of the object.


I don't know how Boehm handles it exactly, but IIRC there are script VMs 
which use Boehm and use similar tagging schemes.


I don't personally use this sort of tagging in my VM, instead having the 
GC keep track of the type, so whenever the VM needs to do something, it 
will fetch the type-name for an object, or maybe fetch an associated 
type-vtable (the VM has tables of function-pointers used to represent 
common operations on objects of various types).

note that there is also a Class/Instance OO system, but this uses its 
own independent vtables, and these objects exist as a single type from 
the POV of the GC.


>>> Many code protection methods rely on storing pointers xor'd with an
>>> obfuscating mask. GCs are not sophisticated enough to track such pointers.
>
> Code protection methods are designed to break the application, but
> the C standard doesn't forbid that.
>

simple: label the case of XOR'ed pointers + GC'ed objects as undefined.


> I'm not sure the C standard allows that specific method of disguising
> pointers, but I think you are allowed to format a pointer with
> sprintf() and %p, erase the original pointer, then later feed the
> buffer sprintf() wrote into sscanf() and %p, and get back the same
> pointer.  While it's in text form, you could do all sorts of things
> to it (encrypt, store it in a file, etc.) , as long as it eventually
> gets back to the original text.
>

possible, but this case can also be labeled as undefined with GC 
objects, or maybe, it can be restricted to certain cases, such as it is 
only allowed if the objects are pinned/locked beforehand, or if they 
were allocated either with malloc or a malloc-like call (and so will not 
be implicitly freed).


>> Yes, that kind of code shouldn't be used with a GC.
>
> People want to know how to recognize "that kind of code" and not
> try to use GC with it, rather than trying it and figuring it out
> the hard way.  And I don't really see a "don't disguise pointers"
> rule as being too onerous if you can clearly define what *ISN'T*
> disguised.  Probably 99.9% of programs would need no change.
>

in my case, it is more like:
pointers stored in global variables are visible (1);
pointers stored in stack variables are visible (2);
pointers stored in other GC managed objects.


1: on both Windows and Linux, the GC will walk the list of loaded 
modules and scan the ".data" and ".bss" sections and similar. this also 
includes any "static" variables within functions.

2: this can be a little harder. in my case, the GC supplies its own 
thread-creation functions, but it is possible to do so without doing 
this (for example, Boehm walks the OS thread list AFAIK). there is a 
difficulty with knowing the exact stack address on WoW64 targets (an 
issue for Boehm AFAIK), but my GC uses a different strategy: it just 
scans the stack until either the known limit is encountered, or Windows 
throws a SEH exception (when the scan function tries to scan into an 
uninitialized guard page), which the function catches and handles.


>>> And what is the gain?
>>
>> The gain is that instead of loosing endless hours tracking that dangling
>> pointer in the debugger you can concentrate on your application instead.
>
> Oh, yes, another rule for a working GC is that the GC has to
> occasionally actually collect some garbage if there's any around
> to collect.  It doesn't have to catch everything immediately, but
> code like:
>
> #define malloc(x) gc_malloc(x)
> int main(void)
> {
> 	while(malloc(6)) { /* loop */ }
> }
> shouldn't terminate the loop because malloc() eventually returns NULL
> due to insufficient collection of garbage.
>
>

usually what happens in this case is that the caller thread will block 
until the GC thread can run and finish (running out of memory triggers 
an immediate GC).

[toc] | [prev] | [next] | [standalone]

#23490

Le 21/07/12 15:57, Gordon Burditt a écrit :
> Jacob, what rules would have to be added for application writers
> to use the GC in lcc-win besides the C standard and "don't invoke
> undefined behavior"?  Hint:  the answer is not "none".  And that's
> not intended as a put-down of lcc's GC or GC in general.

1) Do not hide pointers to the collector, i.e. the collector will NOT 
search the window extra bytes (for instance) for pointers, or the hard
disk. If you need to XOR pointers to hide them do not use the collector
either.

2) In modern machines a collector slows down the program for at most
a milisecond in normal situations, that could be bigger but not much
bigger since the collector tries to spread out the GC time in each
allocation.

That's all.

jacob

[toc] | [prev] | [next] | [standalone]

#23554

>> Jacob, what rules would have to be added for application writers
>> to use the GC in lcc-win besides the C standard and "don't invoke
>> undefined behavior"?  Hint:  the answer is not "none".  And that's
>> not intended as a put-down of lcc's GC or GC in general.
> 
> 1) Do not hide pointers to the collector, i.e. the collector will NOT 
> search the window extra bytes (for instance) for pointers, or the hard
> disk. If you need to XOR pointers to hide them do not use the collector
> either.

Can I count on (unmangled, unencrypted) pointers in auto variables,
even if the compiler puts them in registers, to be found?  How about
register variables?  Function arguments?  If I can't count on those, I
see lots of trouble ahead for programs heavy on linked lists, which
may be in serious need of garbage collection.

How about the return value of a function in code like:
	NODE	*nptr;	/* auto variable */
	...
	nptr = allocatenewnode(....);
if garbage collection happens to be activated (say, by another
thread) just before the return value of allocatenewnode() is stored
in nptr?  Here, allocatenewnode()'s function is to gc_alloc()
something, fill it in with data, and return a pointer to that data.


> 2) In modern machines a collector slows down the program for at most
> a milisecond in normal situations, that could be bigger but not much
> bigger since the collector tries to spread out the GC time in each
> allocation.

The 1,000,000% slowdown I was thinking of could be the result of
scanning huge disk files (if it attempts such a thing, which is
probably hopeless) or, large amounts of compressed, say, video data
in memory.  This assumes the program wasn't modified to be GC-aware
and it just uses gc_alloc() for everything including huge quantities
of data.

[toc] | [prev] | [next] | [standalone]

#23565

Le 22/07/12 09:08, Gordon Burditt a écrit :
> How about the return value of a function in code like:
> 	NODE	*nptr;	/* auto variable */
> 	...
> 	nptr = allocatenewnode(....);
> if garbage collection happens to be activated (say, by another
> thread) just before the return value of allocatenewnode() is stored
> in nptr?  Here, allocatenewnode()'s function is to gc_alloc()
> something, fill it in with data, and return a pointer to that data.

Nothing bad can ever happen because the "allocatenewnode() will have
stored the pointer it received from gc_malloc somewhere and the
collector will see it.

Boehm's collector has been running for YEARS now and it is very stable,
those problems were solved long ago.

[toc] | [prev] | [next] | [standalone]

#23571

On 7/22/2012 5:32 AM, jacob navia wrote:
> Le 22/07/12 09:08, Gordon Burditt a écrit :
>> How about the return value of a function in code like:
>>     NODE    *nptr;    /* auto variable */
>>     ...
>>     nptr = allocatenewnode(....);
>> if garbage collection happens to be activated (say, by another
>> thread) just before the return value of allocatenewnode() is stored
>> in nptr?  Here, allocatenewnode()'s function is to gc_alloc()
>> something, fill it in with data, and return a pointer to that data.
>
> Nothing bad can ever happen because the "allocatenewnode() will have
> stored the pointer it received from gc_malloc somewhere and the
> collector will see it.
>
> Boehm's collector has been running for YEARS now and it is very stable,
> those problems were solved long ago.
>

yep.

Boehm also scans registers, TLS, ..., in addition to globals and the 
stack, ...

raw data also isn't really a big issue either.

[toc] | [prev] | [next] | [standalone]

#23658

On Jul 21, 6:03 pm, jacob navia <ja...@spamsink.net> wrote:

> 2) In modern machines a collector slows down the program for at most
> a milisecond in normal situations, that could be bigger but not much
> bigger since the collector tries to spread out the GC time in each
> allocation.

whilst that's impressive even a milisecond might matter in some real
time systems

[toc] | [prev] | [next] | [standalone]

#23670

בתאריך יום שלישי, 24 ביולי 2012 09:23:03 UTC+1, מאת Nick Keighley:
> On Jul 21, 6:03 pm, jacob navia 'ja...@spamsink.net' wrote:
> 
> whilst that's impressive even a milisecond might matter in some real
> time systems
>
You've got twenty of them per frame in a 50 frames per second video system. So the garbage collector itself won't take down the system. However it's not a negligible consumer of resources.

[toc] | [prev] | [next] | [standalone]

#23934

On Jul 24, 1:36 pm, Malcolm McLean <malcolm.mcle...@btinternet.com>
wrote:
> בתאריך יום שלישי, 24 ביולי 2012 09:23:03 UTC+1, מאת Nick Keighley:> On Jul 21, 6:03 pm, jacob navia 'ja...@spamsink.net' wrote:
>
> > whilst that's impressive even a milisecond might matter in some real
> > time systems
>
> You've got twenty of them per frame in a 50 frames per second video system. So the garbage collector itself won't take down the system. However it's not a negligible consumer of resources.

I didn't say "video" did I?

[toc] | [prev] | [next] | [standalone]

#23676

On 7/24/2012 3:23 AM, Nick Keighley wrote:
> On Jul 21, 6:03 pm, jacob navia <ja...@spamsink.net> wrote:
>
>> 2) In modern machines a collector slows down the program for at most
>> a milisecond in normal situations, that could be bigger but not much
>> bigger since the collector tries to spread out the GC time in each
>> allocation.
>
> whilst that's impressive even a milisecond might matter in some real
> time systems
>

hard real-time, maybe.
then the usual strategy would be not to use a GC.
in any case, it would be best if C preserved the right to choice as to 
whether or not GC is used.


soft real-time is, well, softer. typically, the matter of whether or not 
a GC is used can be more left to the impact it may have on the user 
experience. assuming that the GC is not running all the time (this is 
likely only really in pathological conditions), the negative impact is 
fairly small, but may allow delivering a better user experience overall 
(while at the same time, preventing slower memory leaks from eventually 
crashing the program).

speed critical code may also refrain from allocating memory or using 
dynamic type-checking as well.


actually, as-is, GC libraries supply a choice as to whether or not GC is 
used, apart from people arguing that "there is no choice, since they are 
unusable".

any sort of standard GC would likely still need to preserve choice, and 
hopefully not be too far in "lame duck" territory either.



decided to leave out anecdotes related to my experience plugging my GC 
into the Quake 2 "hunk"-allocator system (basically, memory allocation 
in a manner similar to a large stack).

[toc] | [prev] | [next] | [standalone]

#24133

BGB於 2012年7月24日星期二UTC+8下午10時55分33秒寫道：
> On 7/24/2012 3:23 AM, Nick Keighley wrote:
> 
> > On Jul 21, 6:03 pm, jacob navia <ja...@spamsink.net> wrote:
> 
> >
> 
> >> 2) In modern machines a collector slows down the program for at most
> 
> >> a milisecond in normal situations, that could be bigger but not much
> 
> >> bigger since the collector tries to spread out the GC time in each
> 
> >> allocation.
> 
> >
> 
> > whilst that's impressive even a milisecond might matter in some real
> 
> > time systems
> 
> >
> 
> 
> 
> hard real-time, maybe.
> 
> then the usual strategy would be not to use a GC.
> 
> in any case, it would be best if C preserved the right to choice as to 
> 
> whether or not GC is used.
> 
> 
> 
> 
> 
> soft real-time is, well, softer. typically, the matter of whether or not 
> 
> a GC is used can be more left to the impact it may have on the user 
> 
> experience. assuming that the GC is not running all the time (this is 
> 
> likely only really in pathological conditions), the negative impact is 
> 
> fairly small, but may allow delivering a better user experience overall 
> 
> (while at the same time, preventing slower memory leaks from eventually 
> 
> crashing the program).
> 
> 
> 
> speed critical code may also refrain from allocating memory or using 
> 
> dynamic type-checking as well.
> 
> 
> 
> 
> 
> actually, as-is, GC libraries supply a choice as to whether or not GC is 
> 
> used, apart from people arguing that "there is no choice, since they are 
> 
> unusable".
> 
> 
> 
> any sort of standard GC would likely still need to preserve choice, and 
> 
> hopefully not be too far in "lame duck" territory either.
> 
> 
> 
> 
> 
> 
> 
> decided to leave out anecdotes related to my experience plugging my GC 
> 
> into the Quake 2 "hunk"-allocator system (basically, memory allocation 
> 
> in a manner similar to a large stack).

The GC part can be implememted by reference counts.
But that adds overheads to using every objet.

There is another way to achieve the same funtionalities as the GC.

But that will involve error trapping and serializing  objects to be storable to the hard disk, and perform jobs in a shell way that can be safely exited in all modules.

It is as difficult as building the OS mmu part.



 

[toc] | [prev] | [next] | [standalone]

#24166

On 8/2/2012 8:44 AM, 88888 Dihedral wrote:
> BGB於 2012年7月24日星期二UTC+8下午10時55分33秒寫道：
>> On 7/24/2012 3:23 AM, Nick Keighley wrote:
>>
>>> On Jul 21, 6:03 pm, jacob navia <ja...@spamsink.net> wrote:
>>
>>>
>>
>>>> 2) In modern machines a collector slows down the program for at most
>>
>>>> a milisecond in normal situations, that could be bigger but not much
>>
>>>> bigger since the collector tries to spread out the GC time in each
>>
>>>> allocation.
>>
>>>
>>
>>> whilst that's impressive even a milisecond might matter in some real
>>
>>> time systems
>>
>>>
>>
>>
>>
>> hard real-time, maybe.
>>
>> then the usual strategy would be not to use a GC.
>>
>> in any case, it would be best if C preserved the right to choice as to
>>
>> whether or not GC is used.
>>
>>
>>
>>
>>
>> soft real-time is, well, softer. typically, the matter of whether or not
>>
>> a GC is used can be more left to the impact it may have on the user
>>
>> experience. assuming that the GC is not running all the time (this is
>>
>> likely only really in pathological conditions), the negative impact is
>>
>> fairly small, but may allow delivering a better user experience overall
>>
>> (while at the same time, preventing slower memory leaks from eventually
>>
>> crashing the program).
>>
>>
>>
>> speed critical code may also refrain from allocating memory or using
>>
>> dynamic type-checking as well.
>>
>>
>>
>>
>>
>> actually, as-is, GC libraries supply a choice as to whether or not GC is
>>
>> used, apart from people arguing that "there is no choice, since they are
>>
>> unusable".
>>
>>
>>
>> any sort of standard GC would likely still need to preserve choice, and
>>
>> hopefully not be too far in "lame duck" territory either.
>>
>>
>>
>>
>>
>>
>>
>> decided to leave out anecdotes related to my experience plugging my GC
>>
>> into the Quake 2 "hunk"-allocator system (basically, memory allocation
>>
>> in a manner similar to a large stack).
>
> The GC part can be implememted by reference counts.
> But that adds overheads to using every objet.
>
> There is another way to achieve the same funtionalities as the GC.
>
> But that will involve error trapping and serializing  objects to be storable to the hard disk, and perform jobs in a shell way that can be safely exited in all modules.
>
> It is as difficult as building the OS mmu part.
>

typically, tracing GC's, such as mark/sweep collectors, incremental 
tricolor collectors, ... are what are commonly used in this case.

usually, the stack and global variables and similar are regarded as 
"roots" and are then searched for object references, making it possible 
to determine which objects are and are-not visible.


reference-counting tends to be a PITA to make work without compiler 
assistance, so is not commonly used as a general purpose GC mechanism.

similarly, given that compacting collectors (such as copy-collectors) 
tend not to really be compatible with C (likewise would require compiler 
support), they are not generally used.


I am not really aware of anyone using persistent storage *as* the GC.

[toc] | [prev] | [next] | [standalone]

#23491

Le 21/07/12 15:57, Gordon Burditt a écrit :
> - Collecting non-garbage
> - Aborting the program with things like segfaults.
> - A 1,000,000% slowdown
> - A GC so conservative it never collects anything.

You can collect non garbage when you hide the pointers
from the collector using XOR. Then the collector will
assume that all the pointers not referenced are free
and havoc will ensue.

Also, if you always keep pointers to everything never setting them to
NULL, and all pointers are global (or the roots are in the main()
function, the collector will never collect anything.

For instance a global list will never be collected if its root is a
global pointer. To collect that you have to set that pointer to NULL.

[toc] | [prev] | [next] | [standalone]

#23555

> Le 21/07/12 15:57, Gordon Burditt a écrit :
>> - Collecting non-garbage
>> - Aborting the program with things like segfaults.
>> - A 1,000,000% slowdown
>> - A GC so conservative it never collects anything.
> 
> You can collect non garbage when you hide the pointers
> from the collector using XOR. Then the collector will
> assume that all the pointers not referenced are free
> and havoc will ensue.

So, you make a rule that pointers to GC-memory must be kept in
(a) auto variables,  (you may add or exclude "register variables" here)
(b) static or global variables,
(c) gc_alloc()ed memory, 
(d) function arguments,
or (e) function return values
and they must be in their original binary form.  Violating this causes
undefined behavior.

> Also, if you always keep pointers to everything never setting them to
> NULL, and all pointers are global (or the roots are in the main()
> function, the collector will never collect anything.

Those aren't eligible for collection.  I might have global root pointers
for linked lists, but the nodes I delete out of the list won't have
anything pointing at it.  And if I need to delete the list, I can
assign NULL then.

Do I have to set the pointer to NULL if it's stored in an auto
variable (not in main()) and the ptr = NULL; statement would be
immediately followed by a return statement?  (Don't look at
stack frames for functions that have already returned.)

> For instance a global list will never be collected if its root is a
> global pointer. To collect that you have to set that pointer to NULL.

[toc] | [prev] | [next] | [standalone]

#23564

Le 22/07/12 09:28, Gordon Burditt a écrit :
>> Le 21/07/12 15:57, Gordon Burditt a écrit :

> So, you make a rule that pointers to GC-memory must be kept in
> (a) auto variables,  (you may add or exclude "register variables" here)
    Registers are obviously looked for roots, so it is safe to store
    pointers in registers
> (b) static or global variables,
    Yes
> (c) gc_alloc()ed memory,
     Yes
> (d) function arguments,
    Yes
> or (e) function return values
   Yes
> and they must be in their original binary form.  Violating this causes
> undefined behavior.
>
Yes

>> Also, if you always keep pointers to everything never setting them to
>> NULL, and all pointers are global (or the roots are in the main()
>> function, the collector will never collect anything.
>
> Those aren't eligible for collection.  I might have global root pointers
> for linked lists, but the nodes I delete out of the list won't have
> anything pointing at it.  And if I need to delete the list, I can
> assign NULL then.
>
Yes. In general it is a good idea doing that even if you do not use the
collector. Suppose you do NOT set it to NULL and then (without a
collector) you free() the list. You have now a dangling pointer, that
is much WORST than a NULL pointer since many functions test for NULL but
can't test for dangling pointers.


> Do I have to set the pointer to NULL if it's stored in an auto
> variable (not in main()) and the ptr = NULL; statement would be
> immediately followed by a return statement?  (Don't look at
> stack frames for functions that have already returned.)
>

No, in general set pointers to NULL when using global variables or
functions that run for very long periods of time. In short lived
functions garbage will be collectible when the function exits so it
is a slight optiization to avoid that.

BUT

Setting a pointer to NUL is such a CHEAP operation (some nanoseconds)
that setting unused pointers ALWAYS to NULL will never affect really
the performance of your program.

[toc] | [prev] | [next] | [standalone]

#23656

Gordon Burditt wrote:
> Do I have to set the pointer to NULL if it's stored in an auto
> variable (not in main()) and the ptr = NULL; statement would be
> immediately followed by a return statement?

Most likely in that case the compiler would know the variable is dead 
and not actually store the NULL, so it wouldn't help the GC anyway 
(unless there was some knowledge of the GC built into the compiler, but 
I think the topic under discussion is the case where it isn't).

Any time registers containing pointers are saved in the stack, such as 
before calling another function or in a function prologue, the GC has to 
treat those pointers as live. Then when that stack area is reused for a 
different stack frame, it is often not fully initialized, and the GC 
will still see those pointers as live, even when they are just leftover 
junk on the stack.

Whether or not this significantly limits the GC depends on the way the 
code uses pointers.

[toc] | [prev] | [next] | [standalone]

Page 2 of 4 — ← Prev page 1 [2] 3 4  Next page →

Back to top | Article view | comp.lang.c

csiph-web