Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #64487

Re: Self healthcheck

Show all headers | View raw

"Asaf Las" <roegltd@gmail.com> wrote in message 
news:58c541ab-c6e1-45a8-b03a-8597ed7ecb48@googlegroups.com...
>
> Yes the question was about CPython. But i am not after CPython leaks
> though detecting these would be good, but my own mistakes leading to
> accumulation of data in mutable structures.
> there will be few processes running python code standalone communicating
> across servers and every activity will be spread over time so
> i have to persistently keep record of activity and remove it later when
> activity is finished.

I had a similar concern. My main worry, which turned out to be well-founded, 
was that I would create an object as a result of some user input, but when 
the user had finished with it, and in theory it could be garbage-collected, 
in practice it would not be due to some obscure circular reference 
somewhere.

For short-running tasks this is not a cause for concern, but for a 
long-running server these can build up over time and end up causing a 
problem.

My solution was to log every time an object was created, with some 
self-identifying piece of information, and then log when it was deleted, 
with the same identifier. After running the program for a while I could then 
analyse the log and ensure that each creation had a corresponding deletion.

The tricky bit was logging the deletion. It is a known gotcha in Python that 
you cannot rely on the __del__ method, and indeed it can cause a circular 
reference in itself which prevents the object from being garbage-collected. 
I found a solution somewhere which explained the use of a 'delwatcher' 
class. This is how it works -

class MainObject:
    def __init__(self, identifier):
        self._del = delwatcher('MainObject', identifier)

class delwatcher:
    def __init__(self, obj_type, identifier):
        self.obj_type = obj_type
        self.identifier = identifier
        log('{}: id={} created'.format(self.obj_type, self.identifier))
    def __del__(self):
        log('{}: id={} deleted'.format(self.obj_type, self.identifier))

In this case calling __del__() is safe, as no reference to the main object 
is held.

If you do find that an object is not being deleted, it is then 
trial-and-error to find the problem and fix it. It is probably a circular 
reference

HTH

Frank Millman

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Self healthcheck Asaf Las <roegltd@gmail.com> - 2014-01-21 18:51 -0800
  Re: Self healthcheck Chris Angelico <rosuav@gmail.com> - 2014-01-22 14:08 +1100
    Re: Self healthcheck Asaf Las <roegltd@gmail.com> - 2014-01-22 00:18 -0800
      Re: Self healthcheck Nicholas Cole <nicholas.cole@gmail.com> - 2014-01-22 08:43 +0000
        Re: Self healthcheck Asaf Las <roegltd@gmail.com> - 2014-01-22 07:51 -0800
      Re: Self healthcheck "Frank Millman" <frank@chagford.com> - 2014-01-22 10:56 +0200
        Re: Self healthcheck Asaf Las <roegltd@gmail.com> - 2014-01-22 08:03 -0800
          Re: Self healthcheck Dave Angel <davea@davea.name> - 2014-01-22 13:40 -0500
          Re: Self healthcheck "Frank Millman" <frank@chagford.com> - 2014-01-23 07:36 +0200

csiph-web