Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!aioe.org!feeder.news-service.com!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
To: python-list@python.org
From: Terry Reedy <tjreedy@udel.edu>
Subject: Re: Hooking into Python's memory management
Date: Wed, 04 May 2011 14:58:23 -0400
References: <4DC1841B.2030601@gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.17) Gecko/20110414 Lightning/1.0b2 Thunderbird/3.1.10
In-Reply-To: <4DC1841B.2030601@gmail.com>
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.1166.1304535520.9059.python-list@python.org>
Lines: 66
NNTP-Posting-Host: 82.94.164.166
Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:4644

On 5/4/2011 12:51 PM, Daniel Neilson wrote:
>
> Hello,
> I'm hoping that there will be someone here with sufficient expertise to
> answer a question on Python 3 for me.
>
> I work in the Computer Science department at a large Canadian
> University. We are currently doing a feasibility analysis for switching
> to using Python in our first year major-stream courses.

If you did, I believe you would be following in the footsteps of MIT.

> Part of our first year curriculum requires that students be exposed to
> explicit dynamic memory allocation in the form of C++'s new/delete, C's
> malloc/free, etc. I realize that Python is garbage collected, and that
> there is never a need to explicitly allocate & deallocate objects.
> However, I am trying to determine whether or not it is possible to
> simulate that behaviour within Python via a module for the purposes of
> instruction.

The Python ctypes module allows one to invoke compiled C (or C++, I 
presume) functions in shared libraries (.dll on Windows, .so on *Nix).

> For these purposes, I would like to know whether it is possible within
> Python 3 to write a Python-only module that, essentially, hooks into the
> "constructor" and "destructor" of many of the Python built-in types

Python is compiled as a small startup executable (<30 KB). The builtins 
are all in shared libraries that you can access with ctypes. The 
functions in those libraries are documented in the Python/C API manual.

> (specifically list, dictionary, set, tuple, and string) so that the
> module can:
> 1) Maintain a list of object id()'s for objects that have been created.
> Ideally, this list would also contain the file & line number where the
> object was created.
> 2) Provide a "deallocate" function that will remove a given object's
> id() from the list from (1).
> 3) Print out an error message if the python script terminates with a
> non-empty list from (1). Preferably with a list of objects that are
> still "allocated."

I presume you can do all of this easily. For point 3, a script can 
register an 'atexit' function. As a sidenote, using ctypes 'allows' one 
to crash (segfault, bluescreen) a program just like when using C ;-). It 
thus voids the usual guarantee that one cannot do that.

> Baring a Python-only module, would this behaviour be possible to add via
> a C-language module?

I do not think you will *need* to do this, though you might eventually 
decide to make a custom library with just the functions you want, with 
the names you want.

> A module that hooked in to all memory allocation, and inspected the type
> of the object being allocated to conditionally add the object's id() to
> the list would also suffice.
>
> In either case, if such a module is possible, any pointers you could
> provide regarding how to implement such a module would be appreciated.

I hope the above helps. I think Python is a great learning language.

-- 
Terry Jan Reedy