Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder1.news.weretis.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path: <ian.g.kelly@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.002
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; '*not*': 0.07; 'objects,': 0.07; 'python': 0.09; 'collections': 0.09; 'counting': 0.09; 'garbage': 0.09; 'slices': 0.09; 'subject:()': 0.09; 'bug': 0.10; 'constructs': 0.16; 'gc.collect()': 0.16; 'iteration,': 0.16; 'oct': 0.16; 'subject:array': 0.16; 'true:': 0.16; 'wrote:': 0.17; 'appears': 0.18; 'memory': 0.18; 'import': 0.21; 'object.': 0.22; 'needed.': 0.23; 'seems': 0.23; 'header': 0.24; 'header:In-Reply- To:1': 0.25; 'am,': 0.27; 'expanding': 0.27; 'replace': 0.27; 'message-id:@mail.gmail.com': 0.27; 'contrast,': 0.29; 'objects': 0.29; "i'm": 0.29; 'code': 0.31; 'more,': 0.32; 'running': 0.32; 'certain': 0.33; 'to:addr:python-list': 0.33; 'received:google.com': 0.34; 'list': 0.35; 'received:209.85': 0.35; 'there': 0.35; 'list.': 0.35; 'but': 0.36; 'unable': 0.36; 'does': 0.37; 'why': 0.37; 'received:209': 0.37; 'subject:: ': 0.38; 'some': 0.38; 'to:addr:python.org': 0.39; 'header:Received:5': 0.40; 'your': 0.60; '30,': 0.62; 'between': 0.63; 'special': 0.73; 'cycles.': 0.84; 'to:name:python': 0.84
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=ygqh2w6UiAkyUFrSipgylhIdLs/QxKkO/iS6kaGbArc=; b=MmggHNQvYek0KkJXNQHQB2Y7wJTL5UnY+dt6byXZNB15Yb2+Adk+EAwPntfgL7CEo7 Z6pAXWPH5sPNsdvH4YFGwRxXwWgm7gl6NQ4FjpPisIfh2jB9mmVXSFLd8cLzaHcojRRK fNrkVjRT6sszcL5Wen+tLkPJrmOF/DvqHvFK9UT/5oCnEGpZKiiZOnBbGVDIR9kULJ/U 0Tt2zSZelxQB86wjHrfe47xgKpb3b5K4ufkZNWbhwbYBveEGAR9SPlgliaZf2xMMGnzN nhW2OrN6msQfHnjXgx6d6gr4Wsmp2UMLYgr5WUqEwhw2N7RKAlpifg6hzVk8FB/mdf7v PMeA==
MIME-Version: 1.0
In-Reply-To: <CALwzid=5xkJ-SjzvHDStdGSnwHV-u+_QjZJLNJ8+tAs_L0k-qA@mail.gmail.com>
References: <6998a955-7b34-4f4f-b8d6-62d1028f7561@googlegroups.com> <mailman.2986.1351482209.27098.python-list@python.org> <4c024364-84df-403b-8b9e-4a4c8f06121c@googlegroups.com> <mailman.2991.1351484760.27098.python-list@python.org> <cd108b9d-e951-463e-8e50-a5a024a27c7f@googlegroups.com> <508e6649$0$29967$c3e8da3$5496439d@news.astraweb.com> <508E1BC9.3000308@r3dsolutions.com> <CAPTjJmo3uK0Hfu4h78AkM6BPXXuU9sMuc74QeKFrW4jp54MGMg@mail.gmail.com> <508EC428.5080808@r3dsolutions.com> <CALvWhxvJYfSatk7wg8zcBUHm5rHqeqqJss=N2vjCShGy9XoD5A@mail.gmail.com> <CALwzid=5xkJ-SjzvHDStdGSnwHV-u+_QjZJLNJ8+tAs_L0k-qA@mail.gmail.com>
From: Ian Kelly <ian.g.kelly@gmail.com>
Date: Tue, 30 Oct 2012 02:46:46 -0600
Subject: Re: Negative array indicies and slice()
To: Python <python-list@python.org>
Content-Type: text/plain; charset=ISO-8859-1
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.3083.1351586837.27098.python-list@python.org>
Lines: 31
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1351586837 news.xs4all.nl 6874 [2001:888:2000:d::a6]:44121
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:32477

On Tue, Oct 30, 2012 at 1:21 AM, Ian Kelly <ian.g.kelly@gmail.com> wrote:
> I'm not entirely certain why collection objects get this special
> treatment, but there you have it.

Thinking about it some more, this makes sense.  The GC header is there
to support garbage collection for the object.  Atomic types like ints
do not need this header because they do not reference other objects
and so cannot be involved in reference cycles.  For those types,
reference counting is sufficient.  For types like collections that do
reference other objects, garbage collection is needed.

Expanding on this, I suspect it is actually a bug that slice objects
are not tracked by the garbage collector.  The following code appears
to result in a memory leak:

import gc
gc.disable()
while True:
    for i in range(100):
        l = []
        s = slice(l)
        l.append(s)
        del s, l
    _ = gc.collect()

Try running that and watch your Python memory usage climb and climb.
For contrast, replace the slice with a list and observe that memory
usage does *not* climb.  On each iteration, the code constructs a
reference cycle between a slice and a list.  It seems that because
slices are not tracked by the garbage collector, it is unable to break
these cycles.