Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #12473

Re: PC locks up with list operations

Path csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!aioe.org!feeder.news-service.com!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path <python-python-list@m.gmane.org>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.002
X-Spam-Evidence '*H*': 1.00; '*S*': 0.00; 'raised': 0.07; 'python': 0.08; 'scripts': 0.09; 'locked': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:80.91.229.12': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'received:lo.gmane.org': 0.09; 'tries': 0.15; 'mylist': 0.16; 'received:dip.t-dialin.net': 0.16; 'received:t-dialin.net': 0.16; 'stupid,': 0.16; 'wrote:': 0.16; 'linux': 0.17; 'subject:list': 0.18; 'memory': 0.21; '(most': 0.21; 'versions': 0.23; 'traceback': 0.24; 'up.': 0.26; "i'm": 0.27; 'skip:[ 10': 0.27; 'second': 0.29; 'script': 0.29; 'from:addr:web.de': 0.30; 'hours,': 0.30; 'list': 0.32; 'sort': 0.33; 'too': 0.33; 'there': 0.33; 'to:addr:python-list': 0.33; 'last):': 0.34; 'like:': 0.34; 'header:X-Complaints-To:1': 0.35; 'running': 0.35; 'anything': 0.36; 'starting': 0.36; 'couple': 0.36; 'file': 0.36; 'machine': 0.37; 'using': 0.37; 'but': 0.37; 'something': 0.37; 'received:org': 0.38; 'steven': 0.38; 'subject:: ': 0.39; 'header :Mime-Version:1': 0.39; 'subject:with': 0.39; 'to:addr:python.org': 0.39; 'happens': 0.40; 'more': 0.60; 'waiting': 0.63; 'weeks,': 0.77; 'instruct': 0.84; 'incidents': 0.93
X-Injected-Via-Gmane http://gmane.org/
To python-list@python.org
From Peter Otten <__peter__@web.de>
Subject Re: PC locks up with list operations
Date Wed, 31 Aug 2011 14:56:07 +0200
Organization None
References <4e5e2a0c$0$29965$c3e8da3$5496439d@news.astraweb.com>
Mime-Version 1.0
Content-Type text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding 7Bit
X-Gmane-NNTP-Posting-Host p50849d8f.dip.t-dialin.net
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.12
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.601.1314795359.27778.python-list@python.org> (permalink)
Lines 41
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1314795359 news.xs4all.nl 2406 [2001:888:2000:d::a6]:47199
X-Complaints-To abuse@xs4all.nl
Xref x330-a1.tempe.blueboxinc.net comp.lang.python:12473

Show key headers only | View raw

Steven D'Aprano wrote:

> Twice in a couple of weeks, I have locked up my PC by running a Python 2.5
> script that tries to create a list that is insanely too big.
> 
> In the first case, I (stupidly) did something like:
> 
> mylist = [0]*12345678901234
> 
> After leaving the machine for THREE DAYS (!!!) I eventually was able to
> get to a console and kill the Python process. Amazingly, it never raised
> MemoryError in that time.
> 
> The second time was a little less stupid, but not much:
> 
> mylist = []
> for x in itertools.combinations_with_replacement(some_big_list, 20):
>     mylist.append(func(x))
> 
> After three hours, the desktop is still locked up. I'm waiting to see what
> happens in the morning before rebooting.
> 
> Apart from "Then don't do that!", is there anything I can do to prevent
> this sort of thing in the future? Like instruct Python not to request more
> memory than my PC has?
> 
> I am using Linux desktops; both incidents were with Python 2.5. Do newer
> versions of Python respond to this sort of situation more gracefully?

If you are starting these scripts from the shell, how about ulimit?

$ ulimit -v 40000
$ python -c'print range(10**5)[-1]'
99999
$ python -c'print range(10**6)[-1]'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
MemoryError
$

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

PC locks up with list operations Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-08-31 22:33 +1000
  Re: PC locks up with list operations Chris Withers <chris@simplistix.co.uk> - 2011-08-31 13:40 +0100
    Re: PC locks up with list operations Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-08-31 22:47 +1000
      Re: PC locks up with list operations Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2011-09-01 11:31 +1200
        Re: PC locks up with list operations Tim Chase <python.list@tim.thechases.com> - 2011-08-31 19:53 -0500
      Re: PC locks up with list operations Nobody <nobody@nowhere.com> - 2011-09-12 13:19 +0100
    Re: PC locks up with list operations Robert Spanjaard <spamtrap@arumes.com> - 2011-08-31 16:06 +0200
    Re: PC locks up with list operations John Nagle <nagle@animats.com> - 2011-09-15 12:27 -0700
  Re: PC locks up with list operations Benjamin Kaplan <benjamin.kaplan@case.edu> - 2011-08-31 08:49 -0400
    Re: PC locks up with list operations Carl Banks <pavlovevidence@gmail.com> - 2011-09-13 06:31 -0700
  Re: PC locks up with list operations Peter Otten <__peter__@web.de> - 2011-08-31 14:56 +0200
  Re: PC locks up with list operations Rodrick Brown <rodrick.brown@gmail.com> - 2011-08-31 10:13 -0400
  Re: PC locks up with list operations Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-09-12 18:44 +1000
    Re: PC locks up with list operations Roy Smith <roy@panix.com> - 2011-09-12 07:40 -0400
      Re: PC locks up with list operations Terry Reedy <tjreedy@udel.edu> - 2011-09-12 16:53 -0400

csiph-web